Hello Guys,

 

I found this code really useful and wanted to post it for future reference. At work , at times there is a need to provide data in excel. Unfortunately Microsoft Excel cannot load data which is more than 80K.  In this case scripts like splitting data comes in handy.

The code first creates a function and then its called by passing the file that needs to be split. The code hightlighted in red is all that needs to be edited for your needs .

 

Till next time

Cheers!!

Reference : https://stackoverflow.com/questions/36445193/splitting-one-csv-into-multiple-files-in-python

 

 

Code

import os

def split(filehandler, delimiter=',', row_limit=500000,
 output_name_template='Out_Put_Format_%s.csv', output_path='.', keep_headers=True):
 import csv
 reader = csv.reader(filehandler, delimiter=delimiter)
 current_piece = 1
 current_out_path = os.path.join(
 output_path,
 output_name_template % current_piece
 )
 current_out_writer = csv.writer(open(current_out_path, 'w'), delimiter=delimiter)
 current_limit = row_limit
 if keep_headers:
 headers = next(reader)
 current_out_writer.writerow(headers)
 for i, row in enumerate(reader):
 if i + 1 > current_limit:
 current_piece += 1
 current_limit = row_limit * current_piece
 current_out_path = os.path.join(
 output_path,
 output_name_template % current_piece
 )
 current_out_writer = csv.writer(open(current_out_path, 'w'), delimiter=delimiter)
 if keep_headers:
 current_out_writer.writerow(headers)
 current_out_writer.writerow(row)



split(open('C:\\Users\\Jethin.Abraham\\Documents\\File_to_be_split', 'r'));