I found this code really useful and wanted to post it for future reference. At work , at times there is a need to provide data in excel. Unfortunately Microsoft Excel cannot load data which is more than 80K. In this case scripts like splitting data comes in handy.
The code first creates a function and then its called by passing the file that needs to be split. The code hightlighted in red is all that needs to be edited for your needs .
Till next time
Reference : https://stackoverflow.com/questions/36445193/splitting-one-csv-into-multiple-files-in-python
import os def split(filehandler, delimiter=',', row_limit=500000, output_name_template='Out_Put_Format_%s.csv', output_path='.', keep_headers=True): import csv reader = csv.reader(filehandler, delimiter=delimiter) current_piece = 1 current_out_path = os.path.join( output_path, output_name_template % current_piece ) current_out_writer = csv.writer(open(current_out_path, 'w'), delimiter=delimiter) current_limit = row_limit if keep_headers: headers = next(reader) current_out_writer.writerow(headers) for i, row in enumerate(reader): if i + 1 > current_limit: current_piece += 1 current_limit = row_limit * current_piece current_out_path = os.path.join( output_path, output_name_template % current_piece ) current_out_writer = csv.writer(open(current_out_path, 'w'), delimiter=delimiter) if keep_headers: current_out_writer.writerow(headers) current_out_writer.writerow(row) split(open('C:\\Users\\Jethin.Abraham\\Documents\\File_to_be_split', 'r'));