Sunday, September 25, 2016

Stripping empty lines from a CSV file with python

In some processing I have to do, the comma-separated value (csv) files I'm using occasionally have a line of empty fields.

For example, instead of:

a,b,c,d,e

I get:

,,,,
Sometimes I get fewer commas, like:

,,

So how to strip these lines out of the file?

I have a function like this:
    @staticmethod
    def row_has_data(row):
        '''
        Detect whether a row in a csv file has data, or is blank.
        '''
        has_data = False
        for col in range(0, len(row)):
            if row[col] != '':
                has_data = True
        return has_data

Then I copy the csv file like this:

    def copy_to_csv(self, infile, has_header, outfile):
        '''
        Copy a file to a csv output file, skipping header if present.
        '''
        csv_writer = csv.writer(outfile, dialect='ats_global_dialect', encoding='UTF-8')
        infile.seek(0)
        csv_data = csv.reader(infile)
        if has_header:
            csv_data.next()  # skip header row
        for row in csv_data:
            if self.row_has_data(row):
                csv_writer.writerow(row)
        outfile.close()

No comments:

Post a Comment