Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Python DictWriter header - not writing in first line of existing file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Python DictWriter header - not writing in first line of existing file
# 1  
Old 06-27-2017
Python DictWriter header - not writing in first line of existing file
Hello

I am facing a very unique problem and not able to understand why. I have written a function which will check header of the file. If header is present good else it will write the header on top

Code:
def writeHeaderOutputCSV(fileName):
# See if the file exist already
    try:
        with open(fileName) as f:
            # Find the first line
            f.seek(0)
            head = f.readline()
            headList = [int(e) if e.isdigit() else e for e in head.split(',')]
            if headList == ['Customer', 'Alertkey', 'Node', 'Alertgroup', 'FirstOccurrence', 'TKT_Flag', 'X733SpecificProb', 'TKT_TicketNumber', 'TKT_Keyword\n']:
                # both file exist and header also exist
                logging.debug('writeHeaderOutputCSV - File ' + fileName + ' found and header is also found')
            else:
                  #file found but header not found write header
                with open(fileName,'a') as existFile:
                    writehead='Customer', 'Alertkey', 'Node', 'Alertgroup', 'FirstOccurrence', 'TKT_Flag', 'X733SpecificProb', 'TKT_TicketNumber', 'TKT_Keyword'
                    existFile.seek(0)
                    writer = csv.writer(existFile,delimiter=',')
                    writer.writerow(writehead)
                    # even dictwriter is also a problem not writing in first line but in the last line and then other datas are copied
                    logging.debug('writeHeaderOutputCSV - File ' + fileName + ' found but no header is found. So writing header')
            
    except:
        with open(fileName,'w') as NewFile:
            csv.DictWriter(NewFile,fieldnames=['Customer', 'Alertkey', 'Node', 'Alertgroup', 'FirstOccurrence', 'TKT_Flag', 'X733SpecificProb', 'TKT_TicketNumber', 'TKT_Keyword']).writeheader()
            logging.debug('writeHeaderOutputCSV - File ' + fileName + ' not found so header inserted in the new file')

So both in dictwriter and doing seek(0) and normal writing it is writing on the last line before other lines are written. If the file does not exist (except block) header is written at the top without an issue. But if the file exist but header is not present - then it is writing after the existing data Smilie

The above code screws up the whole writing to csv file also. It looks like it randomly deletes line or override or write from top

Code:
with open(fileName,'a') as existFile:
                    csv.DictWriter(existFile,fieldnames=['Customer', 'Alertkey', 'Node', 'Alertgroup', 'FirstOccurrence', 'TKT_Flag', 'X733SpecificProb', 'TKT_TicketNumber', 'TKT_Keyword']).writeheader()
                    logging.debug('writeHeaderOutputCSV - File ' + fileName + ' found but no header is found. So writing header')
                    existFile.close()

This one doesn't override while writing new data. But header row is written at the last line

---------- Post updated at 08:54 AM ---------- Previous update was at 12:43 AM ----------

Closing the thread : Solution as below
When calling the function it looks like it writes to the last line always

So in place replacement is not possible - we have to create temp file and swap it back

Code:
def writeHeaderOutputCSV(fileName):
    try:
        with open(fileName) as f:
            f.seek(0)
            head = f.readline()
            headList = [int(e) if e.isdigit() else e for e in head.split(',')]
            if headList == ['Customer', 'Alertkey', 'Node', 'Alertgroup', 'FirstOccurrence', 'TKT_Flag', 'X733SpecificProb', 'TKT_TicketNumber', 'TKT_Keyword\n']:
                logging.debug('writeHeaderOutputCSV - File ' + fileName + ' found and header is also found')
                f.close()
            else:
                df = read_csv(fileName)
                tempOutputFile = open(appPath+'output_temp.csv','w')
                df.columns = ['Customer' , 'Alertkey' , 'Node', 'Alertgroup', 'FirstOccurrence', 'TKT_Flag', 'X733SpecificProb', 'TKT_TicketNumber', 'TKT_Keyword']
                df.to_csv(tempOutputFile,index=False)
                tempOutputFile.close()
                with open(fileName,'w') as updateFile:
                    with open(appPath+'output_temp.csv','r') as readUpdateFile:
                        for allRows in readUpdateFile:
                            updateFile.write(allRows)
                logging.debug('writeHeaderOutputCSV - File ' + fileName + ' found but no header is found. So writing header - Investigate who deleted the header slowing down the program')
    except:
        with open(fileName,'w') as NewFile:
            csv.DictWriter(NewFile,fieldnames=['Customer', 'Alertkey', 'Node', 'Alertgroup', 'FirstOccurrence', 'TKT_Flag', 'X733SpecificProb', 'TKT_TicketNumber', 'TKT_Keyword']).writeheader()
            NewFile.close()
            logging.debug('writeHeaderOutputCSV - File ' + fileName + ' not found so header inserted in the new file')

Last edited by radioactive9; 06-27-2017 at 05:21 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Programming

Python Results Converted To C Struct Header File

I created python code that produce output in the form of: moses-red-sea=1.00.03 genesis-snake=2.03 deliverance=5.0.010 I need to take this output and create a "C" header file and have it look like this: struct { char *name; char *fixed_version; } filename_versions... (7 Replies)
Discussion started by: metallica1973
7 Replies

2. Shell Programming and Scripting

Inserting a new line into an already existing file

hello .. I am new to perl scripting, I have a text file, and would like to insert 3 new lines into the same file at different line numbers using perl scripting. Any Help regarding this will be very useful. say the file is sample.txt, contents are aaaaa bbbb ccccc dddd eeeee ffffffff... (4 Replies)
Discussion started by: hemalathak10
4 Replies

3. Shell Programming and Scripting

writing the timestamp to as a header in a file

hello mates, this is my first post. please help me out. i have got a file with some data in it. i am asked to write the timestamp as a header for that file. i mean the time the file created should be mentioned at the top of the file. i know we can use sed to insert a sentence but... (6 Replies)
Discussion started by: jdsony
6 Replies

4. Shell Programming and Scripting

Need help in writing a script to create a new text file with specific data from existing two files

Hi, I have two text files. Need to create a third text file extracting specific data from first two existing files.. Text File 1: Format contains: SQL*Loader: Release 10.2.0.1.0 - Production on Wed Aug 4 21:06:34 2010 some text ............so on...and somwhere text like: Record 1:... (1 Reply)
Discussion started by: shashi143ibm
1 Replies

5. UNIX for Dummies Questions & Answers

Add line with data to existing file

i have a file called motors with 3 columns separated with tabs eg: car make reg i want to create a executable file that will add a line with data eg: car make reg benz s600 t35778 can you please show me how to do this? (7 Replies)
Discussion started by: fletcher
7 Replies

6. UNIX for Dummies Questions & Answers

write new line at the beginning of an existing file

I was trying to find out the easiest way to write new line to the beginning of an exisiting file. I am using KSH. (5 Replies)
Discussion started by: sailussr
5 Replies

7. UNIX for Dummies Questions & Answers

Adding header to an existing file

Dear All, I need to add a header of one line to an already existing file. I know that it can be achieved by the following: echo "Header" > newfile cat file1 >> newfile But my problem is that file is huge and there is no space for creating a new file every time. Is there a way that I can... (5 Replies)
Discussion started by: shash
5 Replies

8. Shell Programming and Scripting

Remove header(first line) and trailer(last line) in ANY given file

Hi, I need some help in removing the header (first line) and the trailer (last line) in a give file... The data file actually comes in EBCDIC format and I converted it into ASCII.. Now I need to strip off the first line and the last line.. I think we can use sed to do something like this:... (2 Replies)
Discussion started by: madhunk
2 Replies

9. Shell Programming and Scripting

Need to add a line of data to already existing file in Unix..

Hello.. I have a text file with 100 lines of data. I need to add 1 line of data to that already existing file at the first line( beginning of the file) , so that the already existing 100 lines will start from 2 nd line.Now the file will have 101 lines of data. Help me on how to add the line... (4 Replies)
Discussion started by: charan81
4 Replies

10. UNIX for Dummies Questions & Answers

Print one line of Existing File

Hi all, I need to know what command/s I can use to print lines from a file. I cannot do pattern matching, so grepping for a string is not possible. I'd like to say "print line 3" etc. Any ideas? Ta. (2 Replies)
Discussion started by: danhodges99
2 Replies
Login or Register to Ask a Question