Troubleshoot awk code that fixes line breaks


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Troubleshoot awk code that fixes line breaks
# 1  
Old 02-17-2016
Troubleshoot awk code that fixes line breaks

Hi. I'm new to UNIX. I've searched this forum and found an example of awk that works for my file to fix line breaks.

awk -F'|' 'NF != 48 || !$48 {printf $0; getline} 1' file1.csv > file2.csv

However, sometimes the line is broken across three lines. But if I execute this again, it fixes that broken line but the last record is repeated two or three times depending on how many times I run it.

1) How do I execute it once and get it to fix all the broken lines, regardless of how many lines the record takes up?
2) How do I get it to stop repeating the last record?
# 2  
Old 02-17-2016
How about (untested)
Code:
 while (NF < 48) {getline X; $0 = $0 X}

Please add a reasonable error checking, as this doesn't account for e.g. NF never reaching 48.
# 3  
Old 02-17-2016
That code alone gives a syntax error. I don't know how to write it correctly yet.
# 4  
Old 02-17-2016
Please post a representative sample.
# 5  
Old 02-17-2016
A sample of the file?
# 6  
Old 02-17-2016
Yes, please.
# 7  
Old 02-17-2016
Below is the header and a few records with one record in the middle that the user has used the enter key on the vendor form, causing it to go over three lines. In this example, all records begin with ,"abc". (Data is not real to protect sensitive information.)

"Account Name","Community Name","Last Name","First Name","ID_number","Admin Gift Pledge ID","Admin Gift Source","Donor Instructions","Gift Other Designation","Matching Gift Company old","Tribute Name","Tribute Notification","Tribute Type","Tribute ID Number","Premiums","Admin Gift Method","Gift Admin Initials","Campaign","Campaign ID","Order Total","Transaction ID","Purchase Date","Authorization Code","Customer Trans Number","Retrieval Code","Appeal Code","From Facebook","Billing Name","Credit Card Type","Billing Email","Billing Phone","Billing Street 1","Billing Street 2","Billing City","Billing State","Billing Zip","Billing Country","Scheduled Payments Active","Payment Frequency","Total Scheduled Amount","Next Payment Amount","Next Payment Date","Payments Remaining","Payment Type","Amount To Date","First Payment Date","Original Transaction Id","Event Name"

,"abc","Anderson","Tom",,,,," Gift for Instruction",,"Mickey","mm@att.net","in honor of someone special",,,,,"API Make a Gift","7440","500.00","56370","1/15/2016 11:29:52 AM","04442D","20160115000004",,"13529","False","Tom Anderson","VISA","tm@hotmail.com",,"1234 S. Dorchester",,"Chicago","IL","60637","US",,,,,,,"OT",,,,
,"abc","Town","Rob",,,,"The Town Family
1234 Grovecrest Cv.
Germantown, TN 38139","Big Event",,"Ray E Town",,"in memory of someone special",,,,,"Make a Gift","4045","50.00","56357","1/15/2016 7:25:48 AM","01530P","20160115000005",,"10063","False","Rob Town","Disc","rob.town@gmail.com",,"2693 Saps Ct.","****************","Burlington","KY","41005","US",,,,,,,"OT",,,,
,"abc","Wills","Alaina","000022333",,,,,,,,,,,,,"Donation Form","4676","15.00","56354","1/15/2016 7:07:29 AM","0332B","20160115000004",,,"False","Alaina Wills","Mast","alaina.wills@gmail.com","4444449986","64 central blvd apt 2","****************","Whitestown","IN","46075","US","True","12",,"15.00","2/15/2016 5:06:27 PM",,"PT","45.00","10/23/2015 5:06:27 PM","45989",
,"abc","Walsh","Tim","0000123400",,,,,,,,,,,,,"Make a Gift","4045","90.00","56316","1/15/2016 7:05:08 AM","0z098","20160000000",,,"False","Tim Walsh","Disc","tw@hotmail.com","****************","5823 Gram Court","****************","Indianapolis","IN","46250","US","True","12",,"90.00","2/15/2016 1:30:01 PM",,"PT","1170.00","1/15/2015 1:30:01 PM","18932",
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Linux

Line breaks in mail spool

Hi, I have an issue with the line breaks in the mail spool- /var/mail/user1. I have set up a script to go through the mail spool on one of the users and parse some parts of the mail, however there doesn't seem to exist the regular line endings CR, LF or both in the lines and this is breaking my... (4 Replies)
Discussion started by: night_watcher
4 Replies

2. Shell Programming and Scripting

[BASH] read 'line' issue with leading tabs and virtual line breaks

Heyas I'm trying to read/display a file its content and put borders around it (tui-cat / tui-cat -t(ypwriter). The typewriter-part is a 'bonus' but still has its own flaws, but thats for later. So in some way, i'm trying to rewrite cat using bash and other commands. But sadly it fails on... (2 Replies)
Discussion started by: sea
2 Replies

3. HP-UX

After using @, line breaks for a particular user in shell

Dear Concern, When we using @ sign, line breaks for a particular user in shell. Please advise how to resolve from the problem in HP UX. tabs@tabsdb02:/ccbs/users/tabs$ cat /etc/passwd|grep tabs tabs:RdCgOsmKee7Ps:221:201::/ccbs/users/tabs:/usr/bin/ksh... (3 Replies)
Discussion started by: makauser
3 Replies

4. UNIX for Dummies Questions & Answers

Page breaks and line breaks

Hi All, Need an urgent solution to an issue . We have created a ksh file or shell script which generates 1 DAT file. the DAT file contains extract of a select statement . Now the issue is , when we are executing the ksh file , the output is coimng with page breaks and line breaks . We have... (4 Replies)
Discussion started by: Ayaskant
4 Replies

5. Programming

Clean and keep line breaks

Hello, I want to keep line spaces in comments but clean more then 2 after each. Example: $sentence="This is my first sentence This will be in a new row This will be too in a new row but not separated with 3line breaks just with one "; And i want to... (1 Reply)
Discussion started by: AimyThomas
1 Replies

6. Shell Programming and Scripting

Remove line breaks after a match

I need to remove all line breaks in a document after a match, until there is a blank line. Example below, after the match "THE GREEN TABLE" remove line breaks until a blank line. Then, after the match "THE BLUE TABLE" do the same. Before: THE GREEN TABLE Lorem ipsum dolor sit amet,... (14 Replies)
Discussion started by: dockline
14 Replies

7. Shell Programming and Scripting

Help with wc and line breaks

Hi everyone, I have gone through the forum trying to find an answer to this question but was unsuccessful. I am hoping that someone can help me with this please. I am trying to get my script to recognise line breaks from a file and to give me a result for wc of each line. So basically, if you... (7 Replies)
Discussion started by: stargazerr
7 Replies

8. Shell Programming and Scripting

any better way to remove line breaks

Hi, I got some log files which print the whole xml message in separate lines: e.g. 2008-10-01 14:21:44,561 INFO do something 2008-10-01 14:21:44,561 INFO print xml : <?xml version="1.0" encoding="UTF-8"?> <a> <b>my data</b> </a> 2008-10-01 14:21:44,563 INFO do something again I want... (3 Replies)
Discussion started by: csmklee
3 Replies

9. UNIX for Dummies Questions & Answers

Newbie ? Need Help with If/Then & Line Breaks...

I hope this makes sense, but I need help with what is indicated with %%, below : A.) ####List active servers and send to file#### # ps -ef | grep jboss | grep sh | awk '{if ($14) {print $12;}else {print $11}}' | sort > /export/home/kthatch/script_results client302 client306 client309... (1 Reply)
Discussion started by: kthatch
1 Replies
Login or Register to Ask a Question