\r getting lost in awk


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting \r getting lost in awk
# 1  
Old 05-21-2015
\r getting lost in awk

I have a DOS .csv file (EOL=\r\n) that also contains some \n characters representing a new line within in some multi-line fields. I want to change the \r\n sequences to \n (like dos2unix does) but change the \n characters not followed by \r to |.

The second gsub in both code examples below seems to do nothing. It's like the \r characters got lost somewhere.
Code:
awk '{RS=""; gsub(/\n/,"|"); gsub(/"\r|"/,"\n"); print}' file

Code:
awk '{RS=""; gsub(/\n/,"|"); gsub(/\r\|/,"\n"); print}' file

Mike
# 2  
Old 05-21-2015
Try setting RS before line is parsed:

Code:
awk '{gsub(/\n/,"|"); gsub(/\r\|/, "\n"); print}' RS='' file

This seem to work fine in GNU awk for my test file
# 3  
Old 05-21-2015
Some more playing around and I'm also seeing the \r characters being truncated - BINMODE seems to help with GNU awk:

Code:
awk '{gsub(/\n/,"|"); gsub(/\r\|/,"\n"); print}' RS='' BINMODE=1 file

This User Gave Thanks to Chubler_XL For This Post:
# 4  
Old 05-21-2015
BINMODE=1

Thanks a lot! I doubt I would have ever figured that out! Did some more research and foundthat BINMOD=1 is the same as BINMODE=r (2: =w, 3: =rw). Letters seem more clear to me.

Mike

PS. An even more straightforward approach I tried and discarded now works as well:

Code:
 
awk '{gsub(/\n/,"|"); print}' RS='\r\n' BINMODE=1 file

---------- Post updated at 02:53 PM ---------- Previous update was at 02:06 PM ----------

This does not work in a pipe (just freezes). I was piping in from a tail command (stripping off headers from multiple files) so I had to change it.
Code:
 
awk 'FNR >=2 {gsub(/\n/,"|"); print}' RS='\r\n' BINMODE=r "$dataDir""Archive_"*".csv" "$dataDir""Exported_XXXXXXXX_Data_"*".csv"

tail was tolorant of missing files, awk is not so in one database I needed to create a blank archive.

Mike

Last edited by Michael Stora; 05-21-2015 at 06:59 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk call in bash function called with arugments not working, something lost in translation?

Hello, I have this awk code in a bash script to perform a find and replace task. This finds one unique line in a file and substitutes the found line with a replacement. #! /bin/bash # value determined elsewhere total_outputs_p1=100 # file being modified... (5 Replies)
Discussion started by: LMHmedchem
5 Replies

2. UNIX for Dummies Questions & Answers

I am so lost !!!

I am taking an online Linux class, which is mandatory for me to graduate. Unfortunately I got the worst online teacher ever, he never respond to any e-mails. I have no idea what going on in this class so I am turning to you guys. Here is the questions for the week, Ok I don't expect anybody to... (1 Reply)
Discussion started by: Thunderbunny
1 Replies

3. UNIX for Dummies Questions & Answers

Lost my Way

I want to execute my own utilities from my bin folder without having to specify paths at run time - i.e. just enter name on command line - to do this my search path needs to be set up at login time - when I do this the path is not getting set right apparently(?) - PATH echos OK but the search... (0 Replies)
Discussion started by: robert stansel
0 Replies

4. UNIX for Dummies Questions & Answers

I am so lost...

I have been using linux now for about 6 months. I like it although it was quite a learning process. Pretty simple for the most part. Here is my prob... The first time I installed linux on my computer it went in just fine, no problems. It did eventually crash though and I had to re-install it.... (2 Replies)
Discussion started by: Treb
2 Replies

5. UNIX for Dummies Questions & Answers

I'm so lost...

Fair warning... I know nothing of Unix. That's why I joined this forum and I need your feedback desperatly. Here is the situation... I built an app using ASP and Northcode's SWF Studio v3 for a cd launcher tool for a client that develops software. They like the app and it works well on... (2 Replies)
Discussion started by: daJabberwocky
2 Replies

6. Filesystems, Disks and Memory

Lost Data Lost Admin

First time so excuse my ignorance please. I may not be accurately describing the issue. I have inherited a small lab mostly SUN V120s. We lost power and are trying to recover. Nope no backups... The primary issue I have is 1 box is an Oracle Server. It has 2 36Gb harddrives. I am able to... (3 Replies)
Discussion started by: murphsr
3 Replies

7. UNIX for Dummies Questions & Answers

So lost

I have been reading up on starting a website, but i am still lost. I am in much need of assistance. Kind of a step by step because i still don't know where to start. please help. thank you (1 Reply)
Discussion started by: sweetie020602
1 Replies

8. AIX

lost errlog

Hi, i have a problem with mi Aix system, the errlog file is initialized to 0, and i can't write in it, when i run the errpt teh error 0315-180 logread: UNEXPECTED EOF 0315-171 Unable to process the error log file /var/adm/ras/errlog. 0315-132 The supplied error log is not valid:... (2 Replies)
Discussion started by: gonmeres
2 Replies

9. Shell Programming and Scripting

lost again

will this script work? I want to use it in unix. clear ans='y' While test $ans='y' do notfound() { echo $response " is not logged in" } found() { echo $response "is logged into " $name } name=`ps -eaf | grep "\<$response\>" | cut -b 80-85` if ; then name=`rsh server2 ps -eaf... (7 Replies)
Discussion started by: azman
7 Replies
Login or Register to Ask a Question