Shell Programming and Scripting

I have a DOS .csv file (EOL=\r\n) that also contains some \n characters representing a new line within in some multi-line fields. I want to change the \r\n sequences to \n (like dos2unix does) but change the \n characters not followed by \r to |.

The second gsub in both code examples below seems to do nothing. It's like the \r characters got lost somewhere.

Code:

awk '{RS=""; gsub(/\n/,"|"); gsub(/"\r|"/,"\n"); print}' file

Code:

awk '{RS=""; gsub(/\n/,"|"); gsub(/\r\|/,"\n"); print}' file

Mike

Try setting RS before line is parsed:

Code:

awk '{gsub(/\n/,"|"); gsub(/\r\|/, "\n"); print}' RS='' file

This seem to work fine in GNU awk for my test file

Some more playing around and I'm also seeing the \r characters being truncated - BINMODE seems to help with GNU awk:

Code:

awk '{gsub(/\n/,"|"); gsub(/\r\|/,"\n"); print}' RS='' BINMODE=1 file

BINMODE=1

Thanks a lot! I doubt I would have ever figured that out! Did some more research and foundthat BINMOD=1 is the same as BINMODE=r (2: =w, 3: =rw). Letters seem more clear to me.

Mike

PS. An even more straightforward approach I tried and discarded now works as well:

Code:

 
awk '{gsub(/\n/,"|"); print}' RS='\r\n' BINMODE=1 file

---------- Post updated at 02:53 PM ---------- Previous update was at 02:06 PM ----------

This does not work in a pipe (just freezes). I was piping in from a tail command (stripping off headers from multiple files) so I had to change it.

Code:

 
awk 'FNR >=2 {gsub(/\n/,"|"); print}' RS='\r\n' BINMODE=r "$dataDir""Archive_"*".csv" "$dataDir""Exported_XXXXXXXX_Data_"*".csv"

tail was tolorant of missing files, awk is not so in one database I needed to create a blank archive.

Mike