Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Replace ^M and the new line that follows it in a delimited file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Replace ^M and the new line that follows it in a delimited file
# 1  
Old 09-25-2014
Lightbulb Replace ^M and the new line that follows it in a delimited file
Hello,


I want to Replace/Remove ^M and the new line that follows it in a delimited file.

So far I have tried following and nothing seems to work for me .
Code:
Tr –d ‘\r\n’ < old.dat > new.dat

-removes all the linefeed, not just the ones after a ^M.

Code:
Sed ‘/^M$/{N; s/.\n//;}’ < old.dat > new.dat

Sed ‘/^M$/{s///; $p; N; s/\n//;}’ < old.dat > new.dat

Sed ‘/^M$/ {$d; N; s/.\n//;}’ < old.dat > new.dat

Sed ‘/^M/{s///; N; s/\n//;}’ < old.dat > new.dat



Any help will be appreciated.


Right now file looks like this
Code:
COL1~|COL2~|COL3~|COL4
app~|ban~|cat~|dog

ABD~|123~|
efg~|456
bill~|gill~|till~|fill


expected result

Code:
COL1~|COL2~|COL3~|COL4
app~|ban~|cat~|dog

ABD~|123~|efg~|456
bill~|gill~|till~|fill

Thanks







Moderator's Comments:
Mod Comment Please use code tags next time for your code and data. Thanks
Last edited by vbe; 09-25-2014 at 12:35 PM..
# 2  
Old 09-25-2014
Code:
awk ' { while(/\r/) { sub(/\r/, ""); A=$0 ; getline ; $0=A$0 } } 1' inputfile > outputfile

# 3  
Old 09-25-2014
Thanks for the reply, I tried the awk cmd, still no luck.
# 4  
Old 09-25-2014
Can you attach a sample of the input file...to peek inside and see if those characters "\r\n" are actually present...
# 5  
Old 09-25-2014
Attach is the sample file.

What I was thinking following cmd gives me the exact row where the problem is
awk -F "~|" 'NF !=37' ADDR1.dat

what if the script does the following work,

counts the number of columns, and, if it's less than however many there should be, tries to merge it with the next line.
And if I can pass in the number of columns expected as a command-line parameter.

# 6  
Old 09-25-2014
File attached
ADDR1.txt (3.1 KB) 
# 7  
Old 09-25-2014
Your file contains no carriage returns at all.

Code:
awk -F"~|" -v OFS="~|" 'NF!=37 { A=$0 ; getline; $0=A$0 } 1' ADDR1.txt

This User Gave Thanks to Corona688 For This Post:
bluestarmoon
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replace delimiter for a particular column in a pipe delimited file

I have an input file as below Emp1|FirstName|MiddleName|LastName|Address|Pincode|PhoneNumber 1234|FirstName1|MiddleName2|LastName3| Add1 || ADD2|123|000000000 Output : 1234|FirstName1|MiddleName2|LastName3| Add1 ,, ADD2|123|000000000 OR 1234,FirstName1,MiddleName2,LastName3, Add1 ||... (2 Replies)
Discussion started by: styris
2 Replies

2. UNIX for Beginners Questions & Answers

Replace a column in tab delimited file with column in other tab delimited file,based on match

Hello Everyone.. I want to replace the retail col from FileI with cstp1 col from FileP if the strpno matches in both files FileP.txt ... (2 Replies)
Discussion started by: YogeshG
2 Replies

3. Shell Programming and Scripting

Replace a number in the last line of a delimited file.

Hi all, I am fairly new to UNIX and I was wondering if you could provide me with some help! Lets say i have a file as below : Line 1 Line 2 Line 3 ABC|12|4|2 Now the number 4 in bold, this number will represent the number of row there is in the file excluding the header and footer... (10 Replies)
Discussion started by: Stinza
10 Replies

4. Shell Programming and Scripting

Replace field in the delimited file

Hi, I have the requirement similar to the one mentioned in the below thread. https://www.unix.com/unix-for-dummies-questions-and-answers/128155-search-replace-string-only-particular-column-delimited-file.html The only difference is that I need to change the field for row 1,2 and the last... (14 Replies)
Discussion started by: chetanojha
14 Replies

5. UNIX for Dummies Questions & Answers

Replace period in a tab delimited file to a number

I have a file like this. It is tab delimited. Unfortunately, the missing data was filled in with a period "." (see the leading lines 1-5 columns) I want to substitute the periods for misisng data with an integer "-999". however, I do not want the global replace to change the other periods seen... (7 Replies)
Discussion started by: genehunter
7 Replies

6. Shell Programming and Scripting

Replace specific column range in a non-delimited file with a string!

Hi All, I will need an help with respect to replacing a range of columns on a non-delimited file using a particular string pattern. Say file input is MYNUMBERD000000-BAN CHUE INSNTS ** N+ MYAREDSDD000000+BAN CHUE INSNTS ** N+ MYDERFFFSD00000-GIR PENT - ACH ** ... (5 Replies)
Discussion started by: navojit dutta
5 Replies

7. Shell Programming and Scripting

Replace col 23 - 26 with new value, non delimited file

hello, i have a undelimited file which contains 229 byte records. i want to change column 23 - 26 with a new value and also change the sign of the data in colulmn 30 - 70. i've tried SED for the first change, but nothing happens: sed 's/\(^.\{22\}\).\{4\}\(.*\)/\0603\2/' inputfile heres an... (8 Replies)
Discussion started by: blt123
8 Replies

8. UNIX for Dummies Questions & Answers

What awk 1-liner will replace value in 1stField of a delimited file with the value of '5' ?

Hi, I am a newbie to awk. Here is my problem. Looking for an awk 1-liner to solve it: My Computing Environment: - Solaris10 - I prefer to use csh or sh shells 1. Lets say my input file is File1.dat (delimter = | ) and looks as follows: (File1.dat) ... (1 Reply)
Discussion started by: andy b
1 Replies

9. Shell Programming and Scripting

replace 3rd field of space delimited text file

how to replace the 3rd colum? Each line begins similarly, but they all ends variously. XX YY 03 variable text here XX YY 03 more variable text here XX YY 03 even more variable text here really long setence XX YY 03 variable numbers also appear 03 11. 123 456 XX YY 03 the occasional comma,... (4 Replies)
Discussion started by: ajp7701
4 Replies

10. UNIX for Dummies Questions & Answers

Search and replace string only in a particular column in a delimited file

I have file with multiple columns. Column values for a record may be same. Now i have to replace a column value(this can be same for the other columns) with new value. File.txt A,B,C,D,A,B,C,D,A,B,C,D A,B,C,D,A,B,C,D,A,B,C,D A,B,C,D,A,B,C,D,A,B,C,D A,B,C,D,A,B,C,D,A,B,C,D... (1 Reply)
Discussion started by: ksailesh
1 Replies
Login or Register to Ask a Question