How to remove just LF from file?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to remove just LF from file?
# 1  
Old 03-01-2018
How to remove just LF from file?

I have a file with the data which looks like this

I open in Notepad++ and see the special characters

Code:
1111,FILE, TEST,787CR/LF
3333,FILE,78LF
7CR/LF

This is csv file transferred to unix. When i open in vi, there are three records with a ^M at the end of each line.

The 2nd and 3rd record are infact a single record. How do I remove the LF from the 2nd record and read 2nd and 3rd record as a single record.

I want the output as

Code:
1111,FILE, TEST,787CR/LF
3333,FILE,787CR/LF


Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!

Last edited by RudiC; 03-01-2018 at 05:31 PM.. Reason: Added CODE tags.
# 2  
Old 03-01-2018
Code:
tr -d '\r' < infile.csv > outfile.csv

This User Gave Thanks to Aia For This Post:
# 3  
Old 03-01-2018
I tried this, but its giving the same output, only this is when I open through vi there is no ^M character at the end..
# 4  
Old 03-01-2018
Please be aware that above is a non-standard file format on *nix which doesn't recognize those <CR> chars. You might need to remove those to get a standard *nix text file; see Aia's proposal.
For exactly your request and your data from post#1, try
Code:
awk '{while (!/\r$/) {getline X; $0 = $0 "," X}} 1' file
1111,FILE, TEST,787
3333,FILE,78,7

The <CR> chars from the input file are still there (as requested) but invisible in the printout.

Last edited by RudiC; 03-01-2018 at 06:08 PM..
# 5  
Old 03-02-2018
Code:
 sed -r  ':l /$/{ N; s/([^/])LF\n/\1/} ;bl' csvfile

This User Gave Thanks to abdulbadii For This Post:
# 6  
Old 03-02-2018
Quote:
Originally Posted by RudiC
Please be aware that above is a non-standard file format on *nix which doesn't recognize those <CR> chars. You might need to remove those to get a standard *nix text file; see Aia's proposal.
For exactly your request and your data from post#1, try
Code:
awk '{while (!/\r$/) {getline X; $0 = $0 "," X}} 1' file
1111,FILE, TEST,787
3333,FILE,78,7

The <CR> chars from the input file are still there (as requested) but invisible in the printout.

Thank you!!. This gives me what I want, but can you tell me how this works?
# 7  
Old 03-02-2018
If a line does not have a trailing <CR> ( = \r = 0x0D = ^M) it keeps getting more lines and appending them until a <CR> is found. Then print the entire construct as one line.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Remove lines from File.A based on criteria in File.B

Hello, I have two files of the following form. I would like to remove from File.A where the first three colum matches values in File.B to give the output in File.C File.A 121 54321 PQR CAT 122 765431 ABC DOG 124 98765 ZXY TIGER 125 86432 GEF LION File.B 122 765431 ABC 125 86432 GEF... (4 Replies)
Discussion started by: Gussifinknottle
4 Replies

2. Shell Programming and Scripting

awk remove/grab lines from file with pattern from other file

Sorry for the weird title but i have the following problem. We have several files which have between 10000 and about 500000 lines in them. From these files we want to remove lines which contain a pattern which is located in another file (around 20000 lines, all EAN codes). We also want to get... (28 Replies)
Discussion started by: SDohmen
28 Replies

3. Shell Programming and Scripting

ksh Loop through file one, remove lines from file two

Good Afternoon, I start with a file named biglist.txt. I have another file smallerlist. txt I want to remove the lines from smallerlist.txt from biglist.txt and leave those lines that do not reside in smallerlist.txt. Thanks !! (2 Replies)
Discussion started by: popeye
2 Replies

4. Shell Programming and Scripting

Remove comments from file with specific file name extensions

Hello Unix board community, I have to program a shell script, but I am a complete noob so I hope I get some help here. The assignment is as follows: The program removes all comments regardless of formatting or language from files with specific file name extensions (php, css, js, ...).... (3 Replies)
Discussion started by: TheZeusMan
3 Replies

5. Shell Programming and Scripting

Remove bad records from file and move them into a file then send those via email

Hi my requirement is that i want pull the bad records from input file and move those records in to a seperate file. that file has to be sent via email.. any suggentions please (1 Reply)
Discussion started by: sxk4999
1 Replies

6. Shell Programming and Scripting

How to remove blank lines in a file and save the file with same name?

I have a text file which has blank lines. I want them to be removed before upload it to DB using SQL *Loader. Below is the command line, i use to remove blank lines. sed '/^ *$/d' /loc/test.txt If i use the below command to replace the file after removing the blank lines, it replace the... (6 Replies)
Discussion started by: vel4ever
6 Replies

7. Shell Programming and Scripting

Remove duplicate lines from first file comparing second file

Hi, I have two files with below data:: file1:- 123|aaa|ppp 445|fff|yyy 999|ttt|jjj 555|hhh|hhh file2:- 445|fff|yyy 555|hhh|hhh The records present in file1, not present in file 2 should be writtent to the out put file. output:- 123|aaa|ppp 999|ttt|jjj Is there any one line... (3 Replies)
Discussion started by: gani_85
3 Replies

8. Shell Programming and Scripting

need to remove 1st line of a file and save the file with same old name

Hi friends, I have a doubt, I am not sure whether it is possible ah nu. I am having a file(sample.txt) which contain 5 lines. I want to remove 1st line in the file and save the file with same old name (sample.txt). For removing 1st line i am using sed 1d filename But dono how to... (3 Replies)
Discussion started by: natraj005
3 Replies

9. UNIX for Dummies Questions & Answers

how to copy a file without remove the contents of the target file?

Hello every body, Kindly support me to "copy a file without remove the contents of the target file" Thanks in advance. :) Ahmed Amer Cairo,Egypt (2 Replies)
Discussion started by: ahmedamer12
2 Replies

10. Shell Programming and Scripting

Batch file to remove double quotes from a file

Hi I have to create a windows batch file which will read an input file and remove "double quotes" from each line of the file. For eg.If input file name is abcd.csv and contents is : "asasd,123123,213213,asd" "esfrf,dsfsdf,234324,sdfsdf" Then the batch file should remove "" from the... (11 Replies)
Discussion started by: akashtcs
11 Replies
Login or Register to Ask a Question