How to remove new line character and append new line character in a file?


 
Thread Tools Search this Thread
Operating Systems HP-UX How to remove new line character and append new line character in a file?
# 1  
Old 10-26-2010
How to remove new line character and append new line character in a file?

Hi Experts,

I have data coming in 4 columns and there are new line characters \n in between the data. I need to remove the new line characters in the middle of the row and keep the \n character at the end of the line.

File is comma (,) seperated.

Eg:
Code:
ID,Client ,SNo,Rank
37,Airtel \n Private \n limited,100,999\n
38,Vodaphone India \n Private Limited,200,888\n
39,Dell Limited,300,777\n
40,HP India Ltd\n,400,666\n

Delete red colour new line characters

Output should be :
Code:
ID,Client ,SNo,Rank
37,Airtel Private  limited,100,999\n
38,Vodaphone India Private Limited,200,888\n
39,Dell Limited,300,777\n
40,HP India Ltd,400,666\n

Here my requirement is , don't delete the \n when it comes after every 3rd comma and delete new line character (\n) in between.

Pl provide the solution to achive this.

Regards
Kari

Last edited by radoulov; 10-26-2010 at 02:48 PM.. Reason: Code tags, please!
# 2  
Old 10-26-2010
Code:
sed 's: *\\n *: :g;s: *, *:,:g' infile

This would also remove space touching coma.

or
Code:
sed 's:\\n: :g;s:  *: :g' infile

s substitute
: separator
\\n pattern to replace (the \ is used to desactivate the \n )
<space> new pattern
g subtitution takes place globaly in the line even if more than 1 occurrence occur

the second substitution crush any number of space character into one in the line so you will never have more than 1 single space


Code:
# cat in
ID,Client ,SNo,Rank
37,Airtel \n Private \n limited,100,999\n
38,Vodaphone India \n Private Limited,200,888\n
39,Dell Limited,300,777\n
40,HP India Ltd\n,400,666\n
# sed 's:\\n: :g;s:  *: :g' in
ID,Client ,SNo,Rank
37,Airtel Private limited,100,999
38,Vodaphone India Private Limited,200,888
39,Dell Limited,300,777
40,HP India Ltd ,400,666
#

... could be gather in one substitution

Code:
sed 's: *\\n *: :g' in

a \n preceeded or followed by any number of space (even 0) is replaced by one single space

Last edited by ctsgnb; 10-26-2010 at 02:35 PM..
These 2 Users Gave Thanks to ctsgnb For This Post:
# 3  
Old 10-27-2010
Hi,
Thanks a lot for the reply.

I also get the data like below ,
Code:
ID,Client ,SNo,Rank
37,Airtel 
 Private  
limited,100,999
38,Vodaphone India 
 Private Limited,200,888
39,Dell Limited,300,777
40,HP India Ltd
,400,666
41,Orange

 India 
Private 


Limited,500,555


In the last row , there are more new lines.

There are new line characters in between the lines, and so when i view the data in test pad i can see the new line characters in between, but actually it shoud be at only endof the line. (This is occuring like this, because in source data is entering in xl doc and there are multiple Alt + Enters in particular fields)

Output should be
Code:
ID,Client ,SNo,Rank
37,Airtel  Private  limited,100,999
38,Vodaphone India Private Limited,200,888
39,Dell Limited,300,777
40,HP India Ltd,400,666
41,Orange India Private Limited,500,555

Here requirement is to delete the new line characters which occur in between the every 3 commas.
Like select first 3 commas and delete the new line characters in between and then again select the next 3 commans and delete if any new line in between (Don't delete new line which is coming after 3 commas) .....like this delete till the end of the file.

Pl provide me the solution for this

Regards
Kari

Last edited by Scott; 10-28-2010 at 01:06 PM.. Reason: Please use code tags
# 4  
Old 10-27-2010
Code:
use strict;
use warnings;

undef $/;
my $file_name="unix.txt";
open (FIN, "$file_name");
my $file=<FIN>;
close (FIN);

$file=~ s{([^,]+,){3}\w+}{&clean_up($&)}ges;

open (FOUT, ">Output_$file_name");
print FOUT $file;
close (FOUT);

sub clean_up
{
	my ($text)=@_;
	$text=~ s{\n}{ }g;
	$text=~ s{  +}{ }g;
	$text=~ s{ ,}{,}g;
	$text=~ s{^ +}{}g;
	return "$text\n";
}

This User Gave Thanks to k_manimuthu For This Post:
# 5  
Old 10-27-2010
Code:
echo `/usr/xpg4/bin/grep -vE "^[:blank:]*$" inputfile | sed 's|^\([0-9]\)|:\1|;s|\([0-9]\)$|\1:|' | tr '\n' ' '`| sed 's|: *:|:|g' | awk -F: '{print$0}' RS=:

Code:
# cat in
ID,Client ,SNo,Rank
37,Airtel
Private
limited,100,999
38,Vodaphone India
Private Limited,200,888
39,Dell Limited,300,777
40,HP India Ltd
,400,666
41,Orange

India
Private


Limited,500,555

# echo `/usr/xpg4/bin/grep -vE "^[:blank:]*$" in | sed 's|^\([0-9]\)|:\1|;s|\([0-9]\)$|\1:|' | tr '\n' ' '`| sed 's|: *:|:|g' | awk -F: '{print$0}' RS=:
ID,Client ,SNo,Rank
37,Airtel Private limited,100,999
38,Vodaphone India Private Limited,200,888
39,Dell Limited,300,777
40,HP India Ltd ,400,666
41,Orange India Private Limited,500,555


#

# 6  
Old 10-27-2010
Thanks a lot , it is perfectly working fine.

And mainly i have 128 columns in my source file , could you please let me know where exactly need changes to this code OR how to handle this scenario when i have 128 columns in the file.

Could you also explain me about this code (Sorry i couldn't understand this code , but it is perfectly working to my scenario).

Please provide me the solution for this.

Thanks a ton for the help

Regards,
Kari
# 7  
Old 10-27-2010
/usr/xpg4/bin/grep -vE "^[:blank:]*$" inputfile remove blank lines
sed 's|^\([0-9]\)|:\1|;s|\([0-9]\)$|\1:|' add ":" at beginning of line starting with a number and at the end of lines ending with a number
| tr '\n' ' ' put everything within one line
sed 's|: *:|:|g' substitute the pattern "two colons separated by any number of space (even 0)" by a single colon
awk -F: '{print$0}' RS=: print result using colon as Field separator and record separator
This User Gave Thanks to ctsgnb For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Append each line based upon the character size

I have a huge file which contains multiple lines. It need to check whether character length is not more than 255 each line. If its not then it should remove the character up to column. I have described in the output below. If its more than that the next line should start with call but if the... (1 Reply)
Discussion started by: JoshvaPeter
1 Replies

2. UNIX for Dummies Questions & Answers

How to remove $ or new line character in a file?

Hi All, Could any one suggest how to remove $ symbol in a text file when i am opening in vi editor. Scenario; For example iam having a file name aaa.txt the data inside the file is like sample name when i am opening in vi editor The same file resembles like below when i am... (1 Reply)
Discussion started by: Chandru_Raj
1 Replies

3. Shell Programming and Scripting

How to Remove comma as last character in end of last line of file?

how to Remove comma as last charector in end of last line of file: example: input file --------------- aaaaaa, bbbbbb, cccc, 12345, ____________ output file : ----------- aaaaaa, bbbbbb, (6 Replies)
Discussion started by: RahulJoshi
6 Replies

4. Shell Programming and Scripting

Remove new line character and add space to convert into fixed width file

I have a file with different record length. The file as to be converted into fixed length by appending spaces at the end of record. The length should be calculated based on the record with maximum length in the file. If the length is less than the max length, the spaces should be appended... (4 Replies)
Discussion started by: Amrutha24
4 Replies

5. Shell Programming and Scripting

Remove the last character (,) for every line in a file

Good afternoon: im working wih 2 files to find differences and use the cmp command cmp file1 file2 file1 file2 are are diifferent char 302 line1 i found what the difference is with the sed command and that is the file1 at the end of every line has a (,) (comma) character. i.e sed -n... (4 Replies)
Discussion started by: alexcol
4 Replies

6. Shell Programming and Scripting

How to remove new line character at end of file.

I need to remove new line character from end of file. Suppose here are content. a|b|c|d|r a|b|c|d|r a|b|c|d|r <new line> that means file contains 4 lines but data is there in 3 lines. so I want that only 3 lines should be there in file. Please help (20 Replies)
Discussion started by: varun940
20 Replies

7. Shell Programming and Scripting

How to remove line break character in a file

Hi, we are trying to process a csv file,in which we are getting data with line breaks.How to remove the line break character in the file? when i try to print the line break charcter using od -c,it gives as '\n' character for both line break and line feed. Please provide your valuable... (6 Replies)
Discussion started by: cnraja
6 Replies

8. Shell Programming and Scripting

How to append a character to the last but one field on a specific line?

Hi Guys, I have a file like this: aaa b c d e f fsss g h i k l qqq r t h n I want: aaa b c d e f fsss g h i k l qqq r t h , n ggg p t e d u qqq i o s , k (2 Replies)
Discussion started by: npatwardhan
2 Replies

9. Shell Programming and Scripting

append a character at end of each line of a file

Hi, i want to append a character '|' at end of each line of a file abc.txt. for example if the file abc.txt conatins: a|b|c 1|2|33 w|2|11 i want result file xyz.txt a|b|c| 1|2|33| w|2|11| I know this is simple but sumhow i am not able to reach end of line. its urgent, thanks for... (4 Replies)
Discussion started by: muaz
4 Replies

10. Shell Programming and Scripting

Using SED to append character to each line

Hey - my first post here, and I'm a total SED newb. I've looked around for previous help on this, but have so far been unsuccessful. I have a program (AMStracker for OS X) that outputs data in the terminal. Output is in this form: . . . 3 0 -75 3 0 -76 3 0 -77 ... (4 Replies)
Discussion started by: c0nn0r
4 Replies
Login or Register to Ask a Question