Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

New line characted issue

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting New line characted issue
# 1  
Old 05-18-2010
Question New line characted issue
Hi

Inorder to eliminate new line characters and the empty lines from our data and I used the below syntax :
Code:
awk '/^[1-9]/{print ""};{gsub("\"\"", "");printf"%s", $0};END{print ""}' | sed '/^$/d'

and eliminated all the new line char and the empty lines in my data.

But except the below sample data. This is one single record and not 2 records. When I use the above mentioned syntax, AWK eliminates the new line char but it considers this as 2 records

Code:
11/18/2009 9:41:48|11/4/2009 0:00:00|"1-237319574"|||"8"|"5"|"4"|"7"|"7"|"3"|"10"|"8"|||"8"|"9"||"

1. Entering the serial number a second time forces me to use phone service rather than online.
2. I noticed that with my new PS3, when I tried to download a movie, it said another PS3 (the one I sent in) is still activated. How am I supposed to deactivate"|10/26/2009 0:00:00.

Can someone pls help me.

Thanks.
Last edited by Franklin52; 05-18-2010 at 05:56 AM.. Reason: Please use code tags!
# 2  
Old 05-18-2010
are lines #1 and #2 supposed to be part of the sample data to be handled by the code? They appear to be unrelated content otherwise...
# 3  
Old 05-18-2010
Data New line character issue
Yes, they are. The entire below data is a single record.

Code:
11/18/2009 9:41:48|11/4/2009 0:00:00|"1-237319574"|||"8"|"5"|"4"|"7"|"7"|"3"|"10"|"8"|||"8"|"9"||"
1. Entering the serial number a second time forces me to use phone service rather than online.
2. I noticed that with my new PS3, when I tried to download a movie, it said another PS3 (the one I sent in) is still activated. How am I supposed to deactivate"|10/26/2009 0:00:00

I want the above kind of data in a single line as a single record but awk treat this as 2 records

Code:
# 1st record -
11/18/2009 9:41:48|11/4/2009 0:00:00|"1-237319574"|||"8"|"5"|"4"|"7"|"7"|"3"|"10"|"8"|||"8"|"9"||"
1. Entering the serial number a second time forces me to use phone service rather than online.

# 2nd record - 
2. I noticed that with my new PS3, when I tried to download a movie, it said another PS3 (the one I sent in) is still activated. How am I supposed to deactivate"|10/26/2009 0:00:00

I already used the syntax -

Code:
awk '/^[1-9]/{print ""};{gsub("\"\"", "");printf"%s", $0};END{print ""}' | sed '/^$/d'

and eliminated new line characters and empty lines from our data. I tested and it worked except for the above mentioned kind of records.


Thanks,
Muruga.
Last edited by Scott; 05-18-2010 at 05:10 PM.. Reason: Code tags, PLEASE!
# 4  
Old 05-18-2010
sorry, looks like you're leaving the tagging up to the moderators, so they may not be parsing it at the same spots. Just verify a) that the following is your bad sample, and b) you'll provide a working sample as well:

Code:
11/18/2009 9:41:48|11/4/2009 0:00:00|"1-237319574"|||"8"|"5"|"4"|"7"|"7"|"3"|"10"|"8"|||"8"|"9"||"
1. Entering the serial number a second time forces me to use phone service rather than online.
2. I noticed that with my new PS3, when I tried to download a movie, it said another PS3 (the one I sent in) is still activated. How am I supposed to deactivate"|10/26/2009 0:00:00.

# 5  
Old 05-18-2010
Data New line character issue
I do not know wht the moderetaors are changing the data I send but here is a sample data and I want to eliminate new line charatcters and this has to be in single line

"1
Entering the serial number a second time forces me to use phone service rather than online.
2. I noticed that with my new PS3, when I tried to download a movie, it said another PS3 (the one I sent in) is still activated. How am I supposed to deactivate"|10/26/2009 0:00:00

Thanks.
# 6  
Old 05-18-2010
try the following [in bold] in addition to what you've been using:

Code:
awk ... |sed '/^$/d;/\"$/N;/.$/N;s/\n//g;'

You may actually need to split it into specific streams, as follows, but the above worked for my testing:

Code:
awk ... |sed -e '/^$/d' -e '/\"$/N;/.$/N;s/\n//g;'

Takes this:
Quote:
-> cat Edit1
11/18/2009 9:41:48|11/4/2009 0:00:00|"1-237319574"|||"8"|"5"|"4"|"7"|"7"|"3"|"10"|"8"|||"8"|"9"||"
1. Entering the serial number a second time forces me to use phone service rather than online.
2. I noticed that with my new PS3, when I tried to download a movie, it said another PS3 (the one I sent in) is still activated. How am I supposed to deactivate"|10/26/2009 0:00:00
and makes this:
Code:
11/18/2009 9:41:48|11/4/2009 0:00:00|"1-237319574"|||"8"|"5"|"4"|"7"|"7"|"3"|"10"|"8"|||"8"|"9"||"1. Entering the serial number a second time forces me to use phone service rather than online.2. I noticed that with my new PS3, when I tried to download a movie, it said another PS3 (the one I sent in) is still activated. How am I supposed to deactivate"|10/26/2009 0:00:00

# 7  
Old 05-19-2010
New line char issue
When I use

Code:
awk '/^[1-9]/{print ""};{gsub("\"\"", "");printf"%s", $0};|sed '/^$/d;/\"$/N;/.$/N;s/\n//g;'

this eliminates carraige returns(new line character) and eliminates empty lines but this combines two lines into one line
Code:
1,ad,"adc
sdfd",edf
2,asd,"def
fde",asd

1,ad,"adcsdfd",edf2,asd,"deffde",asd

I want the this to eliminate carraige returns(new line character) and eliminate empty lines and also seperate 2 lines.

I request you to help me.

Thanks.
Last edited by Scott; 05-19-2010 at 07:51 PM.. Reason: Code tags, PLEASE!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Reading File line by line - Issue

I am trying to read the below file line by line for the below operation i) extract the directory alone and assign it one variable ii) extract the permission available in the line and add comma between the permissions and assign to another variable iii) Finally apply setfacl logic as shown in... (3 Replies)
Discussion started by: sarathy_a35
3 Replies

2. Shell Programming and Scripting

Performance issue - to read line by line

All- We have a performance issue in reading a file line by line. Please find attached scripts for the same. Currently it is taking some 45 min to parse "512444" lines. Could you please have a look at it and provide any suggestions to improve the performance. Thanks, Balu ... (12 Replies)
Discussion started by: balu1729
12 Replies

3. UNIX for Beginners Questions & Answers

Performance issue to read line by line

Hi All- we have performance issue in unix to read line by line. I am looking at processing all the records. description: Our script will read data from a flat file, it will pickup first four character and based on the value it will set up variables accordingly and appended the final output to... (11 Replies)
Discussion started by: balu1729
11 Replies

4. Shell Programming and Scripting

[BASH] read 'line' issue with leading tabs and virtual line breaks

Heyas I'm trying to read/display a file its content and put borders around it (tui-cat / tui-cat -t(ypwriter). The typewriter-part is a 'bonus' but still has its own flaws, but thats for later. So in some way, i'm trying to rewrite cat using bash and other commands. But sadly it fails on... (2 Replies)
Discussion started by: sea
2 Replies

5. UNIX for Dummies Questions & Answers

Broken line issue

Hi, I have a major problem in my source files. I have around 10 source files(around 20 GB).In one of the source files I am getting the broken line issue.My source files needs to be like this Sr.No~Ref.No~Address~Acc.No 1~ABC345~No.2/110~456474 2~ABC786~4w/54~458695 ... (4 Replies)
Discussion started by: Nivas
4 Replies

6. Shell Programming and Scripting

awk new line issue, saying string can't contain new line character

Hi , I am doing some enhancements in an existing shell script. There it used the awk command in a function as below : float_expr() { IFS=" " command eval 'awk " BEGIN { result = $* print result exit(result == 0) }"' } It calls the function float_expr to evaluate two values ,... (1 Reply)
Discussion started by: mady135
1 Replies

7. Shell Programming and Scripting

while read LINE issue

Hi, This is the script and the error I am receiving Can anyone please suggest ? For the exmaple below assume we are using vg01 #!/bin/ksh echo "##### Max Mount Count Fixer #####" echo "Please insert Volume Group name to check" read VG lvs |grep $VG | awk {'print $1'} > /tmp/audit.log ... (2 Replies)
Discussion started by: galuzan
2 Replies

8. Shell Programming and Scripting

Issue with New Line character

Hi, I am reading data from file and storing it in a multiline variable. Every line is seperated with "\n" character. globalstrval="${globalstrval}""${line}""\n" If the value of globalstrval is like: 1234 ABCD EFGH WXYZ .... If I do, YLvar=`echo $globalstrval | grep "ABC"` then... (1 Reply)
Discussion started by: gupt_ash
1 Replies

9. Shell Programming and Scripting

Issue in reading file line by line

I have a file something like: a &nbsp&nbsp&nbsp b c &nbsp&nbsp d sdfsdf &nbsp&nbsp f f &nbsp f f &nbsp fffff dfdf Now when i read it as while read line do echo $line done< file I get the o/p as a b c d sdfsdf f f f f fffff dfdf (5 Replies)
Discussion started by: Shivdatta
5 Replies

10. Shell Programming and Scripting

end of line issue

Hello, I am getting few text files with no EOF ( or end-of-line ) which i need to fix using a command so that i can include it in a script. Now i'm fixing this issue by opening the file in "vi" editor and adding last line. e.g., server1#wc -l temp.txt 9 temp.txt server1#cat ... (6 Replies)
Discussion started by: prvnrk
6 Replies
Login or Register to Ask a Question