Visit The New, Modern Unix Linux Community


Removal of new line character in double quotes


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Removal of new line character in double quotes
# 1  
Removal of new line character in double quotes

Hi,

Could you please help me in removal of newline chracter present in between the double quotes and replacing it with space.

For example ...

Every field is wrapped with double quotes with comma delimiter, so I need to travese from first double quote occerence to till second double quote occurence, if any new line chracter present , I need to replace it with space ..simlarly from double quote occurence 3 to 4, etc.

Input :
Code:
 
 
"ABCD RENT-A-
CAR XYZ LTD","00N0H","Enterprise Lake
View Way"

Output would be like this ...

Code:
 
"ABCD RENT-A -CAR XYZ LTD","00N0H","Enterprise Lake View Way"

# 2  
If you have GNU sed for example and there is no more lines of text before or after this snippet you can try:
Code:
sed -e :a -e 'N; s/\n/ /g; ta' infile
"ABCD RENT-A -CAR XYZ LTD","00N0H","Enterprise Lake View Way"

# 3  
Another approach with awk Smilie:
Code:
awk -F"\"" '!$NF{print;next}{printf("%s ", $0)}' file

This User Gave Thanks to Franklin52 For This Post:
# 4  
Quote:
Originally Posted by Franklin52
Another approach with awk Smilie:
Code:
awk -F"\"" '!$NF{print;next}{printf("%s ", $0)}' file

Like zaxxon, I also like this approach; clever use of FS and NF.

However, it does have a bug. If the value of $NF is the number zero, !$NF will be true (since $NF is evaluated numerically, instead of as a string), which would be incorrect. The solution would be to use length($NF) or concatenate a null string to force conversion to a string type, $NF"".

Example:
Code:
$ cat data
"ABCD RENT-A-
CAR XYZ LTD","00N0H","Enterprise Lake","0
View Way"

$ # Incorrect
awk -F"\"" '!$NF{print;next}{printf("%s ", $0)}' data
"ABCD RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake","0
View Way"

$ # Correct
$  awk -F"\"" '!($NF""){print;next}{printf("%s ", $0)}' data 
"ABCD RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake","0 View Way"

$# Correct
$  awk -F"\"" '!length($NF){print;next}{printf("%s ", $0)}' data
"ABCD RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake","0 View Way"

a golfed version of franklin52's approach:
Code:
$ awk -F'"' '$NF""{printf("%s ", $0);next}1' data
"ABCD RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake","0 View Way"

Even so, I'm not sure this approach meets the original poster's needs. If a line with an odd number of quotes ends on a quote, it will not have the trailing newline replaced with a space.

Regards,
Alister

Last edited by alister; 05-18-2010 at 03:03 PM..
# 5  
Quote:
Originally Posted by alister
Like zaxxon, I also like this approach; clever use of FS and NF.

However, it does have a bug. If the value of $NF is the number zero, !$NF will be true (since $NF is evaluated numerically, instead of as a string), which would be incorrect. The solution would be to use length($NF) or concatenate a null string to force conversion to a string type, $NF"".
Good point!

Regards
# 6  
Code:
# cat cutt
"ABCD RENT-A-
CAR XYZ LTD","00N0H","Enterprise Lake
View Way"

Code:
# a=;while read line; do a="$a $line"; done <cutt ; echo $a
"ABCD RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake View Way"

# 7  
Quote:
Originally Posted by ygemici
Code:
# cat cutt
"ABCD RENT-A-
CAR XYZ LTD","00N0H","Enterprise Lake
View Way"

Code:
# a=;while read line; do a="$a $line"; done <cutt ; echo $a
"ABCD RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake View Way"

I believe the goal is to not naively join all lines in the data, but only those lines which span quoted text. Otherwise, a simple paste command would do the job.

Code:
paste -sd' ' data

Regards,
Alister

---------- Post updated at 03:42 PM ---------- Previous update was at 03:26 PM ----------

Here's a solution that only replaces a newline with a space when that newline occurs between an opening quote character and its corresponding close quote.
Code:
sed -n 'H;g;/^[^"]*"[^"]*\("[^"]*"[^"]*\)*$/d; s/^\n//; y/\n/ /; p; s/.*//; h' data

So long as the total number of quote characters encountered is odd, a line is appended to its predecessor. When finally an even number of quote characters have been seen, the resulting concatenation of lines is printed, with all embedded newlines converted to spaces.

Trial run with sample data:
Code:
$ cat data
"leave me alone"

"ABCD RENT-A-
CAR XYZ LTD","00N0H","Enterprise Lake","
100 View Way"
$ sed -n 'H;g;/^[^"]*"[^"]*\("[^"]*"[^"]*\)*$/d; s/^\n//; y/\n/ /; p; s/.*//; h' data
"leave me alone"

"ABCD RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake"," 100 View Way"

The AWK solutions would mishandle the continutation of "100 View Way":
Code:
$ awk -F'"' '$NF""{printf("%s ", $0);next}1' data
"leave me alone"

"ABCD RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake","
100 View Way"

Regards,
Alister
These 3 Users Gave Thanks to alister For This Post:

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #791
Difficulty: Easy
Logic gates cannot be used to store data.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removal of multiple characters with in double quotes

For one of my need I was going through post "Removal of new line character in double quotes" Which alister has replied like $ cat data "leave me alone" "ABCD RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake"," 100 View Way" $ sed -n 'H;g;/^*"*\("*"*\)*$/d; s/^\n//; y/\n/ /; p; s/.*//; h'... (9 Replies)
Discussion started by: Jag_1981
9 Replies

2. Shell Programming and Scripting

Removal of comma within double quotes

Hi All, I am getting .csv file whenever there is a comma present between a field that field get enclosed with double quotes For eg as below abc,123,xxyy,2178 fgh,123,"x,x"yy",2178 ghi,123,"x,xyy",2178 jkl,123,xx"yy,2178 whereas I want my data as per below abc,123,xxyy,2178... (1 Reply)
Discussion started by: H_bansal
1 Replies

3. Shell Programming and Scripting

Replace Double quotes within double quotes in a column with space while loading a CSV file

Hi All, I'm unable to load the data using sql loader where there are double quotes within the double quotes As these are optionally enclosed by double quotes. Sample Data : "221100",138.00,"D","0019/1477","44012075","49938","49938/15043000","Television - 22" Refurbished - Airwave","Supply... (6 Replies)
Discussion started by: mlavanya
6 Replies

4. Shell Programming and Scripting

Issue with Single Quotes and Double Quotes for prompt PS1

Hi, Trying to change the prompt. I have the following code. export PS1=' <${USER}@`hostname -s`>$ ' The hostname is not displayed <abc@`hostname -s`>$ uname -a AIX xyz 1 6 00F736154C00 <adcwl4h@`hostname -s`>$ If I use double quotes, then the hostname is printed properly but... (3 Replies)
Discussion started by: bobbygsk
3 Replies

5. UNIX for Dummies Questions & Answers

Stripping double quotes from front and end of a line

I have a file and some records may contain double quotes at beginning and at end of line. So how do I strip them? For Example, file is somethings like this Field1;Field2;Field3 01;'Test';'Test Field3' "01;'This is 2nd field';This is 3rd field' " Desired Output is: ... (6 Replies)
Discussion started by: vx04
6 Replies

6. Shell Programming and Scripting

Replace newline character between a double quotes to a space

Hi Guys, I have a file with content as below aj.txt "Iam allfine" abcdef abcd "all is not well" What I'm trying to say is my data has some new line characters in between quoted text. I must get ride of the newline character that comes in between the quoted text. output must be:... (8 Replies)
Discussion started by: ajahuja
8 Replies

7. UNIX for Dummies Questions & Answers

grep single quotes or double quotes

Unix superusers, I am new to unix but would like to learn more about grep. I am very familiar with regular expressions as i have used them for searching text files in windows based text editors. Since I am not very familiar with Unix, I dont understand when one should use GREP with the... (2 Replies)
Discussion started by: george_vandelet
2 Replies

8. Shell Programming and Scripting

Removal of comma(,) present inbetween double quotes(" ")

Hi Experts, I have a file with some of the records contain double quotes. If I found a double quote(") in any particular record , I need to look for the next double quote in that particular record and in between these quotes, if any comma(,) is there I need to replace with Tilde (~) in the same... (12 Replies)
Discussion started by: vsairam
12 Replies

9. UNIX for Advanced & Expert Users

How to remove a character which is enclosed in Double quotes

I want to remove the comma which is present within the double quoted string. All other commas which is present outside double quotes should be present. Input : a,b,"cc,dd,ee",f,ii,"jj,kk",mmm output : a,b,"ccddee",f,ii,"jjkk",mmm (3 Replies)
Discussion started by: mohan_tuty
3 Replies

10. Shell Programming and Scripting

Double quotes or single quotes when using ssh?

I'm not very familiar with the ssh command. When I tried to set a variable and then echo its value on a remote machine via ssh, I found a problem. For example, $ ITSME=itsme $ ssh xxx.xxxx.xxx.xxx "ITSME=itsyou; echo $ITSME" itsme $ ssh xxx.xxxx.xxx.xxx 'ITSME=itsyou; echo $ITSME' itsyou $... (3 Replies)
Discussion started by: password636
3 Replies

Featured Tech Videos