Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Removal of multiple characters with in double quotes

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Removal of multiple characters with in double quotes
# 1  
Old 01-11-2018
Question Removal of multiple characters with in double quotes
For one of my need I was going through post "Removal of new line character in double quotes"

Which alister has replied like

Code:
$ cat data
"leave me alone"

"ABCD RENT-A-
CAR XYZ LTD","00N0H","Enterprise Lake","
100 View Way"
$ sed -n 'H;g;/^[^"]*"[^"]*\("[^"]*"[^"]*\)*$/d; s/^\n//; y/\n/ /; p; s/.*//; h' data
"leave me alone"

"ABCD RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake"," 100 View Way"

In my case requirement is to remove new line character as well as pipe (|) character when it is present inside double quotes. So for that how to enhance this code snippet. I am new to to this area and unable to under stand how mentioned code snippet work. Thanks for any help.
Moderator's Comments:
Mod Comment Please use CODE tags when displaying sample input, output, and code segments (as required by forum rules).
Last edited by Don Cragun; 01-12-2018 at 12:45 AM.. Reason: Add CODE tags and link to referenced thread.
# 2  
Old 01-12-2018
Hello Jag_1981,

If your Input_file is same as shown sample then following may help you in same too.
Code:
awk '{printf("%s%s",$0~/^"/?(FNR==1?"":RS):FS,$0)} END{print ""}'  Input_file

Thanks,
R. Singh
# 3  
Old 01-12-2018
Quote:
Originally Posted by Jag_1981
For one of my need I was going through post "Removal of new line character in double quotes"

Which alister has replied like

Code:
$ cat data
"leave me alone"

"ABCD RENT-A-
CAR XYZ LTD","00N0H","Enterprise Lake","
100 View Way"
$ sed -n 'H;g;/^[^"]*"[^"]*\("[^"]*"[^"]*\)*$/d; s/^\n//; y/\n/ /; p; s/.*//; h' data
"leave me alone"

"ABCD RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake"," 100 View Way"

In my case requirement is to remove new line character as well as pipe (|) character when it is present inside double quotes. So for that how to enhance this code snippet. I am new to to this area and unable to under stand how mentioned code snippet work. Thanks for any help.
Moderator's Comments:
Mod Comment Please use CODE tags when displaying sample input, output, and code segments (as required by forum rules).
You say that your requirement is to remove <newline> and <vertical-bar> characters found in double-quotes, but the sample data you have provided doesn't contain any <vertical-bar> characters (either inside or outside) pairs of double-quotes.

If you can't provide representative sample input data and the sample output that you want to produce from that input (in CODE tags), it makes it hard for us to know whether or not we are on the right track when making suggestions that might work for you.

It is also a very good idea to tell us what operating system and shell you're using whenever you start a new thread in this forum. Although the standard utilities perform the same basic operations, many utilities have additional features on some operating systems. If we know what operating system and shell you're using, we can limit our suggestions for you to things that will work in your environment.

When you read the sed manual page on your system to help you figure out how the code above works, where did you get stuck trying to understand what it does? What modifications have you tried on your own to meet your additional requirements?
# 4  
Old 01-12-2018
My apology for not providing all reqd details..

Operating system - RHEL 6.9

Sample Input file.
Code:
$ cat data
111|"IKJA - SPORTS"|00IIQ|Normal|100 Hall Road|

123|"ABCD RENT-A-
CAR XYZ LTD"|00N0H|Enterprise Lake|"
100 View Way"|

244|"DEFG Travel | Tour
World LTD"|"AK|0Q"|Praire Lake|"
105 NE Main St"|

Expected Output File:
Code:
$ cat data
111|IKJA - SPORTS|00IIQ|Normal|100 Hall Road|

123|ABCD RENT-A-CAR  XYZ LTD|00N0H|Enterprise Lake|100 View Way|

244|DEFG Travel  Tour World LTD|AK0Q|Praire Lake| 105 NE Main St|

The input file is a pipe delimited csv file. However, there are some data which contain a new line character or pipe symbol ( every time enclosed with in double quote). And I do not have the option at present to change the file in source side. So looking for an option to correct the file on myside.
Last edited by Don Cragun; 01-12-2018 at 06:25 PM.. Reason: Add CODE tags, again.
# 5  
Old 01-12-2018
So you want to lose ALL double quotes as well? Try
Code:
awk -F"\"" -vRS= -vOFS= '
    {for (i=2; i<=NF; i+=2) gsub (/[|\n]/, "", $i) 
     $1 = $1
     print $0 ORS
    }
'  file
111|IKJA - SPORTS|00IIQ|Normal|100 Hall Road|

123|ABCD RENT-A-CAR XYZ LTD|00N0H|Enterprise Lake|100 View Way|

244|DEFG Travel  TourWorld LTD|AK0Q|Praire Lake|105 NE Main St|

# 6  
Old 01-12-2018
Thanks Rudic for your help. I tried the same, but it's removing new line character from end of line too. Also, the pipes (|) in following lines are not getting removed as expected.

Code:
$ awk -F"\"" -vRS= -vOFS= '
    {for (i=2; i<=NF; i+=2) gsub (/[|\n]/, "", $i)
     $1 = $1
     print $0 ORS
    }
'  test.csv > test1.csv
$ cat test.csv
111|"IKJA - SPORTS"|00IIQ|Normal|100 Hall Road|
123|"ABCD RENT-A-
CAR XYZ LTD"|00N0H|Enterprise Lake|"
100 View Way"|
244|"DEFG Travel | Tour
World LTD"|"AK|0Q"|Praire Lake|"
105 NE Main St"|

$ cat test1.csv
111|IKJA - SPORTS|00IIQ|Normal|100 Hall Road|123ABCD RENT-A-CAR XYZ LTD|00N0H|Enterprise Lake|100 View Way244|DEFG Travel  TourWorld LTDAK|0QPraire Lake105 NE Main St|

$ cat expected_output.csv
111|IKJA - SPORTS|00IIQ|Normal|100 Hall Road|
123|ABCD RENT-A-CAR XYZ LTD|00N0H|Enterprise Lake|100 View Way|
244|DEFG Travel Tour World LTD|AK0Q|Praire Lake|105 NE Main St|

$

Last edited by Don Cragun; 01-12-2018 at 08:45 PM.. Reason: Change PHP tags to CODE tags.
# 7  
Old 01-12-2018
Your test.csv does not comply to the data structure you posted earlier - data in post#4. There you had a blank line as a record separator on which to rely the script was laid out.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removal of comma within double quotes

Hi All, I am getting .csv file whenever there is a comma present between a field that field get enclosed with double quotes For eg as below abc,123,xxyy,2178 fgh,123,"x,x"yy",2178 ghi,123,"x,xyy",2178 jkl,123,xx"yy,2178 whereas I want my data as per below abc,123,xxyy,2178... (1 Reply)
Discussion started by: H_bansal
1 Replies

2. Shell Programming and Scripting

Extract multiple columns base on double quotes as delimiter

Hi All, I have my data like below "1","abc,db","hac,aron","4","5" Now I need to extract 1,2,4th columns Output should be like "1",abc,db","4" Am trying to use cut command but not able to get the results. Thanks in advance. (4 Replies)
Discussion started by: weknowd
4 Replies

3. Shell Programming and Scripting

Replace Double quotes within double quotes in a column with space while loading a CSV file

Hi All, I'm unable to load the data using sql loader where there are double quotes within the double quotes As these are optionally enclosed by double quotes. Sample Data : "221100",138.00,"D","0019/1477","44012075","49938","49938/15043000","Television - 22" Refurbished - Airwave","Supply... (6 Replies)
Discussion started by: mlavanya
6 Replies

4. Shell Programming and Scripting

Issue with Single Quotes and Double Quotes for prompt PS1

Hi, Trying to change the prompt. I have the following code. export PS1=' <${USER}@`hostname -s`>$ ' The hostname is not displayed <abc@`hostname -s`>$ uname -a AIX xyz 1 6 00F736154C00 <adcwl4h@`hostname -s`>$ If I use double quotes, then the hostname is printed properly but... (3 Replies)
Discussion started by: bobbygsk
3 Replies

5. Shell Programming and Scripting

Multiple double quotes

hi Need to run below command on remote server: cmd -a "1 2" -b 3 If i run below, there's clash matching double quotes and fail. ssh $server "cmd -a "1 2" -b 3" I have few ideas which worked (like keeping the entire cmd in a file and copy it to remote server and then run that file)... (1 Reply)
Discussion started by: reddyr
1 Replies

6. UNIX for Dummies Questions & Answers

how to use grep: finding a string with double quotes and multiple digits

I have a file with a lot of lines (a lot!) that contain 10 digits between double quotes. ie "1726937489". The digits are random throughout, but always contain ten digits. I can not for the life of me, (via scouring the internet and grep how-to manuals) figure out how to find this when I search.... (3 Replies)
Discussion started by: titusbass
3 Replies

7. UNIX for Dummies Questions & Answers

grep single quotes or double quotes

Unix superusers, I am new to unix but would like to learn more about grep. I am very familiar with regular expressions as i have used them for searching text files in windows based text editors. Since I am not very familiar with Unix, I dont understand when one should use GREP with the... (2 Replies)
Discussion started by: george_vandelet
2 Replies

8. Shell Programming and Scripting

Removal of new line character in double quotes

Hi, Could you please help me in removal of newline chracter present in between the double quotes and replacing it with space. For example ... Every field is wrapped with double quotes with comma delimiter, so I need to travese from first double quote occerence to till second double... (7 Replies)
Discussion started by: vsairam
7 Replies

9. Shell Programming and Scripting

Removal of comma(,) present inbetween double quotes(" ")

Hi Experts, I have a file with some of the records contain double quotes. If I found a double quote(") in any particular record , I need to look for the next double quote in that particular record and in between these quotes, if any comma(,) is there I need to replace with Tilde (~) in the same... (12 Replies)
Discussion started by: vsairam
12 Replies

10. Shell Programming and Scripting

Replace multiple blanks within double quotes

I have various column names within double quotes, separated by commas. Example: "column one", "column number two", "this is column number three", anothercolumn, yetanothercolumn I need to eliminate the double quotes and replace the blanks within the double quotes by underscores, giving: ... (5 Replies)
Discussion started by: jgrogan
5 Replies
Login or Register to Ask a Question