Unix/Linux Go Back    


Shell Programming and Scripting BSD, Linux, and UNIX shell scripting — Post awk, bash, csh, ksh, perl, php, python, sed, sh, shell scripts, and other shell scripting languages questions here.

Removal of multiple characters with in double quotes

Shell Programming and Scripting


Reply    
 
Thread Tools Search this Thread Display Modes
    #1  
Old Unix and Linux 1 Week Ago   -   Original Discussion by Jag_1981
Jag_1981's Unix or Linux Image
Jag_1981 Jag_1981 is offline
Registered User
 
Join Date: Jan 2018
Last Activity: 13 January 2018, 2:05 PM EST
Posts: 4
Thanks: 1
Thanked 0 Times in 0 Posts
Unix or Linux Question Removal of multiple characters with in double quotes

For one of my need I was going through post "Removal of new line character in double quotes"

Which alister has replied like



Code:
$ cat data
"leave me alone"

"ABCD RENT-A-
CAR XYZ LTD","00N0H","Enterprise Lake","
100 View Way"
$ sed -n 'H;g;/^[^"]*"[^"]*\("[^"]*"[^"]*\)*$/d; s/^\n//; y/\n/ /; p; s/.*//; h' data
"leave me alone"

"ABCD RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake"," 100 View Way"

In my case requirement is to remove new line character as well as pipe (|) character when it is present inside double quotes. So for that how to enhance this code snippet. I am new to to this area and unable to under stand how mentioned code snippet work. Thanks for any help.
Moderator's Comments:
Removal of multiple characters with in double quotes Please use CODE tags when displaying sample input, output, and code segments (as required by forum rules).

Last edited by Don Cragun; 1 Week Ago at 12:45 AM.. Reason: Add CODE tags and link to referenced thread.
Sponsored Links
    #2  
Old Unix and Linux 1 Week Ago   -   Original Discussion by Jag_1981
RavinderSingh13's Unix or Linux Image
RavinderSingh13 RavinderSingh13 is offline Forum Advisor  
Registered User
 
Join Date: May 2013
Last Activity: 23 January 2018, 11:55 AM EST
Location: Chennai
Posts: 2,689
Thanks: 594
Thanked 1,278 Times in 1,149 Posts
Hello Jag_1981,

If your Input_file is same as shown sample then following may help you in same too.


Code:
awk '{printf("%s%s",$0~/^"/?(FNR==1?"":RS):FS,$0)} END{print ""}'  Input_file

Thanks,
R. Singh
Sponsored Links
    #3  
Old Unix and Linux 1 Week Ago   -   Original Discussion by Jag_1981
Don Cragun's Unix or Linux Image
Don Cragun Don Cragun is online now Forum Staff  
Administrator
 
Join Date: Jul 2012
Last Activity: 23 January 2018, 4:22 PM EST
Location: San Jose, CA, USA
Posts: 10,947
Thanks: 611
Thanked 3,824 Times in 3,268 Posts
Quote:
Originally Posted by Jag_1981 View Post
For one of my need I was going through post "Removal of new line character in double quotes"

Which alister has replied like



Code:
$ cat data
"leave me alone"

"ABCD RENT-A-
CAR XYZ LTD","00N0H","Enterprise Lake","
100 View Way"
$ sed -n 'H;g;/^[^"]*"[^"]*\("[^"]*"[^"]*\)*$/d; s/^\n//; y/\n/ /; p; s/.*//; h' data
"leave me alone"

"ABCD RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake"," 100 View Way"

In my case requirement is to remove new line character as well as pipe (|) character when it is present inside double quotes. So for that how to enhance this code snippet. I am new to to this area and unable to under stand how mentioned code snippet work. Thanks for any help.
Moderator's Comments:
Removal of multiple characters with in double quotes Please use CODE tags when displaying sample input, output, and code segments (as required by forum rules).
You say that your requirement is to remove <newline> and <vertical-bar> characters found in double-quotes, but the sample data you have provided doesn't contain any <vertical-bar> characters (either inside or outside) pairs of double-quotes.

If you can't provide representative sample input data and the sample output that you want to produce from that input (in CODE tags), it makes it hard for us to know whether or not we are on the right track when making suggestions that might work for you.

It is also a very good idea to tell us what operating system and shell you're using whenever you start a new thread in this forum. Although the standard utilities perform the same basic operations, many utilities have additional features on some operating systems. If we know what operating system and shell you're using, we can limit our suggestions for you to things that will work in your environment.

When you read the sed manual page on your system to help you figure out how the code above works, where did you get stuck trying to understand what it does? What modifications have you tried on your own to meet your additional requirements?
    #4  
Old Unix and Linux 1 Week Ago   -   Original Discussion by Jag_1981
Jag_1981's Unix or Linux Image
Jag_1981 Jag_1981 is offline
Registered User
 
Join Date: Jan 2018
Last Activity: 13 January 2018, 2:05 PM EST
Posts: 4
Thanks: 1
Thanked 0 Times in 0 Posts
My apology for not providing all reqd details..

Operating system - RHEL 6.9

Sample Input file.


Code:
$ cat data
111|"IKJA - SPORTS"|00IIQ|Normal|100 Hall Road|

123|"ABCD RENT-A-
CAR XYZ LTD"|00N0H|Enterprise Lake|"
100 View Way"|

244|"DEFG Travel | Tour
World LTD"|"AK|0Q"|Praire Lake|"
105 NE Main St"|

Expected Output File:


Code:
$ cat data
111|IKJA - SPORTS|00IIQ|Normal|100 Hall Road|

123|ABCD RENT-A-CAR  XYZ LTD|00N0H|Enterprise Lake|100 View Way|

244|DEFG Travel  Tour World LTD|AK0Q|Praire Lake| 105 NE Main St|

The input file is a pipe delimited csv file. However, there are some data which contain a new line character or pipe symbol ( every time enclosed with in double quote). And I do not have the option at present to change the file in source side. So looking for an option to correct the file on myside.

Last edited by Don Cragun; 1 Week Ago at 06:25 PM.. Reason: Add CODE tags, again.
Sponsored Links
    #5  
Old Unix and Linux 1 Week Ago   -   Original Discussion by Jag_1981
RudiC's Unix or Linux Image
RudiC RudiC is offline Forum Staff  
Moderator
 
Join Date: Jul 2012
Last Activity: 23 January 2018, 2:47 PM EST
Location: Aachen, Germany
Posts: 11,983
Thanks: 356
Thanked 3,693 Times in 3,391 Posts
So you want to lose ALL double quotes as well? Try


Code:
awk -F"\"" -vRS= -vOFS= '
    {for (i=2; i<=NF; i+=2) gsub (/[|\n]/, "", $i) 
     $1 = $1
     print $0 ORS
    }
'  file
111|IKJA - SPORTS|00IIQ|Normal|100 Hall Road|

123|ABCD RENT-A-CAR XYZ LTD|00N0H|Enterprise Lake|100 View Way|

244|DEFG Travel  TourWorld LTD|AK0Q|Praire Lake|105 NE Main St|

Sponsored Links
    #6  
Old Unix and Linux 1 Week Ago   -   Original Discussion by Jag_1981
Jag_1981's Unix or Linux Image
Jag_1981 Jag_1981 is offline
Registered User
 
Join Date: Jan 2018
Last Activity: 13 January 2018, 2:05 PM EST
Posts: 4
Thanks: 1
Thanked 0 Times in 0 Posts
Thanks Rudic for your help. I tried the same, but it's removing new line character from end of line too. Also, the pipes (|) in following lines are not getting removed as expected.



Code:
$ awk -F"\"" -vRS= -vOFS= '
    {for (i=2; i<=NF; i+=2) gsub (/[|\n]/, "", $i)
     $1 = $1
     print $0 ORS
    }
'  test.csv > test1.csv
$ cat test.csv
111|"IKJA - SPORTS"|00IIQ|Normal|100 Hall Road|
123|"ABCD RENT-A-
CAR XYZ LTD"|00N0H|Enterprise Lake|"
100 View Way"|
244|"DEFG Travel | Tour
World LTD"|"AK|0Q"|Praire Lake|"
105 NE Main St"|

$ cat test1.csv
111|IKJA - SPORTS|00IIQ|Normal|100 Hall Road|123ABCD RENT-A-CAR XYZ LTD|00N0H|Enterprise Lake|100 View Way244|DEFG Travel  TourWorld LTDAK|0QPraire Lake105 NE Main St|

$ cat expected_output.csv
111|IKJA - SPORTS|00IIQ|Normal|100 Hall Road|
123|ABCD RENT-A-CAR XYZ LTD|00N0H|Enterprise Lake|100 View Way|
244|DEFG Travel Tour World LTD|AK0Q|Praire Lake|105 NE Main St|

$


Last edited by Don Cragun; 1 Week Ago at 08:45 PM.. Reason: Change PHP tags to CODE tags.
Sponsored Links
    #7  
Old Unix and Linux 1 Week Ago   -   Original Discussion by Jag_1981
RudiC's Unix or Linux Image
RudiC RudiC is offline Forum Staff  
Moderator
 
Join Date: Jul 2012
Last Activity: 23 January 2018, 2:47 PM EST
Location: Aachen, Germany
Posts: 11,983
Thanks: 356
Thanked 3,693 Times in 3,391 Posts
Your test.csv does not comply to the data structure you posted earlier - data in post#4. There you had a blank line as a record separator on which to rely the script was laid out.
Sponsored Links
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Removal of comma within double quotes H_bansal Shell Programming and Scripting 1 03-12-2016 05:27 AM
Replace Double quotes within double quotes in a column with space while loading a CSV file mlavanya Shell Programming and Scripting 6 05-12-2015 01:05 AM
Multiple double quotes reddyr Shell Programming and Scripting 1 04-11-2011 05:57 PM
Removal of new line character in double quotes vsairam Shell Programming and Scripting 7 05-19-2010 04:44 PM
Removal of comma(,) present inbetween double quotes(" ") vsairam Shell Programming and Scripting 12 07-17-2009 02:03 PM



All times are GMT -4. The time now is 05:52 PM.