Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk to find and replace Double quotes between pipes

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk to find and replace Double quotes between pipes
# 1  
Old 09-15-2014
awk to find and replace Double quotes between pipes
Hello,

Need a AWK command to find and replace Double Quotes in Pipe delimited files

Actully its a CSV file converted to Pipe but the double quotes still exists. So i want to Get rid of them.

Example

Input
Code:
1|2|3|sadsad|"Abc Efg 3"""|dada

Output
Code:
1|2|3|sadsad|Abc Efg 3"|dada

Thanks
Moderator's Comments:
Mod Comment Please CODE tags for sample input and output as well as for code segments.
Last edited by Don Cragun; 09-15-2014 at 06:32 PM.. Reason: Add CODE tags
# 2  
Old 09-15-2014
Code:
sed 's/""|/|/g' < input > output

This User Gave Thanks to Corona688 For This Post:
krux_rap
# 3  
Old 09-15-2014
With awk:
Code:
awk '{gsub(/\"[|]/, "|"); gsub(/[|]\"/, "|"); gsub(/\"\"/, "\""); print}' input

With sed:
Code:
sed -e 's/"|/|/g' -e 's/|"/|/g' -e 's/""/"/g' input

This User Gave Thanks to Don Cragun For This Post:
krux_rap
# 4  
Old 09-15-2014
Another replace is needed if first field can be quoted:

Code:
awk '{gsub(/^"/,"");gsub(/\"[|]/, "|"); gsub(/[|]\"/, "|"); gsub(/\"\"/, "\""); print}' input

Code:
sed -e 's/^"//' -e 's/"|/|/g' -e 's/|"/|/g' -e 's/""/"/g' input

These 2 Users Gave Thanks to Chubler_XL For This Post:
Don Cragun krux_rap
# 5  
Old 09-15-2014
Quote:
Originally Posted by Chubler_XL
Another replace is needed if first field can be quoted:

Code:
awk '{gsub(/^"/,"");gsub(/\"[|]/, "|"); gsub(/[|]\"/, "|"); gsub(/\"\"/, "\""); print}' input

Code:
sed -e 's/^"//' -e 's/"|/|/g' -e 's/|"/|/g' -e 's/""/"/g' input

Good catch, but still not complete. We also have to worry about quotes in the last field:
Code:
awk '{gsub(/^\"|\"$/, ""); gsub(/\"[|]/, "|"); gsub(/[|]\"/, "|"); gsub(/\"\"/, "\""); print}' input
sed -e 's/"$//' -e 's/^"//' -e 's/"|/|/g' -e 's/|"/|/g' -e 's/""/"/g' input

This User Gave Thanks to Don Cragun For This Post:
krux_rap
# 6  
Old 09-16-2014
Why is there a double quote remaining? Is this the kind of CSV escape that a repeated double quote inside double quotes means a single double quote? This is not mentioned in the description, which just says get rid of the double quotes. Try:
Code:
awk -F\" '{for(i=2; i<NF; i++) if($i=="") $(i++)=FS}1' OFS= file

What do we do with possible pipe symbols inside those double quotes?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

sed command to replace consecutive double quotes

I need to replace consecutive double quotes in a csv file, the data in the file is enclosed in double quotes but there are some places where the quotes are repeating Example is below Incoming data is : "Pacific Region"|"PNG"|"Jimmy""|""| Need output as: "Pacific... (10 Replies)
Discussion started by: abhilashnair
10 Replies

2. Shell Programming and Scripting

Replace Double quotes within double quotes in a column with space while loading a CSV file

Hi All, I'm unable to load the data using sql loader where there are double quotes within the double quotes As these are optionally enclosed by double quotes. Sample Data : "221100",138.00,"D","0019/1477","44012075","49938","49938/15043000","Television - 22" Refurbished - Airwave","Supply... (6 Replies)
Discussion started by: mlavanya
6 Replies

3. Shell Programming and Scripting

Replace double quotes with a single quote within a double quoted string

Hi Froum. I have tried in vain to find a solution for this problem - I'm trying to replace any double quotes within a quoted string with a single quote, leaving everything else as is. I have the following data: Before: ... (32 Replies)
Discussion started by: pchang
32 Replies

4. Shell Programming and Scripting

Replace newline character between a double quotes to a space

Hi Guys, I have a file with content as below aj.txt "Iam allfine" abcdef abcd "all is not well" What I'm trying to say is my data has some new line characters in between quoted text. I must get ride of the newline character that comes in between the quoted text. output must be:... (8 Replies)
Discussion started by: ajahuja
8 Replies

5. Shell Programming and Scripting

HELP with AWK or SED. Need to replace the commas between double quotes in CSV file

Hello experts, I need to validate a csv file which contains data like this: Sample.csv "ABCD","I",23,0,9,,"23/12/2012","OK","Street,State, 91135",0 "ABCD","I",23,0,9,,"23/12/2012","OK","Street,State, 91135",0 I just need to check if all the records contain exactly the number of... (5 Replies)
Discussion started by: shell_boy23
5 Replies

6. Shell Programming and Scripting

How to replace spaces excluding those within double quotes and after backslash?

In bash or perl, I would like to know how to substitute a null character (0x00) for every white space without changing the white spaces inside the block of double quotes and the white space immediately following a backslash. Suppose that sample.txt consists of the following line. "b 1" c\ 2 ... (2 Replies)
Discussion started by: LessNux
2 Replies

7. Shell Programming and Scripting

Replace double double quotes using AWK/SED

Hi, I have data as "01/22/97-"aaaaaaaaaaaaaaaaa""aaa""aabbbbbbbbcccccc""zbcd""dddddddddeeeeeeeeefffffff" I want to remove only the Consequitive double quotes and not the one which occurs single. My O/P must be ... (2 Replies)
Discussion started by: Bhuvaneswari
2 Replies

8. UNIX for Dummies Questions & Answers

How to replace double quotes in to ascii value?

Hi, I am getting the data from text file and it contains comma and double quootes. Eg: text file like this: Travelling up Cattai Ridge Rd, going through the "S" bends at the top. Felt a "pull" and pain in groin area, on right side after lifting at work. Repeat "twisting" while working manual... (4 Replies)
Discussion started by: rajesh4851
4 Replies

9. Shell Programming and Scripting

To Replace comma with Pipe inside double quotes

Hi, I have a requirement to replace the comma's inside the double quotes. The comma's inside the double quotes will get changed dynamically. Input Record: "Washington, DC,Prabhu,aju",New York Output Record: "Washington| DC|Prabhu|aju",New York I tried with the below command but it... (3 Replies)
Discussion started by: prabhutkl
3 Replies

10. Shell Programming and Scripting

Replace multiple blanks within double quotes

I have various column names within double quotes, separated by commas. Example: "column one", "column number two", "this is column number three", anothercolumn, yetanothercolumn I need to eliminate the double quotes and replace the blanks within the double quotes by underscores, giving: ... (5 Replies)
Discussion started by: jgrogan
5 Replies
Login or Register to Ask a Question