Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Replace delimeter between quotes with \| using sed or awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Replace delimeter between quotes with \| using sed or awk
# 1  
Old 05-04-2015
Linux Replace delimeter between quotes with \| using sed or awk
I've a pipe delimited data in a file of size 3GB. if the text data conatins pipe delimiter that will be enclose with double quotes. I have to replace delimiter which exists between double quotes with #%@#%@#%@ using awk or sed. can some one provide a better and efficient solution to me.
The below snippet code which I used is taking longer time to execute the file.
.

Code:
awk {if(NR%2-1)gsub(/\|/,"#%@#%@#%@"); else if(!NF) $0=RS $0}1 RS=" ORS=

Code:
123456789|"SYN|THESIS MED CHEM PTY. LTD."|C||100.00|22|"AB""C|Corp"|"""XYZ"|"CDEF"""|""|""'

Result:
Code:
123456789|"SYN\|THESIS MED CHEM PTY. LTD."|C||100.00|22|"AB""C\|Corp"|"""XYZ"|"CDEF"""|""|""'

Last edited by Scrutinizer; 05-04-2015 at 03:49 PM.. Reason: code tags (mod:also for script)
# 2  
Old 05-04-2015
Try:
Code:
awk '!(NR%2){gsub(/\|/,"\\|")}1' RS=\" ORS=\" file

# 3  
Old 05-04-2015
The specification that you give and the sample result are not in agreement, and I doubt the code snippet you present does anything as it hat syntax and semantic errors. Would this come near to your desired result:
Code:
awk -F\" '{for (i=2; i<=NF; i+=2) gsub(/\|/,"#%@#%@#%@",$i)} 1' OFS=\" file
123456789|"SYN#%@#%@#%@THESIS MED CHEM PTY. LTD."|C||100.00|22|"AB""C#%@#%@#%@Corp"|"""XYZ"|"CDEF"""|""|""

# 4  
Old 05-04-2015
Linux Repalce pipe between double quotes with \| using sed or awk comamnd
Code:
my code snippet is working fine. PFB result.
awk '{if(NR%2-1)gsub(/\|/,"#%@#%@#%@"); else if(!NF) $0=RS $0}1' RS=\" ORS= file.txt |sed -e 's/[\"]$//' -e 's/[ \t]*|[ \t]*/|/g' -e '/^$/d'
123456789|SYN#%@#%@#%@THESIS MED CHEM PTY. LTD.|C||100.00|22|AB"C#%@#%@#%@Corp|"XYZ|CDEF"||

But it's taking longer time when I ran a big file.

Please help me out.

---------- Post updated at 04:35 PM ---------- Previous update was at 04:33 PM ----------

Code:
The one which you shared is skipping the characters which doesn't have double quotes

Quote:
Originally Posted by Scrutinizer
Try:
Code:
awk '!(NR%2){gsub(/\|/,"\\|")}1' RS=\" ORS=\" file

# 5  
Old 05-04-2015
Quote:
Originally Posted by BrahmaNaiduA
[..]
The one which you shared is skipping the characters which doesn't have double quotes
I thought that was the idea ... If not, then what exactly are you looking for? Please be specific, there is a contradiction between your written requirement in post #1 and the output sample you provide in post #1...
Last edited by Scrutinizer; 05-04-2015 at 06:02 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

sed command to replace consecutive double quotes

I need to replace consecutive double quotes in a csv file, the data in the file is enclosed in double quotes but there are some places where the quotes are repeating Example is below Incoming data is : "Pacific Region"|"PNG"|"Jimmy""|""| Need output as: "Pacific... (10 Replies)
Discussion started by: abhilashnair
10 Replies

2. Shell Programming and Scripting

Replace every second instance of delimeter

Hi, Need help on replacing every second instance of delimeter. Scenario: var="Name1,Value1,Name2,Value2,Name3,Value3,Name4,Value" I want every second "," to replace with "|" I tried like below echo $var| sed 's/,/|/2' But, it's not working. Expected output: ... (4 Replies)
Discussion started by: Sumanthsv
4 Replies

3. Shell Programming and Scripting

awk substr delimeter

Hi All, I have an awk command that uses the substr function - At the moment I know the length of the values so can use the example below i.e substr(i,0,1) However in future these lengths may change so wondered if you can use a delimiter within a substr? Like in bash you could use cut -d ';',... (6 Replies)
Discussion started by: Ads89
6 Replies

4. Shell Programming and Scripting

Replace Double quotes within double quotes in a column with space while loading a CSV file

Hi All, I'm unable to load the data using sql loader where there are double quotes within the double quotes As these are optionally enclosed by double quotes. Sample Data : "221100",138.00,"D","0019/1477","44012075","49938","49938/15043000","Television - 22" Refurbished - Airwave","Supply... (6 Replies)
Discussion started by: mlavanya
6 Replies

5. Shell Programming and Scripting

sed command to replace string that contain blackslash,double quotes

Hi All, I have been trying to replace a string using the sed command string value contain blackslash and double quotes. I am not a expert writer of unix script but do try not to ask question. I have almost given up. Hope you all can give me some suggestion I want to replace a place string... (6 Replies)
Discussion started by: thanush9sep
6 Replies

6. Shell Programming and Scripting

awk to find and replace Double quotes between pipes

Hello, Need a AWK command to find and replace Double Quotes in Pipe delimited files Actully its a CSV file converted to Pipe but the double quotes still exists. So i want to Get rid of them. Example Input 1|2|3|sadsad|"Abc Efg 3"""|dada Output 1|2|3|sadsad|Abc Efg 3"|dada Thanks... (5 Replies)
Discussion started by: krux_rap
5 Replies

7. Shell Programming and Scripting

Replace space and tab to pipe delimeter

I have file like below abc 12 34 45 code abcdef 451 1 4 code ghtyggg 4 56 3 code I need to change this to abc|12|34|45|code| abcdef|451|1|4|code| ghtyggg|4|56|3|code| I tried replace space with | in sed ... but in the middle some row has... (7 Replies)
Discussion started by: greenworld123
7 Replies

8. Shell Programming and Scripting

HELP with AWK or SED. Need to replace the commas between double quotes in CSV file

Hello experts, I need to validate a csv file which contains data like this: Sample.csv "ABCD","I",23,0,9,,"23/12/2012","OK","Street,State, 91135",0 "ABCD","I",23,0,9,,"23/12/2012","OK","Street,State, 91135",0 I just need to check if all the records contain exactly the number of... (5 Replies)
Discussion started by: shell_boy23
5 Replies

9. Shell Programming and Scripting

Replace double double quotes using AWK/SED

Hi, I have data as "01/22/97-"aaaaaaaaaaaaaaaaa""aaa""aabbbbbbbbcccccc""zbcd""dddddddddeeeeeeeeefffffff" I want to remove only the Consequitive double quotes and not the one which occurs single. My O/P must be ... (2 Replies)
Discussion started by: Bhuvaneswari
2 Replies

10. Shell Programming and Scripting

sed replace spaces between quotes with a variable

I have lines with: elseif (req.http.host ~ "^(www.)?edificationtube.com$|www.edificationtube.org www.edificationtube.net edificationtube.org www.edificationtube.com edificationtube.net") { elseif (req.http.host ~ "^(www.)?collegecontender.com$|www.collegecontender.com collegecontenders.com... (3 Replies)
Discussion started by: EXT3FSCK
3 Replies
Login or Register to Ask a Question