Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

How to ignore Pipe in Pipe delimited file?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to ignore Pipe in Pipe delimited file?
# 8  
Old 08-09-2013
Please do use code tags

I think the only good solution for you is to change the pipe | inside double quotes "" to something else, like yoda does here:
sed 's#\("[^|]*\)\([|]\)\([^|]*"\)#\1,\3#g'

Like this
Code:
echo 'xx|yy|"xyz|xx"|AAA|12...'  |sed 's#\("[^|]*\)\([|]\)\([^|]*"\)#\1,\3#g' | awk 'BEGIN {FS=OFS="|" } $4 ~ /[[:alnum:]]/ { $4="" } {sub(",","|",$3)}1'
xx|yy|"xyz|xx"||12...

This User Gave Thanks to Jotne For This Post:
rohit_shinez
# 9  
Old 08-09-2013
Quote:
Originally Posted by Yoda
You could try something like:
Code:
sed 's#\("[^|]*\)\([|]\)\([^|]*"\)#\1,\3#g' file | awk -F\| '{gsub(/"/,x);sub(",","|",$3);print $3}'

The sed is substituting the pipe to comma so that awk can recognize it as one field.
Quote:
Originally Posted by Jotne
Please do use code tags

I think the only good solution for you is to change the pipe | inside double quotes "" to something else, like yoda does ...
The idea is good: transform the data, before it's seen by the main script, then undo the transform after the main script has done its job.

The implementation, however, is not. Modifying field contents should be avoided. It is much less intrusive to eliminate collisions between delimiter-pipes and quoted-pipes by replacing the delimiter-pipes. Why does this matter? Because delimiters are not visible from AWK script logic. Field contents, however, are visible and used in unpredictable ways.

Instead of changing xx|yy|"xyz|xx"|AAA|12... into xx|yy|"xyz,xx"|AAA|12..., it should become xx,yy,"xyz|xx",AAA,12....

One alternative:
Code:
awk '{for (i=1; i<=NF; i+=2) gsub(/\|/, d, $i); print}' FS=\" OFS=\" d=,

Regards,
Alister
# 10  
Old 08-10-2013
its taking lot of time when using below code since sed is being used , my file contains 27k records with 200 columns pipe separated

Code:
echo 'xx|yy|"xyz|xx"|AAA|12...'  |sed 's#\("[^|]*\)\([|]\)\([^|]*"\)#\1,\3#g' | awk 'BEGIN {FS=OFS="|" } $4 ~ /[[:alnum:]]/ { $4="" } {sub(",","|",$3)}1'xx|yy|"xyz|xx"||12...

# 11  
Old 08-11-2013
You can get hold of GNU Awk 4.0+ and then use FPAT to solve this.
Splitting By Content - The GNU Awk User's Guide
# 12  
Old 08-11-2013
Hi I tried using fpat but couldn't able to frame as per by desired requirement can u help me
# 13  
Old 08-12-2013
This may work
Code:
echo 'xx|yy|"xyz|zzz"|zzz|12' | awk '{c=0
$0=$0"|"
while($0) {
  match($0,/ *"[^"]*" *\||[^\|]*\|/)
  f=substr($0,RSTART,RLENGTH)
  gsub(/^ *"?|"? *\|$/,"",f)
  print "Field " ++c " is " f
  $0=substr($0,RLENGTH+1)
}}'
Field 1 is xx
Field 2 is yy
Field 3 is xyz|zzz
Field 4 is zzz
Field 5 is 12

# 14  
Old 08-12-2013
Try this one:
Code:
awk     '               {for (i=2; i<=NF; i+=2) {                       # every second field is one inside double quotes
                                 gsub (/\|/, "\001", $i)                # replace every delimiter in quoted field by something
                                }
                         FS="|"                                         # set desired field separator
                         $0=$0                                          # rebuild fields
                         for (i=1; i<=NF; i++) {                        # in every new field
                                 gsub ("\001", "|", $i)                 # replace something back to commas
                                }
                        }
                        {for (i=1; i<=NF; i++) print $i}                # print new $i fields
        ' FS="\"" OFS="\"" file
xx
yy
"xyz|zzz"
zzz
12
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Need to convert a pipe delimited text file to tab delimited

Hi, I have a rquirement in unix as below . I have a text file with me seperated by | symbol and i need to generate a excel file through unix commands/script so that each value will go to each column. ex: Input Text file: 1|A|apple 2|B|bottle excel file to be generated as output as... (9 Replies)
Discussion started by: raja kakitapall
9 Replies

2. Shell Programming and Scripting

Help with converting Pipe delimited file to Tab Delimited

I have a file which was pipe delimited, I need to make it tab delimited. I tried with sed but no use cat file | sed 's/|//t/g' The above command substituted "/t" not tab in the place of pipe. Sample file: abc|123|2012-01-30|2012-04-28|xyz have to convert to: abc 123... (6 Replies)
Discussion started by: karumudi7
6 Replies

3. UNIX for Advanced & Expert Users

Check Whether File is Pipe Delimited

Can anybody help me how to check whether a file is Pipe delimited or not? (1 Reply)
Discussion started by: Allwin333
1 Replies

4. UNIX for Dummies Questions & Answers

Reading a pipe delimited file

Hi Guys, i am reading a pipe delimited file using awk command. I have tested the gawk separately. it was fine. But when i execute the script. i am getting the following error saying command not found. Can somebody point out as what i am doing wrong. Cheers!!! (3 Replies)
Discussion started by: mac4rfree
3 Replies

5. Shell Programming and Scripting

How to convert a space delimited file into a pipe delimited file using shellscript?

Hi All, I have space delimited file similar to the one as shown below.. I need to convert it as a pipe delimited, the values inside the pipe delimited file should be as highlighted... AA ATIU2345098809 009697 005374 BB ATIU2345097809 005445 006518 CC ATIU9685098809 003215 003571 DD... (7 Replies)
Discussion started by: nithins007
7 Replies

6. UNIX for Dummies Questions & Answers

Delete last value from pipe delimited file

I have a large(ish) pipe delimited file. The last line of the file contains a total row count and a checksum: END|1537451|1328569446 After making other adjustments to the file, I need to strip out the checksum and apply a new value - I have a script to generate the checksum and 'cat' it... (3 Replies)
Discussion started by: relentl3ss
3 Replies

7. Shell Programming and Scripting

convert a pipe delimited file to a':" delimited file

i have a file whose data is like this:: osr_pe_assign|-120|wg000d@att.com|4| osr_evt|-21|wg000d@att.com|4| pe_avail|-21|wg000d@att.com|4| osr_svt|-11|wg000d@att.com|4| pe_mop|-13|wg000d@att.com|4| instar_ready|-35|wg000d@att.com|4| nsdnet_ready|-90|wg000d@att.com|4|... (6 Replies)
Discussion started by: priyanka3006
6 Replies

8. UNIX for Dummies Questions & Answers

Extracting from pipe delimited file.

Hey, I am new to regualar expression. I wanted to extract the information from a pipe delimited file which has some entries like L|S2CMG1B|||-11178399||1|-8.65|IRCSH|BOND||||N|S|IRDL|AUD||CRP|STD|CRP|M|0|1|||CSH||||OTHER|01|DE|KFW|50418Y9T5|||||||||||2|||||| In this I want to extract the... (1 Reply)
Discussion started by: leepan2008
1 Replies

9. Shell Programming and Scripting

How to generate a pipe ( | ) delimited file?

:)Hi Friends, I have certain log files extracted. I want it to be converted in pipe ( | ) delimited file. How do i do it? E.g. Account Balance : 123456789 Rs O/P (Account Balance: | 123456789 Rs) Account Balance (Last) > 987654321 Rs O/P (Account Balance (Last) | 987654321 Rs) Last... (5 Replies)
Discussion started by: anushree.a
5 Replies

10. Shell Programming and Scripting

How to split pipe delimited file

I have a pipe delimited input file as below. First byte of the each line indicate the record type. Then i need to split the file based on record_type = null,0,1,2,6 and create 5 files. How do i do this in a ksh script? Pls help |sl||SL|SL|SL|1996/04/03|1988/09/15|C|A|sl||||*|... (4 Replies)
Discussion started by: njgirl
4 Replies
Login or Register to Ask a Question