Delimited file problems


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Delimited file problems
# 1  
Old 12-24-2009
Delimited file problems

I am trying to store each value in this delimited file:

"YCEU3600913","29P0","CT",20,"FB","Platform",2700,28000,"O",
"YMLU4366413","42G1","CT",40,"ST","40x8 6,Passive vents at upper",3900,30000,"W",

As you can see, most of the fields are surrounded by double quotes, except fields 4, 7 & 8. Plus, in the 2nd record, field 6 contains a comma in the field which also throws my fields off by one.

Any idea how to capture each field with the problems described above?

For example:

Field1=YMLU4366413
Field2=42G1
Field3=CT
Field4=40
Field5=ST
Field6=40x8 6,Passive vents at upper
Field7=3900
Field8=30000
Field9=W

Obviously I will output the fields in a certain format, but I just need to know how to get those stored correctly in a variable? (Perl, UNIX, awk, sed, anything will do)
# 2  
Old 12-24-2009
Code:
perl -MText::ParseWords -nle'
  $c = 1; print "field", $c++, "=", $_ for parse_line(",",0, $_);
  ' infile

# 3  
Old 12-24-2009
Or try:

Code:
sed -e 's/,\([0-9][0-9]*\)/,"\1"/g' -e 's/^"//g' -e 's/",$//g'  filename | awk -F '","' '{for(i=1;i<=NF;i++)printf("%s%s%s%s%s\n", NR," Field",i,": ",$i)}'

cheers,
Devaraj Takhellambam
# 4  
Old 12-24-2009
Tools just pick off the fields one by one?

Code:
$ cut sample.txt -d'"' -f2
YCEU3600913
YMLU4366413

$ cut sample.txt -d'"' -f4
29P0
42G1

$ cut sample.txt -d'"' -f6
CT
CT

$ cut sample.txt -d'"' -f7 | cut -d"," -f2
20
40

$ cut sample.txt -d'"' -f8
FB
ST

And so on...
# 5  
Old 12-24-2009
This works, but a little silly:
Code:
#!/bin/ksh
awk 'BEGIN {FS=","}
{
        $4="\""$4"\"";
        $(NF-2)="\""$(NF-2)"\"";
        $(NF-3)="\""$(NF-3)"\"";
        OFS=","
        print $0 > "tmp"
}' < FILENAME

awk 'BEGIN {FS="\",\""}
{
        sub(/"/,"")
        sub(/",$/,"")
        for(i=1;i<=NF;i++)
                print "Filed"i"="$i
}' <tmp

# 6  
Old 12-27-2009
Thanks for all the replies. For the first two replies (PERL and AWK solutions), I am having the toughest time trying to output each line.

I have a multi-line file and I would like the output to look like this:
Code:
YMLU4366413 42G1 CT 40 ST 40x8 6,Passive vents at upper 3900 30000 W

I would need the field fixed length, if possible.

Happy Holidays everyone!

Last edited by radoulov; 12-27-2009 at 05:13 AM.. Reason: Use code tags, please!
# 7  
Old 12-27-2009
Code:
% cat infile 
"YCEU3600913","29P0","CT",20,"FB","Platform",2700,28000,"O",
"YMLU4366413","42G1","CT",40,"ST","40x8 6,Passive vents at upper",3900,30000,"W",

Code:
% perl -MText::ParseWords -nle'
  print join " ", parse_line(",",0, $_);
  ' infile
YCEU3600913 29P0 CT 20 FB Platform 2700 28000 O 
YMLU4366413 42G1 CT 40 ST 40x8 6,Passive vents at upper 3900 30000 W

Or may be:

Code:
% perl -MText::ParseWords -nle'
  print join "\t", parse_line(",",0, $_);
  ' infile      
YCEU3600913     29P0    CT      20      FB      Platform        2700    28000   O       
YMLU4366413     42G1    CT      40      ST      40x8 6,Passive vents at upper   3900    30000   W

Quote:
I would need the field fixed length, if possible.
Could you be more specific?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Replace a column in tab delimited file with column in other tab delimited file,based on match

Hello Everyone.. I want to replace the retail col from FileI with cstp1 col from FileP if the strpno matches in both files FileP.txt ... (2 Replies)
Discussion started by: YogeshG
2 Replies

2. Shell Programming and Scripting

Help/Advise please for converting space delimited string variable to comma delimited with quote

Hi, I am wanting to create a script that will construct a SQL statement based on a a space delimited string that it read from a config file. Example of the SQL will be For example, it will read a string like "AAA BBB CCC" and assign to a variable named IN_STRING. I then concatenate... (2 Replies)
Discussion started by: newbie_01
2 Replies

3. UNIX for Dummies Questions & Answers

Need to convert a pipe delimited text file to tab delimited

Hi, I have a rquirement in unix as below . I have a text file with me seperated by | symbol and i need to generate a excel file through unix commands/script so that each value will go to each column. ex: Input Text file: 1|A|apple 2|B|bottle excel file to be generated as output as... (9 Replies)
Discussion started by: raja kakitapall
9 Replies

4. Shell Programming and Scripting

How to make tab delimited file to space delimited?

Hi How to make tab delimited file to space delimited? in put file: ABC kgy jkh ghj ash kjl o/p file: ABC kgy jkh ghj ash kjl Use code tags, thanks. (1 Reply)
Discussion started by: jagdishrout
1 Replies

5. Shell Programming and Scripting

awk read one delimited file, search another delimited file

Hello folks, I have another doozy. I have two files. The first file has four fields in it. These four fields map to different locations in my second file. What I want to do is read the master file (file 2 - 23 fields) and compare each line against each record in file 1. If I get a match in all four... (4 Replies)
Discussion started by: dagamier
4 Replies

6. Shell Programming and Scripting

Help with converting Pipe delimited file to Tab Delimited

I have a file which was pipe delimited, I need to make it tab delimited. I tried with sed but no use cat file | sed 's/|//t/g' The above command substituted "/t" not tab in the place of pipe. Sample file: abc|123|2012-01-30|2012-04-28|xyz have to convert to: abc 123... (6 Replies)
Discussion started by: karumudi7
6 Replies

7. Shell Programming and Scripting

How to convert a space delimited file into a pipe delimited file using shellscript?

Hi All, I have space delimited file similar to the one as shown below.. I need to convert it as a pipe delimited, the values inside the pipe delimited file should be as highlighted... AA ATIU2345098809 009697 005374 BB ATIU2345097809 005445 006518 CC ATIU9685098809 003215 003571 DD... (7 Replies)
Discussion started by: nithins007
7 Replies

8. Shell Programming and Scripting

convert a pipe delimited file to a':" delimited file

i have a file whose data is like this:: osr_pe_assign|-120|wg000d@att.com|4| osr_evt|-21|wg000d@att.com|4| pe_avail|-21|wg000d@att.com|4| osr_svt|-11|wg000d@att.com|4| pe_mop|-13|wg000d@att.com|4| instar_ready|-35|wg000d@att.com|4| nsdnet_ready|-90|wg000d@att.com|4|... (6 Replies)
Discussion started by: priyanka3006
6 Replies

9. UNIX for Dummies Questions & Answers

Converting Space delimited file to Tab delimited file

Hi all, I have a file with single white space delimited values, I want to convert them to a tab delimited file. I tried sed, tr ... but nothing is working. Thanks, Rajeevan D (16 Replies)
Discussion started by: jeevs81
16 Replies

10. Shell Programming and Scripting

Converting Tab delimited file to Comma delimited file in Unix

Hi, Can anyone let me know on how to convert a Tab delimited file to Comma delimited file in Unix Thanks!! (22 Replies)
Discussion started by: charan81
22 Replies
Login or Register to Ask a Question