Parse comma delimited and optionally quotes dimilited file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parse comma delimited and optionally quotes dimilited file
# 1  
Old 09-24-2008
Parse comma delimited and optionally quotes dimilited file

Hi,

Can you please help me?

How to Parse a comma delimited and optionally quotes dimilited file?

sample.dat
----------
"I",+2007,"SANDA, 20, MARTIN PLACE","SANDA 20MARTIN"
"D",+2008,"RANDA, 22, MARTIN PLACE","RANDA 22MARTIN"

Thank you.
Ram
# 2  
Old 09-24-2008
if you have Python and can use it, here's an alternative:
Code:
#!/usr/bin/env python
import csv
reader = csv.reader(open("file.csv", "rb"))
for row in reader:
    print '|'.join(row)
    # do whatever you want with row here..

output:
Code:
# python test.py
I|+2007|SANDA, 20, MARTIN PLACE|SANDA 20MARTIN
D|+2008|RANDA, 22, MARTIN PLACE|RANDA 22MARTIN

# 3  
Old 09-24-2008
Hi,
Thank you for the solution.

But, I would like to know how to do it in a shell script.

Regards,
Ram
# 4  
Old 09-24-2008
sed 's/\",/|/g' <filename> | sed 's/\"//g'

Im not sure of this...i have not tried..tell me if its workin
# 5  
Old 09-24-2008
Perhaps change the field separator to something else, e.g. tab
Code:
awk '{
        for (i = 1; i <= length($0); i++) {
                x = substr($0, i, 1)
                if (x == "\"") {
                        c++
                }
                if (x == "," && c % 2 == 0) {
                        x = "\t"
                }
                printf x
        }
        printf ORS
     }' sample.dat > sample.tsv

# 6  
Old 09-24-2008
There's a pretty detailed analysis of this problem in Friedl's book. It's somewhat more involved that Vijay's solution if you want to cover all the possible quirks of real-life CSV format. Here's one of the simpler approaches (from the first edition, sorry).

Code:
@fields = ();
while (m/"([^"\\]*(\\.[^"\\]*)*)",?|([^,]+),?|,/g) {
  push @fields, defined $1 ? $1 : $3
}
push @fields, undef if m/,$/;
# @ fields now contains parsed line of CSV

You could do the same in sed or awk, although Perl makes some parts of it easier.
# 7  
Old 09-24-2008
Vijay,

The following is the output.

$ sed 's/\",/|/g' sample.dat | sed 's/\"//g'
I|+2007,SANDA, 20, MARTIN PLACE|SANDA 20MARTIN
D|+2008,RANDA, 22, MARTIN PLACE|RANDA 22MARTIN

Not exactly the expected...

Thank you.

Ram
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to parse comma separated field and removing comma in between number and double quotes

Hi Experts, Please support I have below data in file in comma seperated, but 4th column is containing comma in between numbers, bcz of which when i tried to parse the file the column 6th value(5049641141) is being removed from the file and value(222.82) in column 5 becoming value of column6. ... (3 Replies)
Discussion started by: as7951
3 Replies

2. Shell Programming and Scripting

Help/Advise please for converting space delimited string variable to comma delimited with quote

Hi, I am wanting to create a script that will construct a SQL statement based on a a space delimited string that it read from a config file. Example of the SQL will be For example, it will read a string like "AAA BBB CCC" and assign to a variable named IN_STRING. I then concatenate... (2 Replies)
Discussion started by: newbie_01
2 Replies

3. Shell Programming and Scripting

Need a script to convert comma delimited files to semi colon delimited

Hi All, I need a unix script to convert .csv files to .skv files (changing a comma delimited file to a semi colon delimited file). I am a unix newbie and so don't know where to start. The script will be scheduled using cron and needs to convert each .csv file in a particular folder to a .skv... (4 Replies)
Discussion started by: CarpKing
4 Replies

4. Shell Programming and Scripting

Comma delimited file manipulation

Question about how to change the first & last name in column one & two so that the names have a capital letter for just the first letter. Example: asdf@asdf.com,asdf,asdfasdf,176.23.22.345,4/12/2012 changed to: asdf@asdf.com,Asdf,Asdfasdf,176.23.22.345,4/12/2012 Thank you kindly, Nick (2 Replies)
Discussion started by: nickytcom
2 Replies

5. UNIX for Dummies Questions & Answers

Flat File - Comma Delimited

I have a flat file whose contents are comma delimited and there are 84 columns in total, so everytime I try to view the contents, things get over lapped it becomes diffcult to read through the result set. Is there a command / what would be the best way...if I want to view the results alligned... (4 Replies)
Discussion started by: priya33184
4 Replies

6. Shell Programming and Scripting

Replacing comma with in double quotes in a csv file

Hello, I need to read a csv file and I am trying to replace a comma with a text DSEE?DSEE. Example Input "Chapter","NewTrains, "oldTrains","Delayed",10,"London" "Chapter","Newbuses,oldbuses","On Time",20,"London" Output "Chapter","NewTrainsDSEE?DSEE... (5 Replies)
Discussion started by: venkatvani
5 Replies

7. UNIX for Dummies Questions & Answers

Comma delimited file

Hi All, I have output of sql saved in comma separated file. Now i need to read line by line this file and assign word to a unix variable for further processing Eg: Test file world, 1, 3, 4 earth,2,3,4,5 moon,1,2,3,4 Output should be word1= world word2=1 echo " first word... (7 Replies)
Discussion started by: gwrm
7 Replies

8. Shell Programming and Scripting

Parsing comma delimited text file

I need to delete a set of files in certain directories if there're older than a certain number of days. So I have a text file, with each line containing the directory & number of days. The format is like this: dirA,5 dirB,7 How do I write script to iteratively parse this text file & delete... (5 Replies)
Discussion started by: chengwei
5 Replies

9. Shell Programming and Scripting

Converting Tab delimited file to Comma delimited file in Unix

Hi, Can anyone let me know on how to convert a Tab delimited file to Comma delimited file in Unix Thanks!! (22 Replies)
Discussion started by: charan81
22 Replies

10. Shell Programming and Scripting

Comma Delimited file

I have a comma delimited file that sometimes has addresses details in. The problem is that the address detail can be seen as: "Sample House, Sample Road". When I run a script specifying the file is comma delimited I would like it to ignore comma's that are in between speech marks. Is this... (2 Replies)
Discussion started by: dbrundrett
2 Replies
Login or Register to Ask a Question