Visit Our UNIX and Linux User Community


Parse apart strings of comma separated data with varying number of fields


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parse apart strings of comma separated data with varying number of fields
# 1  
Old 09-12-2008
Parse apart strings of comma separated data with varying number of fields

I have a situation where I am reading a text file line-by-line. Those lines of data contain comma separated fields of data. However, each line can vary in the number of fields it can contain. What I need to do is parse apart each line and write each field of data found (left to right) into a file.

Example Lines of data
---------------------
abc,123,def,456,ghi,789
123,def,456
def,456,ghi,789,jkl
abc
def,456,ghi,789

After the first read the variable $LINE would equal "abc,123,def,456,ghi,789".
Then the fields of data should be parsed and written to FILE.TXT and should look like this :
abc
123
def
456
ghi
789

The second line is read and the variable $LINE would equal "123,def,456" and it's fields of data would be parsed and written to FILE.TXT and would look like so :
123
def
456

and so on for the remaining lines that are read into $LINE.

I have no problems with reading in the lines of data and placing each line into the $LINE variable. I need assistance with how to parse apart each line as it's read into $LINE and writing the fields to a file one-by-one from left to right. I'm pretty sure awk would probably be the best method but I'm open to suggestions on any more efficient methods/commands.

Can anyone help?
# 2  
Old 09-12-2008
Something like this?

Code:
tr ',' '\n' < file > FILE.TXT

Regards
# 3  
Old 09-12-2008
Try:

Code:
tr ',' '\n' < filename

or,

Code:
sed 's/,/\n/g' filename

# 4  
Old 09-12-2008
sed -e 's/\,/^M/g' filename >newfilename
# 5  
Old 09-12-2008
Quote:
Originally Posted by n1djs
sed -e 's/\,/^M/g' filename >newfilename
this will not work. ^M is the DOS LF character.
# 6  
Old 09-12-2008
Sorry, but, maybe I wasn't clear enough in my original thread. tr won't work for me because I need to do processing on each string after it's been parsed and before reading of the next record. If I'm reading this right the tr command will read input from 'file' and write the results to FILE.TXT. This will not allow me to do other processing after parsing the line and writing to FILE.TXT. After each read and subsequent parsing of $LINE I am going to do other processing with the parsed data before reading the next record and parsing it. And so on....

Process Flow/Steps/Loop
------------------------
1) Read a record in $LINE (which I already handle via a while loop)
2) Parse $LINE and output fields to FILE.TXT (which is what I need help with)
3) Then I will do other processing using the field data written out to FILE.TXT (which I will handle)
4) Step 1, until end of file.

Step 2 is the only thing I need assistance with. I need to take what's in $LINE, no matter how many comma separated elements there may be, parse them out and write them, one at-a-time, to FILE.TXT. Once all elements have been written out for $LINE I will then use that data for other processing. Then I will read the next record into $LINE. And so on.
# 7  
Old 09-12-2008
Tools what about nested looping?

where I echo "$yf" you could do anything you need to do

Code:
> cat infile2
abc,123,def,456,ghi,789
123,def,456
def,456,ghi,789,jkl
abc
def,456,ghi,789

> cat 2break
#! /usr/bin/bash

while read zf
  do
  echo "$zf" | tr "," "\n" >infile2_t
  while read yf
    do
    echo "$yf"
  done<infile2_t
done <infile2

> 2break
abc
123
def
456
ghi
789
123
def
456
def
456
ghi
789
jkl
abc
def
456
ghi
789


Previous Thread | Next Thread
Test Your Knowledge in Computers #567
Difficulty: Medium
The ternary if-then-else operator generally has the following syntax: E ? X : Y where E is a Boolean expression and if true, the statement is assigned the value Y.
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to extract fields from a CSV i.e comma separated where some of the fields having comma as value?

can anyone help me!!!! How to I parse the CSV file file name : abc.csv (csv file) The above file containing data like abv,sfs,,hju,',',jkk wff,fst,,rgr,',',rgr ere,edf,erg,',',rgr,rgr I have a requirement like i have to extract different field and assign them into different... (4 Replies)
Discussion started by: J.Jena
4 Replies

2. Shell Programming and Scripting

Convert fixed value fields to comma separated values

Hi All, Hope you are doing Great!!!. Today i have came up with a problem to say exactly it was for performance improvement. I have written code in perl as a solution for this to cut in specific range, but it is taking time to run for files thousands of lines so i am expecting a sed... (9 Replies)
Discussion started by: mad man
9 Replies

3. Shell Programming and Scripting

awk to parse comma separated field and removing comma in between number and double quotes

Hi Experts, Please support I have below data in file in comma seperated, but 4th column is containing comma in between numbers, bcz of which when i tried to parse the file the column 6th value(5049641141) is being removed from the file and value(222.82) in column 5 becoming value of column6. ... (3 Replies)
Discussion started by: as7951
3 Replies

4. Shell Programming and Scripting

awk print - fields separated with comma's need to ignore inbetween double quotes

I am trying to re-format a .csv file using awk. I have 6 fields in the .csv file. Some of the fields are enclosed in double quotes and contain comma's inside the quotes. awk is breaking this into multiple fields. Sample lines from the .csv file: Device Name,Personnel,Date,Solution... (1 Reply)
Discussion started by: jxrst
1 Replies

5. Shell Programming and Scripting

reading comma separated data and reorder

hey guys! i need to read data from a file that are comma separated then reorder them in another file to be generated for example: x,y,z a,b,c l,m,n o,p,q and transform this into: x,a,l,o y,b,m,p z,c,n,q Will appreciate your fast reply Regards! (5 Replies)
Discussion started by: maiooi90
5 Replies

6. Shell Programming and Scripting

Perl script to parse output and print it comma separated

I need to arrange output of SQL query into a comma separated format and I'm struggling with processing the output... The output is something like this: <Attribute1 name><x amount of white spaces><Atribute value> <Attribute2 name><x amount of white spaces><Atribute value> <Attribute3... (2 Replies)
Discussion started by: Juha
2 Replies

7. Shell Programming and Scripting

Help parse comma separated list

I have a list of files with the same name, but they have a different date stamp in the name. I can find the first file, but I need to find the second file. I am using this information to create a variable I use later. Here is a example of how I find the first file. "ls -mr... (11 Replies)
Discussion started by: NoMadBanker
11 Replies

8. Shell Programming and Scripting

Varying number of awk search strings

I've created an awk script that handles a varying number of search strings handed to it as command line parameters ($1 $2 etc). There may be 1, or 2 or 3 or more. A simplified version of the script is: awk -v TYP="$1 $2 $3 $4 $5 $6" ' BEGIN { CTYP = split (TYP,TYPP," ") } ... (2 Replies)
Discussion started by: CarlosNC
2 Replies

9. Shell Programming and Scripting

Unix shell script to parse the contents of comma-separated file

Dear All, I have a comma-separated file. 1. The first line of the file(header) should have 4 commas(5 fields). 2. The last line of the file should have 1 comma(2 fields). Pls help me in checking this condition in a shell script. And the number of lines between the first line and last... (11 Replies)
Discussion started by: KrishnaSaran
11 Replies

10. UNIX for Dummies Questions & Answers

Remove whitespaces between comma separated fields from file

Hello all, I am a unix dummy. I am trying to remove spaces between fields. I have the file in the following format 12332432, 2345 , asdfsdf ,100216 , 9999999 12332431, 2341 , asdfsd2 ,100213 , 9999999 &... (2 Replies)
Discussion started by: nitinbjoshi
2 Replies

Featured Tech Videos