Sponsored Content
Top Forums Shell Programming and Scripting Parse apart strings of comma separated data with varying number of fields Post 302235619 by 2reperry on Friday 12th of September 2008 09:39:36 AM
Old 09-12-2008
Parse apart strings of comma separated data with varying number of fields

I have a situation where I am reading a text file line-by-line. Those lines of data contain comma separated fields of data. However, each line can vary in the number of fields it can contain. What I need to do is parse apart each line and write each field of data found (left to right) into a file.

Example Lines of data
---------------------
abc,123,def,456,ghi,789
123,def,456
def,456,ghi,789,jkl
abc
def,456,ghi,789

After the first read the variable $LINE would equal "abc,123,def,456,ghi,789".
Then the fields of data should be parsed and written to FILE.TXT and should look like this :
abc
123
def
456
ghi
789

The second line is read and the variable $LINE would equal "123,def,456" and it's fields of data would be parsed and written to FILE.TXT and would look like so :
123
def
456

and so on for the remaining lines that are read into $LINE.

I have no problems with reading in the lines of data and placing each line into the $LINE variable. I need assistance with how to parse apart each line as it's read into $LINE and writing the fields to a file one-by-one from left to right. I'm pretty sure awk would probably be the best method but I'm open to suggestions on any more efficient methods/commands.

Can anyone help?
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Remove whitespaces between comma separated fields from file

Hello all, I am a unix dummy. I am trying to remove spaces between fields. I have the file in the following format 12332432, 2345 , asdfsdf ,100216 , 9999999 12332431, 2341 , asdfsd2 ,100213 , 9999999 &... (2 Replies)
Discussion started by: nitinbjoshi
2 Replies

2. Shell Programming and Scripting

Unix shell script to parse the contents of comma-separated file

Dear All, I have a comma-separated file. 1. The first line of the file(header) should have 4 commas(5 fields). 2. The last line of the file should have 1 comma(2 fields). Pls help me in checking this condition in a shell script. And the number of lines between the first line and last... (11 Replies)
Discussion started by: KrishnaSaran
11 Replies

3. Shell Programming and Scripting

Varying number of awk search strings

I've created an awk script that handles a varying number of search strings handed to it as command line parameters ($1 $2 etc). There may be 1, or 2 or 3 or more. A simplified version of the script is: awk -v TYP="$1 $2 $3 $4 $5 $6" ' BEGIN { CTYP = split (TYP,TYPP," ") } ... (2 Replies)
Discussion started by: CarlosNC
2 Replies

4. Shell Programming and Scripting

Help parse comma separated list

I have a list of files with the same name, but they have a different date stamp in the name. I can find the first file, but I need to find the second file. I am using this information to create a variable I use later. Here is a example of how I find the first file. "ls -mr... (11 Replies)
Discussion started by: NoMadBanker
11 Replies

5. Shell Programming and Scripting

Perl script to parse output and print it comma separated

I need to arrange output of SQL query into a comma separated format and I'm struggling with processing the output... The output is something like this: <Attribute1 name><x amount of white spaces><Atribute value> <Attribute2 name><x amount of white spaces><Atribute value> <Attribute3... (2 Replies)
Discussion started by: Juha
2 Replies

6. Shell Programming and Scripting

reading comma separated data and reorder

hey guys! i need to read data from a file that are comma separated then reorder them in another file to be generated for example: x,y,z a,b,c l,m,n o,p,q and transform this into: x,a,l,o y,b,m,p z,c,n,q Will appreciate your fast reply Regards! (5 Replies)
Discussion started by: maiooi90
5 Replies

7. Shell Programming and Scripting

awk print - fields separated with comma's need to ignore inbetween double quotes

I am trying to re-format a .csv file using awk. I have 6 fields in the .csv file. Some of the fields are enclosed in double quotes and contain comma's inside the quotes. awk is breaking this into multiple fields. Sample lines from the .csv file: Device Name,Personnel,Date,Solution... (1 Reply)
Discussion started by: jxrst
1 Replies

8. Shell Programming and Scripting

awk to parse comma separated field and removing comma in between number and double quotes

Hi Experts, Please support I have below data in file in comma seperated, but 4th column is containing comma in between numbers, bcz of which when i tried to parse the file the column 6th value(5049641141) is being removed from the file and value(222.82) in column 5 becoming value of column6. ... (3 Replies)
Discussion started by: as7951
3 Replies

9. Shell Programming and Scripting

Convert fixed value fields to comma separated values

Hi All, Hope you are doing Great!!!. Today i have came up with a problem to say exactly it was for performance improvement. I have written code in perl as a solution for this to cut in specific range, but it is taking time to run for files thousands of lines so i am expecting a sed... (9 Replies)
Discussion started by: mad man
9 Replies

10. UNIX for Beginners Questions & Answers

How to extract fields from a CSV i.e comma separated where some of the fields having comma as value?

can anyone help me!!!! How to I parse the CSV file file name : abc.csv (csv file) The above file containing data like abv,sfs,,hju,',',jkk wff,fst,,rgr,',',rgr ere,edf,erg,',',rgr,rgr I have a requirement like i have to extract different field and assign them into different... (4 Replies)
Discussion started by: J.Jena
4 Replies
join(1) 							   User Commands							   join(1)

NAME
join - relational database operator SYNOPSIS
join [-a filenumber | -v filenumber] [-1 fieldnumber] [-2 fieldnumber] [-o list] [-e string] [-t char] file1 file2 join [-a filenumber] [-j fieldnumber] [-j1 fieldnumber] [-j2 fieldnumber] [-o list] [-e string] [-t char] file1 file2 DESCRIPTION
The join command forms, on the standard output, a join of the two relations specified by the lines of file1 and file2. There is one line in the output for each pair of lines in file1 and file2 that have identical join fields. The output line normally con- sists of the common field, then the rest of the line from file1, then the rest of the line from file2. This format can be changed by using the -o option (see below). The -a option can be used to add unmatched lines to the output. The -v option can be used to output only unmatched lines. The default input field separators are blank, tab, or new-line. In this case, multiple separators count as one field separator, and leading separators are ignored. The default output field separator is a blank. If the input files are not in the appropriate collating sequence, the results are unspecified. OPTIONS
Some of the options below use the argument filenumber. This argument should be a 1 or a 2 referring to either file1 or file2, respectively. -a filenumber In addition to the normal output, produce a line for each unpairable line in file filenumber, where filenumber is 1 or 2. If both -a 1 and -a 2 are specified, all unpairable lines will be output. -e string Replace empty output fields in the list selected by option -o with the string string. -j fieldnumber Equivalent to -1fieldnumber -2fieldnumber. -j1 fieldnumber Equivalent to -1fieldnumber. -j2 fieldnumber Equivalent to -2fieldnumber. Fields are numbered starting with 1. -o list Each output line includes the fields specified in list. Fields selected by list that do not appear in the input will be treated as empty output fields. (See the -e option.) Each element of which has the either the form filenum- ber.fieldnumber, or 0, which represents the join field. The common field is not printed unless specifically requested. -t char Use character char as a separator. Every appearance of char in a line is significant. The character char is used as the field separator for both input and output. With this option specified, the collating term should be the same as sort without the -b option. -v filenumber Instead of the default output, produce a line only for each unpairable line in filenumber, where filenumber is 1 or 2. If both -v 1 and -v 2 are specified, all unpairable lines will be output. -1 fieldnumber Join on the fieldnumberth field of file 1. Fields are decimal integers starting with 1. -2fieldnumber Join on the fieldnumberth field of file 2. Fields are decimal integers starting with 1. OPERANDS
The following operands are supported: file1 file2 A path name of a file to be joined. If either of the file1 or file2 operands is -, the standard input is used in its place. file1 and file2 must be sorted in increasing collating sequence as determined by LC_COLLATE on the fields on which they are to be joined, normally the first in each line (see sort(1)). USAGE
See largefile(5) for the description of the behavior of join when encountering files greater than or equal to 2 Gbyte (2**31 bytes). EXAMPLES
Example 1: Joining the password file and group file The following command line will join the password file and the group file, matching on the numeric group ID, and outputting the login name, the group name and the login directory. It is assumed that the files have been sorted in ASCII collating sequence on the group ID fields. example% join -j1 4-j2 3 -o 1.1 2.1 1.6 -t:/etc/passwd /etc/group Example 2: Using the -o option The -o 0 field essentially selects the union of the join fields. For example, given file phone: !Name Phone Number Don +1 123-456-7890 Hal +1 234-567-8901 Yasushi +2 345-678-9012 and file fax: !Name Fax Number Don +1 123-456-7899 Keith +1 456-789-0122 Yasushi +2 345-678-9011 where the large expanses of white space are meant to each represent a single tab character), the command: example% join -t"tab" -a 1 -a 2 -e '(unknown)' -o 0,1.2,2.2 phone fax would produce !Name Phone Number Fax Number Don +1 123-456-7890 +1 123-456-7899 Hal +1 234-567-8901 (unknown Keith (unknown) +1 456-789-012 Yasushi +2 345-678-9012 +2 345-678-9011 ENVIRONMENT VARIABLES
See environ(5) for descriptions of the following environment variables that affect the execution of join: LANG, LC_ALL, LC_CTYPE, LC_MES- SAGES, LC_COLLATE, and NLSPATH. EXIT STATUS
The following exit values are returned: 0 All input files were output successfully. >0 An error occurred. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ |CSI |Enabled | +-----------------------------+-----------------------------+ |Interface Stability |Standard | +-----------------------------+-----------------------------+ SEE ALSO
awk(1), comm(1), sort(1), uniq(1), attributes(5), environ(5), largefile(5), standards(5) NOTES
With default field separation, the collating sequence is that of sort -b; with -t, the sequence is that of a plain sort. The conventions of the join, sort, comm, uniq, and awk commands are wildly incongruous. SunOS 5.10 8 Feb 2000 join(1)
All times are GMT -4. The time now is 05:31 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy