Sponsored Content
Top Forums UNIX for Advanced & Expert Users How to Parse a CSV file into a Different Format Post 88457 by Abhishek Ghose on Friday 4th of November 2005 08:11:21 AM
Old 11-04-2005
Bug

You can also use this:

sed 's/,[ ]*$//g' original
head -1 original|tr ',' '\n'|tail +2|sed 's/^/,/'>file1
tail +2 original|join -1 100 -2 100 - file1|awk -F"," '{print $1","$NF}'>file2
tail +2 original|cut -d"," -f2-|tr "," "\n">file3
paste -d"," file2 file3 >final
rm file1 file2 file3


"original" is the name of the file you start with (the csv file).
"final" contains the transformed data.

(There is a small hitch with the join statement used above (statement 3). It uses columns 100 in both files two join. This yields a cartesian product. This wont work if either file contains more than or equal to 100 columns. In such a situation change the 100 to column number that doesnt exist. I know, this is not a very neat method...but didnt work on join much :-) )
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

CSV File parse help in Perl

Folks, I have a bit of an issue trying to obtain some data from a csv file using PERL. I can sort the file and remove any duplicates leaving only 4 or 5 rows containing data. My problem is that the data contained in the original file contains a lot more columns and when I try ro run this script... (13 Replies)
Discussion started by: lodey
13 Replies

2. Shell Programming and Scripting

parse csv file, sha1 hash and output

I have a file, not really a csv, but containing delineated data just the same. Lets call that file "raw_data.txt". It contains data in the format of company name:fein number like this: first company name:123456789 second company name:987654321 what i need to do is read this file, apply... (11 Replies)
Discussion started by: FreddyG
11 Replies

3. Shell Programming and Scripting

Parse XML file into CSV with shell?

Hi, It's been a few years since college when I did stuff like this all the time. Can someone help me figure out how to best tackle this problem? I need to parse a file full of entries that look like this: <eq action="A" sectyType="0" symbol="PGR" exch="CA" curr="VEF" sess="NORM"... (7 Replies)
Discussion started by: Pcushing
7 Replies

4. Shell Programming and Scripting

Parse csv file

Hi, Our requirement is to parse the input file(.csv format). The each column in the file is delimited with comma. We need to take each column and apply some business validation rule. If data itself contains comma, then those fields are enclosed with double quotes ("). We can see this double... (7 Replies)
Discussion started by: vfrg
7 Replies

5. Shell Programming and Scripting

how to parse the file in xml format using awk/nawk

Hi All, I have an xml file with the below format. <a>111</a><b>222</b><c>333<c><d><e>123</e><f>234</f><d><e>456</e><f>789</f> output needed is 111,222,333,123,234 111,222,333,456,789 nawk 'BEGIN{FS="<|>"} {print a,b,c,e,f a="" ... (7 Replies)
Discussion started by: natalie23
7 Replies

6. Shell Programming and Scripting

How to read and parse the content of csv file containing # as delimeter into fields using Bash?

#!/bin/bash i=0 cat 1.csv | while read fileline do echo "$fileline" IFS="#" flds=( $fileline ) nrofflds=${#flds} echo "noof fields$nrofflds" fld=0 while do echo "noof counter$fld" echo "$nrofflds" #fld1="${flds}" trying to store the content of line to fields but i... (4 Replies)
Discussion started by: barani75
4 Replies

7. Shell Programming and Scripting

How to parse csv format?

Hi, I have a file with 3 fields in csv format: /tmp/foo/,MODIFY,bar/toto "/tmp/foo, bar/","ATTRIB,ISDIR","toto, tata/foobar"I would like to split fields in order to obtain the following: Line1: /tmp/foo/ MODIFY bar/totoLine2: /tmp/foo, bar/ ATTRIB,ISDIR toto, tata/foobarCan't find my way... (11 Replies)
Discussion started by: chebarbudo
11 Replies

8. Shell Programming and Scripting

how to parse this file and obtain a .csv or .xls

Hello Expert, I have a file in the following format: SYNTAX_VERSION 5 MONITOR "NAME_TEMPLATES" DESCRIPTION "Monitors for contents of error " INTERVAL "1m" MONPROG "script.sh NAME_TEMPLATES" MAXTHRESHOLD GEN_BELOW_RESET SEVERITY Major ... (17 Replies)
Discussion started by: Ant-one
17 Replies

9. UNIX for Dummies Questions & Answers

Help to parse csv file with shell script

Hello ! I am very aware that this is not the first time this question is asked here, because I have already read a lot of previous answers, but none of them worked, so... As said in the title, I want to read a csv file with a bash script. Here is a sample of the file: ... (4 Replies)
Discussion started by: Grhyll
4 Replies

10. Shell Programming and Scripting

How to parse this file using awk and output in CSV format?

My source file looks like this: Cust-Number = "101" Cust-Name="Joe" Cust-Town="London" Cust-hobby="tennis" Cust-purchase="200" Cust-Number = "102" Cust-Name="Mary" Cust-Town="Newyork" Cust-hobby="reading" Cust-purchase="125" Now I want to parse this file (leaving out hobby) and... (10 Replies)
Discussion started by: Balav
10 Replies
diff3(1)						      General Commands Manual							  diff3(1)

NAME
diff3 - 3-way differential file comparison SYNOPSIS
file1 file2 file3 DESCRIPTION
compares three versions of a file, and prints disagreeing ranges of text flagged with these codes: all three files differ file1 is different file2 is different file3 is different The type of change required to convert a given range of a given file to some other is indicated in one of these ways: Text is to be appended after line number n1 in file f, where f = or Text is to be changed in the range line n1 through line n2. If n1 = n2, the range can be abbreviated to n1. The original contents of the range follows immediately after a indication. When the contents of two files are identical, the contents of the lower-numbered file is suppressed. Produces a script for the editor that can be used to incorporate into file1 all changes between file2 and file3 (see ed(1)); i.e., the changes that normally would be flagged and Produces a script to incorporate only changes flagged Produces a script to incorporate only changes flagged Produces a script that will incorporate all changes between file2 and file3, but treat overlapping changes (that is, changes that would be flagged with in normal listing) differently. The overlapping lines in both files will be inserted by the edit script bracketed by and lines. Produces a script that will incorporate only changes flagged , but treat these changes in the manner of option. The following command applies the resulting script to file1. EXTERNAL INFLUENCES
International Code Set Support Single- and multi-byte character code sets are supported. WARNINGS
Text lines that consist of a single period defeat Files longer than 64K bytes do not work. FILES
SEE ALSO
diff(1). diff3(1)
All times are GMT -4. The time now is 01:23 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy