Bash script to reorder csv


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Bash script to reorder csv
# 1  
Old 01-15-2011
Bash script to reorder csv

hi guys,

im fairly new to unix and bash scripts and therefore your help would really be appreciated.

i need to write a bash script that will take a csv file, and reorder the data and output to another csv file.

The source csv file will look something like this:
HEAD,671061,Add,SS 2011,1,??? NEW Hanger,IPL/CML/BDCM/BCA
LOT,671061,1,30/12/2010,31/01/2011,,
LINE,671061,43825012,2,30/12/2010,,
LINE,671061,43825174,1,30/12/2010,,
LINE,671061,43825256,1,30/12/2010,,
LINE,671061,43825262,1,30/12/2010,,
LOT,671061,2,20/01/2011,22/02/2011,,
LINE,671061,43825012,2,20/01/2011,,
LINE,671061,43825029,2,20/01/2011,,
LINE,671061,43825256,1,20/01/2011,,
LINE,671061,43825262,1,20/01/2011,,
LOT,671061,3,03/03/2011,04/04/2011,,
LINE,671061,43825012,2,03/03/2011,,
LINE,671061,43825262,1,28/04/2011,,
HEAD,671532,Add,4,MS-CWE-5 CUFF,,
LOT,671532,1,07/04/2011,09/05/2011,,
LINE,671532,76595017,1,07/04/2011
LINE,671532,76595052,2,07/04/2011
LINE,671532,76595069,1,07/04/2011

i need to then sort it by field 2, and then group by header type, and then sort all line items by date, to produce the following:

HEAD,671061,Add,SS 2011,1,??? NEW Hanger,IPL/CML/BDCM/BCA
LINE,671061,43825012,2,30/12/2010,,
LINE,671061,43825174,1,30/12/2010,,
LINE,671061,43825256,1,30/12/2010,,
LINE,671061,43825262,1,30/12/2010,,
LINE,671061,43825012,2,20/01/2011,,
LINE,671061,43825029,2,20/01/2011,,
LINE,671061,43825256,1,20/01/2011,,
LINE,671061,43825262,1,20/01/2011,,
LINE,671061,43825012,2,03/03/2011,,
LINE,671061,43825262,1,28/04/2011,,
LOT,671061,1,30/12/2010,31/01/2011,,
LOT,671061,2,20/01/2011,22/02/2011,,
LOT,671061,3,03/03/2011,04/04/2011,,
HEAD,671532,Add,4,MS-CWE-5 CUFF,,
LINE,671532,76595017,1,07/04/2011,,
LINE,671532,76595052,2,07/04/2011
LINE,671532,76595069,1,07/04/2011
LOT,671532,1,07/04/2011,09/05/2011
HEAD,671533,Add,4,MS-CWE-5 CUFF
LINE,671533,76595017,1,07/04/2011
LINE,671533,76595023,4,07/04/2011
LINE,671533,76595069,1,07/04/2011
LINE,671533,76595017,1,21/04/2011
LINE,671533,76595023,4,21/04/2011
LOT,671533,1,07/04/2011,09/05/2011
LOT,671533,2,21/04/2011,23/05/2011

Any help would be much appreciated

thanks
# 2  
Old 01-16-2011
Try the sort command:
Code:
sort -t, -k2,2n -k1,1 file

# 3  
Old 01-16-2011
@scrutinizer,

can you please explain the below sort command

sort -t, -k2,2n -k1,1 file
# 4  
Old 01-16-2011
Using a comma as a separator (-t,) , numerically sort on the second column (-k2,2n) and then sort on the first column (-k1,1).
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Reorder of fields

I need to reorder the fields of an input file. I was using the following awk statement but I am stumped on how to get it exactly as I need. I need the output to be tab delimited except for the CITY, NAME and ID. Those last set of columns should be a single field separated by a space. And they... (5 Replies)
Discussion started by: ncwxpanther
5 Replies

2. Shell Programming and Scripting

awk to reorder lines in file

The output of an awk script is the below file. Line 1,3 that starts with the Ion... need to be under line 2,4 that starts with R_. The awk runs but no output results. Thank you :). file IonXpress_007 MEV37 R_2016_09_20_12_47_36_user_S5-00580-7-Medexome IonXpress_007 MEV40... (6 Replies)
Discussion started by: cmccabe
6 Replies

3. Shell Programming and Scripting

Csv download in a bash script

I am attempting to download a url in csv format. When I download this url in a browser excel opens up and automatically populates with comma separated values. When I try to use curl or wget I get nothing or garbage. This on the command line just hangs wget -b... (2 Replies)
Discussion started by: bash_in_my_head
2 Replies

4. Shell Programming and Scripting

Bash script help - removing certain rows from .csv file

Hello Everyone, I am trying to find a way to take a .csv file with 7 columns and a ton of rows (over 600,000) and remove the entire row if the cell in forth column is blank. Just to give you a little background on why I am doing this (just in case there is an easier way), I am pulling... (3 Replies)
Discussion started by: MrTuxor
3 Replies

5. Shell Programming and Scripting

BASH script to parse XML and generate CSV

Hi All, Hope all you are doing good! Need your help. I have an XML file which needs to be converted CSV file. I am not an expert of awk/sed so your help is highly appreciated!! XML file looks like this: <l:event dateTime="2013-03-13 07:15:54.713" layerName="OSB" processName="ABC"... (2 Replies)
Discussion started by: bhaskar_m
2 Replies

6. UNIX for Dummies Questions & Answers

Reorder column in Unix

I have a flat file with the 3 columns and separate by tab delimiter, and the last column with special character. A B C D F G Now I would like to swap the third column with second column to following format: A B C D F G I know this cannot be done in Unix command with using cut... (3 Replies)
Discussion started by: newbie2011
3 Replies

7. Shell Programming and Scripting

Reorder and insert column using awk

Hi friends, I am beginner in shell scripting. I am trying to modify (Reorder and insert column) a BIG .txt file using awk to create a .bim file. My start file has 25 columns But I need to make a new file with only 5 columns from the start file and insert a new column in position 3 with value... (5 Replies)
Discussion started by: smitra
5 Replies

8. Shell Programming and Scripting

Reorder the Cut characters

Hi, I have a fixed width flatfile, I want to view this file specific to it's character position and in order I want to...example as below ABCDE.txt 01COLTSMANNING18 02PATS BRADY 12 03PACKSROGERS 12I used unix cut command to see specific field based on length but unable to order them as... (6 Replies)
Discussion started by: okkadu
6 Replies

9. UNIX for Advanced & Expert Users

Unix Bash: substitute columns in .csv using other .csv columns

Hi All, I have two .csv's input.csv having values as (7 columns) ABC,A19907103,ABC DEV YUNG,2.17,1000,2157,07/07/2006 XYZ,H00213850,MM TRUP HILL,38.38,580,23308,31/08/2010 output.csv having (25 columns) A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U,V,W,X,Y... (4 Replies)
Discussion started by: abhivyas
4 Replies

10. Shell Programming and Scripting

Need to modify csv-file with bash script

Hi Guys, I need to write a script, that exports the "moz_places" table of the "places.sqlite"-file (firefox browser history) into a csv-file. That part works. After the export, my csv looks like this: ... 4429;http://www.sqlite.org/sqlite.html;"Command Line Shell For... (11 Replies)
Discussion started by: Sebi0815
11 Replies
Login or Register to Ask a Question