Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Bash script to reorder csv
Top Forums Shell Programming and Scripting Bash script to reorder csv Post 302488169 by daz_20 on Saturday 15th of January 2011 04:05:43 PM
Old 01-15-2011
Bash script to reorder csv
hi guys,

im fairly new to unix and bash scripts and therefore your help would really be appreciated.

i need to write a bash script that will take a csv file, and reorder the data and output to another csv file.

The source csv file will look something like this:
HEAD,671061,Add,SS 2011,1,??? NEW Hanger,IPL/CML/BDCM/BCA
LOT,671061,1,30/12/2010,31/01/2011,,
LINE,671061,43825012,2,30/12/2010,,
LINE,671061,43825174,1,30/12/2010,,
LINE,671061,43825256,1,30/12/2010,,
LINE,671061,43825262,1,30/12/2010,,
LOT,671061,2,20/01/2011,22/02/2011,,
LINE,671061,43825012,2,20/01/2011,,
LINE,671061,43825029,2,20/01/2011,,
LINE,671061,43825256,1,20/01/2011,,
LINE,671061,43825262,1,20/01/2011,,
LOT,671061,3,03/03/2011,04/04/2011,,
LINE,671061,43825012,2,03/03/2011,,
LINE,671061,43825262,1,28/04/2011,,
HEAD,671532,Add,4,MS-CWE-5 CUFF,,
LOT,671532,1,07/04/2011,09/05/2011,,
LINE,671532,76595017,1,07/04/2011
LINE,671532,76595052,2,07/04/2011
LINE,671532,76595069,1,07/04/2011

i need to then sort it by field 2, and then group by header type, and then sort all line items by date, to produce the following:

HEAD,671061,Add,SS 2011,1,??? NEW Hanger,IPL/CML/BDCM/BCA
LINE,671061,43825012,2,30/12/2010,,
LINE,671061,43825174,1,30/12/2010,,
LINE,671061,43825256,1,30/12/2010,,
LINE,671061,43825262,1,30/12/2010,,
LINE,671061,43825012,2,20/01/2011,,
LINE,671061,43825029,2,20/01/2011,,
LINE,671061,43825256,1,20/01/2011,,
LINE,671061,43825262,1,20/01/2011,,
LINE,671061,43825012,2,03/03/2011,,
LINE,671061,43825262,1,28/04/2011,,
LOT,671061,1,30/12/2010,31/01/2011,,
LOT,671061,2,20/01/2011,22/02/2011,,
LOT,671061,3,03/03/2011,04/04/2011,,
HEAD,671532,Add,4,MS-CWE-5 CUFF,,
LINE,671532,76595017,1,07/04/2011,,
LINE,671532,76595052,2,07/04/2011
LINE,671532,76595069,1,07/04/2011
LOT,671532,1,07/04/2011,09/05/2011
HEAD,671533,Add,4,MS-CWE-5 CUFF
LINE,671533,76595017,1,07/04/2011
LINE,671533,76595023,4,07/04/2011
LINE,671533,76595069,1,07/04/2011
LINE,671533,76595017,1,21/04/2011
LINE,671533,76595023,4,21/04/2011
LOT,671533,1,07/04/2011,09/05/2011
LOT,671533,2,21/04/2011,23/05/2011

Any help would be much appreciated

thanks
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need to modify csv-file with bash script

Hi Guys, I need to write a script, that exports the "moz_places" table of the "places.sqlite"-file (firefox browser history) into a csv-file. That part works. After the export, my csv looks like this: ... 4429;http://www.sqlite.org/sqlite.html;"Command Line Shell For... (11 Replies)
Discussion started by: Sebi0815
11 Replies

2. UNIX for Advanced & Expert Users

Unix Bash: substitute columns in .csv using other .csv columns

Hi All, I have two .csv's input.csv having values as (7 columns) ABC,A19907103,ABC DEV YUNG,2.17,1000,2157,07/07/2006 XYZ,H00213850,MM TRUP HILL,38.38,580,23308,31/08/2010 output.csv having (25 columns) A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U,V,W,X,Y... (4 Replies)
Discussion started by: abhivyas
4 Replies

3. Shell Programming and Scripting

Reorder the Cut characters

Hi, I have a fixed width flatfile, I want to view this file specific to it's character position and in order I want to...example as below ABCDE.txt 01COLTSMANNING18 02PATS BRADY 12 03PACKSROGERS 12I used unix cut command to see specific field based on length but unable to order them as... (6 Replies)
Discussion started by: okkadu
6 Replies

4. Shell Programming and Scripting

Reorder and insert column using awk

Hi friends, I am beginner in shell scripting. I am trying to modify (Reorder and insert column) a BIG .txt file using awk to create a .bim file. My start file has 25 columns But I need to make a new file with only 5 columns from the start file and insert a new column in position 3 with value... (5 Replies)
Discussion started by: smitra
5 Replies

5. UNIX for Dummies Questions & Answers

Reorder column in Unix

I have a flat file with the 3 columns and separate by tab delimiter, and the last column with special character. A B C D F G Now I would like to swap the third column with second column to following format: A B C D F G I know this cannot be done in Unix command with using cut... (3 Replies)
Discussion started by: newbie2011
3 Replies

6. Shell Programming and Scripting

BASH script to parse XML and generate CSV

Hi All, Hope all you are doing good! Need your help. I have an XML file which needs to be converted CSV file. I am not an expert of awk/sed so your help is highly appreciated!! XML file looks like this: <l:event dateTime="2013-03-13 07:15:54.713" layerName="OSB" processName="ABC"... (2 Replies)
Discussion started by: bhaskar_m
2 Replies

7. Shell Programming and Scripting

Bash script help - removing certain rows from .csv file

Hello Everyone, I am trying to find a way to take a .csv file with 7 columns and a ton of rows (over 600,000) and remove the entire row if the cell in forth column is blank. Just to give you a little background on why I am doing this (just in case there is an easier way), I am pulling... (3 Replies)
Discussion started by: MrTuxor
3 Replies

8. Shell Programming and Scripting

Csv download in a bash script

I am attempting to download a url in csv format. When I download this url in a browser excel opens up and automatically populates with comma separated values. When I try to use curl or wget I get nothing or garbage. This on the command line just hangs wget -b... (2 Replies)
Discussion started by: bash_in_my_head
2 Replies

9. Shell Programming and Scripting

awk to reorder lines in file

The output of an awk script is the below file. Line 1,3 that starts with the Ion... need to be under line 2,4 that starts with R_. The awk runs but no output results. Thank you :). file IonXpress_007 MEV37 R_2016_09_20_12_47_36_user_S5-00580-7-Medexome IonXpress_007 MEV40... (6 Replies)
Discussion started by: cmccabe
6 Replies

10. Shell Programming and Scripting

Reorder of fields

I need to reorder the fields of an input file. I was using the following awk statement but I am stumped on how to get it exactly as I need. I need the output to be tab delimited except for the CITY, NAME and ID. Those last set of columns should be a single field separated by a space. And they... (5 Replies)
Discussion started by: ncwxpanther
5 Replies

LEARN ABOUT FREEBSD

atf-sh

ATF-SH(1)						    BSD General Commands Manual 						 ATF-SH(1)

NAME

     atf-sh [-s shell] -- interpreter for shell-based test programs

SYNOPSIS

     atf-sh script

DESCRIPTION

     atf-sh is an interpreter that runs the test program given in script after loading the atf-sh(3) library.

     atf-sh is not a real interpreter though: it is just a wrapper around the system-wide shell defined by ATF_SHELL.  atf-sh executes the inter-
     preter, loads the atf-sh(3) library and then runs the script.  You must consider atf-sh to be a POSIX shell by default and thus should not
     use any non-standard extensions.

     The following options are available:

     -s shell	  Specifies the shell to use instead of the value provided by ATF_SHELL.

ENVIRONMENT

     ATF_LIBEXECDIR    Overrides the builtin directory where atf-sh is located.  Should not be overridden other than for testing purposes.
     ATF_PKGDATADIR    Overrides the builtin directory where libatf-sh.subr is located.  Should not be overridden other than for testing purposes.
     ATF_SHELL	       Path to the system shell to be used in the generated scripts.  Scripts must not rely on this variable being set to select a
		       specific interpreter.

EXAMPLES

     Scripts using atf-sh(3) should start with:

	   #! /usr/bin/env atf-sh

     Alternatively, if you want to explicitly choose a shell interpreter, you cannot rely on env(1) to find atf-sh.  Instead, you have to hardcode
     the path to atf-sh in the script and then use the -s option afterwards as a single parameter:

	   #! /path/to/bin/atf-sh -s/bin/bash

ENVIRONMENT

     ATF_SHELL	  Path to the system shell to be used in the generated scripts.

SEE ALSO

     atf-sh(3)

BSD
								September 27, 2014							       BSD
All times are GMT -4. The time now is 10:35 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy