Sponsored Content
Full Discussion: sort csv file
Top Forums Shell Programming and Scripting sort csv file Post 302165988 by drl on Sunday 10th of February 2008 09:43:06 AM
Old 02-10-2008
Hi, mozart.

Welcome to the forum. In the future please surround data with code tags so that we can read them easily, and do the same for scripts.

I added a few extra lines to exercise the sorting of fields. This script uses commands often found on *nix. The standard sort has an option for named months:
Code:
#!/usr/bin/env sh

# @(#) s1       Demonstrate sorting fields by transforming.

#  ____
# /
# |   Infrastructure BEGIN

echo
set -o nounset

debug=":"
debug="echo"

## The shebang using "env" line is designed for portability. For
#  higher security, use:
#
#  #!/bin/sh -

## Use local command version for the commands in this demonstration.

set +o nounset
echo "(Versions displayed with local utility \"version\")"
version >/dev/null 2>&1 && version =o $(_eat $0 $1) tr cut sort
set -o nounset

echo

FILE=${1-data1}
echo " Input file $FILE:"
cat $FILE

# |   Infrastructure END
# \
#  ---

echo
echo " Results from processing:"
tr ' ' '_' <$FILE |
tr ',-' ' ' |
cut -d" " -f1-5 |
sort --key=5,5 --key=4M,4 --key=3,3

exit 0

Producing:
Code:
% ./s1 data2

(Versions displayed with local utility "version")
Linux 2.6.11-x1
GNU bash, version 2.05b.0(1)-release (i386-pc-linux-gnu)
tr (coreutils) 5.2.1
cut (coreutils) 5.2.1
sort (coreutils) 5.2.1

 Input file data2:
Bryant,Kobe4,23-sep-1979,L.A. California,USA,kobe@lakers.comk+1(310)87890
Sharapova,Maria,19-apr-1987,Brandenton FLorida,USA,admin@mariasharapova.com,+1(954)3387488
Bryant,Kobe1,23-aug-1978,L.A. California,USA,kobe@lakers.comk+1(310)87890
Spielberg,Steven,18-dec-1946,L.A.California,USA,steven@dreamworks.com,+1(389)89098
Harnois,Elisabeth,26-may-1979,Detroit Michigan,USA,liz@1treehill.net,+1(313)74564536
Ryan,Michelle,22-apr-1984,Enfield London,UK,mryan@nbc.com,+44(207)2988410
Belle,Camilla,2-oct-19886,Miami Florida,USA,cam_65@hollywood.org, +1(954)3790491
Gellar,Sarah Michelle,14-apr-1977,L.A. California,USA,smg@buffy.com,+1(213)3398848m
Underwood,Carrie,10-mar-1983,Checotah Oklahoma,USA,carrieunderwood@rca.com,+1(918)655423
Bryant,Kobe3,23-sep-1978,L.A. California,USA,kobe@lakers.comk+1(310)87890
Bryant,Kobe2,24-aug-1978,L.A. California,USA,kobe@lakers.comk+1(310)87890

 Results from processing:
Spielberg Steven 18 dec 1946
Gellar Sarah_Michelle 14 apr 1977
Bryant Kobe1 23 aug 1978
Bryant Kobe2 24 aug 1978
Bryant Kobe3 23 sep 1978
Harnois Elisabeth 26 may 1979
Bryant Kobe4 23 sep 1979
Underwood Carrie 10 mar 1983
Ryan Michelle 22 apr 1984
Sharapova Maria 19 apr 1987
Belle Camilla 2 oct 19886

I like your music, I hope you like my opus s1 Smilie. See man pages for details ... cheers, drl
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sort the csv file

hello , the content of a csv file is looking like this... vah1 , imk4 , LDAP Secure. vah1 , fmk1 , Not able to SSH vah1 , fk1 ,Not able to SSH vah1 , imk1 , LDAP Secure. vah1 , imk3 , Un-Secured vah1 , fmk1 , LDAP Secure. vah1 , fk1 , LDAP Secure. vah1 , fmk1 , LDAP Secure. vah1 ,... (2 Replies)
Discussion started by: nagendramv
2 Replies

2. Shell Programming and Scripting

How to sort columns in excel(csv) file

i want sort columns with headers based on another file headers file1 eg: i'm having an empty file with only coumn names like lastname firstname title expirydate stlcno status etc... another file with same column names and some other as well but in different order... file2 eg:firstname... (2 Replies)
Discussion started by: Man83Nagesh
2 Replies

3. UNIX for Dummies Questions & Answers

Sort and join all .csv files

Hi , I have a dump of .csv files in a folder, I need to sort all files by the first 3 columns and merge them all in one big csv file. I am using a cat command to join them, but how can i sort before appending all the files as one big file, Regards, Shruthi (4 Replies)
Discussion started by: shruthidwh
4 Replies

4. Shell Programming and Scripting

Sort csv file

Hello, I am facing the following problem: I have a file in csv format. So several records where data fields are separated by comma ',' The lenght of the fields is variable, as well as the number of fields per record. But there are at least 7 fields per record. I need to sort the file based... (1 Reply)
Discussion started by: BSF
1 Replies

5. Shell Programming and Scripting

Sort on mm/dd/yyyy csv

I have the following csv data that I would like to sort on the 4th field by date from oldest to newest. data: ,,,02/06/2012,Carry H,Add,Add,Sys_RTH,111.222.333.777 4/7/12,,,03/03/2013,Parry H,Add,Add,Sys_RTH,111.222.333.776 ,,,12/06/2012,Harry H,Add,Add,Sys_RTH,111.222.333.778... (3 Replies)
Discussion started by: wbrunc
3 Replies

6. UNIX for Dummies Questions & Answers

Sort csv file by duplicated column value

hello, I have a large file (about 1gb) that is in a file similar to the following: I want to make it so that I can put all the duplicates where column 3 (delimited by the commas) are shown on top. Meaning all people with the same age are listed at the top. The command I used was ... (3 Replies)
Discussion started by: jl487
3 Replies

7. Shell Programming and Scripting

Sort and list values from CSV

I have a CSV with below values name,city,2,country name,city,15,country abc,wq,10,afdfd, qeqe,ewqre,1,wqew I need to sort them in ascending order based on the value of column 3 and then , pick the rows with values less than 10 and be able to display the columns in that row. Can anyone... (11 Replies)
Discussion started by: Nevergivup
11 Replies

8. UNIX for Beginners Questions & Answers

How to sort a column in excel/csv file?

I have to sort the 4th column of an excel/csv file. I tried the following command sort -u --field-separator=, --numeric-sort -k 2 -n dinesh.csv > test.csv But, it's not working. Moreover, I have to do the same for more than 30 excel/csv file. So please help me to do the same. (6 Replies)
Discussion started by: dineshkumarsrk
6 Replies

9. UNIX for Beginners Questions & Answers

How to align/sort the column pairs of an csv file, based on keyword word specified in another file?

I have a csv file as shown below, xop_thy 80 avr_njk 50 str_nyu 60 avr_irt 70 str_nhj 60 avr_ngt 50 str_tgt 80 xop_nmg 50 xop_nth 40 cyv_gty 40 cop_thl 40 vir_tyk 80 vir_plo 20 vir_thk 40 ijk_yuc 70 cop_thy 70 ijk_yuc 80 irt_hgt 80 I need to align/sort the csv file based... (7 Replies)
Discussion started by: dineshkumarsrk
7 Replies
CSV(3pm)						User Contributed Perl Documentation						  CSV(3pm)

NAME
Class::CSV - Class based CSV parser/writer SYNOPSIS
use Class::CSV; my $csv = Class::CSV->parse( filename => 'test.csv', fields => [qw/item qty sub_total/] ); foreach my $line (@{$csv->lines()}) { $line->sub_total('$'. sprintf("%0.2f", $line->sub_total())); print 'Item: '. $line->item(). " ". 'Qty: '. $line->qty(). " ". 'SubTotal: '. $line->sub_total(). " "; } my $cvs_as_string = $csv->string(); $csv->print(); my $csv = Class::CSV->new( fields => [qw/userid username/], line_separator => " "; ); $csv->add_line([2063, 'testuser']); $csv->add_line({ userid => 2064, username => 'testuser2' }); DESCRIPTION
This module can be used to create objects from CSV files, or to create CSV files from objects. Text::CSV_XS is used for parsing and creating CSV file lines, so any limitations in Text::CSV_XS will of course be inherant in this module. EXPORT None by default. METHOD
CONSTRUCTOR parse the parse constructor takes a hash as its paramater, the various options that can be in this hash are detailed below. Required Options o fields - an array ref containing the list of field names to use for each row. there are some reserved words that cannot be used as field names, there is no checking done for this at the moment but it is something to be aware of. the reserved field names are as follows: "string", "set", "get". also field names cannot contain whitespace or any characters that would not be allowed in a method name. Source Options (only one of these is needed) o filename - the path of the CSV file to be opened and parsed. o filehandle - the file handle of the CSV file to be parsed. o objects - an array ref of objects (e.g. Class::DBI objects). for this to work properly the field names provided in fields needs to correspond to the field names of the objects in the array ref. o classdbi_objects - depreciated use objects instead - using classdbi_objects will still work but its advisable to update your code. Optional Options o line_separator - the line seperator to be included at the end of every line. defaulting to " " (unix carriage return). new the new constructor takes a hash as its paramater, the same options detailed in parse apply to new however no Source Options can be used. this constructor creates a blank CSV object of which lines can be added via add_line. ACCESSING lines returns an array ref containing objects of each CSV line (made via Class::Accessor). the field names given upon construction are available as accessors and can be set or get. for more information please see the notes below or the perldoc for Class::Accessor. the lines accessor is also able to be updated/retrieved in the same way as individual lines fields (examples below). Example retrieving the lines: my @lines = @{$csv->lines()}; removing the first line: pop @lines; $csv->lines(@lines); sorting the lines: @lines = sort { $a->userid() <=> $b->userid() } @lines: $csv->lines(@lines); sorting the lines (all-in-one way): $csv->lines([ sort { $a->userid() <=> $b->userid() } @{$csv->lines()} ]); Retrieving a fields value there is two ways to retrieve a fields value (as documented in Class::Accessor). firstly you can call the field name on the object and secondly you can call "get" on the object with the field name as the argument (multiple field names can be specified to retrieve an array of values). examples are below. my $value = $line->test(); OR my $value = $line->get('test'); OR my @values = $line->get(qw/test test2 test3/); Setting a fields value setting a fields value is simmilar to getting a fields value. there are two ways to set a fields value (as documented in Class::Accessor). firstly you can simply call the field name on the object with the value as the argument or secondly you can call "set" on the object with a hash of fields and their values to set (this isn't standard in Class::Accessor, i have overloaded the "set" method to allow this). examples are below. $line->test('123'); OR $line->set( test => '123' ); OR $line->set( test => '123', test2 => '456' ); Retrieving a line as a string to retrieve a line as a string simply call "string" on the object. my $string = $line->string(); new_line returns a new line object, this can be useful for to "splice" a line into lines (see example below). you can pass the values of the line as an ARRAY ref or a HASH ref. Example my $line = $csv->new_line({ userid => 123, domainname => 'splicey.com' }); my @lines = $csv->lines(); splice(@lines, 1, 0, $line); OR splice(@{$csv->lines()}, 1, 0, $csv->new_line({ userid => 123, domainname => 'splicey.com' })); add_line adds a line to the lines stack. this is mainly useful when the new constructor is used but can of course be used with any constructor. it will add a new line to the end of the lines stack. you can pass the values of the line as an ARRAY ref or a HASH ref. examples of how to use this are below. Example $csv->add_line(['house', 100000, 4]); $csv->add_line({ item => 'house', cost => 100000, bedrooms => 4 }); OUTPUT string returns the object as a string (CSV file format). print calls "print" on string (prints the CSV to STDOUT). SEE ALSO
Text::CSV_XS, Class::Accessor AUTHOR
David Radunz, <david@boxen.net> COPYRIGHT AND LICENSE
Copyright 2004 by David Radunz This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.10.0 2007-02-08 CSV(3pm)
All times are GMT -4. The time now is 07:20 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy