Sponsored Content
Special Forums Hardware Filesystems, Disks and Memory manipulate csv file to add columns Post 302098443 by oscarmon on Monday 4th of December 2006 09:37:51 AM
Old 12-04-2006
manipulate csv file to add columns

Hi, I have a csv file with a key composed by 3 columns and some other numeric fields and I need to obtain the partial amounts by some part of the key. This may be some difficult to understand, so better see an example, where my input file is:

name,surname,department,y2004,y2005,y2006
John,Smith,sales,300,200,100
Mary,Grant,sales,100,0,100
John,Smith,IT,0,50,250
George,Parker,IT,200,150,250
Sarah,Lee,marketing,300,400,100
John,Smith,marketing,200,0,250

And would like to obtain an output file with the total amounts added by department like:

department,y2004,y2005,y2006
sales,400,200,200
IT,200,200,500
marketing,500,400,350

If you imagine this is a database table, this could easily be done in sql with a group by, but unix scripting is something different... Could anyone help me with this?
Thanks a lot.

Oscar.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Add Empty columns at the end of csv file

Hi, Can you please tell me how to add empty columns at the end csv file? Currently there are 6 columns in the csv file. I want to add 35 empty columns at the end of this csv file. Thanks, Tushar (17 Replies)
Discussion started by: Tushar Bendale
17 Replies

2. Shell Programming and Scripting

Add columns in csv file

Hi everybody, I am new here and I need a reel help please A have a csv file and I want to add new in the end of the file Devisce Model,VMGuest Name,Host OS Name, Memory Size Value1,Value2,Value3,Value4 Value5,Value6,Value7,Value8 Value9,Value10,Value11,Value12 And I want to add to new... (3 Replies)
Discussion started by: villebonnais
3 Replies

3. Shell Programming and Scripting

AWK : Add columns in the end of csv file

Hi everybody, I need some help please I have a csv file named masterFile1.csv header1,header2,header3 value1,value2,value3 value4,value5,value6 I am trying to add new columns in the end of the csv to have a new csv file named masterFile2.csv like this :... (3 Replies)
Discussion started by: villebonnais
3 Replies

4. UNIX for Dummies Questions & Answers

Manipulate and move columns in a file

Hello Unix Gurus, I have a request 2 perform several functions on a file, delete columns, delete rows based on column value, and finally move around columns in the final output. Consider the following input file with 12 columns; ... (1 Reply)
Discussion started by: chumsky
1 Replies

5. Linux

Add empty columns at the end of a CSV file

I have a .CSV file (lets say named as file.csv) with numeric and string values. The string might contain commas hence they are enclosed in double quotes as in the below format. column1,column2,column3,column4,column5,column6,column7 12,455,"string, with, quotes, and with, commas, in... (3 Replies)
Discussion started by: dhruuv369
3 Replies

6. Shell Programming and Scripting

Manipulate a CSV File

Hello, How do i manipulate .csv file to this format? Thank you very much. Source: john,5 marco,7 john,4 paul,3 marco,8 Output: john,9 marco,15 (5 Replies)
Discussion started by: tara123
5 Replies

7. Shell Programming and Scripting

awk command to manipulate csv file in UNIX

Hi, I am new to awk/unix and am trying to put together a script to manipulate the date column in a csv file. I have file1.csv with the following contents: Date,ID,Number,Amount,Volume,Size 01-Apr-2014,WERFG,998,105873.96,10873.96,1342.11 01-Apr-2014,POYFR,267,5681.44,5681.44,462.96 I... (2 Replies)
Discussion started by: Prit Siv
2 Replies

8. Shell Programming and Scripting

Add 8 columns at the end of .csv file using awk

Hello all, I have a .csv file of 16 columns consists of bunch of numbers. 6.45E+01 1.17E+01 8.10E+04 8.21E+01 8.50E+00 1.20E+01 1.02E+01 1.88E+01 1.86E+04 3.53E+03 1.09E+07 3.82E+04 2.09E+03 3.57E+03 2.98E+03 3.93E+03 6.34E+01 3.23E+01 9.24E+04 ... (5 Replies)
Discussion started by: Zam_1234
5 Replies

9. Shell Programming and Scripting

awk command to manipulate csv file in UNIX

Hi, I am new to awk and unix programming and trying to manipulate a csv file. My current csv file looks like this: col1,col2,col3,col4,col5,col4,col5,col6,col7,col8 223,V,c,2,4,f,r,,y,z 223,V,c,3,2,f,r,,y,z 223,V,c,1,4,f,r,,y,z 223,V,c,4,3,f,r,,y,z 227,V,c,3,1,f,r,,y,z... (8 Replies)
Discussion started by: Prit Siv
8 Replies

10. Shell Programming and Scripting

To add multiple columns in csv file

Hi Guys, Can you help to add static values at the end of the csv file with headers input_file id,name 1,neo 2,pull Expected id,name,status,entry,g_id 1,neo,done,2019-11-01T07:14:23,pass 2,pull,done,2019-11-01T07:14:23,pass My try but not able replacing properly and unable... (1 Reply)
Discussion started by: Master_Mind
1 Replies
CSV(3pm)						User Contributed Perl Documentation						  CSV(3pm)

NAME
Class::CSV - Class based CSV parser/writer SYNOPSIS
use Class::CSV; my $csv = Class::CSV->parse( filename => 'test.csv', fields => [qw/item qty sub_total/] ); foreach my $line (@{$csv->lines()}) { $line->sub_total('$'. sprintf("%0.2f", $line->sub_total())); print 'Item: '. $line->item(). " ". 'Qty: '. $line->qty(). " ". 'SubTotal: '. $line->sub_total(). " "; } my $cvs_as_string = $csv->string(); $csv->print(); my $csv = Class::CSV->new( fields => [qw/userid username/], line_separator => " "; ); $csv->add_line([2063, 'testuser']); $csv->add_line({ userid => 2064, username => 'testuser2' }); DESCRIPTION
This module can be used to create objects from CSV files, or to create CSV files from objects. Text::CSV_XS is used for parsing and creating CSV file lines, so any limitations in Text::CSV_XS will of course be inherant in this module. EXPORT None by default. METHOD
CONSTRUCTOR parse the parse constructor takes a hash as its paramater, the various options that can be in this hash are detailed below. Required Options o fields - an array ref containing the list of field names to use for each row. there are some reserved words that cannot be used as field names, there is no checking done for this at the moment but it is something to be aware of. the reserved field names are as follows: "string", "set", "get". also field names cannot contain whitespace or any characters that would not be allowed in a method name. Source Options (only one of these is needed) o filename - the path of the CSV file to be opened and parsed. o filehandle - the file handle of the CSV file to be parsed. o objects - an array ref of objects (e.g. Class::DBI objects). for this to work properly the field names provided in fields needs to correspond to the field names of the objects in the array ref. o classdbi_objects - depreciated use objects instead - using classdbi_objects will still work but its advisable to update your code. Optional Options o line_separator - the line seperator to be included at the end of every line. defaulting to " " (unix carriage return). new the new constructor takes a hash as its paramater, the same options detailed in parse apply to new however no Source Options can be used. this constructor creates a blank CSV object of which lines can be added via add_line. ACCESSING lines returns an array ref containing objects of each CSV line (made via Class::Accessor). the field names given upon construction are available as accessors and can be set or get. for more information please see the notes below or the perldoc for Class::Accessor. the lines accessor is also able to be updated/retrieved in the same way as individual lines fields (examples below). Example retrieving the lines: my @lines = @{$csv->lines()}; removing the first line: pop @lines; $csv->lines(@lines); sorting the lines: @lines = sort { $a->userid() <=> $b->userid() } @lines: $csv->lines(@lines); sorting the lines (all-in-one way): $csv->lines([ sort { $a->userid() <=> $b->userid() } @{$csv->lines()} ]); Retrieving a fields value there is two ways to retrieve a fields value (as documented in Class::Accessor). firstly you can call the field name on the object and secondly you can call "get" on the object with the field name as the argument (multiple field names can be specified to retrieve an array of values). examples are below. my $value = $line->test(); OR my $value = $line->get('test'); OR my @values = $line->get(qw/test test2 test3/); Setting a fields value setting a fields value is simmilar to getting a fields value. there are two ways to set a fields value (as documented in Class::Accessor). firstly you can simply call the field name on the object with the value as the argument or secondly you can call "set" on the object with a hash of fields and their values to set (this isn't standard in Class::Accessor, i have overloaded the "set" method to allow this). examples are below. $line->test('123'); OR $line->set( test => '123' ); OR $line->set( test => '123', test2 => '456' ); Retrieving a line as a string to retrieve a line as a string simply call "string" on the object. my $string = $line->string(); new_line returns a new line object, this can be useful for to "splice" a line into lines (see example below). you can pass the values of the line as an ARRAY ref or a HASH ref. Example my $line = $csv->new_line({ userid => 123, domainname => 'splicey.com' }); my @lines = $csv->lines(); splice(@lines, 1, 0, $line); OR splice(@{$csv->lines()}, 1, 0, $csv->new_line({ userid => 123, domainname => 'splicey.com' })); add_line adds a line to the lines stack. this is mainly useful when the new constructor is used but can of course be used with any constructor. it will add a new line to the end of the lines stack. you can pass the values of the line as an ARRAY ref or a HASH ref. examples of how to use this are below. Example $csv->add_line(['house', 100000, 4]); $csv->add_line({ item => 'house', cost => 100000, bedrooms => 4 }); OUTPUT string returns the object as a string (CSV file format). print calls "print" on string (prints the CSV to STDOUT). SEE ALSO
Text::CSV_XS, Class::Accessor AUTHOR
David Radunz, <david@boxen.net> COPYRIGHT AND LICENSE
Copyright 2004 by David Radunz This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.10.0 2007-02-08 CSV(3pm)
All times are GMT -4. The time now is 06:05 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy