Sponsored Content
Top Forums Shell Programming and Scripting Create a pivot table from CSV file Post 302938398 by RudiC on Sunday 15th of March 2015 03:08:23 PM
Old 03-15-2015
Mayhap sth. in this direction:
Code:
awk     'NR>1   {LN[$28]; HD[$1]; MX[$28,$1]+=$21; SUM[$28]+=$21; TOT[$1]+=$21; TOTAL+=$21}
         END    {               printf "%10s", "";      for (i in HD) printf "%10s", i;         printf "     Total\n";
                 for (j in LN) {printf "%10s",j;        for (i in HD) printf "%10s", MX[j,i]+0; printf "%10s\n", SUM[j]}
                                printf "%10s", "Total"; for (i in HD) printf "%10s", TOT[i] ;   printf "%10s\n", TOTAL;
                }
        ' FS="," file
                  10        11        12         1         2         3         4         5         6         7         8         9     Total
        60        12         0         1         0         0         0         0         0         0         0         0         0        13
        61        27         5         8         1         6        11         0         1         0         0         0         0        59
        62        21         5         0         0         1         4         0         0         1         0         0         0        32
     Total        60        10         9         1         7        15         0         1         1         0         0         0       104

? May need some beautifying...


AND - PLEASE make very sure <CR> characters are removed before running *nix scripts on or before posting data!

Last edited by RudiC; 03-15-2015 at 04:14 PM..
This User Gave Thanks to RudiC For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Pivot table

Hello everyone, I would like to use awk to parse a file with three columns in, like: monday 0 1 monday 1 1 monday 2 1 monday 3 1 monday 4 1 monday 5 1 tuesday 0 5 tuesday 1 1 tuesday 2 1 tuesday 3 1 tuesday 4 1 wednesday 0 1 monday 5 25 they represent the day the hour and the... (2 Replies)
Discussion started by: gio001
2 Replies

2. Shell Programming and Scripting

to create an output file as a table

Hi, I have four input files and would like to create an output file as a table. Please check the example below. File 1. 111111 222222 333333 444444 File 2. 555555 666666 777777 888888 File 3. aaaaa bbbbb ccccc ddddd (2 Replies)
Discussion started by: marcelus
2 Replies

3. Shell Programming and Scripting

Store table contents in csv file

I need to write a script to store the contents of a table in a csv file I'm using Toad, it's a Oracle database. (5 Replies)
Discussion started by: ladyAnne
5 Replies

4. UNIX for Dummies Questions & Answers

Storing data from a table into a csv file

Hi I need to write a bash script to take the data stored in 3 oracle tables .. and filter them and store the results in a csv file. It is an Oracle database Thank you (1 Reply)
Discussion started by: ladyAnne
1 Replies

5. Shell Programming and Scripting

Create Pivot table

I would like to use awk to parse a file with three columns in, like: Chennai,01,1 Chennai,07,1 Chennai,08,3 Chennai,09,6 Chennai,10,12 Chennai,11,19 Chennai,12,10 Chennai,13,12 Kerala,09,2 AP,10,1 AP,11,1 Delhi,13,1 Kerala,13,3 Chennai,00,3 Chennai,01,1 Chennai,02,1 Chennai,07,5 (3 Replies)
Discussion started by: boston_nilesh
3 Replies

6. Shell Programming and Scripting

Convert file in csv or table

Hi there, i have a file like that in attachment (PLEVA3_280711_SAP.txt), i would extract some basic information from it and report in a new file or table like this: i try to use bash and i extract the single object in this way (see attach scriptino.sh), but i receive a strange... (5 Replies)
Discussion started by: alen192
5 Replies

7. Shell Programming and Scripting

Update the table using values from a csv file

i want to run update query for oracle which is in up.sql taking values from a.csv. I have implemented shell script to do it. extn="perl" ls -1 | while read file do echo "$file,$extn" > a.csv done up.sql contains update file_list set filename=$1 where extn=$2; The code to update is... (2 Replies)
Discussion started by: millan
2 Replies

8. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

9. Shell Programming and Scripting

Awk- Pivot Table Averages

Hi everyone, Has anyone figured out yet how to do pivot table averages using AWK. I didn't see anything with regards to doing averages. For example, suppose you have the following table with various individuals and their scores in round1 and round2: SAMPLE SCORE1 SCORE2 British ... (6 Replies)
Discussion started by: Geneanalyst
6 Replies

10. UNIX for Beginners Questions & Answers

1. This will insert the records into db table by reading from ta csv file

I have this code with me but the condition is If any of the mandatory columns are null then entire file will be rejected. LOAD DATA infile ' ' #specifies the name of a datafile containing data that you want to load BADFILE ' ' #specifies the name of... (1 Reply)
Discussion started by: raka123
1 Replies
DBD::File(3)						User Contributed Perl Documentation					      DBD::File(3)

NAME
DBD::File - Base class for writing DBI drivers SYNOPSIS
This module is a base class for writing other DBDs. It is not intended to function as a DBD itself. If you want to access flatfiles, use DBD::AnyData, or DBD::CSV, (both of which are subclasses of DBD::File). DESCRIPTION
The DBD::File module is not a true DBI driver, but an abstract base class for deriving concrete DBI drivers from it. The implication is, that these drivers work with plain files, for example CSV files or INI files. The module is based on the SQL::Statement module, a simple SQL engine. See DBI for details on DBI, SQL::Statement for details on SQL::Statement and DBD::CSV or DBD::IniFile for example drivers. Metadata The following attributes are handled by DBI itself and not by DBD::File, thus they all work like expected: Active ActiveKids CachedKids CompatMode (Not used) InactiveDestroy Kids PrintError RaiseError Warn (Not used) The following DBI attributes are handled by DBD::File: AutoCommit Always on ChopBlanks Works NUM_OF_FIELDS Valid after "$sth-"execute> NUM_OF_PARAMS Valid after "$sth-"prepare> NAME Valid after "$sth-"execute>; undef for Non-Select statements. NULLABLE Not really working, always returns an array ref of one's, as DBD::CSV doesn't verify input data. Valid after "$sth-"execute>; undef for Non-Select statements. These attributes and methods are not supported: bind_param_inout CursorName LongReadLen LongTruncOk Additional to the DBI attributes, you can use the following dbh attribute: f_dir This attribute is used for setting the directory where CSV files are opened. Usually you set it in the dbh, it defaults to the current directory ("."). However, it is overwritable in the statement handles. f_ext This attribute is used for setting the file extension where (CSV) files are opened. There are several possibilities. DBI:CSV:f_dir=data;f_ext=.csv In this case, DBD::File will open only "table.csv" if both "table.csv" and "table" exist in the datadir. The table will still be named "table". If your datadir has files with extensions, and you do not pass this attribute, your table is named "table.csv", which is probably not what you wanted. The extension is always case-insensitive. The table names are not. DBI:CSV:f_dir=data;f_ext=.csv/r In this case the extension is required, and all filenames that do not match are ignored. f_schema This will set the schema name. Default is the owner of the folder in which the table file resides. "undef" is allowed. my $dbh = DBI->connect ("dbi:CSV:", "", "", { f_schema => undef, f_dir => "data", f_ext => ".csv/r", }) or die $DBI::errstr; The effect is that when you get table names from DBI, you can force all tables into the same (or no) schema: my @tables $dbh->tables (); # no f_schema "merijn".foo "merijn".bar # f_schema => "dbi" "dbi".foo "dbi".bar # f_schema => undef foo bar Driver private methods data_sources The "data_sources" method returns a list of subdirectories of the current directory in the form "DBI:CSV:f_dir=$dirname". If you want to read the subdirectories of another directory, use my ($drh) = DBI->install_driver ("CSV"); my (@list) = $drh->data_sources (f_dir => "/usr/local/csv_data" ); list_tables This method returns a list of file names inside $dbh->{f_dir}. Example: my ($dbh) = DBI->connect ("DBI:CSV:f_dir=/usr/local/csv_data"); my (@list) = $dbh->func ("list_tables"); Note that the list includes all files contained in the directory, even those that have non-valid table names, from the view of SQL. KNOWN BUGS
o The module is using flock () internally. However, this function is not available on all platforms. Using flock () is disabled on MacOS and Windows 95: There's no locking at all (perhaps not so important on MacOS and Windows 95, as there's a single user anyways). AUTHOR
This module is currently maintained by H.Merijn Brand < h.m.brand at xs4all.nl > and Jens Rehsack < rehsack at googlemail.com > The original author is Jochen Wiedmann. COPYRIGHT AND LICENSE
Copyright (C) 2009 by H.Merijn Brand & Jens Rehsack Copyright (C) 2004 by Jeff Zucker Copyright (C) 1998 by Jochen Wiedmann All rights reserved. You may freely distribute and/or modify this module under the terms of either the GNU General Public License (GPL) or the Artistic License, as specified in the Perl README file. SEE ALSO
DBI, Text::CSV, Text::CSV_XS, SQL::Statement perl v5.12.1 2009-06-05 DBD::File(3)
All times are GMT -4. The time now is 01:44 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy