Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Reconciling two CSV files using shell scripting Post 303041099 by RudiC on Thursday 14th of November 2019 04:35:39 PM
Old 11-14-2019
Code:
awk -F, '
FNR == 1        {FCNT++                                                                                         # inc file counter with every new file
                }
FNR > 1         {KEYS[$1]                                                                                       # keep $1 in an array; overwrite duplicates
                 for (i=2; i<=NF; i++) W[$1,FCNT,i] = $i                                                        # keep fields in array indexed by key, file No., field No.
                }
END             {printf "Key, diff columns"                                                                     # start printing header
                 for (i=1; i<NF; i++) printf ",f1_value%d,f2_value%d,value%d_diff", i, i, i                     # complete header line for all fields
                 printf RS
                 for (k in KEYS)        {for (i=NF; i>1; i--)   {N1 = W[k,1,i]                                  # for all keys, for all fields, get values
                                                                 N2 = W[k,2,i]                                  # for both files,
                                                                 D1 = N2 - N1                                   # and calc difference
                                                                 OUT = sprintf (",%s,%s,%s", N1, N2, D1) OUT    # collect all those in temp var OUT
                                                                 if (D1) COLS = ",Value" i-1 COLS               # if diff exist, collect fields in temp var COLS
                                                                }
                                         print k, "\"" substr (COLS,2) "\"", substr (OUT, 2)                    # print all those, cutting off leading comma
                                         OUT = COLS = ""                                                        # reset temp vars
                                        }
                }
' file[12]                                                                                                      # OFS and SUBSEP relict from development, not needed

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with shell scripting for accepting .csv files as CLA

I want to automate test script on shell scripting. There are 2 .csv files named account.csv and balance.csv.These files needs to passed as command line arguments and the following logic needs to applied further. Any account with a balance that was due before Oct 23, 2007 has an overdue... (2 Replies)
Discussion started by: coolguy123
2 Replies

2. Shell Programming and Scripting

How to insert a sequence number column inside a pipe delimited csv file using shell scripting?

Hi All, I need a shell script which could insert a sequence number column inside a dat file(pipe delimited). I have the dat file similar to the one as shown below.. |A|B|C||D|E |F|G|H||I|J |K|L|M||N|O |P|Q|R||S|T As shown above, the column 4 is currently blank and i need to insert sequence... (5 Replies)
Discussion started by: nithins007
5 Replies

3. Shell Programming and Scripting

Shell scripting:from text file to CSV

Hello friends, I have a file as follows: "empty line" content1 content2 content3 content1 content2 content3 content1 content2 content3 It starts with an empty line, how can i get a csv like this: (12 Replies)
Discussion started by: kraterions
12 Replies

4. Shell Programming and Scripting

How to create or convert to pdf files from csv files using shell script?

Hi, Can anyone help me how to convert a .csv file to a .pdf file using shell script Thanks (2 Replies)
Discussion started by: ssk250
2 Replies

5. Shell Programming and Scripting

How to calculate avg values of csv file using shell scripting .?

hi all i have a reporting work and i want it to be automated using shell scripting kindly let me know how can i make that possibe . eg data are :... (2 Replies)
Discussion started by: Avinash shaw
2 Replies

6. Shell Programming and Scripting

How to calculate average of csv using shell scripting?

Hi, I need to calculate the average of the following values using shell scripitng. Can anyone please suggest a solution? ... (10 Replies)
Discussion started by: karan pratap si
10 Replies

7. Shell Programming and Scripting

Shell script for field wise record count for different Files .csv files

Hi, Very good wishes to all! Please help to provide the shell script for generating the record counts in filed wise from the .csv file My question: Source file: Field1 Field2 Field3 abc 12f sLm 1234 hjd 12d Hyd 34 Chn My target file should generate the .csv file with the... (14 Replies)
Discussion started by: Kirands
14 Replies

8. Shell Programming and Scripting

Need a piece of shell scripting to remove column from a csv file

Hi, I need to remove first column from a csv file and i can do this by using below command. cut -f1 -d, --complement Mytest.csv I need to implement this in shell scripting, Whenever i am using the above command alone in command line it is working fine. I have 5 files in my directory and... (3 Replies)
Discussion started by: Samah
3 Replies

9. Shell Programming and Scripting

Read csv file, convert the data and make one text file in UNIX shell scripting

I have input data looks like this which is a part of a csv file 7,1265,76548,"0102:04" 8,1266,76545,"0112:04" I need to make the output data should look like this and the output data will be part of text file: 7|1265000 |7654899 |A| 8|12660000 |76545999 |B| The logic behind the... (6 Replies)
Discussion started by: RJG
6 Replies

10. UNIX for Beginners Questions & Answers

Export Oracle multiple tables to multiple csv files using UNIX shell scripting

Hello All, just wanted to export multiple tables from oracle sql using unix shell script to csv file and the below code is exporting only the first table. Can you please suggest why? or any better idea? export FILE="/abc/autom/file/geo_JOB.csv" Export= `sqlplus -s dev01/password@dEV3... (16 Replies)
Discussion started by: Hope
16 Replies
counter(n)						      Counters and Histograms							counter(n)

NAME
counter - Procedures for counters and histograms SYNOPSIS
package require Tcl 8 package require counter ?2.0? ::counter::init tag args ::counter::count tag ?delta? ?instance? ::counter::start tag instance ::counter::stop tag instance ::counter::get tag args ::counter::exists tag ::counter::names ::counter::histHtmlDisplay tag args DESCRIPTION
The counter package provides a counter facility and can compute statistics and histograms over the collected data. ::counter::init tag args This defines a counter with the name tag. The args determines the characteristics of the counter. The args are -group name Keep a grouped counter where the name of the histogram bucket is passed into ::counter::count. -hist bucketsize Accumulate the counter into histogram buckets of size bucketsize. For example, if the samples are millisecond time values and bucketsize is 10, then each histogram bucket represents time values of 0 to 10 msec, 10 to 20 msec, 20 to 30 msec, and so on. -hist2x bucketsize Accumulate the statistic into histogram buckets. The size of the first bucket is bucketsize, each other bucket holds values 2 times the size of the previous bucket. For example, if bucketsize is 10, then each histogram bucket represents time values of 0 to 10 msec, 10 to 20 msec, 20 to 40 msec, 40 to 80 msec, and so on. -hist10x bucketsize Accumulate the statistic into histogram buckets. The size of the first bucket is bucketsize, each other bucket holds values 10 times the size of the previous bucket. For example, if bucketsize is 10, then each histogram bucket represents time val- ues of 0 to 10 msec, 10 to 100 msec, 100 to 1000 msec, and so on. -lastn N Save the last N values of the counter to maintain a "running average" over the last N values. -timehist secsPerMinute Keep a time-based histogram. The counter is summed into a histogram bucket based on the current time. There are 60 per- minute buckets that have a size determined by secsPerMinute, which is normally 60, but for testing purposes can be less. Every "hour" (i.e., 60 "minutes") the contents of the per-minute buckets are summed into the next hourly bucket. Every 24 "hours" the contents of the per-hour buckets are summed into the next daily bucket. The counter package keeps all time-based histograms in sync, so the first secsPerMinute value seen by the package is used for all subsequent time-based histograms. ::counter::count tag ?delta? ?instance? Increment the counter identified by tag. The default increment is 1, although you can increment by any value, integer or real, by specifying delta. You must declare each counter with ::counter::init to define the characteristics of counter before you start to use it. If the counter type is -group, then the counter identified by instance is incremented. ::counter::start tag instance Record the starting time of an interval. The tag is the name of the counter defined as a -hist value-based histogram. The instance is used to distinguish this interval from any other intervals that might be overlapping this one. ::counter::stop tag instance Record the ending time of an interval. The delta time since the corresponding ::counter::start call for instance is recorded in the histogram identified by tag. ::counter::get tag args Return statistics about a counter identified by tag. The args determine what value to return: -total Return the total value of the counter. This is the default if args is not specified. -totalVar Return the name of the total variable. Useful for specifying with -textvariable in a Tk widget. -N Return the number of samples accumulated into the counter. -avg Return the average of samples accumulated into the counter. -avgn Return the average over the last N samples taken. The N value is set in the ::counter::init call. -hist bucket If bucket is specified, then the value in that bucket of the histogram is returned. Otherwise the complete histogram is returned in array get format sorted by bucket. -histVar Return the name of the histogram array variable. -histHour Return the complete hourly histogram in array get format sorted by bucket. -histHourVar Return the name of the hourly histogram array variable. -histDay Return the complete daily histogram in array get format sorted by bucket. -histDayVar Return the name of the daily histogram array variable. -resetDate Return the clock seconds value recorded when the counter was last reset. -all Return an array get of the array used to store the counter. This includes the total, the number of samples (N), and any type-specific information. This does not include the histogram array. ::counter::exists tag Returns 1 if the counter is defined. ::counter::names Returns a list of all counters defined. ::counter::histHtmlDisplay tag args Generate HTML to display a histogram for a counter. The args control the format of the display. They are: -title string Label to display above bar chart -unit unit Specify minutes, hours, or days for the time-base histograms. For value-based histograms, the unit is used in the title. -images url URL of /images directory. -gif filename Image for normal histogram bars. The filename is relative to the -images directory. -ongif filename Image for the active histogram bar. The filename is relative to the -images directory. -max N Maximum number of value-based buckets to display. -height N Pixel height of the highest bar. -width N Pixel width of each bar. -skip N Buckets to skip when labeling value-based histograms. -format string Format used to display labels of buckets. -text boolean If 1, a text version of the histogram is dumped, otherwise a graphical one is generated. counter 2.0 counter(n)
All times are GMT -4. The time now is 01:05 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy