Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Reconciling two CSV files using shell scripting Post 303041099 by RudiC on Thursday 14th of November 2019 04:35:39 PM
Old 11-14-2019
Code:
awk -F, '
FNR == 1        {FCNT++                                                                                         # inc file counter with every new file
                }
FNR > 1         {KEYS[$1]                                                                                       # keep $1 in an array; overwrite duplicates
                 for (i=2; i<=NF; i++) W[$1,FCNT,i] = $i                                                        # keep fields in array indexed by key, file No., field No.
                }
END             {printf "Key, diff columns"                                                                     # start printing header
                 for (i=1; i<NF; i++) printf ",f1_value%d,f2_value%d,value%d_diff", i, i, i                     # complete header line for all fields
                 printf RS
                 for (k in KEYS)        {for (i=NF; i>1; i--)   {N1 = W[k,1,i]                                  # for all keys, for all fields, get values
                                                                 N2 = W[k,2,i]                                  # for both files,
                                                                 D1 = N2 - N1                                   # and calc difference
                                                                 OUT = sprintf (",%s,%s,%s", N1, N2, D1) OUT    # collect all those in temp var OUT
                                                                 if (D1) COLS = ",Value" i-1 COLS               # if diff exist, collect fields in temp var COLS
                                                                }
                                         print k, "\"" substr (COLS,2) "\"", substr (OUT, 2)                    # print all those, cutting off leading comma
                                         OUT = COLS = ""                                                        # reset temp vars
                                        }
                }
' file[12]                                                                                                      # OFS and SUBSEP relict from development, not needed

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with shell scripting for accepting .csv files as CLA

I want to automate test script on shell scripting. There are 2 .csv files named account.csv and balance.csv.These files needs to passed as command line arguments and the following logic needs to applied further. Any account with a balance that was due before Oct 23, 2007 has an overdue... (2 Replies)
Discussion started by: coolguy123
2 Replies

2. Shell Programming and Scripting

How to insert a sequence number column inside a pipe delimited csv file using shell scripting?

Hi All, I need a shell script which could insert a sequence number column inside a dat file(pipe delimited). I have the dat file similar to the one as shown below.. |A|B|C||D|E |F|G|H||I|J |K|L|M||N|O |P|Q|R||S|T As shown above, the column 4 is currently blank and i need to insert sequence... (5 Replies)
Discussion started by: nithins007
5 Replies

3. Shell Programming and Scripting

Shell scripting:from text file to CSV

Hello friends, I have a file as follows: "empty line" content1 content2 content3 content1 content2 content3 content1 content2 content3 It starts with an empty line, how can i get a csv like this: (12 Replies)
Discussion started by: kraterions
12 Replies

4. Shell Programming and Scripting

How to create or convert to pdf files from csv files using shell script?

Hi, Can anyone help me how to convert a .csv file to a .pdf file using shell script Thanks (2 Replies)
Discussion started by: ssk250
2 Replies

5. Shell Programming and Scripting

How to calculate avg values of csv file using shell scripting .?

hi all i have a reporting work and i want it to be automated using shell scripting kindly let me know how can i make that possibe . eg data are :... (2 Replies)
Discussion started by: Avinash shaw
2 Replies

6. Shell Programming and Scripting

How to calculate average of csv using shell scripting?

Hi, I need to calculate the average of the following values using shell scripitng. Can anyone please suggest a solution? ... (10 Replies)
Discussion started by: karan pratap si
10 Replies

7. Shell Programming and Scripting

Shell script for field wise record count for different Files .csv files

Hi, Very good wishes to all! Please help to provide the shell script for generating the record counts in filed wise from the .csv file My question: Source file: Field1 Field2 Field3 abc 12f sLm 1234 hjd 12d Hyd 34 Chn My target file should generate the .csv file with the... (14 Replies)
Discussion started by: Kirands
14 Replies

8. Shell Programming and Scripting

Need a piece of shell scripting to remove column from a csv file

Hi, I need to remove first column from a csv file and i can do this by using below command. cut -f1 -d, --complement Mytest.csv I need to implement this in shell scripting, Whenever i am using the above command alone in command line it is working fine. I have 5 files in my directory and... (3 Replies)
Discussion started by: Samah
3 Replies

9. Shell Programming and Scripting

Read csv file, convert the data and make one text file in UNIX shell scripting

I have input data looks like this which is a part of a csv file 7,1265,76548,"0102:04" 8,1266,76545,"0112:04" I need to make the output data should look like this and the output data will be part of text file: 7|1265000 |7654899 |A| 8|12660000 |76545999 |B| The logic behind the... (6 Replies)
Discussion started by: RJG
6 Replies

10. UNIX for Beginners Questions & Answers

Export Oracle multiple tables to multiple csv files using UNIX shell scripting

Hello All, just wanted to export multiple tables from oracle sql using unix shell script to csv file and the below code is exporting only the first table. Can you please suggest why? or any better idea? export FILE="/abc/autom/file/geo_JOB.csv" Export= `sqlplus -s dev01/password@dEV3... (16 Replies)
Discussion started by: Hope
16 Replies
XML::LibXML::Devel(3)					User Contributed Perl Documentation				     XML::LibXML::Devel(3)

NAME
XML::LibXML::Devel - makes functions from LibXML.xs available SYNOPSIS
/********************************************** * C functions you want to access */ xmlNode *return_node(); void receive_node(xmlNode *); ############################################### # XS Code void * xs_return_node CODE: RETVAL = return_node(); OUTPUT: RETVAL void xs_receive_node void *n CODE: receive_node(n); ############################################### # Perl code use XML::LibXML::Devel; sub return_node { my $raw_node = xs_return_node(); my $node = XML::LibXML::Devel::node_to_perl($raw_node); XML::LibXML::Devel::refcnt_inc($raw_node); return $node; } sub receive_node { my ($node) = @_; my $raw_node = XML::LibXML::Devel::node_from_perl($node); xs_receive_node($raw_node); XML::LibXML::Devel::refcnt_inc($raw_node); } DESCRIPTION
"XML::LibXML::Devel" makes functions from LibXML.xs available that are needed to wrap libxml2 nodes in and out of XML::LibXML::Nodes. This gives cleaner dependencies than using LibXML.so directly. To XS a library that uses libxml2 nodes the first step is to do this so that xmlNodePtr is passed as void *. These raw nodes are then turned into libxml nodes by using this "Devel" functions. Be aware that this module is currently rather experimental. The function names may change if I XS more functions and introduce a reasonable naming convention. Be also aware that this module is a great tool to cause segfaults and introduce memory leaks. It does however provide a partial cure by making "xmlMemUsed" available as "mem_used". FUNCTIONS
NODE MANAGEMENT node_to_perl node_to_perl($raw_node); Returns a LibXML::Node object. This has a proxy node with a reference counter and an owner attached. The raw node will be deleted as soon as the reference counter reaches zero. If the C library is keeping a pointer to the raw node, you need to call refcnt_inc immediately. You also need to replace xmlFreeNode by a call to refcnt_dec. node_to_perl node_from_perl($node); Returns a raw node. This is a void * pointer and you can do nothing but passing it to functions that treat it as an xmlNodePtr. The raw node will be freed as soon as its reference counter reaches zero. If the C library is keeping a pointer to the raw node, you need to call refcnt_inc immediately. You also need to replace xmlFreeNode by a call to refcnt_dec. refcnt_inc refcnt_inc($raw_node); Increments the raw nodes reference counter. The raw node must already be known to perl to have a reference counter. refcnt_dec refcnt_dec($raw_node); Decrements the raw nodes reference counter and returns the value it had before. if the counter becomes zero or less, this method will free the proxy node holding the reference counter. If the node is part of a subtree, refcnt_dec will fix the reference counts and delete the subtree if it is not required any more. refcnt refcnt($raw_node); Returns the value of the reference counter. fix_owner fix_owner($raw_node, $raw_parent); This functions fixes the reference counts for an entire subtree. it is very important to fix an entire subtree after node operations where the documents or the owner node may get changed. this method is aware about nodes that already belong to a certain owner node. MEMORY DEBUGGING $ENV{DEBUG_MEMORY} BEGIN {$ENV{DEBUG_MEMORY} = 1;}; use XML::LibXML; This turns on libxml2 memory debugging. It must be set before XML::LibXML is loaded. mem_used mem_used(); Returns the number of bytes currently allocated. EXPORT None by default. SEE ALSO
This was created to support the needs of Apache2::ModXml2. So this can serve as an example. AUTHOR
Joachim Zobel <jz-2011@heute-morgen.de> COPYRIGHT AND LICENSE
Copyright (C) 2011 by Joachim Zobel This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.10.1 or, at your option, any later version of Perl 5 you may have available. perl v5.18.2 2014-02-01 XML::LibXML::Devel(3)
All times are GMT -4. The time now is 04:53 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy