Combine and complete multiple CSV files based on 1 parameter


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Combine and complete multiple CSV files based on 1 parameter
# 8  
Old 08-20-2018
There you are:
Code:
awk -F, '                                                                                       # set field separator to comma
FNR == 1        {FLNR++                                                                         # on 1. line of each file: count files
                 FN[FLNR] = FILENAME                                                            # save respective file name
                 next                                                                           # don't process each file's first line further
                }
                {V[FLNR, $2] = $3                                                               # save Volume data in array indexed by file No. and Source Well No.
                 D[FLNR, $2] = $4                                                               # save Dest Well with same index
                 if ($2 > MX2) MX2 = $2                                                         # save max. Source Well No.
                }
END             {print "Source,Well_Source,Volume,Destination_Well,Destination"                 # print header
                 for (f=1; f<=FLNR; f++)                                                        # process files 1 - max. file No.
                   for (i=1; i<=MX2; i++)       {ft = f                                         # process 1 - MX2 wells in files; assign temp file No.
                                                 while ((ft <= FLNR) && (V[ft,i] == 0)) ft++    # test if Volume is not zero in actual and following files
                                                                                                # and we don't exceed max file No.
                                                 if (ft > FLNR) ft = f                          # if exceeded, use original f No.
                                                 print FN[ft], i, V[ft,i], D[f,i], "Deadpool"   # print actual (saved) file name, source well No., first non-
                                                                                                # zero Volume, respective dest well, and string constant
                                                 V[ft, i] = 0                                   # set consumed Volume to zero
                                                }
                }
'  OFS=, file[1-4]                                                                              #  set output field separator to comma, specify 1 - n file names


Last edited by RudiC; 08-21-2018 at 03:09 AM..
These 2 Users Gave Thanks to RudiC For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

CSV File with Multiple Search Parameter

Dear Team Members, I have a unique problem. Below is the dataset which I have. I am writing a script which will read through the file and pull the invoice no. (Field 2 of C1 row). "C1",990001,"L1","HERO","MOTORCYCLE","ASIA-PACIFIC","BEIJING" "C2","CLUTCH","HYUNDAI",03032017... (13 Replies)
Discussion started by: chetanojha
13 Replies

2. Shell Programming and Scripting

Combine multiple rows based on selected column keys

Hello I want to collapse a file with multiple rows into consolidated lines of entries based on selected columns as the 'key'. Example: 1 2 3 Abc def ghi 1 2 3 jkl mno p qrts 6 9 0 mno def Abc 7 8 4 Abc mno mno abc 7 8 9 mno mno abc 7 8 9 mno j k So if columns 1, 2 and 3 are... (6 Replies)
Discussion started by: linuxlearner123
6 Replies

3. Shell Programming and Scripting

Awk: Combine multiple lines based on number of fields

If a file has following kind of data, comma delimited 1,2,3,4 1 1 1,2,3,4 1,2 2 2,3,4 My required output must have only 4 columns with comma delimited 1,2,3,4 111,2,3,4 1,222,3,4 I have tried many awk command using ORS="" but couldnt progress (10 Replies)
Discussion started by: mdkm
10 Replies

4. Shell Programming and Scripting

Combine 3 files based on a pattern

HI, I have 3 files that contain the following information (sql output from Oracle database stored in a txt file): File1.txt : alter table "SYS"."INT_COST_PRICE" enable row movement; alter table "SYS"."INT_SOH" enable row movement; alter table "SYSMAN"."XX_ACI_SKURTP" enable row movement;... (6 Replies)
Discussion started by: rparavastu
6 Replies

5. Shell Programming and Scripting

Combine multiple lines in file based on specific field

Hi, I have an issue to combine multiple lines of a file. I have records as below. Fields are delimited by TAB. Each lines are ending with a new line char (\n) Input -------- ABC 123456 abcde 987 890456 7890 xyz ght gtuv ABC 5tyin 1234 789 ghty kuio ABC ghty jind 1234 678 ght ... (8 Replies)
Discussion started by: ratheesh2011
8 Replies

6. Shell Programming and Scripting

combine lines from two files based on an if statement

I'm rather new to programming, and am attempting to combine lines from 2 files in a way that is way beyond my expertise - any help would be appreciated! I need to take a file (file1) and add columns to it from another file (file2). However, a line from file2 should only be added to a given line... (3 Replies)
Discussion started by: Cheri
3 Replies

7. Shell Programming and Scripting

How to combine two files based on fields?

I have two files which are as follows: File 1: 1 abc 250 2 pqr 300 3 xyz 100 File 2: 1 abc 230 2 pqr 700 3 xyz 500 Now I need output File, File 3as: S.No Name Count1 Count2 1 abc 250 230 2 pqr 300 700 3 xyz 100 500 NOTE: (13 Replies)
Discussion started by: karumudi7
13 Replies

8. Shell Programming and Scripting

Combine two files and put it in .csv file

Hi Freinds I have two .txt file gem1.txt and gem2.txt, Sample gem1.txt abstract (1.0.0) actionmailer (2.3.5, 2.2.2) actionpack (2.3.5, 2.2.2) activerecord (2.3.5, 2.2.2) activerecord-oracle_enhanced-adapter (1.1.9) activerecord-sqlserver-adapter (2.3.4) activeresource (2.3.5, 2.2.2)... (3 Replies)
Discussion started by: ankit_view24
3 Replies

9. Shell Programming and Scripting

Combine Multiple text or csv files column-wise

Hi All I am trying to combine columns from multiple text files into a single file using paste command but the record length being unequal in the different files the data is running over to the closest empty cell on the left. Please see below. What can i do to resolve this ? File 1 File... (15 Replies)
Discussion started by: venky_ibm
15 Replies

10. Shell Programming and Scripting

how to combine 2 lines in same files based on any text

hi, I want to combine two lines in same file. If the line ends with '&' it should belongs to previous line only Here i am writing example. Ex1: line 1 : return abcdefgh& line 2 : ijklmnopqr& line 3 : stuvw& line 4 : xyz output should be line 1: return abcdefghijklmnopqrstuvwxyz ... (11 Replies)
Discussion started by: spc432
11 Replies
Login or Register to Ask a Question