Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk --> math-operation in data-record and joining with second file data

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk --> math-operation in data-record and joining with second file data
# 1  
Old 01-08-2014
awk --> math-operation in data-record and joining with second file data
Hi!

I have a pretty complex job - at least for me!
i have two csv-files with meassurement-data:
fileA
Code:
...
01/07/14,01:10:00,20960,1993,2122,2052,1960,1778,1568,1376,1367,1282,1161,1186,1013,921,708,645,536,441,354,336,323,348,375,203,117,109,84,49,58,34,33,26,16,10,7,5,4,1,4,1,0,0,0,1,0,1,0,0,1,0,0,0,10527,22,0,119,974.6,5,3.98,0,0,0,0,0,75,58.8,
3.299,2.773,29.5,26.6,199,0000 0000 0000 0000,0.842718,1.04962,0.93209,0.582942,1.57034,4.5951(#/cm263)
01/07/14,01:20:00,21235,1985,2100,2215,2114,1943,1838,1653,1554,1463,1444,1448,1305,1069,969,820,677,587,477,417,435,470,466,280,138,118,116,58,43,40,19,26,13,13,10,7,4,1,1,2,2,1,1,0,0,1,0,0,0,0,0,0,10742,8,0,132,974.7,4.99,3.97,0,0,0,0,0,75,
58.8,3.299,2.772,27,26.1,199,0000 0000 0000 0000,0.879983,1.08077,0.962225,0.626434,1.57077,4.99936(#/cm263)
01/07/14,01:30:00,24639,2239,2474,2576,2527,2426,2195,1947,1790,1853,1737,1680,1645,1367,1248,1075,868,708,614,564,540,517,586,412,185,149,142,77,68,39,37,28,20,12,9,8,5,9,2,0,2,2,2,1,0,0,0,0,1,2,0,0,12659,30,0,164,974.8,4.99,3.97,0,0,0,0,0,7
5,58.5,3.301,2.771,26.5,25.1,199,0000 0000 0000 0000,0.89578,1.09963,0.976291,0.626434,1.57876,5.95218(#/cm263)
01/07/14,01:40:00,25306,2285,2509,2715,2685,2614,2492,2215,2110,1967,1974,1986,1894,1691,1525,1245,994,868,723,648,608,677,725,500,193,151,164,98,52,56,38,37,22,10,8,8,7,1,3,4,2,1,3,2,0,0,0,1,0,0,0,0,12985,29,0,174,974.7,4.99,3.99,0,0,0,0,0,7
5,58.4,3.284,2.771,29,25.6,199,0000 0000 0000 0000,0.919853,1.11922,0.995875,0.626434,1.57731,6.4352(#/cm263)
01/07/14,01:50:00,22003,2115,2336,2370,2485,2356,2304,2041,2021,1967,1939,1953,1789,1610,1439,1189,1005,803,690,604,528,609,660,437,200,172,141,108,74,34,41,30,20,16,15,7,6,2,0,2,0,0,1,0,0,1,0,0,0,0,0,0,11683,14,0,158,976.1,4.97,3.99,0,0,0,0,
0,75,59,3.3,2.771,29,27,199,0000 0000 0000 0000,0.932305,1.12304,1.00107,0.67317,1.57366,5.86103(#/cm263)
01/07/14,02:00:00,23419,2252,2449,2529,2629,2618,2404,2225,2272,2208,2157,2095,1977,1810,1567,1305,1110,915,794,658,625,706,772,494,236,181,150,100,67,44,36,35,16,13,9,3,4,3,1,1,0,0,0,2,1,0,0,0,0,0,0,0,12339,25,0,172,975.4,4.99,3.96,0,0,0,0,0
,75,58.8,3.299,2.773,27,26.1,199,0000 0000 0000 0000,0.942454,1.12894,1.0082,0.67317,1.57104,6.34192(#/cm263)
01/07/14,02:10:00,26960,2511,2689,2820,2900,2753,2573,2419,2381,2316,2268,2219,2174,1934,1621,1419,1227,976,841,774,627,780,830,591,229,179,198,106,59,35,37,33,24,16,17,5,3,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,13432,46,0,191,975.6,5.01,4.02,0,0,0,0,
0,75,58.5,3.298,2.773,26.1,25.1,199,0000 0000 0000 0000,0.938614,1.12763,1.00614,0.67317,1.5745,7.0131(#/cm263)
....

fileB
Code:
...
07/01/2014,01:10,233.40,10.70,2,2,3,1,271020.0,223
07/01/2014,01:20,231.20,9.20,2,2,3,1,276540.0,223
07/01/2014,01:30,231.90,10.30,2,2,3,1,282720.0,223
07/01/2014,01:40,231.70,11.10,2,2,3,1,289380.0,223
07/01/2014,01:50,233.30,11.80,2,2,3,1,296460.0,223
07/01/2014,02:00,233.20,10.50,2,2,3,1,302760.0,223
07/01/2014,02:10,235.90,11.40,2,2,3,1,309600.0,223
...

fileA: in each record the sum of $4+$5+$6+$7 and the sum of $11+$12+$13+$14+$14+$16 should be calculated and be pasted under the csv-conditions at the end of the record.
after that step, the data-records of fileA and fileB should be joined after a check of date and time. priority has the date of fileA.
I try to do this job by a EXCEL-Script, but EXCEL is in this case not working reliable.
So i want to solve it in awk - i do some small jobs with awk, but herefor i have no idea how to start and transcript the challange - can someone help me?

Thanks in advance,
IMPe
Last edited by IMPe; 01-08-2014 at 06:51 AM..
# 2  
Old 01-08-2014
Have you started with anything yet??
# 3  
Old 01-08-2014
Quote:
Originally Posted by PikK45
Have you started with anything yet??
hi PikK45

no, i have not, because i have no idea, how i start in the right way. Should i put the data in a array... how do i calculate the data sum=$4+$5+$6+$6 and past it at the end - record by record.
and finaly, how to compare the date and time-marks of fileA and fileB and let the script decide, to join the record of fileB.
Can you please give me a hint,
thanks in advance
IMPe
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Copy last few lines of a file, perform math operation and iterate further

Hi, I am trying to generate a data of following order: 4 0 1 642 643 4 642 643 1283 1284 4 1283 1284 1924 1925 4 1924 1925 2565 2566 4 2565 2566 3206 3207 4 3206 3207 3847 3848 4 3847 3848 4488 4489 4 4488 4489 5129 5130 ---------------------- 4 1 2 643 644 4 643 644 1284... (6 Replies)
Discussion started by: SaPa
6 Replies

2. Shell Programming and Scripting

awk script to find data in three file and perform replace operation

Have three files. Any other approach with regards to file concatenation or splitting, etc is appreciated If column55(billngtype) of file1 contains YMNC or YPBC then pick the value of column13(documentnumber). Now find this documentnumber in column1(Billdoc) of file2 and grep the corresponding... (4 Replies)
Discussion started by: as7951
4 Replies

3. Shell Programming and Scripting

Modifying text file records, find data in one place in the record and print it elsewhere

Hello, I have some text data that is in the form of multi-line records. Each record ends with the string $$$$ and the next record starts on the next line. RDKit 2D 15 14 0 0 0 0 0 0 0 0999 V2000 5.4596 2.1267 0.0000 C 0 0 0 0 0 0 0 0 0 0 0 ... (5 Replies)
Discussion started by: LMHmedchem
5 Replies

4. Shell Programming and Scripting

awk --> math-operation in a array

Hi main object is categorize the difference of data-values (TLUFT02B - TLUFT12B). herefor i read out data-files which are named acording to the timeformat yyyymmddhhmm. WR030B 266.48 Grad 0 WR050B 271.46 Grad 0 WR120B 268.11 Grad 0 WV030B 2.51 m/s ... (6 Replies)
Discussion started by: IMPe
6 Replies

5. Shell Programming and Scripting

Help with parsing data with awk , eliminating unwanted data

Experts , Below is the data: --- Physical volumes --- PV Name /dev/dsk/c1t2d0 VG Name /dev/vg00 PV Status available Allocatable yes VGDA 2 Cur LV 8 PE Size (Mbytes) 8 Total PE 4350 Free PE 2036 Allocated PE 2314 Stale PE 0 IO Timeout (Seconds) default --- Physical volumes ---... (5 Replies)
Discussion started by: rveri
5 Replies

6. Shell Programming and Scripting

Grab data between 2 keywords any do an array operation and write the file intact

Hi Unix Gurus, I need to grep for a block that is between a start and end keyword and then in between I need to find and replace a keyword. for eg: I need to search between Test = 000; and Test = 000; and find K9 and replace with M9 INPUT FILE Define { Replace = K9; Test =... (6 Replies)
Discussion started by: naveen@
6 Replies

7. Shell Programming and Scripting

Replace field in one file with whole record data of another

Hello Group, I need to replace the city field in “File 1 (fileld 3), with the entire record line of “File 2” (including delimiters) where the “city” field (File 1, Field 3)matches city field (File 2, Field1). All of the other data in “File 1” should remain intact(Fields 1,2,4,5,6). Only field... (1 Reply)
Discussion started by: vestport
1 Replies

8. Shell Programming and Scripting

awk math operation on two files

Hi, I need your help. I've got two files and i need to add 2nd line after occurrence of "Group No X" from data2.txt to 3rd line (after occurrence of "Group No X") from data1.txt. There is the same number of "Groups" in both files and the numbers of groups have the same pattern. data1.txt Group... (2 Replies)
Discussion started by: killerbee
2 Replies

9. Shell Programming and Scripting

Extract data based on match against one column data from a long list data

My input file: data_5 Ali 422 2.00E-45 102/253 140/253 24 data_3 Abu 202 60.00E-45 12/23 140/23 28 data_1 Ahmad 256 7.00E-45 120/235 140/235 22 data_4 Aman 365 8.00E-45 15/65 140/65 20 data_10 Jones 869 9.00E-45 65/253 140/253 18... (12 Replies)
Discussion started by: patrick87
12 Replies

10. Shell Programming and Scripting

how to delete record in file data with index in another file?

I want to deal with several data, i.e., data.*.txt with following structure MSG|20010102|123 125 4562 409|SEND MSG|20010102|120 230|SEND MSG|20010102|120 204 5071|SEND MSG|20010103|2 11 1098 9810|SEND ...... index file index.txt is 11 201 298 100 ...... What I want to do is: 1)... (0 Replies)
Discussion started by: zhynxn
0 Replies
Login or Register to Ask a Question