awk 3 files to one based on multiple columns


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk 3 files to one based on multiple columns
# 1  
Old 12-16-2008
awk 3 files to one based on multiple columns

Hi all,

I have three files, one is a navigation file, one is a depth file and one is a file containing the measured field of gravity. The formats of the files are;

navigation file:

2006 320 17 39 0 0 *nav 21.31542 -157.887
2006 320 17 39 10 0 *nav 21.31542 -157.887
2006 320 17 39 20 0 *nav 21.31542 -157.887

depth file:

2006 321 19 17 16 681 dpth 4744.62 0
2006 321 19 17 31 419 dpth 4741.73 0
2006 321 19 17 46 973 dpth 4744.53 0
2006 321 19 18 2 26 dpth 4745.93 0

gravity file:

2006 320 17 39 30 0 cgrv 6992.7 205.6 0 978923
2006 320 17 40 0 0 cgrv 6992.7 205.6 -0.1 978923.1
2006 320 17 40 30 0 cgrv 6992.8 205.7 0 978923.1
2006 320 17 41 0 0 cgrv 6992.8 205.7 0 978923.2

The first five columns of each file are time stamps (year, day, hour, min, seconds). I want to find the latitude (column 8, file 1), longitude (column 9, file 1) and depth (column 9, file 2) for the matching times of file 3 (gravity file). So I need to find when the first 5 columns of file 1 are equal to the first five columns of file 3 and append column 8 and 9 of file 1 to a new version of file 3. I then need to find when the first 5 columns of file 2 are equal to the first five columns of file 3 and append column 9 of file 2 to the new file.

Did that make any sense?

Thanks!
# 2  
Old 12-16-2008
Code:
#!/bin/sh
FILE1="gravity.txt"
FILE2="navigation.txt"
FILE3="depth.txt"

while read line1
do
  key=`echo $line1 | cut -d ' ' -f 1-5`
  line2=`grep "$key" $FILE2 | head -1 | cut -d ' ' -f 8-9`
  line3=`grep "$key" $FILE3 | head -1 | cut -d ' ' -f 9`
  echo "$line1 $line2 $line3"
done < $FILE1

(Untested)

Last edited by Smiling Dragon; 12-16-2008 at 10:49 PM..
# 3  
Old 12-16-2008
The way I'd do it would be to read file 3 line by line using sed or awk to get get the columns, untested as I'm not near a Unix machine.

file3=`cat $filename | awk '{FS="(Whatever it is)"};{OFS="Whatever"};{print $1 - $5}`

for i in $file3
do
awk '{OFS=""};{OFS=""};{/$i/};{print $i $9}' file1 > opfile.
do for file 2
done

The syntax is worng but the general idea works.

Then sort the output file by date.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Paste columns based on common column: multiple files

Hi all, I've multiple files. In this case 5. Space separated columns. Each file has 12 columns. Each file has 300-400K lines. I want to get the output such that if a value in column 2 is present in all the files then get all the columns of that value and print it side by side. Desired output... (15 Replies)
Discussion started by: genome
15 Replies

2. Shell Programming and Scripting

Appending different columns of multiple files in awk

Hello All, I have three input files cat file1 col1|col2|col3 a|1|A b|2|B cat file2 col1|col2|col3 c|3|C cat file3 col1|col2|col3 d|4|D e|5|E i want below output file4 col1|col2 a|1 (6 Replies)
Discussion started by: looney
6 Replies

3. Shell Programming and Scripting

awk Parse And Create Multiple Files Based on Field Value

Hello: I am working parsing a large input file which will be broken down into multiples based on the second field in the file, in this case: STORE. The idea is to create each file with the corresponding store number, for example: Report_$STORENUM_$DATETIMESTAMP , and obtaining the... (7 Replies)
Discussion started by: ec012
7 Replies

4. Shell Programming and Scripting

awk arrays comparing multiple columns across two files.

Hi, I'm trying to use awk arrays to compare values across two files based on multiple columns. I've attempted to load file 2 into an array and compare with values in file 1, but success has been absent. If anyone has any suggestions (and I'm not even sure if my script so far is on the right lines)... (4 Replies)
Discussion started by: hubleo
4 Replies

5. Shell Programming and Scripting

Match files based on either of the two columns awk

Dear Shell experts, I have 2 files with structure: File 1: ID and count head test_GI_count1.txt 1000094 2 10039307 1 10039641 1 10047177 11 10047359 1 1008555 2 10120302 1 10120672 13 10121776 1 10121865 32 And 2nd file: head Protein_gi_GeneID_symbol.txt protein_gi GeneID... (11 Replies)
Discussion started by: smitra
11 Replies

6. Shell Programming and Scripting

awk script to split file into multiple files based on many columns

So I have a space delimited file that I'd like to split into multiple files based on multiple column values. This is what my data looks like 1bc9A02 1 10 1000 FTDLNLVQALRQFLWSFRLPGEAQKIDRMMEAFAQRYCQCNNGVFQSTDTCYVLSFAIIMLNTSLHNPNVKDKPTVERFIAMNRGINDGGDLPEELLRNLYESIKNEPFKIPELEHHHHHH 1ku1A02 1 10... (9 Replies)
Discussion started by: viored
9 Replies

7. Shell Programming and Scripting

Extracting columns from multiple files with awk

hi everyone! I'd like to extract a single column from 5 different files and put them together in an output file. I saw a similar question for 2 input files, and the line of code workd very well, the code is: awk 'NR==FNR{a=$2; next} {print a, $2}' file1 file2 I added the file3, file4 and... (10 Replies)
Discussion started by: orcaja
10 Replies

8. UNIX for Dummies Questions & Answers

Extracting columns from multiple files with awk

hi everyone! I already posted it in scripts, I'm sorry, it's doubled I'd like to extract a single column from 5 different files and put them together in an output file. I saw a similar question for 2 input files, and the line of code workd very well, the code is: awk 'NR==FNR{a=$2; next}... (1 Reply)
Discussion started by: orcaja
1 Replies

9. Shell Programming and Scripting

sort by based on multiple columns

Hi, Is there any way to sort a file in cshell by sort command, sorting it by multiple fields, like to sort it first by the second column and then by the first column. Thanks forhead (1 Reply)
Discussion started by: Takeeshe
1 Replies

10. Shell Programming and Scripting

Sorting based on Multiple columns

Hi, I have a requirement whereby I have to sort a flat file based on Multiple Columns (similar to ORDER BY Clause of Oracle). I am getting 10 columns in the flat file and I want the file to be sorted on 1st, 3rd, 4th, 7th and 9th columns in ascending order. The flat file is pipe seperated. Any... (15 Replies)
Discussion started by: dharmesht
15 Replies
Login or Register to Ask a Question