Need help in column comparison & adding extra line to files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need help in column comparison & adding extra line to files
# 1  
Old 11-22-2012
Need help in column comparison & adding extra line to files

Hi,

I wanted to check whether the x,y,z coordinates of two files are equal or not. At times, when one file is converted to another suitable file extension , there are some chances that the data mismatch would happen during the conversion. In order to avoid the data misfit, i would like to compare the co-ordinates of the both files and check whether they are equal or not.

For example, following are the sample files :


File1 (here additional columns are intentional left, but in real file, there are around 10 columns)

Code:
x_coord   y_coord     z_coord     ID            

1.01     2.56       8.32         1  
1.06     2.50       7.36         2
1.08     2.69       4.25         1


File2 (here additional columns are intentional left, but in real file, there are around 10 columns)

Code:
ID      x_coord        y_coord     z_coord                        

1          1.01        2.56      8.32
2          1.06        2.50      7.36
1          1.08        2.69      4.25

Expected Output:

comp

Code:
0     0     0     0
0     0     0     0
0     0     0     0

Below, is the code i tried.


Code:
awk '{print $2,$3,$4,$5}}' file1 > xyz
awk '{print $2,$3,$4,$5}}' file2 > data
paste xyz data > inn
awk '{print $0,$1-$6,$2-$7,$3-$8,$4-$5}' inn > core
awk '{print $9,$10,$11,$12}' core > comp
rm xyz data inn core

The above code gives the desired result as displayed in the expected output. But, is there any other way of using the shell commands more efficiently. !!!!

Last edited by b@l@ji; 11-22-2012 at 08:55 PM.. Reason: need to amend the code
# 2  
Old 11-22-2012
Quote:
Originally Posted by b@l@ji
Hi,

I need help in following two tasks.

Tasks:1 Column comparison between two files.

There is a file_1 with columns $1 , $2 , $3 , $4 , $5
Another is file_2 with columns $1 , $2 , $3 , $4 , $5

1.) i wish to compare whether values in $2 in file_1 is equal to $3 in file_2 and $3 in file_1 is equal to $5 in file_2

2.) i wish to replace the column $1 in file_2 using the values in the column $5 from file_1

Tasks:2 How to add extra lines to an existing file ?

for example, i want to add following two lines to an existing file at bottom. Is it possible to do it, without having to write the below two lines into a separate file and use "cat" command ?

Numbers
5678
This sounds like a homework item; if it is it shouldn't be posted here.

If this isn't a homework item, what is supposed to happen if the values are equal and what is supposed to happen if they are not equal?

Yes, it is possible to add lines to the end of a file without using cat.

If you want help figuring out how to do something, you will be much more likely to get useful results if you provide sample input files and show us the output that should be produced (all using code tags).
# 3  
Old 11-22-2012
@ Don Cragun: As you can see that , I have amended the post with the existing code which I use.

Can you throw your expert opinion on how further the script can be handled efficiently ?
# 4  
Old 11-22-2012
b@l@ji,
Please post sample contents for file1 and file2 and show us the output you want when processing those sample files.
# 5  
Old 11-22-2012
@ Don Cragun :

As suggested, I have modified it with the sample contents of file1 file2 and the expected output comp.
# 6  
Old 11-22-2012
The following should work the same as your six line script reading the data from each input file once (instead of four times), call awk once (instead of four times), and not calling paste or rm at all:
Code:
awk -v f2=file2 'BEGIN {OFS = "\t"}
{       o1 = $2; o2 = $3; o3 = $4;  o4 = $5
        getline < f2
        print o1 - $3, o2 - $4, o3 - $5, o4 - $2
}' file1 > comp

As with your script, surprising things may happen if file1 and file2 don't contain the same number of lines.
------------------
PS for the 2nd part of your original posting (which has been lost in all of your edits), two portable ways to add:
Code:
Numbers
5678

to the end of a file without using cat include:
Code:
printf "Numbers\n5678\n" >> file

and
Code:
echo Numbers >> file;echo 5678 >> file

Note that printf is a built-in in ksh and echo is a built-in in most shells. Some versions of echo always recognize backslash escape sequences, some versions of echo never recognize backslash escape sequences, and some versions of echo violate POSIX standard and Single UNIX Specification requirements by accepting an option that determines whether or not backslash escape sequences are recognized. The uses of echo shown above should do what you want here with any version of echo.

Last edited by Don Cragun; 11-23-2012 at 02:57 AM..
# 7  
Old 11-23-2012
FIW, printf is not only a shell built-in ksh, but also in bash and dash, for example.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Adding an extra date column in UNIX file

Hi All, I have a file with only one column of data (without delimiter). For Ex: cat temp.txt 22055 21088 93840 30990 50990 50950 I want to insert an additional column with current date as value. So, i have used below command but didn't get the result as excepted. Could onyone over... (5 Replies)
Discussion started by: Suresh
5 Replies

2. Shell Programming and Scripting

awk - 2 files comparison without for loop - multi-line issue

Greetings Experts, I need to handle the views created over monthly retention tables for which every new table in YYYYMMDD format, there is equivalent view created and the older table which might be dropped, the view over it has to be re-created over a dummy table so that it doesn't fail.... (2 Replies)
Discussion started by: chill3chee
2 Replies

3. Shell Programming and Scripting

Adding Extra Column in txt file base on Condition

HI Guys, I have below input. Output Base on Below Condition. 1> if forth column is empty and next coming line have same name with \es then add that column name on all rows 2>rest of all are es:vsDataEUtranCellFDD Input:- CCL01736 CCL01736_7A_1 es:vsDataEUtranCellFDD ... (3 Replies)
Discussion started by: pareshkp
3 Replies

4. Shell Programming and Scripting

Merge two files line by line and column by column

Hi All, I have two files having oracle query result. I want to merge to files line by line and also with column File1 23577|SYNC TYPE 23578|Order Number|ConnectionState 23585|Service State|Service NameFile2 23577|AR Alarm Sync 23578|A5499|9 23585|7|test_nov7Result... (18 Replies)
Discussion started by: Harshal22
18 Replies

5. Shell Programming and Scripting

Faster Line by Line String/Date Comparison of 2 Files

Hello, I was wondering if anyone knows a faster way to search and compare strings and dates from 2 files? I'm currently using "for loop" but seems sluggish as i have to cycle through 10 directories with 10 files each containing thousands of lines. Given: -10 directories -10 files... (4 Replies)
Discussion started by: agentgrecko
4 Replies

6. Shell Programming and Scripting

adding a column of unique string to multiple files

Hi, I need to add a column of unique string to individual csv file. my individual csv file looks like this and each file has variable row length: 178.52,39.4,train,0.003355544375334351,39.15752753933254,0.4895933968953914... (7 Replies)
Discussion started by: ida1215
7 Replies

7. Solaris

Adding Extra Hard Disk

Hi Solaris users - I have an Ultra10 SPARC machine, with IIe processor. To prepare for the Solaris10 admin exam PartII I need to set up the metadb/mirroring in my machine, but do not know how to do this properly. I need this to practice the mirroring tasks. If anyone could help it would be... (3 Replies)
Discussion started by: patcom
3 Replies

8. Shell Programming and Scripting

Column comparison between two files: moved from another post

I have two files as follows: Pval.txt ID Pvalue 91500 0.004 91700 0.007 91800 0.12 91900 0.05 92000 0.99 Freq.txt ID Frequency 90000 56 91500 29 91600 78 91700 60 91800 77 91900 66 92000 70 92100 50 92200 60I need to compare 1st... (19 Replies)
Discussion started by: cs_novice
19 Replies

9. IP Networking

Adding an extra route to the ip routing table

In my college dorm, there is a file sharing network in the entire building. Problem is, there is only a manual for windows with the settings on how to connect... :mad: They say that you have to give the following command in cmd in windows: route add 172.16.71.0 mask 255.255.255.0... (2 Replies)
Discussion started by: Japie89
2 Replies

10. Shell Programming and Scripting

Help in adding positive & negative values in a column

Hi Gurus, In my file I have an amount field from position 74 to 87, which contains values starting with '+' as well as '-'. I want to add all positive values in a varible called "CREDIT" and all negative values in a variable "DEBIT". I know, we can use grep to identify values with positive and... (4 Replies)
Discussion started by: berlin_germany
4 Replies
Login or Register to Ask a Question