Comparing lines of data


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Comparing lines of data
# 1  
Old 12-03-2012
Apple Comparing lines of data

Total UNIX Rookie, but I'm learning. I have columns of integer data separated by spaces, and I'm using a Mac terminal.

What I want to do:
1. Compare "line 1 column 2" (x) to "line 2 column 2" (y); is y-x>=100?
2. If yes, display difference and y's line number
3. If no, increment x and y by one and repeat

How I am going about it:
Code:
sed -n '1,2 p' datafile | cut -d' ' -f1 #gives me the two fields of interest

Where do I go from here?
# 2  
Old 12-03-2012
Is this a homework?
# 3  
Old 12-03-2012
This is for research. The data is power and its odd harmonics, and I am looking for a way to ID transients without having to graph everything. I just chose simple numbers so I can understand the mechanics. Thanks.
# 4  
Old 12-03-2012
OK,
please post a sample input file and an example of the expected output. Please use code tags: Video tutorial on how to use code tags in The UNIX and Linux Forums.
# 5  
Old 12-03-2012
With assumptions about your data and its structure:
Code:
awk '!(NR%2) && (d=$2-x)>=100{print d,NR}{x=$2}' file


Last edited by elixir_sinari; 12-03-2012 at 12:12 PM..
# 6  
Old 12-03-2012
data sample, I modified the second column for simplicity:

Code:
1.94183e+05  7.8e+04  1.09609e+04  -2.51962e+03  2.58938e+03  -1.78130e+04  8.17326e+03  9.09750e+03  
1.95223e+05  7.8e+04  1.08129e+04  -2.50996e+03  2.23601e+03  -1.76457e+04 8.53983e+03  8.87947e+03  
1.95826e+05  7.8e+04  1.08992e+04  -2.80696e+03  2.35323e+03  -1.77662e+04  8.43230e+03  8.83312e+03  
1.94827e+05  7.62589e+04  1.11074e+04  -2.79142e+03  2.60123e+03  -1.80548e+04  8.05580e+03  8.95096e+03  
1.92552e+05  7.8e+04  1.10656e+04  -2.03271e+03  3.18014e+03  -1.78507e+04  7.77928e+03  9.32247e+03  
1.91165e+05  7.9e+04  1.07475e+04  -1.78148e+03  3.61320e+03  -1.76183e+04  7.69183e+03  9.39564e+03  
1.91355e+05  7.9e+04  1.07990e+04  -2.17165e+03  3.17912e+03  -1.79394e+04  7.87802e+03  9.13235e+03  
1.90294e+05  7.8e+04  1.08781e+04  -2.42501e+03  2.58355e+03  -1.82862e+04  8.15985e+03  8.91582e+03  
1.88968e+05  7.8e+04  1.03374e+04  -2.25564e+03  2.31198e+03  -1.81390e+04  8.48529e+03  8.75009e+03  
1.89367e+05  7.8e+04  1.01039e+04  -1.94181e+03  2.55877e+03  -1.76577e+04  8.55628e+03  8.80059e+03

---------- Post updated at 11:21 AM ---------- Previous update was at 11:19 AM ----------

and I expect that the output will be:
line 5: 1741.1
line 6: 1000

---------- Post updated at 11:27 AM ---------- Previous update was at 11:21 AM ----------

elixir_sinari, would you mind explaining?
# 7  
Old 12-03-2012
Quote:
Originally Posted by markymarkg123
Total UNIX Rookie, but I'm learning. I have columns of integer data separated by spaces, and I'm using a Mac terminal.

What I want to do:
1. Compare "line 1 column 2" (x) to "line 2 column 2" (y); is y-x>=100?
2. If yes, display difference and y's line number
3. If no, increment x and y by one and repeat

How I am going about it:
Code:
sed -n '1,2 p' datafile | cut -d' ' -f1 #gives me the two fields of interest

Where do I go from here?
Your step 3 makes no sense to me. If y-x is not >= 100, then (y+1)-(x+1) can't be >= 100 either. This will just create an very large loop until one of the values overflows. So, until you can explain why this should be done, I'm going to ignore this requirement.

The proposal elixir_sinari makes the assumption that you are only comparing x values on odd numbered lines against y values on even numbered lines. Seeing your sample data and comments in message #6 in this thread, it looks like you want to compare the value in the 2nd column of your input file for every line except the 1st.

This slight modification to elixir_sinari's proposal:
Code:
awk '(NR>1) && (d=$2-x)>=100{print d,NR}{x=$2}' file

produces the output:
Code:
1741.1 5
1000 6
1741.1 14
1000 15
1741.1 23
1000 24

with your sample data. Is this what you wanted?
This User Gave Thanks to Don Cragun For This Post:
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Comparing multiple lines in same file

Hello, I would like to write a /bin/ksh script to manipulate a file and compare its contexts. Comparing lines 1 & 2, 3 & 4, 5 & 6, and so forth until the end of the file. This is what I would like the script to compare (using line 1 & 2 as an example): 1. Verify if the last column in line 1 is... (10 Replies)
Discussion started by: seekryts15
10 Replies

2. UNIX for Dummies Questions & Answers

Comparing lines within a word list

Hello all- New to this forum, and relatively new to using grep at the Terminal command line to work with regular expressions. I've got a background in math and some programming experience, so it's not been too difficult to learn the basics of searching through my word lists for particular types of... (13 Replies)
Discussion started by: dtalvacchio
13 Replies

3. Shell Programming and Scripting

Comparing to 3 data

# cat list.txt server1 server2 server3 server4 # data to be compared of. #dns address 1.1.1.1 2.2.2.2 3.3.3.3 #for i in `cat list.txt` do grep dns $ i done (1 Reply)
Discussion started by: invinzin21
1 Replies

4. Shell Programming and Scripting

Comparing the data in a 2 files

Hi Friends, I have a file 1 CREATE MULTISET TABLE TEYT_Q9_T.TEST ,NO FALLBACK , NO BEFORE JOURNAL, NO AFTER JOURNAL, CHECKSUM = DEFAULT, DEFAULT MERGEBLOCKRATIO ( XYZ DECIMAL(10,0), ABC VARCHAR(5) CHARACTER SET LATIN NOT CASESPECIFIC, PQR... (3 Replies)
Discussion started by: i150371485
3 Replies

5. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ... (3 Replies)
Discussion started by: Wynner
3 Replies

6. Shell Programming and Scripting

Comparing lines of two different files

Hello, Please help me with this problem if you have a solution. I have two files: <file1> : In each line, first word is an Id and then other words that belong to this Id piMN-1 abc pqr xyz py12 niLM y12 FY4 pqs fiRLym F12 kite red <file2> : same as file1, but can have extra lds... (3 Replies)
Discussion started by: mira
3 Replies

7. Shell Programming and Scripting

comparing lines in file

i have 2 files and i want to compare i currently cat the files and awk print $1, $2 and doing if file1=file2 then fail, else exit 0 what i want to do is compare values, with column 1 being a reference i want to compare line by line and then still be able to do if then statement to see if worked... (1 Reply)
Discussion started by: sigh2010
1 Replies

8. Shell Programming and Scripting

Comparing data inside file

Hi Everyone, I will try to explain my question please forgive my english here. I am looking for shell script or command that can compare data in the files. I have 50 files in one directory test1 test2 test3 ....so on. I want to compare data in each files with each other and output each... (4 Replies)
Discussion started by: email-lalit
4 Replies

9. Shell Programming and Scripting

comparing lines from 2 files

Hi Friends, I have 2 files A and B . I want to compare the 3rd line of file A and B . (I dont want to compare the 2 files, using diff or cmp). I just want to know whether 3rd line of A matches the 3 rd line of B. Can anybody share their knowledge on the same? Thanks , Vijaya (12 Replies)
Discussion started by: vijaya2006
12 Replies

10. UNIX for Dummies Questions & Answers

Comparing data list...

I have a list of files that I want to compare to another list of files, how do I do that? The first list will be my known list and hard coded, for example: mylist="janfile.tar jarfile.jar jan.rpt.Z" etc. The second list will be found by doing an 'ls' piped to a file: ls > filelist.dat ... (4 Replies)
Discussion started by: giannicello
4 Replies
Login or Register to Ask a Question