Finding difference between two columns of unequal length


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Finding difference between two columns of unequal length
# 1  
Old 11-28-2014
Finding difference between two columns of unequal length

Hi,

I have two files which look like this

Code:
cat waitstate.txt 
18.2
82.1

Code:
cat gostate.txt 
5.6
5.8
6.1
6.3
6.6
6.9
7.2
7.5
7.7
9.7
22.4
27.1
30.4
33.4
37.5
46.6
50.7
89.4

I want to find difference between these two files. The values that I am interested in are as follows. If my subtraction formulation is (A-B), B is a value in the waitstate file.
The value of A will be the one which is the smallest number that is larger than B in the gostate file.

In this case, the solutions should be 22.4-18.2 and 89.4-82.1.

I am not sure how to generate the subset of file 2 based on file 1.

Much appreciated!
# 2  
Old 11-28-2014
What be the desired output? Try
Code:
awk 'FNR==NR {T[NR]=$1; CNT=1; next} {D=$1-T[CNT];if (D>0) {print D, $1, T[CNT]; CNT++}}' file1 file2
4.2 22.4 18.2
7.3 89.4 82.1

---------- Post updated at 11:20 ---------- Previous update was at 11:17 ----------

In case there's more values in file2, try
Code:
awk     'FNR==NR        {T[NR]=$1; CNT=1; MAX=NR; next}
                        {D=$1-T[CNT]
                         if (D>0) {print D, $1, T[CNT]; CNT++}}
         CNT > MAX      {exit}
        ' file1 file2

This User Gave Thanks to RudiC For This Post:
# 3  
Old 11-28-2014
Works like a treat @RudiC. Thanks!
# 4  
Old 11-28-2014
Alternatively:
Code:
awk '$1+0>p+0{if(NR>1)print $1-p,$1 "-" p; if(!((getline p<f)>0)) exit}'  f=file1 file2


Last edited by Scrutinizer; 11-28-2014 at 06:35 AM..
These 3 Users Gave Thanks to Scrutinizer For This Post:
# 5  
Old 11-28-2014
@ Scrutinizer : elegant solution Smilie

Not one liner solution, I just wanted to share this function, you can try this

Code:
awk '

# Array, nearest neighbour using simple min-max method
function _get_keyMnMx(Arr,key,arg,   low,upp,i)
{	
	for(i in Arr)
	{ 
                # Make it Numeric
                i+=0

		if(i < key)
		{ 
			low =  (low !="" && low > i) ? low : i
		}
		if(i > key)
		{   
			upp =  (upp !="" && upp < i) ? upp : i
		}  
	}		    
	low  =  (low == "") ? "NaN" : low 
	upp  =  (upp == "") ? "NaN" : upp 
			 
	return ( tolower(arg)=="up" ? upp : tolower(arg) == "dw" ? low : low " " upp )	 
}

# Reading your gostate.txt
FNR==NR{
	Array[$1]
	next
}

#  Reading waitstate.txt
{
	# larger than column1 value from gostate.txt file
	largerthanthis = _get_keyMnMx(Array,$1,"up")


	print $1, largerthanthis, largerthanthis - $1
}
   ' gostate.txt waitstate.txt


Resulting


Code:
18.2 22.4 4.2
82.1 89.4 7.3


Last edited by Akshay Hegde; 11-28-2014 at 11:56 AM..
This User Gave Thanks to Akshay Hegde For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Finding difference in 2 different timestamps

Legends, I have a requirement to run the script exactly after one hour of completion of dependent script. Eg: Script B should run after one hour on the completion of Script A. I got the time stamps using following variables. these scripts runs in autosys > DATE=`date +%H:%M` >... (4 Replies)
Discussion started by: sdosanjh
4 Replies

2. Shell Programming and Scripting

Finding the length of the longest column

Hi, I am trying to figure out how to get the length of the longest column in the entire file (because the length varies from one row to the other) I was doing this at first to check how many fields I have for the first row: awk '{print NF; exit}' file Now, I can do this: awk '{ if... (4 Replies)
Discussion started by: MIA651
4 Replies

3. Shell Programming and Scripting

Need help in finding in date difference

Hi, My date is coming as STARTDATE=Sun Jul 15 00:34:23 2012 ENDDATE=Sun Jul 15 00:50:04 2012I want difference between these two dates,anyone's helps will be appriciated. Thanks Prasoon (3 Replies)
Discussion started by: prasson_ibm
3 Replies

4. Shell Programming and Scripting

Finding the records with a specified length

I have a sample txt file which has different variable lengths of 2,10,3,15. What is the command that I need use in order to get the record count that has length '3' Thanks (3 Replies)
Discussion started by: bobby1015
3 Replies

5. Shell Programming and Scripting

finding difference between two files

Hi, I have two files one with 12486 lines second one with 13116 As per the comparsion between two files the count have 630 difference I used diff command to find the difference between two files but it's not understandable could any one suggest any command to get 630 records in a new... (4 Replies)
Discussion started by: thelakbe
4 Replies

6. Shell Programming and Scripting

Finding the length of a string in a field

I have a field MPN-BANK-NUMBERO:006,00:N in a file FILE1 How can i display the length of the Bank Number.Any idea ? The output shud take the following format FILE 1 6 Where 1 is the starting position and 6 is the Ending position. (9 Replies)
Discussion started by: bobby1015
9 Replies

7. Shell Programming and Scripting

Finding multiple column values and match in a fixed length file

Hi, I have a fixed length file where I need to verify the values of 3 different fields, where each field will have a different value. How can I do that in a single step. (6 Replies)
Discussion started by: naveen_sangam
6 Replies

8. Shell Programming and Scripting

Finding the time difference

Hi, I have two files A.txt and B.txt. And i have the following attributes in both the files. <date and time> <a unique id> For eg: <2007 May 30 20:29:36:034 GMT> <ID1> in A.txt <2007 May 30 20:42:36:038 GMT> <ID1> in B.txt Now, i need to find the time difference... (0 Replies)
Discussion started by: padma.raajesh
0 Replies

9. Shell Programming and Scripting

Help with finding length of a field

I have a pipe delimited file. I need to check that the first and second fields are 5 characters long and if not i need to append 0 in front of them to make them 5 characters long. can some body let mwe know how i can find the length of the two fields and then make them 5 characters long if they... (6 Replies)
Discussion started by: dsravan
6 Replies

10. Shell Programming and Scripting

Finding out the length of a string held within a variable

:confused: Does anyone know which command I can use to find out the length of a string held within a variable? (5 Replies)
Discussion started by: dbrundrett
5 Replies
Login or Register to Ask a Question