Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Finding difference in 1st field for rows of data Post 302767765 by co21ss on Thursday 7th of February 2013 11:52:11 AM
Old 02-07-2013
Finding difference in 1st field for rows of data

I have a file that has multiple lines, of grouped data, that typically all have the same values in the 1st field, however, I would like to search the 1st field for any differences and set a flag to use in an "if" statement to run some other routine.

An example of the typical file is below, which is a comma separated file:

Code:
1st field     2nd field     3rd field
   "20"  ,       "file1"     ,   "xxxx"
   "20"  ,       "file1"     ,   "xxxx"
   "20"  ,       "file1"     ,   "xxxx"
   "20"  ,       "file1"     ,   "xxxx"
   "20"  ,       "file2"     ,   "yyyy"
   "20"  ,       "file2"     ,   "yyyy"
   "20"  ,       "file2"     ,   "yyyy"
   "20"  ,       "file2"     ,   "yyyy"
   "20"  ,       "file3"     ,   "zzzz"
   "20"  ,       "file3"     ,   "zzzz"
   "20"  ,       "file3"     ,   "zzzz"
   "20"  ,       "file3"     ,   "zzzz"

I want to set a flag for each data set of the 3rd field, if the file has differences in the 1st field like below:
Code:
1st field     2nd field     3rd field
   "20"  ,       "file1"     ,   "xxxx"
   "1"   ,       "file1"     ,   "xxxx"
   "20"  ,       "file1"     ,   "xxxx"
   "20"  ,       "file1"     ,   "xxxx"
   "1"  ,       "file2"     ,   "yyyy"
   "20"  ,       "file2"     ,   "yyyy"
   "20"  ,       "file2"     ,   "yyyy"
   "20"  ,       "file2"     ,   "yyyy"
   "20"  ,       "file3"     ,   "zzzz"
   "1"  ,       "file3"     ,   "zzzz"
   "20"  ,       "file3"     ,   "zzzz"
   "20"  ,       "file3"     ,   "zzzz"

One method I've tried so far is to strip out the lines where the 3rd field is all the same, and then try to run through each line and check the 1st field for differences.

I've tried the following, but can't get it to work.
Code:
 awk -F ',' '{f = $1} {for (i=1;i<=NR;i++) if ($1 != $f) {printf "%s\n", "FAIL"} }' tmpfile > fail.log


Last edited by co21ss; 02-07-2013 at 02:29 PM.. Reason: Add awk method used
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Difference between two rows

Dears, I have a list as follows, 2 4 8 If I want to find the difference between two consecutive rows. Then I have to store the specific rows in two variables and then find the difference. Could someone tell how this can be done. Regards, (7 Replies)
Discussion started by: JimJim
7 Replies

2. Shell Programming and Scripting

Finding different rows

I have two files. They are prety much the same, except a few lines. I would like to get the differences between the two file, but only those differences and nothing more. "diff" and "sdiff" cannot seem to do it. "sdiff -s" gets close, but I still get "less than" or "greater than" signs in the... (2 Replies)
Discussion started by: dombi
2 Replies

3. Shell Programming and Scripting

Extract difference of two columns from different rows

Hello guys, Please help me to solve this problem. I have tried some awk commands but couldn't succeed. I have a tab delimited file where each record is separated by ------ and 4th column of each record is same. <INPUT FILE> ------ peon 53931587 53931821 ... (12 Replies)
Discussion started by: sam_2921
12 Replies

4. Shell Programming and Scripting

Difference between corresponding elements of successive rows

Hi, I have a file in the following format a1 b1 c1 d1 a2 b2 c2 d2 a3 b3 c3 d3 a4 b4 c4 d4 I need a script to find the difference between corresponding values of successive rows. So the output would have one less row than the input file and should look like: a2-a1 b2-b1 c2-c1 d2-d1... (4 Replies)
Discussion started by: sajal.bhatia
4 Replies

5. UNIX for Dummies Questions & Answers

Suggestion to convert data in rows to data in columns

Hello everyone! I have a huge dataset looking like this: nameX nameX 0 1 2 2 2 2 2 2 2 2 2 2 2 2 1 2 2 2 1 2 2 2 ............... nameY nameY 2 2 2 2 2 2 2 2 2 2 1 2 2 2 1 2 2 2 ..... nameB nameB 0 1 2 2 2 2 2 2 2 2 1 2 2 2 1 2 2 2 ..... (can be several thousands of codes) and I need... (8 Replies)
Discussion started by: kush
8 Replies

6. Shell Programming and Scripting

finding difference between two files

Hi, I have two files one with 12486 lines second one with 13116 As per the comparsion between two files the count have 630 difference I used diff command to find the difference between two files but it's not understandable could any one suggest any command to get 630 records in a new... (4 Replies)
Discussion started by: thelakbe
4 Replies

7. Shell Programming and Scripting

Summing up rows data regarding 1st column

Dear all, I have one file like LABEL A B C D E F G H I J K L M N G02100 64651.3 25630.7 8225.21 51238 267324 268005 234001 52410.9 18598.2 10611 10754.7 122535 267170 36631.4 G02100 12030.3 8260.15 8569.91 ... (4 Replies)
Discussion started by: AAWT
4 Replies

8. Shell Programming and Scripting

Need help in finding in date difference

Hi, My date is coming as STARTDATE=Sun Jul 15 00:34:23 2012 ENDDATE=Sun Jul 15 00:50:04 2012I want difference between these two dates,anyone's helps will be appriciated. Thanks Prasoon (3 Replies)
Discussion started by: prasson_ibm
3 Replies

9. Shell Programming and Scripting

Finding first difference between two files

Hi! I'd like to know if it is possible for a command to find the first difference between two large files, output that line from both file and stop, so no need to continue after that to save some computation time. I don't think looping through it will be efficient enough but that's the only... (6 Replies)
Discussion started by: Mojing
6 Replies

10. Shell Programming and Scripting

Finding difference in 2 different timestamps

Legends, I have a requirement to run the script exactly after one hour of completion of dependent script. Eg: Script B should run after one hour on the completion of Script A. I got the time stamps using following variables. these scripts runs in autosys > DATE=`date +%H:%M` >... (4 Replies)
Discussion started by: sdosanjh
4 Replies
diff3(1)						      General Commands Manual							  diff3(1)

NAME
diff3 - 3-way differential file comparison SYNOPSIS
file1 file2 file3 DESCRIPTION
compares three versions of a file, and prints disagreeing ranges of text flagged with these codes: all three files differ file1 is different file2 is different file3 is different The type of change required to convert a given range of a given file to some other is indicated in one of these ways: Text is to be appended after line number n1 in file f, where f = or Text is to be changed in the range line n1 through line n2. If n1 = n2, the range can be abbreviated to n1. The original contents of the range follows immediately after a indication. When the contents of two files are identical, the contents of the lower-numbered file is suppressed. Produces a script for the editor that can be used to incorporate into file1 all changes between file2 and file3 (see ed(1)); i.e., the changes that normally would be flagged and Produces a script to incorporate only changes flagged Produces a script to incorporate only changes flagged Produces a script that will incorporate all changes between file2 and file3, but treat overlapping changes (that is, changes that would be flagged with in normal listing) differently. The overlapping lines in both files will be inserted by the edit script bracketed by and lines. Produces a script that will incorporate only changes flagged , but treat these changes in the manner of option. The following command applies the resulting script to file1. EXTERNAL INFLUENCES
International Code Set Support Single- and multi-byte character code sets are supported. WARNINGS
Text lines that consist of a single period defeat Files longer than 64K bytes do not work. FILES
SEE ALSO
diff(1). diff3(1)
All times are GMT -4. The time now is 10:50 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy