Finding difference in 1st field for rows of data | Unix Linux Forums | UNIX for Dummies Questions & Answers

  Go Back    


UNIX for Dummies Questions & Answers If you're not sure where to post a UNIX or Linux question, post it here. All UNIX and Linux newbies welcome !!

Finding difference in 1st field for rows of data

UNIX for Dummies Questions & Answers


Tags
awk

Closed Thread    
 
Thread Tools Search this Thread Display Modes
    #1  
Old 02-07-2013
co21ss co21ss is offline
Registered User
 
Join Date: Dec 2012
Last Activity: 3 October 2013, 3:23 PM EDT
Posts: 5
Thanks: 1
Thanked 0 Times in 0 Posts
Finding difference in 1st field for rows of data

I have a file that has multiple lines, of grouped data, that typically all have the same values in the 1st field, however, I would like to search the 1st field for any differences and set a flag to use in an "if" statement to run some other routine.

An example of the typical file is below, which is a comma separated file:


Code:
1st field     2nd field     3rd field
   "20"  ,       "file1"     ,   "xxxx"
   "20"  ,       "file1"     ,   "xxxx"
   "20"  ,       "file1"     ,   "xxxx"
   "20"  ,       "file1"     ,   "xxxx"
   "20"  ,       "file2"     ,   "yyyy"
   "20"  ,       "file2"     ,   "yyyy"
   "20"  ,       "file2"     ,   "yyyy"
   "20"  ,       "file2"     ,   "yyyy"
   "20"  ,       "file3"     ,   "zzzz"
   "20"  ,       "file3"     ,   "zzzz"
   "20"  ,       "file3"     ,   "zzzz"
   "20"  ,       "file3"     ,   "zzzz"

I want to set a flag for each data set of the 3rd field, if the file has differences in the 1st field like below:

Code:
1st field     2nd field     3rd field
   "20"  ,       "file1"     ,   "xxxx"
   "1"   ,       "file1"     ,   "xxxx"
   "20"  ,       "file1"     ,   "xxxx"
   "20"  ,       "file1"     ,   "xxxx"
   "1"  ,       "file2"     ,   "yyyy"
   "20"  ,       "file2"     ,   "yyyy"
   "20"  ,       "file2"     ,   "yyyy"
   "20"  ,       "file2"     ,   "yyyy"
   "20"  ,       "file3"     ,   "zzzz"
   "1"  ,       "file3"     ,   "zzzz"
   "20"  ,       "file3"     ,   "zzzz"
   "20"  ,       "file3"     ,   "zzzz"

One method I've tried so far is to strip out the lines where the 3rd field is all the same, and then try to run through each line and check the 1st field for differences.

I've tried the following, but can't get it to work.

Code:
 awk -F ',' '{f = $1} {for (i=1;i<=NR;i++) if ($1 != $f) {printf "%s\n", "FAIL"} }' tmpfile > fail.log


Last edited by co21ss; 02-07-2013 at 01:29 PM.. Reason: Add awk method used
Sponsored Links
    #2  
Old 02-07-2013
RudiC RudiC is offline Forum Advisor  
Registered User
 
Join Date: Jul 2012
Last Activity: 1 September 2014, 3:01 PM EDT
Location: Aachen, Germany
Posts: 4,159
Thanks: 69
Thanked 995 Times in 943 Posts
Not clear. The file seems to be comma AND multiple space delimited. AND, first four lines (and next four ... etc.) seem to contain identical values. It's easy to set a flag on a field changing, but to what avail? Where does $1 == "1" into play? I don't see a connection between your input sample and the desired output you post. Pls be more specific, post more detailed input / output samples.
And, btw, don't use $f in your awk script. That's a shell'ism; use f to reference the variable itself. In awk, $f would address the f'th field of the input line...
Sponsored Links
    #3  
Old 02-07-2013
co21ss co21ss is offline
Registered User
 
Join Date: Dec 2012
Last Activity: 3 October 2013, 3:23 PM EDT
Posts: 5
Thanks: 1
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by RudiC View Post
Not clear. The file seems to be comma AND multiple space delimited. AND, first four lines (and next four ... etc.) seem to contain identical values. It's easy to set a flag on a field changing, but to what avail? Where does $1 == "1" into play? I don't see a connection between your input sample and the desired output you post. Pls be more specific, post more detailed input / output samples.
And, btw, don't use $f in your awk script. That's a shell'ism; use f to reference the variable itself. In awk, $f would address the f'th field of the input line...
Sorry for not being clear. The code is not space separated, but just shown as such to clarify the input. Here is a better example:

Typical input -

Code:
"20","file1","xxxx"
"20","file1","xxxx"
"20","file1","xxxx"
"20","file1","xxxx"
"20","file2","yyyy"
"20","file2","yyyy"
"20","file2","yyyy"
"20","file2","yyyy"
"20","file3","zzzz"
"20","file3","zzzz"
"20","file3","zzzz"
"20","file3","zzzz"

Non-typical input -

Code:
"1","file1","xxxx"
"20","file1","xxxx"
"20","file1","xxxx"
"20","file1","xxxx"
"20","file2","yyyy"
"20","file2","yyyy"
"5","file2","yyyy"
"20","file2","yyyy"
"20","file3","zzzz"
"20","file3","zzzz"
"20","file3","zzzz"
"20","file3","zzzz"

In the non-typical input case, I would like to create an output, such as "FAIL", to use in a later routine if the 1st field is different from the rest for any of the individual files.

The 1st field will not always be known, but however is a value input from another file entirely. The goal here is run a script to automate having to visually look through the file for differences in the 1st field for any "file1" or "file2" lines of the file.

Last edited by co21ss; 02-07-2013 at 04:08 PM.. Reason: clarification
Sponsored Links
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Summing up rows data regarding 1st column AAWT Shell Programming and Scripting 4 04-10-2012 08:18 AM
Remove 1st two rows and last 2 rows kam786sim Shell Programming and Scripting 12 08-31-2010 08:28 AM
Sort alpha on 1st field, numerical on 2nd field (sci notation) FrancoisCN Shell Programming and Scripting 1 06-12-2009 10:45 AM
Finding different rows dombi Shell Programming and Scripting 2 03-06-2006 04:01 PM
Difference between two rows JimJim Shell Programming and Scripting 7 07-05-2005 07:52 AM



All times are GMT -4. The time now is 10:01 AM.