Uniq code in sorted order | Unix Linux Forums | Shell Programming and Scripting

  Go Back    


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

Uniq code in sorted order

Shell Programming and Scripting


Closed Thread    
 
Thread Tools Search this Thread Display Modes
    #1  
Old 11-19-2012
irfanmemon irfanmemon is offline
Registered User
 
Join Date: Nov 2012
Last Activity: 22 November 2012, 4:49 AM EST
Posts: 10
Thanks: 2
Thanked 0 Times in 0 Posts
Uniq code in sorted order

Hi All,
A small question:

I have a file check.txt which looks like below:


Code:
KRPROD2,2012-11-20 08:46:50:408,ODK3325688102
KRPROD2,2012-11-20 08:47:35:289,ODK3325688102
KRPROD2,2012-11-20 08:47:35:446,ODK3325688102
KRPROD2,2012-11-20 08:48:32:973,ODK3325689120
KRPROD2,2012-11-20 09:29:17:833,ODK3325689120
KRPROD2,2012-11-20 09:29:17:912,ODK3325689120

I want to get below output

Code:
KRPROD2,2012-11-20 08:46:50:408,ODK3325688102
KRPROD2,2012-11-20 08:48:32:973,ODK3325689120

which is first instance of field3 in the file. Please help

Last edited by Scrutinizer; 11-20-2012 at 01:19 AM.. Reason: code tags
Sponsored Links
    #2  
Old 11-19-2012
elixir_sinari's Avatar
elixir_sinari elixir_sinari is offline Forum Advisor  
Registered User
 
Join Date: Mar 2012
Last Activity: 9 October 2014, 4:50 PM EDT
Location: India
Posts: 1,412
Thanks: 101
Thanked 496 Times in 473 Posts

Code:
awk -F, '!c[$3]++' file

Sponsored Links
    #3  
Old 11-19-2012
irfanmemon irfanmemon is offline
Registered User
 
Join Date: Nov 2012
Last Activity: 22 November 2012, 4:49 AM EST
Posts: 10
Thanks: 2
Thanked 0 Times in 0 Posts
Thanks so much...it works.
Can you please help to explain as i m new bie in unix
    #4  
Old 11-19-2012
Don Cragun's Avatar
Don Cragun Don Cragun is online now Forum Staff  
Moderator
 
Join Date: Jul 2012
Last Activity: 26 November 2014, 12:20 PM EST
Location: San Jose, CA, USA
Posts: 5,098
Thanks: 196
Thanked 1,705 Times in 1,448 Posts

Code:
awk -F, '!c[$3]++' file

The -F, sets the field separator to a <comma>.
The c[$3] is an array indexed by the contents of field 3. An unintialized variable in awk has value 0 (or empty string) depending on context).
The first time it is evaluated !c[$3] (NOT)(value of c[$3]), is (NOT)zero which evaluates to TRUE. When the value is TRUE the action associated with this awk statement is performed. Since there is no action given, the default action (print the line) is performed.
Then the ++ increments the value of c[$3] so that when this test is performed again on lines with the same 3rd field, the test !c[$3] will evaluate to FALSE because (NOT)(positive integer value) evaluates to zero. When the test evaluates to FALSE, the action associated with this line is not performed so subsequent lines with the same value in the 3rd field are not printed.
This test is performed once for each line in the file in order from beginning to end.
Sponsored Links
    #5  
Old 11-19-2012
irfanmemon irfanmemon is offline
Registered User
 
Join Date: Nov 2012
Last Activity: 22 November 2012, 4:49 AM EST
Posts: 10
Thanks: 2
Thanked 0 Times in 0 Posts
Thanks for detailed explanation

But when I am using below line/command its giving me error;


Code:
ssh ${fmsServerUserName}@${fmsServerName} "awk -F, '!c[$3]++' 10_FMS_CRXtoFMS.csv" >> 10_FMS_CRXtoFMS.csv

Error:
awk: !c[]++
awk:    ^ syntax error
awk: fatal: invalid subscript expression

Please help

Last edited by Scrutinizer; 11-20-2012 at 01:19 AM.. Reason: code tags
Sponsored Links
    #6  
Old 11-20-2012
subramanian subramanian is offline
Registered User
 
Join Date: Oct 2012
Last Activity: 10 November 2013, 4:55 AM EST
Posts: 9
Thanks: 0
Thanked 2 Times in 2 Posts
Hi
escape ! and $ chars. it will work.


Code:
ssh ${fmsServerUserName}@${fmsServerName} "awk -F, '\!c[\$3]++' 10_FMS_CRXtoFMS.csv" >> 10_FMS_CRXtoFMS.csv


Last edited by Scrutinizer; 11-20-2012 at 01:21 AM.. Reason: code tags
Sponsored Links
    #7  
Old 11-20-2012
Don Cragun's Avatar
Don Cragun Don Cragun is online now Forum Staff  
Moderator
 
Join Date: Jul 2012
Last Activity: 26 November 2014, 12:20 PM EST
Location: San Jose, CA, USA
Posts: 5,098
Thanks: 196
Thanked 1,705 Times in 1,448 Posts
Quote:
Originally Posted by irfanmemon View Post
Thanks for detailed explanation

But when I am using below line/command its giving me error;

ssh ${fmsServerUserName}@${fmsServerName} "awk -F, '!c[$3]++' 10_FMS_CRXtoFMS.csv" >> 10_FMS_CRXtoFMS.csv

Error:
awk: !c[]++
awk: ^ syntax error
awk: fatal: invalid subscript expression

Please help
By putting the !c[$e]++ in a double quoted string (i.e., "awk -F, '!c[$3]++' 10_FMS_CRXtoFMS.csv" ), you had the shell expand $3 instead of awk. Given the error message, it would appear that the shell you were executing when you invoked this command either didn't have three positional parameters or the third positional parameter at that time was an empty string.

Then when you invoked awk, the script that it was given was '!c[]++' and you got the syntax error from awk.

I don't use ssh much, but adding the backslash escape as shown in red in the following should get rid of the syntax error for you:

Code:
ssh ${fmsServerUserName}@${fmsServerName} "awk -F, '!c[\$3]++' 10_FMS_CRXtoFMS.csv" >> 10_FMS_CRXtoFMS.csv

It isn't clear to me whether the two references to 10_FMS_CRXtoFMS.csv are two references to the same file or references to different files with the same name on different servers. If you are appending output to a file that you're using for input, bad things are very likely to happen.
Sponsored Links
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Read filenames in sorted order nishantrk Shell Programming and Scripting 10 02-01-2012 07:24 AM
sort the files based on timestamp and execute sorted files in order saidutta123 Shell Programming and Scripting 1 09-18-2011 03:47 PM
Sorted file ksailesh UNIX for Advanced & Expert Users 3 11-17-2009 10:59 PM
executing code on files in the sorted order -help! epi8 Shell Programming and Scripting 1 05-20-2008 04:30 AM
sorted processes pro Shell Programming and Scripting 7 05-19-2007 04:52 AM



All times are GMT -4. The time now is 01:21 PM.