Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Text Proccessing with sort,uniq,awk
Top Forums Shell Programming and Scripting Text Proccessing with sort,uniq,awk Post 302482486 by anurag.singh on Tuesday 21st of December 2010 04:02:16 PM
Old 12-21-2010
1. !a[$3] == >> $3 is date. This is to make sure that we are processing only 1st line for a given date (a[$3] will be set for other reords for same data)
2. a[$3]++ == >> This is just to set array with index $3.
3. v="before 09:10" == >> v is a variable, This is used in print statement (last argument to print)
4. substr == >> substr(string, start_index, lenght_of_substring)

Hope this helps !!
If you don't want before lines, just modify if condition where time value is being checked.
Code:
sort -t ',' -k 3,3 -k 4,4 file.log | awk -F, '{if($1 != "X" && !a[$3]) {a[$3]++;if($4 > "09:10:00") print "ID number " $2 " came in on",substr($3,6,5)," at",substr($4,0,5),"after 09:10";}}'

command I gave earlier, (In post #7) doing following:
1. Process only 1st line for a given date (after sorting the file) and ignore all other lines for same date
2. Compate time value with "09:10:00" and display before/after message accordingly

If you still stuck, you may post a proper input and proper output.
Last edited by anurag.singh; 12-22-2010 at 04:25 AM.. Reason: typo
This User Gave Thanks to anurag.singh For This Post:
rollyah
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

sort/uniq

I have a file: Fred Fred Fred Jim Fred Jim Jim If sort is executed on the listed file, shouldn't the output be?: Fred Fred Fred Fred Jim Jim Jim (3 Replies)
Discussion started by: jimmyflip
3 Replies

2. UNIX for Dummies Questions & Answers

Help with Last,uniq, sort and cut

Using the last, uniq, sort and cut commands, determine how many times the different users have logged in. I know how to use the last command and cut command... i came up with last | cut -f1 -d" " | uniq i dont know if this is right, can someone please help me... thanks (1 Reply)
Discussion started by: jay1228
1 Replies

3. Shell Programming and Scripting

sort and uniq in perl

Does anyone have a quick and dirty way of performing a sort and uniq in perl? How an array with data like: this is bkupArr BOLADVICE_VN this is bkupArr MLT6800PROD2A this is bkupArr MLT6800PROD2A this is bkupArr BOLADVICE_VN_7YR this is bkupArr MLT6800PROD2A I want to sort it... (4 Replies)
Discussion started by: reggiej
4 Replies

4. Shell Programming and Scripting

Sort, Uniq, Duplicates

Input File is : ------------- 25060008,0040,03, 25136437,0030,03, 25069457,0040,02, 80303438,0014,03,1st 80321837,0009,03,1st 80321977,0009,03,1st 80341345,0007,03,1st 84176527,0047,03,1st 84176527,0047,03, 20000735,0018,03,1st 25060008,0040,03, I am using the following in the script... (5 Replies)
Discussion started by: Amruta Pitkar
5 Replies

5. Shell Programming and Scripting

Help with Uniq and sort

The key is first field i want only uniq record for the first field in file. I want the output as or output as Appreciate help on this (4 Replies)
Discussion started by: pinnacle
4 Replies

6. Shell Programming and Scripting

Sort and uniq after comparision

Hi All, I have a text file with the format shown below. Some of the records are duplicated with the only exception being date (Field 15). I want to compare all duplicate records using subscriber number (field 7) and keep only those records with greater date. ... (1 Reply)
Discussion started by: nua7
1 Replies

7. Shell Programming and Scripting

Sort uniq or awk

Hi again, I have files with the following contents datetime,ip1,port1,ip2,port2,number How would I find out how many times ip1 field shows up a particular file? Then how would I find out how many time ip1 and port 2 shows up? Please mind the file may contain 100k lines. (8 Replies)
Discussion started by: LDHB2012
8 Replies

8. Shell Programming and Scripting

File comparison and proccessing using awk

Hi Guys, I am having two requirement in one of my scripts. please help out to find a fast solution using AWK (since there is lot of data to be processed) 1) First snippet - File1 has two columns and file2 has three columns If any value of column 1 of file1 matches with column 1... (4 Replies)
Discussion started by: stormfield
4 Replies

9. Shell Programming and Scripting

Uniq or sort -u or similar only between { }

Hi ! I am trying to remove doubbled entrys in a textfile only between delimiters. Like that example but i dont know how to do that with sort or similar. input: { aaa aaa } { aaa aaa } output: { aaa } { (8 Replies)
Discussion started by: fugitivus
8 Replies

10. UNIX for Dummies Questions & Answers

Uniq and sort -u

Hello all, Need to pick your brains, I have a 10Gb file where each row is a name, I am expecting about 50 names in total. So there are a lot of repetitions in clusters. So I want to do a sort -u file Will it be considerably faster or slower to use a uniq before piping it to sort... (3 Replies)
Discussion started by: senhia83
3 Replies

LEARN ABOUT DEBIAN

perl::critic::policy::builtinfunctions::prohibitlvaluesubstr

Perl::Critic::Policy::BuiltinFunctions::ProhibitLvalueSuUser(Contributed Perl DocPerl::Critic::Policy::BuiltinFunctions::ProhibitLvalueSubstr(3pm)

NAME

       Perl::Critic::Policy::BuiltinFunctions::ProhibitLvalueSubstr - Use 4-argument "substr" instead of writing "substr($foo, 2, 6) = $bar".

AFFILIATION

       This Policy is part of the core Perl::Critic distribution.

DESCRIPTION

       Conway discourages the use of "substr()" as an lvalue, instead recommending that the 4-argument version of "substr()" be used instead.

	   substr($something, 1, 2) = $newvalue;     # not ok
	   substr($something, 1, 2, $newvalue);      # ok

       The four-argument form of "substr()" was introduced in Perl 5.005.  This policy does not report violations on code which explicitly
       specifies an earlier version of Perl (e.g. "use 5.004;").

CONFIGURATION

       This Policy is not configurable except for the standard options.

SEE ALSO

       "substr" in perlfunc (or "perldoc -f substr").

       "4th argument to substr" in perl5005delta

AUTHOR

       Graham TerMarsch <graham@howlingfrog.com>

COPYRIGHT

       Copyright (c) 2005-2011 Graham TerMarsch.  All rights reserved.

       This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

perl v5.14.2							    2012-06-07	 Perl::Critic::Policy::BuiltinFunctions::ProhibitLvalueSubstr(3pm)
All times are GMT -4. The time now is 06:09 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy