Sponsored Content
Full Discussion: Uniq code in sorted order
Top Forums Shell Programming and Scripting Uniq code in sorted order Post 302733163 by Don Cragun on Monday 19th of November 2012 10:45:01 PM
Old 11-19-2012
Code:
awk -F, '!c[$3]++' file

The -F, sets the field separator to a <comma>.
The c[$3] is an array indexed by the contents of field 3. An unintialized variable in awk has value 0 (or empty string) depending on context).
The first time it is evaluated !c[$3] (NOT)(value of c[$3]), is (NOT)zero which evaluates to TRUE. When the value is TRUE the action associated with this awk statement is performed. Since there is no action given, the default action (print the line) is performed.
Then the ++ increments the value of c[$3] so that when this test is performed again on lines with the same 3rd field, the test !c[$3] will evaluate to FALSE because (NOT)(positive integer value) evaluates to zero. When the test evaluates to FALSE, the action associated with this line is not performed so subsequent lines with the same value in the 3rd field are not printed.
This test is performed once for each line in the file in order from beginning to end.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sorted processes

Hi, I am trying to make a script that creates a list of all active (alive) processes sorted by size and then print this list on screen. Could anyone help me? Thaks a lot (7 Replies)
Discussion started by: pro
7 Replies

2. Shell Programming and Scripting

Compare 2 sorted files

Hi all, please give me the commands using which i can compare 2 sorted files and get the difference in third file, indiating where the difference is from either file1 or file2. as: File1 (Original file) GARRY JOHN JULIE SAM --------------- File2 DEV GARRY JOHN JOHNIEE (7 Replies)
Discussion started by: varungupta
7 Replies

3. Shell Programming and Scripting

executing code on files in the sorted order -help!

Say i have 2 files in the giving format: file1 1 2 3 4 1 2 3 4 1 2 3 4 file2 1 2 3 4 1 2 3 4 1 2 3 4 I have a PERL code (loaned by one of u -i forgot who - thanks!) that extracts the 2nd column from each file and append horizontally to a new file: perl -ane 'push @{$L->}, $F; close... (1 Reply)
Discussion started by: epi8
1 Replies

4. UNIX for Advanced & Expert Users

Sorted file

Hi Is there any unix shell command or utility to know if the file is sorted or not? Thanks (3 Replies)
Discussion started by: ksailesh
3 Replies

5. UNIX for Dummies Questions & Answers

numerically sorted filenames

How do you sort filenames: 1 2 3 4 5 6 7 8 9 10 12 13 14 15 16 17 18 19 20 21 as: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 (6 Replies)
Discussion started by: kenneth.mcbride
6 Replies

6. UNIX for Dummies Questions & Answers

Help with printing sorted expression

Hi All, How can I print the sorted results of the following expression in Perl ?? print "$i\t$h{$i}\n"; I tried print (sort ("$i\t$h{$i}")"\n"); and other variations of the same but failed. Can someone suggest how to solve this problem, as I'm tryin print sorted results of my script, which... (11 Replies)
Discussion started by: pawannoel
11 Replies

7. Shell Programming and Scripting

sort the files based on timestamp and execute sorted files in order

Hi I have a requirement like below I need to sort the files based on the timestamp in the file name and run them in sorted order and then archive all the files which are one day old to temp directory My files looks like this PGABOLTXML1D_201108121235.xml... (1 Reply)
Discussion started by: saidutta123
1 Replies

8. Shell Programming and Scripting

Read filenames in sorted order

Hi , My requirement is to scan a directory for file names with LTR.PDF* and send those files via ftp to another server one by one. Now the the problem is file names are like LTR.PDF ,LTR.PDF1 ,LTR.PDF2.....LTR.PDF10..upto 99 and these needs to be sent in sorted order. is there a way to get... (10 Replies)
Discussion started by: nishantrk
10 Replies

9. UNIX for Beginners Questions & Answers

How to create a summary file of all files in a directory sorted in reverse alphabetical order.?

I have an interactive script which works terrific at processing a folder of unsorted files into new directories. I am wondering how I could modify my script so that( upon execution) it provides an additional labelled summary file on my desktop that lists all of the files in each directory that... (4 Replies)
Discussion started by: Braveheart
4 Replies

10. Shell Programming and Scripting

Matrix to 3 col sorted

Hello experts, I have matrices sorted by position, there are 400k rows, 3000 columns. ID CHR POS M1 M2 M3 M4 M5 ID1 1 1 4.6 2.6 2.1 3.5 4.2 ID2 1 100 3.6 2.9 3.2 2.6 2.5 ID3 1 1000 4.1... (9 Replies)
Discussion started by: senhia83
9 Replies
INNWATCH.CTL(5) 						File Formats Manual						   INNWATCH.CTL(5)

NAME
innwatch.ctl - control Usenet supervision by innwatch DESCRIPTION
The file /etc/news/innwatch.ctl is used to determine what actions are taken during the periodic supervisions by innwatch. The file consists of a series of lines; blank lines and lines beginning with a number sign (``#'') are ignored. All other lines consist of seven fields, each preceded by a delimiting character: :label:state:condition:test:limit:command:reason The delimiter can be any one of several non-alphanumeric characters that does not appear elsewhere in the line; there is no way to quote it to include it in any of the fields. Any of ``!'', ``,'', ``:'', ``@'', ``;'', or ``?'' is a good choice. Each line can have a different delimiter; the first character on each line is the delimiter for that line. White space surrounding delimiters, except before the first, is ignored, and does not form part of the fields, white space within fields is permitted. All delimiters must be present. The first field is a label for the control line. It is used as an internal state indicator and in ctlinnd messages to control the server. If omitted, the line number is used. The second field specifies when this control line should be used. It consists of a list of labels, and special indicators, separated by whitespace. If the current state matches against any of the labels in this field, this line will be used as described below. The values that may be used are: - This line matches if the current state is the same as the label on this line, or if the current state is ``run,'' the initial state. This is also the default state if this field is empty. + This line matches if the current state is ``run.'' * This line always matches. label This line matches if the current state is the specified ``label.'' -label This line matches if the current state is not the specified ``label.'' The third field specifies a shell command that is invoked if this line matches. Do not use any shell filename expansion characters such as ``*'', ``?'', or ``['' (even quoted, they're not likely to work as intended). If the command succeeds, as indicated by its exit status, it is expected to have printed a single integer to standard output. This gives the value of this control line, to be used below. If the com- mand fails, the line is ignored. The command is executed with its current directory set to the news spool directory, /var/spool/news. The fourth field specifies the operator to use to test the value returned above. It should be one of the two letter numeric test operators defined in test(1) such as ``eq'', ``lt'' and the like. The leading dash (`'-'') should not be included. The fifth field specifies a constant with which to compare the value using the operator just defined. This is done by invoking the command test value -operator constant The line is said to ``succeed'' if it returns true. The sixth field specifies what should be done if the line succeeds, and in some cases if it fails. Any of the following words may be used: throttle Causes innwatch to throttle the server if this line succeeds. It also sets the state to the value of the line's label. If the line fails, and the state was previously equal to the label on this line (that is, this line had previously succeeded), then a go command will be sent to the server, and innwatch will return to the ``run'' state. The ``throttle'' is only performed if the current state is ``run'' or a state other than the label of this line, regardless of whether the command succeeds. pause Is identical to ``throttle'' except that the server is paused. shutdown Sends a ``shutdown'' command to the server. It is for emergency use only. flush Sends a ``flush'' command to the server. go Causes innwatch to send a ``go'' command to the server and to set the state to ``run.'' exit Causes innwatch to exit. skip The result of the control file is skipped for the current pass. The last field specifies the reason that is used in those ctlinnd commands that require one. More strictly, it is part of the reason -- innwatch appends some information to it. In order to enable other sites to recognize the state of the local innd server, this field should usually be set to one of several standard values. Use ``No space'' if the server is rejecting articles because of a lack of filesystem resources. Use ``loadav'' if the server is rejecting articles because of a lack of CPU resources. Once innwatch has taken some action as a consequence of its control line, it skips the rest of the control file for this pass. If the action was to restart the server (that is, issue a ``go'' command), then the next pass will commence almost immediately, so that innwatch can discover any other condition that may mean that the server should be suspended again. EXAMPLES
@@@df .|awk 'NR==2 {print $4}'@lt@10000@throttle@No space @@@df -i .|awk 'NR==2 {print $4}'@lt@1000@throttle@No space (inodes) The first line causes the server to be throttled if the free space drops below 10000 units (using whatever units df uses), and restarted again when free space increases above the threshold. The second line does the same for inodes. The next three lines act as a group and should appear in the following order. It is easier to explain them, however, if they are described from the last up. !load!load hiload!loadavg!lt!5!go! :hiload:+ load:loadavg:gt:8:throttle:loadav /load/+/loadavg/ge/6/pause/loadav The final line causes the server to be paused if innwatch is in the ``run'' state and the load average rises to, or above, six. The state is set to ``load'' when this happens. The previous line causes the server to be throttled when innwatch is in the ``run'' or ``load'' state, and the load average rises above eight. The state is set to ``hiload'' when this happens. Note that innwatch can switch the server from ``paused'' to ``throttled'' if the load average rises from below six to between six and seven, and then to above eight. The first line causes the server to be sent a ``go'' command if innwatch is in the ``load'' or ``hiload'' state, and the load average drops below five. Note that all three lines assume a mythical command loadavg that is assumed to print the current load average as an integer. In more prac- tical circumstances, a pipe of uptime into awk is more likely to be useful. BUGS
This file must be tailored for each individual site, the sample supplied is truly no more than a sample. The file should be ordered so that the more common problems are tested first. The ``run'' state is not actually identified by the label with that three letter name, and using it will not work as expected. Using an ``unusual'' character for the delimiter such as ``('', ``*'', ``&'', ```'', ``''', and the like, is likely to lead to obscure and hard to locate bugs. HISTORY
Written by <kre@munnari.oz.au> for InterNetNews. This is revision 1.5, dated 1996/09/06. SEE ALSO
innd(8), ctlinnd(8), news.daily(8). INNWATCH.CTL(5)
All times are GMT -4. The time now is 08:25 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy