The first part:
reads the first line from file into the shell variable A (read A) and the second line from file into the shell variable B (read B), reprints those two lines unchanged (printf '%s\n' "$A" "$B" and sorts the remaining lines from file (sort). This sorts the data in file while keeping the headers (unsorted) at the start of the output.
All of the output from the first part is then piped (|) into the second part:
which creates an array indexed by the first field on each input line (T[$1]) and sets the value of that array to the number of times that index has been seen (after using the previous value of the variable). In awk, statements consist of a condition section and an action section. In this case the condition is !T[$1]++ and there is no action section. When there is no action section, the default action (print the line) is performed if, and only if, the condition evaluates to a non-zero numeric value or a non-empty string value. Since undefined elements of an array are treated as a zero or an empty string (depending on context), the condition !T[$1]++ is evaluated as follows the first time a line is seen with a given string in the first field:
T[$1] is seen as a zero value
!T[$1] yields a value of 1 for the condition
T[$1]++ increments the value of the array element to 1
and since the condition's value is non-zero, the line is printed. On subsequent lines with the same first field the evaluation is:
T[$1] is seen as a non-zero value (a count of the number of times this value has been seen before)
!T[$1] yields a value of 0 for the condition
T[$1]++ increments the value of the array element
and since the condition's value is zero, the line is not printed.
Note that the sort in the first part grouped all lines with the same 1st field together and (if both an "add" and a "delete" appeared for that value, the line(s) with "add" for that 1st field value will come before any line(s) with "delete".
Nice, simple, straight-forward magic. The only likely problem with this approach is that if the data in your file contains a 1st field value that is identical to the 1st field in one of the header lines, those data lines will be lost. But, as long as you don't have a user named "user", that shouldn't be a problem for you.
These 2 Users Gave Thanks to Don Cragun For This Post:
Note that if you don't care about the order of the data lines in the output, you can also try something like:
which doesn't need sort and only reads the input file once. (It will print all users with "add" in the 2nd field in the order in which they were first found in the input file followed by users who have no "add" but do have a "delete" in the 2nd field in random order.) With your latest sample input file, the output will be:
Last edited by Don Cragun; 03-06-2016 at 04:12 PM..
Reason: Remove single quotes in comments.
Hi - I want to interrogate information about my poker hands, sessions are all recorded in a text file in a particular format. Each hand starts with the string <PokerStars> followed by a unique hand reference and other data like date/time. There is then all the information about each hand. My first... (5 Replies)
I have one input file ABC.txt and one output DEF.txt. After the ABC is processed and created output, I want to rename ABC.txt to ABC.orig and DEF to ABC.txt. Currently when I am doing this, it does not process the input file as it cannot read and write to the same file. How can I achieve this?
... (12 Replies)
Hello,
I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this :
This is the output of ls command : I stored the output in a file filelist
1.1M... (5 Replies)
Input File:
1234, 2345,abc
1,24141,gw
222,rff,sds
2232145,sdsd,121
Output file to be generated:
000001234,2345,abc
000000001,24141,gw
000000222,rff,sds
002232145,sdsd,121
i.e; the first column is padded to get 9 digits.
I tried with following: (3 Replies)
Input File:
1234, 2345,abc
1,24141,gw
222,rff,sds
2232145,sdsd,121
Output file to be generated:
000001234,2345,abc
000000001,24141,gw
000000222,rff,sds
002232145,sdsd,121
i.e; the first column is padded to get 9 digits.
I tried with following: (1 Reply)
I have a 3-column data file, for which I wish to print certain parts of $3
PHI PSI A(x)
-177.5 -177.5 1.0625
-177.5 -172.5 0.55
-177.5 -167.5 0.0478125
-177.5 -162.5 0
-177.5 -157.5 0.284375
-177.5 -152.5 0.187188
-177.5 -147.5 0.236875
-177.5 -142.5 0.383438
-177.5 ... (3 Replies)
I am trying to process file which has following data
#23456789012345
ACNASPSA13N0N0
ACNAPCPA05N0N0
ACNAFATS11N0N0
I want to take out each line from the file and what to put in the file by name which if part of the line starting from offset 10 to 15. It means I want to create three file... (3 Replies)
Hello,
I'm running a script on AIX to process lines in a file. I need to enclose the second column in quotation marks and write each line to a new file. I've come up with the following:
#!/bin/ksh
filename=$1
exec >> $filename.new
cat $filename | while read LINE
do
echo $LINE | awk... (2 Replies)
Hello,
say suppose i am processing an file emp.dat the field of which are
deptno empno empname etc
now say suppose i want to change the file to emp.lst then how can i do it? Here i what i attempted but in vain
BEGIN{
system("sort emp.dat > emp.lst")
FILENAME="emp.lst"
}
{
print... (2 Replies)
Hello,
I have what is probably a simple task in text manipulation, but I just can't wrap my brain around it.
I have a text file that looks something like the following. Note that some have middle initials in the first field and some don't.
john.r.smith:john.smith@yahoo.com... (4 Replies)