02-28-2020
A more detailed explanation follows.
The main awk code runs for each input line.
!a[$0]++ is ultra-condensed, quick and dirty.
A bit more explicit is !($0 in A) { A[$0]; print }:
If not $0 in array A (A[$0] not defined) then define A[$0] (no A[$0]=value needed here) and print $0.
The array A is associative (string-addressed). So if the same $0 will occur in another input line it will see a defined A[$0] and won't print.
If there is a pre-condition and no { action code } following then the default for a true condition is { print }, and print without arguments defaults to print $0.
Now to the quick and dirty !A[$0]++:
Define A[$0] with value 0 if undefined, if the negated value is non-zero (true) then default-print. Also post-increment A[$0].
If the same $0 will occur then the A[$0] value will be 1, negated 0 (false), won't print, but post-incremented.
If the same $0 will occur then the A[$0] value will be 2, negated 0 (false), won't print, but post-incremented.
...
This User Gave Thanks to MadeInGermany For This Post:
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi,
Let's say that I have a file called table, I know that if I need to see a the second column for exampls I use:
awk ' {print $2}' table.txt
Is there anyway to use awk to actually cut a column and put it somewhere else in the table?:confused: (8 Replies)
Discussion started by: cosmologist
8 Replies
2. Shell Programming and Scripting
Hi i need a favour
i have a file which has some trillions of records. The file is like this
11111000000000192831840914000000000000000000000000000
45789899090000000000000000011111111111111111111111111
I want to cut specific postions in each line like cut1-3 and assisgn it to a variable and... (5 Replies)
Discussion started by: richa2.m
5 Replies
3. Shell Programming and Scripting
hi,
I have a file with content like this for an employee:
EmployeeID
101
Day_type, day
vacation,1/2/2009
sick day, 3/2/2009
personal day, 4/5/2009
jury duty day, 5/5/2009
how do I make the result to show:
EmployeeID,Day_type,day
101,vacation,1/2/2009
101,sick day,... (6 Replies)
Discussion started by: jbchen
6 Replies
4. UNIX for Advanced & Expert Users
Hello,
I had posted earlier about printing fields using AWK, but now I have a slightly different problem. I have text files in the format:
1*2,3,4,5
and wish to print the first, third, and fifth fields, including the asterisk and commas. In other words, after filtering it should look... (1 Reply)
Discussion started by: Jahn
1 Replies
5. Shell Programming and Scripting
Hi,
Need a help with shell script. I have to search for a string in one of the file, if match found, copy the line to a new file and delete the line from the exisiting file.
eg:
83510000000000063800000.1800000.1600000.1600000.2400000.1800000.2000000.21... (6 Replies)
Discussion started by: gpaulose
6 Replies
6. Shell Programming and Scripting
Hi Everyone,
# cat 1.txt
1321631,77770132976455,19,20091001011859,20091001011907
1321631,77770132976455,19,20091001011859,20091001011907
1321631,77770132976455,19,20091001011859,20091001011907
# cat 1.txt | awk -F, '{OFS=",";print $1,$3,$4,$5}'
1321631,19,20091001011859,20091001011907... (7 Replies)
Discussion started by: jimmy_y
7 Replies
7. Shell Programming and Scripting
I have a file and need to only select users that have a shell of “/bin/bash” in the line using awk or sed please help (4 Replies)
Discussion started by: boyboy1212
4 Replies
8. UNIX for Advanced & Expert Users
Hi,
I've got a query regarding which of the following is more efficient & why -
cat <filename>|cut -d'*' -f2- > <newfilename>
or
cut -d'*' -f2- <filename> > <newfilename>
Thanks. (17 Replies)
Discussion started by: sumoka
17 Replies
9. Shell Programming and Scripting
I have a file which contains 3 fields separated by tabs example
andrew kid baker
I need to swap kid and baker using cut and paste commands how is this to be done?
Thanks (3 Replies)
Discussion started by: drew211
3 Replies
10. Shell Programming and Scripting
i have file as with the below content
aaa.bbb.cc.dd
aaa.fff.bb
yyyyy.rrrrr.ggggg.iii
wwww.w.r.ty
i want the o/p as below
dd
bb
iii
ty
but i dont want to use awk. is there any other way to do this ? (5 Replies)
Discussion started by: anandgodse
5 Replies
LEARN ABOUT OPENSOLARIS
regex
regex(1F) FMLI Commands regex(1F)
NAME
regex - match patterns against a string
SYNOPSIS
regex [-e] [-v "string"] [pattern template] ...
pattern [template]
DESCRIPTION
The regex command takes a string from the standard input, and a list of pattern / template pairs, and runs regex() to compare the string
against each pattern until there is a match. When a match occurs, regex writes the corresponding template to the standard output and
returns TRUE. The last (or only) pattern does not need a template. If that is the pattern that matches the string, the function simply
returns TRUE. If no match is found, regex returns FALSE.
The argument pattern is a regular expression of the form described in regex(). In most cases, pattern should be enclosed in single quotes
to turn off special meanings of characters. Note that only the final pattern in the list may lack a template.
The argument template may contain the strings $m0 through $m9, which will be expanded to the part of pattern enclosed in ( ... )$0 through
( ... )$9 constructs (see examples below). Note that if you use this feature, you must be sure to enclose template in single quotes so that
FMLI does not expand $m0 through $m9 at parse time. This feature gives regex much of the power of cut(1), paste(1), and grep(1), and some
of the capabilities of sed(1). If there is no template, the default is $m0$m1$m2$m3$m4$m5$m6$m7$m8$m9.
OPTIONS
The following options are supported:
-e Evaluates the corresponding template and writes the result to the standard output.
-v "string" Uses string instead of the standard input to match against patterns.
EXAMPLES
Example 1 Cutting letters out of a string
To cut the 4th through 8th letters out of a string (this example will output strin and return TRUE):
`regex -v "my string is nice" '^.{3}(.{5})$0' '$m0'`
Example 2 Validating input in a form
In a form, to validate input to field 5 as an integer:
valid=`regex -v "$F5" '^[0-9]+$'`
Example 3 Translating an environment variable in a form
In a form, to translate an environment variable which contains one of the numbers 1, 2, 3, 4, 5 to the letters a, b, c, d, e:
value=`regex -v "$VAR1" 1 a 2 b 3 c 4 d 5 e '.*' 'Error'`
Note the use of the pattern '.*' to mean "anything else".
Example 4 Using backquoted expressions
In the example below, all three lines constitute a single backquoted expression. This expression, by itself, could be put in a menu defini-
tion file. Since backquoted expressions are expanded as they are parsed, and output from a backquoted expression (the cat command, in this
example) becomes part of the definition file being parsed, this expression would read /etc/passwd and make a dynamic menu of all the login
ids on the system.
`cat /etc/passwd | regex '^([^:]*)$0.*$' '
name=$m0
action=`message "$m0 is a user"`'`
DIAGNOSTICS
If none of the patterns match, regex returns FALSE, otherwise TRUE.
NOTES
Patterns and templates must often be enclosed in single quotes to turn off the special meanings of characters. Especially if you use the
$m0 through $m9 variables in the template, since FMLI will expand the variables (usually to "") before regex even sees them.
Single characters in character classes (inside []) must be listed before character ranges, otherwise they will not be recognized. For exam-
ple, [a-zA-Z_/] will not find underscores (_) or slashes (/), but [_/a-zA-Z] will.
The regular expressions accepted by regcmp differ slightly from other utilities (that is, sed, grep, awk, ed, and so forth).
regex with the -e option forces subsequent commands to be ignored. In other words, if a backquoted statement appears as follows:
`regex -e ...; command1; command2`
command1 and command2 would never be executed. However, dividing the expression into two:
`regex -e ...``command1; command2`
would yield the desired result.
ATTRIBUTES
See attributes(5) for descriptions of the following attributes:
+-----------------------------+-----------------------------+
| ATTRIBUTE TYPE | ATTRIBUTE VALUE |
+-----------------------------+-----------------------------+
|Availability |SUNWcsu |
+-----------------------------+-----------------------------+
SEE ALSO
awk(1), cut(1), grep(1), paste(1), sed(1), regcmp(3C), attributes(5)
SunOS 5.11 12 Jul 1999 regex(1F)