Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Awk counting lines with field match Post 302314488 by fredted40x on Friday 8th of May 2009 11:49:25 AM
Old 05-08-2009
Awk counting lines with field match

Hi,

Im trying to create a script that reads throught every line in a file and then counts how many lines there with a certain field that matches a input, and also ausing another awk it has to do the same as the above but to then use sort anduniq to get rid of all the unique lines with another field.

Heres an example line that i am going through

345.345.345.345 Mon June 03 02:33:44 GMT 2007

and here is my awk's
as i havnt managed to compress them into one line yet
awk '$2 == "'"$DDD"'" { print $5 }' ~/folder/home.hhh | awk -F: '$1 >= VAR1 && $1 <= VAR2 { print $0 }' VAR1=$BEGIN VAR2=$END | wc -l

awk '$2 == "'"$DDD"'" { print $5 }' ~/folder/home.hhh | awk -F: '$1 >= VAR1 && $1 <= VAR2 { print $0 }' VAR1=$BEGIN VAR2=$END | sort | uniq | wc -l;;

DDD = A user entered day, e.g. Mon
BEGIN each earliest hour in a range e.g. 01
END is the last hour in a hour range e.g. 03

The first awk works bril, but the second doesnt as im trying to use the uniq on the 345.345.345.345 field ($1 on the original lines), but the first awk doesnt pass this foreward to the next awk. is it possible to still check if $2 is equal to DAYS and the time has an hour between a time range and then use the uniq to delete any lines where $1 of the original line is the same.

Sorry if this is confusing and please ask questions if needed.

Thanks

Last edited by fredted40x; 05-08-2009 at 12:57 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk - Counting number of similar lines

Hi All I have the input file OMAK_11. OMAK 000002EXCLUDE 1341 OMAK 000002EXCLUDE 1341 OMAK 000002EXCLUDE 1341 OMAK 000003EXCLUDE 1341 OMAK 000003EXCLUDE 1341 OMAK 000003EXCLUDE ... (8 Replies)
Discussion started by: dhanamurthy
8 Replies

2. Shell Programming and Scripting

Counting lines of code in a directory with awk

I've never toyed with awk, but it seems every time I present an elegant 2- to 8-line script, someone comes back with an awk 1-liner. I just came up with this to count all the lines of source code in a directory. How would I do it in awk? LINES=0 for n in $(wc -l *.cpp *.h | cut -b-7); do ... (2 Replies)
Discussion started by: KenJackson
2 Replies

3. Shell Programming and Scripting

counting lines containing two column field values with awk

Hello everybody, I'm trying to count the number of consecutive lines in a text file which have two distinctive column field values. These lines may appear in several line blocks within the file, but I only want a single block to be counted. This was my first approach to tackle the problem (I'm... (6 Replies)
Discussion started by: origamisven
6 Replies

4. Shell Programming and Scripting

AWK: Pattern match between 2 files, then compare a field in file1 as > or < field in file2

First, thanks for the help in previous posts... couldn't have gotten where I am now without it! So here is what I have, I use AWK to match $1 and $2 as 1 string in file1 to $1 and $2 as 1 string in file2. Now I'm wondering if I can extend this AWK command to incorporate the following: If $1... (4 Replies)
Discussion started by: right_coaster
4 Replies

5. UNIX for Dummies Questions & Answers

awk display the match and 2 lines after the match is found.

Hello, can someone help me how to find a word and 2 lines after it and then send the output to another file. For example, here is myfile1.txt. I want to search for "Error" and 2 lines below it and send it to myfile2.txt I tried with grep -A but it's not supported on my system. I tried with awk,... (4 Replies)
Discussion started by: eurouno
4 Replies

6. Shell Programming and Scripting

counting lines that match pattern

I have a file of 1.3 millions lines. some are with the same word twice on the line, some line have two diffrent words. each line has two words, one in brackets. example: foo (foo) bar (bar) thae (awvd) beladf (vswvw) I am sure this can be done with one line of... (6 Replies)
Discussion started by: robsonde
6 Replies

7. Shell Programming and Scripting

Counting lines in a file using awk

I want to count lines of a file using AWK (only) and not in the END part like this awk 'END{print FNR}' because I want to use it. Does anyone know of a way? Thanks a lot. (7 Replies)
Discussion started by: guitarist684
7 Replies

8. Shell Programming and Scripting

awk repeat one field at all lines and modify field repetitions

Hello experts I have a file with paragraphs begining with a keeping date and ending with "END": 20120301 num num John num num A keepnum1 num num kathrin num num A keepnum1 num num kathrin num num B keepnum2 num num Pete num num A keepnum1 num num Jacob num... (2 Replies)
Discussion started by: phaethon
2 Replies

9. Shell Programming and Scripting

awk to match value to a field within +/- value

In the awk below I use $2 of filet to search filea for a match. If the values in $2 are exact match this works great. However, that is not always the case, so I need to perform the search using a range of + or - 2. That is if the value in filea $2 is within + or - 2 of filet $2 then it is matched.... (6 Replies)
Discussion started by: cmccabe
6 Replies

10. Shell Programming and Scripting

awk to match field between two files and use conditions on match

I am trying to look for $2 of file1 (skipping the header) in $2 of file2 (skipping the header) and if they match and the value in $10 is > 30 and $11 is > 49, then print the line from file1 to a output file. If no match is foung the line is not printed. Both the input and output are tab-delimited.... (3 Replies)
Discussion started by: cmccabe
3 Replies
AWK  is  a  programming  language devised by Aho, Weinberger, and
Kernighan at Bell Labs (hence the  name).   Awk  programs  search
files for specific patterns and performs actions for every occur-
rence of these patterns.  The patterns can be regular expressions
as used in the ed editor.  The actions are expressed using a sub-
set of the C language.	The  patterns  and  actions  are  usually
placed	in  a rules file whose name must be the first argument in
the command line, preceded by the flag -f.  Otherwise, the  first
argument  on  the command line is taken to be a string containing
the rules themselves. All other arguments are  taken  to  be  the
names  of text files on which the rules are to be applied, with -
being the standard input.  To take rules from the standard input,
use -f -.  The command: would read the patterns and actions rules
from the file rules and apply them to  all  the  arguments.   The
general  format  of  a	rules  file is: ~~~<pattern> { <action> }
~~~<pattern> { <action> } ~~~...  There  may  be  any  number  of
these  <;pattern>  {  <action> } sequences in the rules file.  Awk
reads a line of input from the current input file and applies ev-
ery <;pattern> { <action> } in sequence to the line.  If the <pat-
tern>; corresponding to any { <action> } is missing, the action is
applied  to  every line of input.  The default { <action> } is to
print the matched input line.  The <;pattern>s may consist of  any
valid C expression.  If the <;pattern> consists of two expressions
separated by a comma, it is taken to be a range and the  <;action>
is  performed  on all lines of input that match the range.  <;pat-
tern>;s may contain regular expressions delimited by an @  symbol.
Regular  expressions  can be thought of as a generalized wildcard
string matching mechanism, similar to that used by many operating
systems  to  specify file names.  Regular expressions may contain
any of the following characters:
  x    An ordinary character
      The backslash quotes any character
  ^    A circumflex at the beginning of an expr matches  the  be-
       ginning of a line.
  $    A  dollar-sign at the end of an expression matches the end
       of a line.
  .    A period matches any single character except newline.
  *    An expression followed by an asterisk matches zero or more
       occurrences  of	that  expression: fo* matches f, fo, foo,
       fooo, etc.
  +    An expression followed by a plus sign matches one or  more
       occurrences of that expression: fo+ matches fo, foo, fooo,
       etc.
  []   A string enclosed in square brackets  matches  any  single
       character  in  that  string,  but no others.  If the first
       character in the string is a  circumflex,  the  expression
       matches any character except newline and the characters in
       the string.  For example, [xyz] matches xx and zyx,  while
       [^xyz] matches abc but not axb.	A range of characters may
       be specified by two characters separated by -.
Actions are expressed as a subset of the C language.   All  vari-
ables  are  global and default to int's if not formally declared.
Only char's and int's and pointers and arrays of char and int are
allowed.   Awk allows only decimal integer constants to be used--
no hex (0xnn) or octal (0nn). String and character constants  may
contain  all  of  the special C escapes (
, 
, etc.).  Awk sup-
ports the if, else, while and break flow of  control  constructs,
which  behave  exactly as in C.  Also supported are the following
unary and binary operators, listed in order from highest to  low-
est precedence:
  Operator	 Type	      Associativity
  () [] 	 unary	      left to right
  ! ~ ++ -- - * &	      unaryright to left
  * / % 	 binary       left to right
  + -		 binary       left to right
  << >> 	 binary       left to right
  < <= > >=	 binary       left to right
  == != 	 binary       left to right
  &		 binary       left to right
  ^		 binary       left to right
  |		 binary       left to right
  &&		 binary       left to right
  ||		 binary       left to right
  =		 binary       right to left
Comments are introduced by a '#' symbol and are terminated by the
first newline character.  The standard /* and */  comment  delim-
iters  are not supported and will result in a syntax error.  When
awk reads a line from the current input file, the record is auto-
matically  separated  into fields.  A field is simply a string of
consecutive characters delimited by either the beginning  or  end
of  line,  or  a field separator character.  Initially, the field
separators are the space and tab character.   The  special  unary
operator  '$'  is used to reference one of the fields in the cur-
rent input record (line).  The fields are  numbered  sequentially
starting  at  1.   The	expression $0 references the entire input
line.  Similarly, the record separator is used to  determine  the
end of an input line, initially the newline character.	The field
and record separators may be changed programatically  by  one  of
the  actions and will remain in effect until changed again.  Mul-
tiple (up to 10) field separators are allowed at a time, but only
one  record  separator.   Fields behave exactly like strings; and
can be used in the same context as a character array.  These  ar-
rays  can be considered to have been declared as:      char ($n)[
128 ]; In other words, they are 128 bytes long.  Notice that  the
parentheses  are  necessary  because the operators [] and $ asso-
ciate from right to left; without them, the statement would  have
parsed	as:	 char $(1[ 128 ]); which is obviously ridiculous.
If the contents of one of these field arrays is altered,  the  $0
field  will  reflect  this change.  For example, this expression:
     *$4 = 'A'; will change the first  character  of  the  fourth
field to an upper- case letter 'A'.  Then, when the following in-
put line:      120 PRINT  "Name 	 address	 Zip"  is
processed,   it   would  be  printed  as:	120  PRINT  "Name
Address        Zip"; Fields may also be modified with the strcpy()
function  (see below).	For example, the expression:	  strcpy(
$4, "Addr." ); applied	to  the  same  line  above  would  yield:
     120  PRINT  "Name		Addr.	      Zip"  The following
variables are pre-defined:
   FS	       Field separator (see below).
   RS	       Record separator (see below also).
   NF	       Number of fields in current input record (line).
   NR	       Number of records processed thus far.
   FILENAME    Name of current input file.
   BEGIN       A special <pattern> that matches the beginning  of
	       input text.
   END	       A  special <pattern> that matches the end of input
	       text.
Awk also provides some useful built-in functions for  string  ma-
nipulation and printing:
   print(arg)  Simple  printing  of  strings  only, terminated by
	       '
'.
   printf(arg...)Exactly the printf() function from C.
   getline()   Reads the next record and  returns  0  on  end  of
	       file.
   nextfile()  Closes  the current input file and begins process-
	       ing the next file
   strlen(s)   Returns the length of its string argument.
   strcpy(s,t) Copies the string t to the string s.
   strcmp(s,t) Compares the s to t and returns 0 if they match.
   toupper(c)  Returns its character argument converted to upper-
	       case.
   tolower(c)  Returns its character argument converted to lower-
	       case.
   match(s,@re@)Compares the string s to the  regular  expression
	       re  and	returns the number of matches found (zero
	       if none).
Awk was written by Saeko Hirabauashi and Kouichi Hirabayashi.
All times are GMT -4. The time now is 09:44 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy