awk not working for calculating no of lines with criteria


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk not working for calculating no of lines with criteria
# 1  
Old 07-15-2013
awk not working for calculating no of lines with criteria

I have tar.gz file and i want to count the lines which are matching the criteria as well as which are not matching the criteria. Following is the code

Output Requirement:
Match the input from zcat with 26th filed having 02 value, in case it matches then print the output in a file & increase the match counter by 1 & in case it doesnt matches then increase the not match counter. At last i have 2 files one having the records in a.txt file & another file having match counter & not match counter values.

But this is not working, please help

Code:
zcat filename.tar.gz | awk -v mon="07" '
BEGIN {
 if (( (substr($0,26,2)=="02") && substr($0,84,2) == month  ))
  print $0 >> "a.txt"
  ++matchcounter
 else 
 ++notmatch 
 ;}
END { print matchcounter","notmatch >> "countfile"}
'

# 2  
Old 07-15-2013
Quote:
Originally Posted by siramitsharma
I have tar.gz file and i want to count the lines which are matching the criteria as well as which are not matching the criteria. Following is the code

Output Requirement:
Match the input from zcat with 26th filed having 02 value, in case it matches then print the output in a file & increase the match counter by 1 & in case it doesnt matches then increase the not match counter. At last i have 2 files one having the records in a.txt file & another file having match counter & not match counter values.

But this is not working, please help

Code:
zcat filename.tar.gz | awk -v mon="07" '
BEGIN {
 if (( (substr($0,26,2)=="02") && substr($0,84,2) == month  ))
  print $0 >> "a.txt"
  ++matchcounter
 else 
 ++notmatch 
 ;}
END { print matchcounter","notmatch >> "countfile"}
'

Try without BEGIN
Code:
zcat filename.tar.gz | awk -v mon="07" '
    {
 if (( (substr($0,26,2)=="02") && substr($0,84,2) == month  ))
  print $0 >> "a.txt"
  ++matchcounter
 else 
 ++notmatch}
    END { print matchcounter","notmatch >> "countfile"}'

# 3  
Old 07-15-2013
Seems to be working, thanks.

One more thing, if i want to extract the filename as well from filename.tar.gz file & get the count accordingly for match counter and not match counter, how would i do?

---------- Post updated at 10:00 PM ---------- Previous update was at 06:00 PM ----------

hi pamu,
when i tried removing BEGIN it gave an error, below is the code

Code:
 cat filename.tar.gz | awk -v mon="07" '
    {
 if (( (substr($0,26,2)=="02") && substr($0,84,2) == month  ))
  print $0 >> "a.txt"
  ++matchcounter
 else
 ++notmatch};
    END { print matchcounter","notmatch >> "countfile"}'
awk: cmd. line:5:  else
awk: cmd. line:5:  ^ syntax error

Can you please suggest
# 4  
Old 07-15-2013
Quote:
Originally Posted by siramitsharma
Seems to be working, thanks.

One more thing, if i want to extract the filename as well from filename.tar.gz file & get the count accordingly for match counter and not match counter, how would i do?

---------- Post updated at 10:00 PM ---------- Previous update was at 06:00 PM ----------

hi pamu,
when i tried removing BEGIN it gave an error, below is the code

Code:
 cat filename.tar.gz | awk -v mon="07" '
    {
 if (( (substr($0,26,2)=="02") && substr($0,84,2) == month  ))
  print $0 >> "a.txt"
  ++matchcounter
 else
 ++notmatch};
    END { print matchcounter","notmatch >> "countfile"}'
awk: cmd. line:5:  else
awk: cmd. line:5:  ^ syntax error

Can you please suggest
I don't know what you're trying to do, but there are a few obvious problems. Since I don't know what you're trying to do, I haven't made any attempt to test the following suggestion.

In earlier posts in this thread you were using zcat to unzip a compressed tar file. The code in orange in your script is now using cat instead of zcat???

The code in red in your script sets a variable named "mon" (and never uses it) and uses a variable named "month" (that has never been set).

And as shown by the diagnostic messages you're getting from awk, your if statement, is not using the correct syntax.

The following seems to fix the obvious issues above:
Code:
zcat filename.tar.gz | awk -v mon="07" '
{       if(substr($0,26,2)=="02" && substr($0,84,2) == mon) {
                print $0 >> "a.txt"
                ++matchcounter
        } else 
                ++notmatch
}
END {   print matchcounter","notmatch >> "countfile"}'

The changes marked in red are crucial; the other changes are editorial.

I don't see how this script can do anything useful with a tar file, but I haven't made any changes to account for that. Your if statement might make sense (although I didn't look at the definition of a tar header block to confirm it) if it was only looking at tar headers, but this script is looking at every line in the tar file (headers and archived file contents).
# 5  
Old 07-16-2013
hey don,
that's correct, script is looking at each line of of archived files in the tar file. Can you please suggest how to get the archived filename which is being parsed in the code below for each line
# 6  
Old 07-16-2013
Shell scripts aren't particularly well suited to skipping over chunks of data that may contain data that is binary (rather than textual) contents of a file. Even if all of the files that are included in the tar file are text files, the shell and awk aren't necessarily a good fit for this job. Instead of showing us a broken awk script that doesn't do what you want to do; why don't you tell us what you are trying to do and show us output from the command:
Code:
tar -tvf filename.tar.gz

or if that doesn't work:
Code:
zcat filename.tar.gz | tar -tvf -

and show us exactly what output you want your shell/awk script to produce when given this gzipped tar archive (or the unzipped tar archive) as input?
# 7  
Old 07-16-2013
hi don,
I want to parse the tar file having multiple files(more than 5000+) using the conditions mentioned in awk & also need to print the filename from where the condition has matched. In case condition in theawk(if condition) is not matched then notmatch counter should increase along with the filename in which it has not matched.

Code 1:
Code:
zcat filename.tar.gz

This would read each line of archived files

Code 2:
Code:
awk -v mon="07" '
{       if(substr($0,26,2)=="02" && substr($0,84,2) == mon) {
                print $0 >> "a.txt"
                ++matchcounter
        } else 
                ++notmatch
}

This condition will then match the condition and in case matched then output should be written in text file & also increase the match counter, in case condition is not matched, then counter of notmatch should increase

Code 3:
Code:
END {   print matchcounter","notmatch >> "countfile"}'

This condition will print the value of match and notmatch counter


Now requirement is to print the archived filename in the above condition(code no 2)
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need To Delete Lines Based On Search Criteria

Hi All, I have following input file. I wish to retain those lines which match multiple search criteria. The search criteria is stored in a variable seperated from each other by comma(,). SEARCH_CRITERIA = "REJECT, DUPLICATE" Input File: ERROR,MYFILE_20130214_11387,9,37.75... (3 Replies)
Discussion started by: angshuman
3 Replies

2. Shell Programming and Scripting

Select lines from a file based on a criteria

Hi I need to select lines from a txt file, I have got a line starting with ZMIO:MSISDN= and after a few line I have another line starting with 'MOBILE STATION ISDN NUMBER' and another one starting with 'VLR-ADDRESS' I need to copy these three lines as three different columns in a separate... (3 Replies)
Discussion started by: Tlcm sam
3 Replies

3. Shell Programming and Scripting

Calculating the epoch time from standard time using awk and calculating the duration

Hi All, I have the following time stamp data in 2 columns Date TimeStamp(also with milliseconds) 05/23/2012 08:30:11.250 05/23/2012 08:30:15.500 05/23/2012 08:31.15.500 . . etc From this data I need the following output. 0.00( row1-row1 in seconds) 04.25( row2-row1 in... (5 Replies)
Discussion started by: ks_reddy
5 Replies

4. Shell Programming and Scripting

Merging Lines based on criteria

Hello, Need help with following scenario. A file contains following text: {beginning of file} New: This is a new record and it is not on same line. Since I have lost touch with script take this challenge and bring all this in one line. New: Hello losttouch. You seem to be struggling... (4 Replies)
Discussion started by: losttouch
4 Replies

5. Shell Programming and Scripting

Print lines that match certain criteria

Hi all I have a text file with the following format: id col1 col2 col3 col4 col5 col6 col7 ... row1 0 0 0 0 0 0 0 row2 0 0 0 0 0 0 0 row3 0 0 0 0 0 0.2 0 row4 0 0 0 0 0 0 0 row5 0 0 0 0 0 0 0 row6 0 0 0 0.1 0 0 0 row7 0 0 0 0 0 0 0 row8 0 0 0 0 0 0 0 row9 0 0 0 0 0 0 0 ... The file... (2 Replies)
Discussion started by: gautig
2 Replies

6. Shell Programming and Scripting

[Solved] awk calculating between lines

Hey guys, maybe you can help me with this... I want to read input.dat line by line, while doing a simple calculation between the second column value of the current line and the second column value of the next line (like a difference). input is something like this: 0 3.945757 1 ... (1 Reply)
Discussion started by: origamisven
1 Replies

7. Shell Programming and Scripting

Calculating 12th working day

I have a business requirement in my project where I need to calculate the 12th working day of every month. Can any please tell me the solution to my problem. Thanks in advance (7 Replies)
Discussion started by: ami_smart
7 Replies

8. Shell Programming and Scripting

Replacing lines which match certain criteria

Hi, I have code which is like this <TABLE name="UsageDetail_24> <ROW> <Date24><!]></Date24> <Time24><!]></Time24> <Destination24><!]></Destination24> <Rate24><!]></Rate24> <Duration24><!]></Duration24> <Cost24><!]></Cost24> <Allowance24><!]></Allowance24> </ROW> <ROW>... (3 Replies)
Discussion started by: legolad
3 Replies

9. Shell Programming and Scripting

Delete new lines based on search criteria

Hi all! A bit of background: I am trying to create a script that formats SQL statements. I have gotten so far as to add new lines based on certain match criteria like commas, keywords etc. In the process, I end up adding newlines where I don't want. For example: substr(colName, 1, 10)... (3 Replies)
Discussion started by: jayarkay
3 Replies

10. Windows & DOS: Issues & Discussions

selection criteria in Access query not working

Attached are views of the components of a dummy Access database. The database represents an example of the problem that has reared its ugly head. The query example is a simple "Selection" query, which, after getting it to work, will become an "Append" query. The selected data will be appended... (1 Reply)
Discussion started by: hipockets
1 Replies
Login or Register to Ask a Question