Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Help fixing awk code

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help fixing awk code
# 1  
Old 01-18-2017
Help fixing awk code
can someone please help me spot and fix the issue with the following code:

Code:
awk -F, -v SEARCHPATT="(Wed|Tue)" -v ADDISTR="Mon|Tue|Wed|Thu|Fri|Sat|Sun" -vVF="$VALFOUND"
"BEGIN{           {D[10] = D[13] = 1
                 D[11] = D[14] = 2
                }

$0 ~ "," VF "," {L = 1                                  # start output only if VALFOUND is matched
                }

!L              {next                                   # skip line if NOT VALFOUND
                }

                {A = strftime("%a %b %d %T %Y,%s",$3)
                }

A ~ SEARCHPATT &&
NF in D         {TOT = 0
                 for (n = split ($(NF-D[NF]), T, "="); n>1; n--)    {sub (/_.*/, _, T[n]); TOT += T[n]}

                 if (NF == 11 || A ~ ADDISTR)   print TOT "-" $3 "_" $(NF-D[NF]+1) "----" A
                }
}" datafile.txt

it's supposed to be a optimized, better version of the following code which i slapped together:

Code:
gawk -v SEARCHPATT="${SEARCHPATT}" -v ADDISTR="${INCEXCSTR}" -F, '/,'"${VALFOUND}"',/,0 {A=strftime("%a %b %d %T %Y,%s",$3);if((NF == 13) && (A ~ ADDISTR) && (A ~ SEARCHPATT)) {print $12"-"$3"_0""-" $13"----"A} else if ((NF == 14) && (A ~ ADDISTR) && (A ~ SEARCHPATT)) {print $12"-"$3"_0""-" $13"----"A} else if ((NF == 10) && (A ~ ADDISTR) && (A ~ SEARCHPATT)) {print $9"-"$3"_"$10"----"A} else if ((NF == 11) && (A ~ SEARCHPATT)) {print $9"-"$3"_"$10"----"A} }' datafile.txt | awk -F"----" '{print $1}'

the content of the datafile being read here could look like this:

typeA
Code:
0,greenscreen_pc10,1484711626,335086,/PROD/NOA/cicsmrch/sys/unikixmain.log,25M,greenscreen_pc10,25638056,0,333183--335086,-1
0,greenscreen_pc10,1484711922,337099,/PROD/NOA/cicsmrch/sys/unikixmain.log,25M,greenscreen_pc10,25796338,0,335086--337099,0
0,greenscreen_pc10,1484712222,338253,/PROD/NOA/cicsmrch/sys/unikixmain.log,25M,greenscreen_pc10,25887414,0,337099--338253,2

or like this:

typeB
Code:
0,plm_tomcat_logcheck,1484756597,12685,/opt/apps/plm/logs/catalina.out,964K,plm_tomcat_logcheck,985770,Master_Item_Service_is_down=0_njava_lang_NoClassDefFoundError=0_njava_lang_OutOfMemoryError=0_nemxCommonAppInitialization__Error_while_initializing=0_nINFO__Stopping_Coyote_HTTP_1_1_on_http_8080=0_nThe_file_or_directory_is_corrupted_and_unreadable=0_n,11713--12685,2
0,plm_tomcat_logcheck,1484756898,12865,/opt/apps/plm/logs/catalina.out,980K,plm_tomcat_logcheck,999773,Master_Item_Service_is_down=0_njava_lang_NoClassDefFoundError=0_njava_lang_OutOfMemoryError=0_nemxCommonAppInitialization__Error_while_initializing=0_nINFO__Stopping_Coyote_HTTP_1_1_on_http_8080=0_nThe_file_or_directory_is_corrupted_and_unreadable=0_n,12685--12865,8
0,plm_tomcat_logcheck,1484757197,13076,/opt/apps/plm/logs/catalina.out,996K,plm_tomcat_logcheck,1017418,Master_Item_Service_is_down=0_njava_lang_NoClassDefFoundError=0_njava_lang_OutOfMemoryError=0_nemxCommonAppInitialization__Error_while_initializing=0_nINFO__Stopping_Coyote_HTTP_1_1_on_http_8080=0_nThe_file_or_directory_is_corrupted_and_unreadable=0_n,12865--13076,0

or like this:

typeC
Code:
0,plm_tomcat_logcheck,1424392034,81033,/opt/apps/plm/logs/catalina.out,6.3M,plm_tomcat_logcheck,6539198,Master_Item_Service_is_down=0 java_lang_NoClassDefFoundError=0 java_lang_OutOfMemoryError=0 emxCommonAppInitialization__Error_while_initializing=0 INFO__Stopping_Coyote_HTTP_1_1_on_http_8080=0 The_file_or_directory_is_corrupted_and_unreadable=0,80801--81033
0,plm_tomcat_logcheck,1424392334,81307,/opt/apps/plm/logs/catalina.out,6.3M,plm_tomcat_logcheck,6561051,Master_Item_Service_is_down=0 java_lang_NoClassDefFoundError=0 java_lang_OutOfMemoryError=0 emxCommonAppInitialization__Error_while_initializing=0 INFO__Stopping_Coyote_HTTP_1_1_on_http_8080=0 The_file_or_directory_is_corrupted_and_unreadable=0,81033--81307
0,plm_tomcat_logcheck,1424392634,81367,/opt/apps/plm/logs/catalina.out,6.3M,plm_tomcat_logcheck,6565967,Master_Item_Service_is_down=0 java_lang_NoClassDefFoundError=0 java_lang_OutOfMemoryError=0 emxCommonAppInitialization__Error_while_initializing=0 INFO__Stopping_Coyote_HTTP_1_1_on_http_8080=0 The_file_or_directory_is_corrupted_and_unreadable=0,81307--81367

in the case of typeB and typeC, notice field 9 does not just contain a number. it contains a string and a 'equal-to' number. what i want to do is account for scenarios of typeA, typeB and typeC. meaning, add up the numbers in field 9, so that the resulting output looks similar to this:

Code:
0-1424534260_8--8
0-1424534560_8--8
0-1424534860_8--8
0-1413868231_12043--12043

# 2  
Old 01-18-2017
The output you present doesn't match any of the input files or types. Please specify exactly how to tell the type[ABC] from each other.
# 3  
Old 01-18-2017
the text of type[ABC] can be in the same datafile.

so when running the awk code on the data file, here are the different ways to differentiate a line that is being read:

Code:
1). 

How many fields does the line have, if there are 11 fields on a line, do xyz, if there are 10 fields on a line, do abc, if there are 13 fields on a line do cbd.  
if there are 14 fields on a line, do this and that.  the lines are all comma delimited. 

2). 

In the 9th field of each line, is there just a numerical number in it, or is there a few texts.  
If there are texts (or alphanumeric characters) in the 9th field, get the number directly in front of the equal sign for each pattern.  
And add up all the number so u get a total number.  if there is just a numerical number in the 9th field, no need to do any adding.

when the above is done, grab the epoch time from the 3rd field, and also the second to last field. so after each line is processed, the result should be:

${totalcountFrom9thfield}-${thirdfield}_${the-secondto-last-field-HOWEVER-if-the-second to last field does not contain the string "--", then grab the last field instead because that will have the "--"}
# 4  
Old 01-18-2017
Now THIS is precise a specification! Do you really want generic pseudo code as a solution proposal?

If you look into the code in post#1, it already reacts to the field count, and it calculates the numbers from $9, albeit with an algorithm different from what you describe. So - WHAT is the problem? Describe exactly and in detail, supply sample input line, its output and where it doesn't fulfill your needs.

BTW, how about trying to understand how the script works, and applying some corrections yourself?
This User Gave Thanks to RudiC For This Post:
SkySmart
# 5  
Old 01-18-2017
I added the text from typeA B and C to one datafile.

but when i run the code from post one on the combined datafile, i get the following:

Code:
 ./check
Usage: awk [POSIX or GNU style options] -f progfile [--] file ...
Usage: awk [POSIX or GNU style options] [--] 'program' file ...
POSIX options:          GNU long options: (standard)
        -f progfile             --file=progfile
        -F fs                   --field-separator=fs
        -v var=val              --assign=var=val
Short options:          GNU long options: (extensions)
        -b                      --characters-as-bytes
        -c                      --traditional
        -C                      --copyright
        -d[file]                --dump-variables[=file]
        -D[file]                --debug[=file]
        -e 'program-text'       --source='program-text'
        -E file                 --exec=file
        -g                      --gen-pot
        -h                      --help
        -i includefile          --include=includefile
        -l library              --load=library
        -L [fatal]              --lint[=fatal]
        -n                      --non-decimal-data
        -M                      --bignum
        -N                      --use-lc-numeric
        -o[file]                --pretty-print[=file]
        -O                      --optimize
        -p[file]                --profile[=file]
        -P                      --posix
        -r                      --re-interval
        -S                      --sandbox
        -t                      --lint-old
        -V                      --version

To report bugs, see node `Bugs' in `gawk.info', which is
section `Reporting Problems and Bugs' in the printed version.

gawk is a pattern scanning and processing language.
By default it reads standard input and writes standard output.

Examples:
        gawk '{ sum += $1 }; END { print sum }' file
        gawk -F: '{ print $1 }' /etc/passwd
./check: 19: ./check: NF-D[NF]: not found
./check: 21: ./check: NF-D[NF]+1: not found
./check: 14: ./check: BEGIN{           {D[10] = D[13] = 1
                 D[11] = D[14] = 2
                }

./check ~ , VF , {L = 1                                  # start output only if VALFOUND is matched
                }

!L              {next                                   # skip line if NOT VALFOUND
                }

                {A = strftime(%a: not found


i have tried playing with this many different ways. i suspect the issue may be related to the quotes but ive tried escaping them, adding them, removing them but the issue remains unresolved.

sorry if im making this complicated.
# 6  
Old 01-18-2017
Show your code, and then maybe we can tell you why it's not working.

Don't, and we'll never know.
# 7  
Old 01-18-2017
these are problems:
Code:
"BEGIN{
....
}"

use single quotes (for starters):
Code:
awk -F, -v SEARCHPATT="(Wed|Tue)" -v ADDISTR="Mon|Tue|Wed|Thu|Fri|Sat|Sun" -vVF="$VALFOUND" '
BEGIN {
....

}' datafile.txt

Last edited by vgersh99; 01-18-2017 at 04:15 PM..
This User Gave Thanks to vgersh99 For This Post:
SkySmart
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Help with fixing screen position

Hey guys, I am trying to make print a pattern with * on a 10*10 two dimensional array in a for loop and I want the incoming 10*10 to overlap the previous 10*10 so that the * look like it is moving. is there a way to fix the screen position? ever time it prints a 10*10 the screen moves. ... (3 Replies)
Discussion started by: amit14august
3 Replies

2. Shell Programming and Scripting

Fixing a shell script

I have this shell script that I wrote to check an input file to see if it is empty or not, and then clean the file from any line that starts with the sign "<" (without quotation marks" and then spell the number of line of the file, and the empty lines, too. The script then will create two output... (11 Replies)
Discussion started by: faizlo
11 Replies

3. AIX

Fixing security problem

Hi I use Rapid 7 to check some servers ( AIX 5.3 ) for security problems. There are 2 problems I don't know to deal with 1. Problem : TCP Sequence Number Approximation Vulnerability Solution : _Enable TCP MD5 Signature 2. Problem : HTTP Basic Authentication Enable Solution : _ Use... (5 Replies)
Discussion started by: bobochacha29
5 Replies

4. Homework & Coursework Questions

Help fixing my database script

1. The problem statement, all variables and given/known data: I need help I get a variant of syntax errors when compiling my script to maintain a database. It's a simple database meant to create/view/maintain vehicles. 2. Relevant commands, code, scripts, algorithms: my if statements have... (5 Replies)
Discussion started by: gamernerd101
5 Replies

5. Shell Programming and Scripting

Awk Problem - Fixing needed

os: sunos/linux shell: bash awk -v PCT="$PERCENTAGE" -v UWARN="$UWARNING" -v UCRIT="$UCRITICAL" 'BEGIN { PCT=PCT+0; UWARN=UWARN+0; UCRIT=UCRIT+0 ; if(PCT<UWARN) { printf \" '${FILESYS}':... (3 Replies)
Discussion started by: SkySmart
3 Replies

6. Shell Programming and Scripting

help fixing awk statement

awk "BEGIN {if($MessageREAD<$ThresholdW) {print \"OK\" ; exit 0} else if(($MessageREAD>=$ThresholdW) && ($MessageREAD<$ThresholdC)) {print \"WARNING\" ; exit 1}" else if($MessageREAD<=$ThresholdC) {print \"CRITICAL\" ;... (4 Replies)
Discussion started by: SkySmart
4 Replies

7. Shell Programming and Scripting

Fixing the width of a word

Is there a way to fix the width of the word being printed to a file? I am trying to create an output to a file with columns , like a spread sheet. I have used "\t" to adjust the columns but still it does not show well in the file, mainly due to the variable length values in the column so \t does... (1 Reply)
Discussion started by: davidtd
1 Replies

8. Shell Programming and Scripting

Help fixing awk code to print values from 2 files

Hi everyone, Please help on this: I have file1: <file title="Title 1 and 2"> <report> <title>Title 1</title> <number>No. 1234</number> <address>Address 1</address> <date>October 07, 2009</date> <description>Some text</description> </report> ... (6 Replies)
Discussion started by: Ophiuchus
6 Replies

9. Linux

fixing with sed

I am trying to replace the value of $f3 but its not working . I don't know what I am missing here . cat dim_copy.20080516.sql | grep -i "create view" | grep -v OPSDM002 | while read f1 f2 f3 f4 f5 f6 f7 f8 f9 do echo " $f3 " sed -e... (13 Replies)
Discussion started by: capri_drm
13 Replies
Login or Register to Ask a Question