Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

AWK failing to parse on certain files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting AWK failing to parse on certain files
# 1  
Old 09-26-2011
AWK failing to parse on certain files
Dear Unix Gurus,

need your expertise to help troubleshoot a certain problem i'm having. I crated a shell script which will ftp get 1 crash log from multiple servers (listed in a text file). Each log will then be parsed by calling an awk script. The problem is, for certain log its parsing perfectly, and for some, its not parsing at all.

I have checked the input file, they are FTPed without any error into desired input directory. Disk space is also not a problem. However the AWK is throwing out below errors :

Code:
$ awk -f crash.awk /home/icpapp/scripts/crash/input/crash.OM5 /home/icpapp/scripts/crash/output/crash.OM5
awk: Cannot find or open file /home/icpapp/scripts/crash/output/crash.OM5.
 The input line number is 42440. The file is /home/icpapp/scripts/crash/output/crash.OM5.
 The source line number is 1.

SmilieSmilie

below is my shell script :

Code:
#!/bin/sh

APPHOME=$HOME/scripts/crash

for lines in `cat $APPHOME/crash.servers`
do
  #echo $lines
  SERVER=`echo $lines | cut -f1 -d=`
  IP=`echo $lines | cut -f2 -d=`
  #echo $IP
  cd $APPHOME
  echo "Getting crash log from : "$SERVER
  $APPHOME/ftp.sh "$IP"
  mv crash.txt $APPHOME/input/crash.$SERVER
  echo "Input file : "$APPHOME/input/crash.$SERVER
  echo "Output file : "$APPHOME/output/crash.$SERVER
  awk -f crash.awk $APPHOME/input/crash.$SERVER $APPHOME/output/crash.$SERVER
  if [ $? -eq 0 ]; then
    echo "Exit status : Success"
  else
    echo "Exit Status : Error"
  fi
echo "----------------------------------------"
done

exit 0

And below is my awk script :

Code:
BEGIN {
scope=0
crashinfo=""
pidtid=""
FS=" "
}

{

if ($0 == "-")
{
  scope++
}

if ($1 == "Process")
{
  scope++
}

if ($1 == "PROCESS")
{
  scope++
}

if (substr($0,0,5) == " /app")
{
  if (scope == 4)
  {
    crashinfo=$0
    scope++
  }
}

if ($1 == "Itanium" && $2 == "Register" && $3 == "dump" && $4 == "for")
{
  if (scope == 5)
  {
    pidtid=$5" "$6" "$7" "$8
    print pidtid"|"crashinfo >> ARGV[2]
    scope=0
  }
}

}

END {

}

I'm not sure if i should put in my put my input file in here as it is VERY huge. However, it would look like something like this :

Code:
-
Process 3120, thread 0x0000005e
-
PROCESS CRASHED AFTER RECEIVING SIGNAL - 11!!

 /app/mw/lib/libkernel32.so +0x138370 = MwThread(void*) +0x310 at 0xe44c4370
 /usr/lib/hpux32/libpthread.so.1 +0x2a040 = __pthread_bound_body +0x170 at 0xe3db5040

        Itanium Register dump for Process 3120  Thread 94

  ip: 0000005300000050    um: 12

gr00: 0000000000000000 0000005200000059 60000000cfbbf1a4 60000000cfbbf1a0
gr04: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
gr08: 0000005300000054 0000000000001370 00000000000032c8 0000000000000000
gr12: 60000000cfbbf1d0 000000000e283e40 0000000000e42658 60000000cfbbf1cc
gr16: 60000000cfbbf1d8 0000000000000000 60000000cfbbf174 60000000cfbbf170
gr20: 60000000cfbbf16c 60000000cfbbf168 60000000cfbbf198 60000000cfbbf194
gr24: 60000000cfbbf190 60000000cfbbf18c 60000000cfbbf188 60000000cfbbf184
gr28: 60000000cfbbf180 60000000cfbbf17c 60000000cfbbf178 60000000cfbbf1a8
gr32: 0000000000000000 0000000000000000 ................ 60000000e2e83400
gr36: ................ 0000000000b2c8f0

and it goes on and on for each crash occurrence.

This is the output when i run the full script
Code:
Getting crash log from : ISS1
Interactive mode off.
Input file : /home/icpapp/scripts/crash/input/crash.ISS1
Output file : /home/icpapp/scripts/crash/output/crash.ISS1
Exit status : Success
----------------------------------------
Getting crash log from : ISS2
Interactive mode off.
Input file : /home/icpapp/scripts/crash/input/crash.ISS2
Output file : /home/icpapp/scripts/crash/output/crash.ISS2
Exit status : Success
----------------------------------------
Getting crash log from : ISS3
Interactive mode off.
Input file : /home/icpapp/scripts/crash/input/crash.ISS3
Output file : /home/icpapp/scripts/crash/output/crash.ISS3
Exit status : Success
----------------------------------------
Getting crash log from : ISS4
Interactive mode off.
Input file : /home/icpapp/scripts/crash/input/crash.ISS4
Output file : /home/icpapp/scripts/crash/output/crash.ISS4
Exit status : Success
----------------------------------------
Getting crash log from : ISS5
Interactive mode off.
Input file : /home/icpapp/scripts/crash/input/crash.ISS5
Output file : /home/icpapp/scripts/crash/output/crash.ISS5
Exit status : Success
----------------------------------------
Getting crash log from : OM1
Interactive mode off.
Input file : /home/icpapp/scripts/crash/input/crash.OM1
Output file : /home/icpapp/scripts/crash/output/crash.OM1
Exit status : Success
----------------------------------------
Getting crash log from : OM2
Interactive mode off.
Input file : /home/icpapp/scripts/crash/input/crash.OM2
Output file : /home/icpapp/scripts/crash/output/crash.OM2
Exit status : Success
----------------------------------------
Getting crash log from : OM3
Interactive mode off.
Input file : /home/icpapp/scripts/crash/input/crash.OM3
Output file : /home/icpapp/scripts/crash/output/crash.OM3
Exit status : Success
----------------------------------------
Getting crash log from : OM4
Interactive mode off.
Input file : /home/icpapp/scripts/crash/input/crash.OM4
Output file : /home/icpapp/scripts/crash/output/crash.OM4
Exit status : Success
----------------------------------------
Getting crash log from : OM5
Interactive mode off.
Input file : /home/icpapp/scripts/crash/input/crash.OM5
Output file : /home/icpapp/scripts/crash/output/crash.OM5
awk: Cannot find or open file /home/icpapp/scripts/crash/output/crash.OM5.
 The input line number is 42440. The file is /home/icpapp/scripts/crash/output/crash.OM5.
 The source line number is 1.
Exit Status : Error
----------------------------------------
Getting crash log from : OM6
Interactive mode off.
Input file : /home/icpapp/scripts/crash/input/crash.OM6
Output file : /home/icpapp/scripts/crash/output/crash.OM6
awk: Cannot find or open file /home/icpapp/scripts/crash/output/crash.OM6.
 The input line number is 2548. The file is /home/icpapp/scripts/crash/output/crash.OM6.
 The source line number is 1.
Exit Status : Error
----------------------------------------

it only errors out on the last 2 log servers. Below are the input file details :
Code:
$ ll
total 86816
-rw-rw-rw-   1 icpapp     icpapp     4603273 Sep 26 22:28 crash.ISS1
-rw-rw-rw-   1 icpapp     icpapp     4756421 Sep 26 22:28 crash.ISS2
-rw-rw-rw-   1 icpapp     icpapp     4229522 Sep 26 22:28 crash.ISS3
-rw-rw-rw-   1 icpapp     icpapp     4240746 Sep 26 22:28 crash.ISS4
-rw-rw-rw-   1 icpapp     icpapp      121643 Sep 26 22:28 crash.ISS5
-rw-rw-rw-   1 icpapp     icpapp        8192 Sep 26 22:28 crash.OM1
-rw-rw-rw-   1 icpapp     icpapp     3365032 Sep 26 22:28 crash.OM2
-rw-rw-rw-   1 icpapp     icpapp     3368729 Sep 26 22:28 crash.OM3
-rw-rw-rw-   1 icpapp     icpapp     3014289 Sep 26 22:28 crash.OM4
-rw-rw-rw-   1 icpapp     icpapp     3351178 Sep 26 22:28 crash.OM5
-rw-rw-rw-   1 icpapp     icpapp      129630 Sep 26 22:28 crash.OM6


your help in this matter is greatly appreciated Smilie
# 2  
Old 09-26-2011
Code:
awk -f crash.awk $APPHOME/input/crash.$SERVER $APPHOME/output/crash.$SERVER

should be
Code:
awk -f crash.awk $APPHOME/input/crash.$SERVER > $APPHOME/output/crash.$SERVER

if you meant to write to the output file instead of reading it as input.
# 3  
Old 09-27-2011
hi binlib,

Thanks for your reply, but i put it that way because i use the second argument as an input argument, where i will use it as output file in the awk script ARGV[2]
# 4  
Old 09-27-2011
Quote:
Originally Posted by tarj
hi binlib,

Thanks for your reply, but i put it that way because i use the second argument as an input argument, where i will use it as output file in the awk script ARGV[2]
but it will still be read as input.. unless you adjust ARGC and delete ARGV[2] in BEGIN {}
# 5  
Old 09-27-2011
Hi neutronscott,

yes it will be treated as input argument, where inside the awk script will be used in :

Code:
print pidtid"|"crashinfo >> ARGV[2]

# 6  
Old 09-27-2011
It will also be treated as an input file, not just an argument.
# 7  
Old 09-27-2011
See The AWK Manual - Other Arguments
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parse input of two files to be the same in awk

I have two files that I am going to use diff to find the differences but need to parse them before I do that. I have include the format of each file1 and file2 with the desired output of each (the first 5 fields in each file). The first file has a "chr" before the # that needs to be removed. I... (1 Reply)
Discussion started by: cmccabe
1 Replies

2. Shell Programming and Scripting

awk Parse And Create Multiple Files Based on Field Value

Hello: I am working parsing a large input file which will be broken down into multiples based on the second field in the file, in this case: STORE. The idea is to create each file with the corresponding store number, for example: Report_$STORENUM_$DATETIMESTAMP , and obtaining the... (7 Replies)
Discussion started by: ec012
7 Replies

3. Shell Programming and Scripting

awk conditions failing (special characters?)

This is really frustrating because I can't figure it out. I'm running a health check script. One of the items I'm checking is the amount of memory on a server. I use the free command, which outputs something like this (excerpt) Mem: 100 100 100 100 Swap: 100 100 100 100 In my debugging... (5 Replies)
Discussion started by: JustaDude
5 Replies

4. Shell Programming and Scripting

Using awk to Parse File

Hi all, I have a file that contains a good hundred of these job definitions below: Job Name Last Start Last End ST Run Pri/Xit ________________________________________________________________ ____________________... (7 Replies)
Discussion started by: atticuss
7 Replies

5. Shell Programming and Scripting

awk to parse huge files

Hello All, I have a situation as below: (1) Read a source file (a single file of 1.2 million rows in it ) (2) Read Destination files one by one and replace the content ( few fields in it ) with the corresponding matching field from source file. I tried as below: ( please note I am not... (4 Replies)
Discussion started by: panyam
4 Replies

6. Shell Programming and Scripting

Parse a file using awk

Hi Experts, I am trying to parse the following file; FILEA a|b|c|c|c|c a|b|d|d|d|d e|f|a|a|a|a e|f|b|b|b|boutput expected: a<TAB>b <TAB><TAB>c<TAB>c<TAB>c<TAB>c<TAB> <TAB><TAB>d<TAB>d<TAB>d<TAB>d<TAB> e<TAB>f <TAB><TAB>a<TAB>a<TAB>a<TAB>a<TAB> <TAB><TAB>b<TAB>b<TAB>b<TAB>b<TAB>*... (7 Replies)
Discussion started by: rajangupta2387
7 Replies

7. UNIX for Dummies Questions & Answers

AWK w/ if statements failing.

I'm converting some code from ksh on my macbook (Version M 1993-12-28 s+) to an older solaris machine with ksh 88. I can't seem to figure out this line, it worked on the new shell version. set -A combo -- $(for x in ${ImageIDs}; do nawk -v s=$x 'if($2 == s) getline ; getline if ($1 ==... (2 Replies)
Discussion started by: nerdcurious
2 Replies

8. Shell Programming and Scripting

awk script to parse results from TWO files

I am trying to parse two files and get data that does not match in one of the columns ( column 3 in my case ) Data for two files are as follows A.txt ===== abc 10 5 0 1 16 xyz 16 1 1 0 18 efg 30 8 0 2 40 ijk 22 2 0 1 25 B.txt ===== abc... (6 Replies)
Discussion started by: roger67
6 Replies

9. Shell Programming and Scripting

AWK: Parse lvdisplay

Hi, I would like to parse the next output of an lvdisplay -v using awk: --- Logical volumes --- LV Name /dev/vg01/lvol3 VG Name /dev/vg01 LV Permission read/write LV Status available/syncd Mirror copies ... (2 Replies)
Discussion started by: RuBiCK
2 Replies

10. Shell Programming and Scripting

Parse file using awk and work in awk output

hi guys, i want to parse a file using public function, the file contain raw data in the below format i want to get the output like this to load it to Oracle DB MARWA1,BSS:26,1,3,0,0,0,0,0.00,22,22,22.00 MARWA2,BSS:26,1,3,0,0,0,0,0.00,22,22,22.00 this the file raw format: Number of... (6 Replies)
Discussion started by: dagigg
6 Replies
Login or Register to Ask a Question