Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Parsing a log file and creating a report script

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parsing a log file and creating a report script
# 1  
Old 08-10-2017
Parsing a log file and creating a report script
The log file is huge and lot of information, i would like to parse and make a report .

below is the log file looks like:

Code:
REPORT DATE: Mon Aug  10 04:16:17 CDT 2017
SYSTEN VER: v1.3.0.9
TERMINAL TYPE: prod
SYSTEM: nb11cu51
UPTIME:   04:16AM   up 182 days 57 mins min
MODEL, TYPE, and SN: IBM,9026-P70,01100699F
PROCESSOR TYPE: PowerPC_RS64-II
NUMBER OF PROCESSORS: 4





rmt0             P2/Z1-A5                 SCSI 8mm Tape Drive (20000 MB)

        Manufacturer................EXABYTE
        Machine Type and Model......IBM-20GB
        Device Specific.(Z1)........41eA
        Serial Number...............60171713
        Device Specific.(LI)........A0000004
        Part Number.................59H4117
        FRU Number..................59H4120
        EC Level....................E30396
        Device Specific.(Z0)........0180020283000030
        Device Specific.(Z3)........


rmt1             P2-I6/Z1-A0              IBM Magstar MP Tape Subsystem

        Manufacturer................IBM     
        Machine Type and Model......03570C11        
        Serial Number...............0000000A6844
        Device Specific.(FW)........544D
        Loadable Microcode Level....A0B00E27

  tmscsi1          P2-I6/Z1-A6              SCSI I/O Controller Initiator Device

        Device Specific.(Z0)........0300
        Manufacturer................IBM     
        Machine Type and Model......PCI SCSI-2UW-0  

  pci2             P2                       PCI Bus

        Device Specific.(YL)........P2

        Device Specific.(YL)........P2-I7/Q1

  ssa0             P2-I7/Q1                 IBM SSA 160 SerialRAID Adapter (14109100)

        Part Number................. 09L569B
        FRU Number.................. 34L5318
        Serial Number...............S0237219
        EC Level....................    F23699
        Manufacturer................IBM053
        ROS Level and ID............B800    0000
        Loadable Microcode Level....05
        Device Driver Level.........00
        Displayable Message.........SSA-ADAPTER
        Device Specific.(Z0)........SDRAM=064
        Device Specific.(Z1)........CACHE=00
        Device Specific.(Z2)........UID=006094BE000037F4
        Device Specific.(YL)........P2-I7/Q1

  ssa1             P2-I8/Q1                 IBM SSA 160 SerialRAID Adapter (14109100)

        Part Number................. 09L5695
        FRU Number.................. 34L5388
        Serial Number...............S0270187
        EC Level....................    F23699
        Manufacturer................IBM053
        ROS Level and ID............B800    0000
        Loadable Microcode Level....05
        Device Driver Level.........00
        Displayable Message.........SSA-ADAPTER
        Device Specific.(Z0)........SDRAM=064
        Device Specific.(Z1)........CACHE=



  rmt1             P2-I6/Z1-A0              IBM Magstar MP Tape Subsystem

        Manufacturer................IBM
        Machine Type and Model......03570C11
        Serial Number...............0000000A6844
        Device Specific.(FW)........544D
        Loadable Microcode Level....A0B00E27

  tmscsi1          P2-I6/Z1-A6              SCSI I/O Controller Initiator Device

        Device Specific.(Z0)........0300
        Manufacturer................IBM
        Machine Type and Model......PCI SCSI-2UW-0

  pci2             P2                       PCI Bus

        Device Specific.(YL)........P2

  ssa0             P2-I7/Q1                 IBM SSA 160 SerialRAID Adapter (14109100)






 hdisk0           P2/Z1-A9                 16 Bit SCSI Disk Drive (9100 MB)

        Manufacturer................IBM     
        Machine Type and Model......DNES-309170W    
        FRU Number..................25L3101     
        ROS Level and ID............53414755
        Serial Number...............AJJ55889
        EC Level....................F42017    
        Part Number.................25L1861     
        Device Specific.(Z0)........000003029F00013A
        Device Specific.(Z1)........25L2871     
        Device Specific.(Z2)........0933
        Device Specific.(Z3)........00176
        Device Specific.(Z4)........0001
        Device Specific.(Z5)........22
        Device Specific.(Z6)........F42036    

  hdisk1           P2/Z1-Aa                 16 Bit LVD SCSI Disk Drive (9100 MB)

        Manufacturer................SEAGATE     
        Machine Type and Model......DPSS-309170N    
        FRU Number..................07N3675     
        ROS Level and ID............53393347
        Serial Number...............ZD11B560
        EC Level....................F79851    
        Part Number.................07N3721     
        Device Specific.(Z0)........000003029F00013A
        Device Specific.(Z1)........07N4912     
        Device Specific.(Z2)........0933
        Device Specific.(Z3)........00231
        Device Specific.(Z4)........0001
        Device Specific.(Z5)........22
        Device Specific.(Z6)........F79851    

  pdisk0          USSAB540-D4              SSA160 Physical Disk Drive (9100 MB)

        Manufacturer................IBM     
        Machine Type and Model......DRVC09B
        Part Number.................34L8483     
        ROS Level and ID............0023
        Serial Number...............680BA636SA
        EC Level....................E32094    
        Device Specific.(Z2)........CUSHA023  
        Device Specific.(Z3)........34L8483     
        Device Specific.(Z4)........00068


pdisk01          USSAB540-D4              SSA160 Physical Disk Drive (9100 MB)

        Manufacturer................MAC     
        Machine Type and Model......DRVC09A
        Part Number.................34L8483     
        ROS Level and ID............0023
        Serial Number...............680BA636TT
        EC Level....................E32094    
        Device Specific.(Z2)........CUSHA023  
        Device Specific.(Z3)........34L8483     
        Device Specific.(Z4)........00068




The output should like this :

Code:
SYSTEM: nb11cu51
MODEL: IBM
TYPE: 7026-H70
SN:0110068BF
NUMBER OF PROCESSORS: 4
Total pdisk:2
Total hdisk:2
Total Tape Drive (rmt0,1): 2
SearialRAID adapeter (ssa0,1): 2


Device  Manufacturer   Machine Type Model     FRU Number     Serial Number     Part Number
hdisk0    IBM           DNES-309170W        25L3101        AJJ55889    25L1861
hdisk1  SEAGATE           DPSS-309170N        07N3675        ZD11B560    07N3721
pdisk0  IBM            DRVC09B            NA        680BA636SA      NA
pdisk01 MAC           DRVC09A            NA        680BA636TT    NA
rmt0    EXABYTE           IBM-20GB            59H4120        A0000004        59H4117
ssa0    09L569A           IBM053            34L5318     S0237219    09L569A

Last edited by amir07; 08-14-2017 at 04:28 PM.. Reason: Add CODE tags again.
# 2  
Old 08-10-2017
First...
Please note that learning to correctly use CODE tags is not that difficult..

Use Code Tags when you post any code or data samples so others can easily read your code. You can easily do this by highlighting your code and then clicking on the # in the editing menu. (You can also type code tags [code] and [/code] by hand.) The following tutorial demonstrates how to do this.



And then...
  1. What operating system and shell are you using?
  2. What have you tried to solve this problem on your own?
  3. Where are you stuck?
  4. Do the devices in your output have to be sorted, or can the output be in the same order as your input?
This User Gave Thanks to Don Cragun For This Post:
rbatte1
# 3  
Old 08-14-2017
IBM AIX 5.3
shell script
device needs to be sorted
i will put my code what I have
Last edited by Don Cragun; 08-14-2017 at 04:33 PM.. Reason: Remove unneeded ICODE tags.
# 4  
Old 08-14-2017
Please only put sample input, sample output, and code segments in CODE tags; not everything in your post.

We look forward to seeing what code you currently have, the output it produces from your sample input, and a description of where you are stuck trying to get the output you want.
# 5  
Old 08-14-2017
How are devices selected for the report? rmt1 exists twice but is listed once only, ssa0 is reported, but ssa1 is not. Why should we ignore tmscsi1?
# 6  
Old 08-15-2017
This is a very old system (if i had to guess i'd say 15+ years) and i am astonished that it even runs AIX 5.3. My old 43P won't run anything newer than 5.2. SSA (IBMs idea of obsoleting SCSI) went out of fashion about the turn of the century. (Btw.: i don't know the "9026-P70" in your source report but the "7026-H70" from your sample output. Might it be that your cut&paste didn't work well?)

AS it seems this an output from either the lscfg -vp command or from prtconf (not sure if this was already present in 5.3). Wouldn't it be easier - instead of parsing this output - to use the lsdev and lsattr commands to get exactly the specifications you want to know about every individual device?

For this, of course, one would need to know what exactly you want to see in your report. Right now this is only vaguely defined.

I hope this helps.

bakunin
Last edited by bakunin; 08-15-2017 at 05:39 AM..
# 7  
Old 08-16-2017
The reason tmscsi ignored because there is no manufacturer name, serial number and part number.
ssa1 also need to be reported.
all devices with manufacturer name, seriall number and part number need to be reported!

---------- Post updated at 09:30 AM ---------- Previous update was at 09:11 AM ----------

Code:
for h in nb11cu51
do
ssh root@$h lsattr -Elsys0 | awk -v h=$h '
echo { print "Device  Manufacturer   Machine_Type Model     FRU_Number     Serial_Number     Part_Number" }
$1=="Manufacturer   "{s=$2}
$1=="Manufacturer"{m=$2}
$1=="Model"{n=$2}
$1=="Fru_Number"{o=$2}
$1=="Serial_number"{p=$2}
$1=="Part_Number"{q=$2}
END {print h,m,s,n,o,q}'
done

I could not figure out the heading detail of the report and counts.

---------- Post updated at 09:31 AM ---------- Previous update was at 09:30 AM ----------

Code:
for h in nb11cu51
do
ssh root@$h lsattr -Elsys0 | awk -v h=$h '
echo { print "Device  Manufacturer   Machine_Type Model     FRU_Number     Serial_Number     Part_Number" }
$1=="Manufacturer   "{s=$2}
$1=="Manufacturer"{m=$2}
$1=="Model"{n=$2}
$1=="Fru_Number"{o=$2}
$1=="Serial_number"{p=$2}
$1=="Part_Number"{q=$2}
END {print h,m,s,n,o,q}'
done

I could not figure out the heading detail of the report and counts.


Moderator's Comments:
Mod Comment Thanks for trying, but youve been TOO generous with the ICODE tags.
Please use CODE tags as required by forum rules!
Last edited by RudiC; 08-16-2017 at 12:01 PM.. Reason: Changed copious amounts of ICODE to single CODE tags.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Issue with awk script parsing log file

Hello All, I am trying to parse a log file and i got this code from one of the good forum colleagues, However i realised later there is a problem with this awk script, being naive to awk world wanted to see if you guys can help me out. AWK script: awk '$1 ~ "^WRITER_" {p=1;next}... (18 Replies)
Discussion started by: Ariean
18 Replies

2. Shell Programming and Scripting

Shell script for creating log file and sending mail?

Hi , I am trying to create shell script which will help me to compare file name in two folder. There is a multiple file store in 2 folder.I want to compare that with the name. If all the file are same then send a mail that "all date is same" if not then create one log file which contain... (4 Replies)
Discussion started by: san_dy123
4 Replies

3. Shell Programming and Scripting

Parsing with Name value pair and creating a normalized file

I have url string as follows and I need to parse the name value pair into fields /rows event_id date time payload 1329130951 20120214 22.30.40... (1 Reply)
Discussion started by: smee
1 Replies

4. Shell Programming and Scripting

Script for Parsing Log File

Working on a script that inputs an IP, parses and outputs to another file. A Sample of the log is as follows: I need the script to be able to input IP and print the data in an output file in the following format or something similar: Thanks for any help you can give me! (8 Replies)
Discussion started by: Winsarc
8 Replies

5. UNIX for Dummies Questions & Answers

Creating a report from csv file.

Hi Gurus, I need your help in transforming the CSV file into some what a report format. My source file looks like below Date,ProdID,TimeID,LevelID 2010-08-31,200,M,1 2010-08-31,201,Q,2 2010-08-31,202,Y,1 2010-08-31,203,M,5 Output required is ... (9 Replies)
Discussion started by: naveen.kuppili
9 Replies

6. Shell Programming and Scripting

Parsing of file for Report Generation (String parsing and splitting)

Hey guys, I have this file generated by me... i want to create some HTML output from it. The problem is that i am really confused about how do I go about reading the file. The file is in the following format: TID1 Name1 ATime=xx AResult=yyy AExpected=yyy BTime=xx BResult=yyy... (8 Replies)
Discussion started by: umar.shaikh
8 Replies

7. UNIX for Dummies Questions & Answers

Script for parsing details in a log file to a seperate file

Hi Experts, Im a new bee for scripting, I would ned to do the following via linux shell scripting, I have an application which throws a log file, on each action of a particular work with the application, as sson as the action is done, the log file would vanish or stops updating there, the... (2 Replies)
Discussion started by: pingnagan
2 Replies

8. Shell Programming and Scripting

Help with script parsing a log file

I have a large log file, which I want to first use grep to get the specific lines then send it to awk to print out the specific column and if the result is zero, don't do anything. What I have so far is: LOGDIR=/usr/local/oracle/Transcription/log ERRDIR=/home/edixftp/errors #I want to be... (3 Replies)
Discussion started by: mevasquez
3 Replies

9. Shell Programming and Scripting

Shell script for parsing 300mb log file..

am relatively new to Shell scripting. I have written a script for parsing a big file. The logic is: Apart from lot of other useless stuffs, there are many occurances of <abc> and corresponding </abc> tags. (All of them are properly closed) My requirement is to find a particular tag (say... (3 Replies)
Discussion started by: gurpreet470
3 Replies

10. Shell Programming and Scripting

Help with a shell script for creating a log file

I have a schell script that runs continously on an AIX system. It is actually started from another shell script with the "ksh -x" command and then I just write the output to a log file. This causes the log files to be filled with mostly useless information. I would like to modify this script to... (2 Replies)
Discussion started by: heprox
2 Replies
Login or Register to Ask a Question