Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk, sed or similar log repair help

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk, sed or similar log repair help
# 1  
Old 08-13-2008
awk, sed or similar log repair help
I have a log file that for some reason, once or two time a month, line foods are missing.

This log is generated from vmstat everyminute. I dont know why sometimes it does this.

Each line in the log should have 18 columns separated by one or more spaces.

Good Log: (not actual log)
Code:
1 1 1 123456 1234 123 1 1 1 100
1 2 1 123 1234 1234 1 1 1 10
2 1 1 123 12345 123 1 1 1 3
1 1 3 123 1234 123 1 1 1 1

bad log
Code:
1 1 1 123456 1234 123 1 1 1 100 1 2 1 123 1234 1234 1 1 1 10 
2 1 1 123 12345 123 1 1 1 3
1 1 3 123 1234 123 1 1 1 1

There could be more than 2 rows on one line.

If I know there will ALWAYS be 18 columns how acan I quickly go though the log and IF there is more than 18 columns put a linefeed in and rewite the log?

thanks...
# 2  
Old 08-13-2008
Are you trying to do this while the logfile is being written to by vmstat?
# 3  
Old 08-13-2008
Code:
awk '{ for(i=1; i <=NF; i++) 
        {
        printf("%s ", $i)
        if ( !(i%18) ) { printf("\n") }
        }
     }'  logfile

try something like this - assuming the file is not being written to....
# 4  
Old 08-13-2008
No this is done the following day. So it is not being written to anymore.

another caveat

there are headers every few rows. so I need to ignore those (anything with ANYthing other than 0-9.

But these rows with letters are NOT appended to any of the numeric only rows
# 5  
Old 08-13-2008
There are alot of rows put together in this log so I get this:

In this particular row there are 360 fields. That is actally the most there will ever be.

Code:
1 1 0 617894 2987960 8 1 0 0 0 0 0 1069 4232 217 0 0 99
1 1 0 662317 2987993 9 2 0 0 0 0 0 1099 5219 254 0 0 99
1 1 0 624608 2987993 16 1 0 0 0 0 0 1191 89839 338 1 1 98
awk: Line     1     1     0    cannot have more than 199 fields.
 The input line number is 711. The file is vmstat.12.
 The source line number is 1.



Quote:
Originally Posted by jim mcnamara
Code:
awk '{ for(i=1; i <=NF; i++) 
        {
        printf("%s ", $i)
        if ( !(i%18) ) { printf("\n") }
        }
     }'  logfile

try something like this - assuming the file is not being written to....
# 6  
Old 08-13-2008
Curious as to what vmstat command is throwing off the output altogether and what platform are you on?
From the error message it looks like a job for Perl as awk has an upper limit on the number of fields it can process.
# 7  
Old 08-13-2008
This is running on HP-UX 11.11 and 11.23

This is what we run via cron:

Code:
vmstat 60 1440 > /path/to/vmstat.`date +%d`

This usually works fine. I dont know what is causing the linefeeds to disappear.

Part of the log will be correct then it will start having the problem and then it sometimes starts working correctly again.

I can do perl, was hoping awk or something would be very quick and easy.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Vlookup using awk non similar files

I need to vlookup and check the server not found. Source file 1 server1 server2 server3 server4 server5_root server6_silver server7 server7-test server7-temp Source file 2 server1_bronze (6 Replies)
Discussion started by: ranjancom2000
6 Replies

2. Shell Programming and Scripting

Optimizing for loop with awk or anything similar and portable

The variable COUNTPRO contains: COUNTPRO='Error__posting__message__to__EMR__Queue=0 Error__parsing__ReceiptSummary=0 xinetd__=4327 HTTP__1_1__500___=0 START__=2164 Marshaller__exception__while__converting__to__Receipt__xml=0 MessagePublisher__is__not__configured__correctly=0... (9 Replies)
Discussion started by: SkySmart
9 Replies

3. Shell Programming and Scripting

Add values of similar patterns with awk

so my output is this: session_closed=157 session_opened=151 session_closed=18 session_opened=17 there are two patterns here, but with different values. the two patterns are "session_opened" and "session_closed". i expect there will be many more other patterns. what i want to do is... (8 Replies)
Discussion started by: SkySmart
8 Replies

4. Shell Programming and Scripting

Isolate text with sed or similar utility

All, I'm getting a list like the following and I'd like to kill each PID in turn. pid (17797) pid (21748) pid (21754) pid (21704) pid (2199) pid (2159) pid (17809) pid (21769) pid (21778) pid (21715) ... (3 Replies)
Discussion started by: ejianu
3 Replies

5. Answers to Frequently Asked Questions

Why Parsing Can't be Done With sed ( or similar tools)

Regularly we have questions like: i have an XML (C, C++, ...) file with this or that property and i want to extract the content of this or that tag (function, ...). How do i do it in sed? Yes, in some (very limited) cases this is possible, but in general this can't be done. That is: you can do... (0 Replies)
Discussion started by: bakunin
0 Replies

6. Shell Programming and Scripting

Editing files with sed or something similar

{ "AFafa": "FAFA","AFafa": "FAFA" "baseball":"soccer","wrestling":"dancing" "rhinos":"crocodiles","roles":"foodchain" } I need to insert a new line before the closing brackets "}" so that the final output looks like this: { "AFafa": "FAFA","AFafa": "FAFA"... (6 Replies)
Discussion started by: SkySmart
6 Replies

7. AIX

Change Log Repair Action without using diag?

Hi Gurus, I want to change Log Repair Action remotely, or in a script. In these cases, the dial menu can not be used. Is there any command to do the job? Thank you in advance! (4 Replies)
Discussion started by: aixlover
4 Replies

8. Shell Programming and Scripting

Help with awk or something similar

i have a file like this: wedd01A1 1 wedd01A2 2 wedd01A3 1 wedd02A2 3 wedd02A3 4 wadd02A1 1 wadd02A2 5 wqdd01A1 3 wsdd01A3 1 i want out like this: A1 A2 A3 wedd01 1 2 1 wedd02 0 3 4 wadd02 1 5 0 wqdd01 3 0 0 wsdd01 0 0 1 (8 Replies)
Discussion started by: aydj
8 Replies

9. Shell Programming and Scripting

Help in grep function or similar using awk

I have a list of id; for example: file 1 dfghd dfghe dfgey dfgeu I have another data file that contain this ids as headers; for ex. file2 >dfghd gfdgfddl;klfkld;ld;lgl;dld'l'dv >dfghe gkwhjhsgdjdjdjhjddj >dfgey jdkjfhdjhfdkjhfdkhkdk I wanted to compare file 1 and file 2... (1 Reply)
Discussion started by: Lucky Ali
1 Replies

10. Shell Programming and Scripting

Using sed (or similar) to rename variable headings

Hello, I'm rather new to the world of regular expressions and sed, though am excited by its possibilities. I have a particular task I'd like to achieve, and have googled the topic quite a bit. However, having found some codes that perform a task very similar to what I'd like to do, I can't for... (2 Replies)
Discussion started by: redseventyseven
2 Replies
Login or Register to Ask a Question