Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

processing the output of AWK

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting processing the output of AWK
# 1  
Old 10-04-2011
processing the output of AWK
Hi

my input file is
Code:
<so   > < Time  >      <Pid>  <some ro><Job Name> 
111004 04554447      26817  JOB03275 MBPDVLOI
111004 04554473      26817  JOB03275 MBPDVLOI
111004 04554778      26807  JOB03276 MBPDVAWD
111004 04554779      26807  JOB03276 MBPDVAWD
111004 04554780      26817  JOB03276 MBPDVAWD
111004 04554783      26817  JOB03276 MBPDVAWD
111004 04555113      26807  JOB03277 MBPDD090
111004 04555116      26807  JOB03277 MBPDD090
111004 04555117      26817  JOB03277 MBPDD090
111004 04555159      26817  JOB03277 MBPDD090
111004 04555447      26807  JOB03278 MBPDD201
111004 04555449      26807  JOB03278 MBPDD201

the First occurrence of <time> for <Job Name> is starting time
the Fourth occurrence of <time> for <Job Name> is end time of that job

i want to calculate the difference between start time and end time for all the jobs
Note : Every job has exactly 4 entries where 1st entry denotes start time and 4th denotes the end time , there is no rule that all the four occurrences are consecutive

i have tried the
Code:
awk 'x[$2,$5]++' FS=" " file.txt

i dont know how to catch each occurrences to variables so that we can process in script

please help me
Last edited by vbe; 10-04-2011 at 09:27 AM.. Reason: code tags please
# 2  
Old 10-04-2011
Code:
$ cat testfile
#!/bin/bash
for i in `awk '{print $NF|"sort|uniq"}' infile`
do
        echo "$i -- `sed -n "/$i/,/$i/p" infile | sed -n '1p;$p' | awk '{print $2|"xargs"}' | awk '{print $2-$1}'`"
done
$ sh testfile
MBPDD090 -- 46
MBPDD201 -- 42
MBPDVAWD -- 5
MBPDVLOI -- 26
$

# 3  
Old 10-04-2011
You can do something like that :
Code:
awk '
    {
        Times[$5,++Job[$5]] = $2;
    }
    END {
        for (j in Job) {
           printf "%-8s  : %s\n", j, (Job[j] >= 4 ? Times[j,4]-Times[j,1] : "?");
        }
    }
' file.txt

Output for your sample file :
Code:
MBPDD201  : ?
MBPDD090  : 46
MBPDVLOI  : ?
MBPDVAWD  : 5

Jean-Pierre.
These 2 Users Gave Thanks to aigles For This Post:
rakeshkumar vgersh99
# 4  
Old 10-04-2011
solution is amazing .... thanks a ton aigles

---------- Post updated at 03:19 PM ---------- Previous update was at 02:50 PM ----------

can anyone please explain the below multi dimensional array (if am not wrong) , i tried but could not get exactly how its working
Times[$5,++Job[$5]] = $2;
# 5  
Old 10-04-2011
What you have here are two arrays:

Job[] uses Jobname as an index and counts how many lines have been seen for a job sofar.
Times[] has and index of JobName + Line sequence (from Job[]) and stores the Time value (field 2).

When processing line 6 of your input file the arrays will be as follows:
Code:
Job[MBPDVLOI]=2
Job[MBPDVAWD]=4

Code:
Times[MBPDVLOI,1]=04554447
Times[MBPDVLOI,2]=04554473
Times[MBPDVAWD,1]=04554778
Times[MBPDVAWD,2]=04554779
Times[MBPDVAWD,3]=04554780
Times[MBPDVAWD,4]=04554783

This User Gave Thanks to Chubler_XL For This Post:
rakeshkumar
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

awk - Rename output file, after processing, same as input file

I have one input file ABC.txt and one output DEF.txt. After the ABC is processed and created output, I want to rename ABC.txt to ABC.orig and DEF to ABC.txt. Currently when I am doing this, it does not process the input file as it cannot read and write to the same file. How can I achieve this? ... (12 Replies)
Discussion started by: High-T
12 Replies

2. Programming

awk processing / Shell Script Processing to remove columns text file

Hello, I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this : This is the output of ls command : I stored the output in a file filelist 1.1M... (5 Replies)
Discussion started by: ajayram
5 Replies

3. Shell Programming and Scripting

Processing diff output

How to get diff to not print the chevrons and the dashes? In this case the differences are all single line differences. Also the first few lines don't matter. How to get the output to always exclude the first few lines? Thanks! (1 Reply)
Discussion started by: stevensw
1 Replies

4. Shell Programming and Scripting

processing with awk

I have many lines like the following in a file(there are also other kinds of lines) Host: 72.52.104.74 (tserv1.fmt2.he.net) Ports: 22/open/tcp//tcpwrapped///, 53/open/tcp//domain//PowerDNS 3.3/, 179/open/tcp//tcpwrapped/// Ignored State: closed (997) Seq Index: 207 IP ID Seq: All... (9 Replies)
Discussion started by: esolvepolito
9 Replies

5. UNIX for Dummies Questions & Answers

Take output of processing in text file

Hi ALL, I am presently using perl script mukesh.pl I just want to catch its output into another text file . So I am using > File.txt . I am getting output but i want the whole processing of the script into that file please let me know . Thanks in advance Cheers Mukesh (1 Reply)
Discussion started by: mumakhij
1 Replies

6. Shell Programming and Scripting

processing db2 output

db2 command produces this output: Database 1 entry: Database alias = DB1 Database name = DB1 Node name = ND1 Database release level = a.00 Comment = Directory entry type ... (5 Replies)
Discussion started by: valero
5 Replies

7. UNIX for Dummies Questions & Answers

single output of awk script processing multiple files

Helllo UNIX Forum :) Since I am posting on this board, yes, I am new to UNIX! I read a copy of "UNIX made easy" from 1990, which felt like a making a "computer-science time jump" backwards ;) So, basically I have some sort of understanding what the basic concept is. Problem Description:... (6 Replies)
Discussion started by: Kasimir
6 Replies

8. Shell Programming and Scripting

Writing output into different files while processing file using AWK

Hi, I am trying to do the following using AWK program. 1. Read the input data file 2. Parse the record and see if it contains errors 3. If the record contains errors, then write it into Reject file, else, write into usual output file or display it on the screen Here is what I have done -... (6 Replies)
Discussion started by: vidyak
6 Replies

9. IP Networking

processing tcpflow output

I'm using tcpflow to analyze traffic traces. When I launch tcpflow with -r option it creates some files, one for each flow. The problem is that some of these files are not readable. I tryed to process them with awk, but also using it i cannot visualize them correctly. Can anyone suggest me a... (5 Replies)
Discussion started by: littleboyblu
5 Replies

10. Shell Programming and Scripting

processing tab-formated output of command w/bash

I have a command that when ran it will have an output such as string LongerString string2 longerString2 More MoreStrings seperated by tabs. The command lists domains and their accounts set up in my server admin software (interworx). The end result will be that it will run rsync for... (2 Replies)
Discussion started by: sweede
2 Replies
Login or Register to Ask a Question