Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Pattern Matching and creating output

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Pattern Matching and creating output
# 1  
Old 03-17-2016
Pattern Matching and creating output
HI Unix Forum,

My requirement

I have two set of Patterns UBA and CIE for which different Phases are there which will have Start and End time. They are not in same order.

I want the o/p in the below mentioned format.

Quote:
UBA
Phase 0 Start_Time End_Time Time_Consumed(End_Time -Start_Time)
upto
Phase X Start_Time End_Time Time_Consumed(End_Time -Start_Time)

CIE
Phase 0 Start_Time End_Time Time_Consumed(End_Time -Start_Time)
upto
Phase Y Start_Time End_Time Time_Consumed(End_Time -Start_Time)
Eg: Mangolia Alien 03:04:56 Phase 0 started (10 seconds)
In the above stmt --- 03:04:36 is Start_Time for Phase0


Hope my Requirement is clear.

Below is an Sample LOg --- That may not make sense as I have changed to protect the data.

Code:
 Log (This log is not exact but just to indicate the pattern)
  <<Any number of lines inbetween>>
[YYYY-MM-DD hh:mm:ss] Standard Output for '/Task: CIE':
--------------------------------------------------------------------------------
Mangolia Alien 03:04:56   Phase 0 started (10 seconds)
 <<Any number of lines inbetween>>
 --------------------------------------------------------------------------------
[YYYY-MM-DD 03:04:58] Standard Output for '/Task: CIE':
--------------------------------------------------------------------------------
Mangolia Alien 03:04:58   Phase 1 started (0 seconds)
 <<Any number of lines inbetween>>
--------------------------------------------------------------------------------
[YYYY-MM-DD 03:05:07] Standard Output for '/Task: CIE':
--------------------------------------------------------------------------------
Mangolia Alien 03:05:07   Phase 2 started (7 seconds)
 <<Any number of lines inbetween>>
--------------------------------------------------------------------------------
[YYYY-MM-DD 03:05:12] Standard Output for '/Task: UBA':
--------------------------------------------------------------------------------
Mangolia Alien 03:05:12   Phase 0 started (14 seconds)
 <<Any number of lines inbetween>>
--------------------------------------------------------------------------------
[YYYY-MM-DD 03:05:16] Standard Output for '/Task: CIE':
Mangolia Alien 03:05:16   Phase 2 ended (16 seconds)
 <<Any number of lines inbetween>>
--------------------------------------------------------------------------------
[YYYY-MM-DD 03:05:19] Standard Output for '/Task: UBA':
Mangolia Alien 03:05:19   Phase 0 ended (21 seconds)
 <<Any number of lines inbetween>>
--------------------------------------------------------------------------------
[YYYY-MM-DD 03:05:20] Standard Output for '/Task: UBA':
--------------------------------------------------------------------------------
Mangolia Alien 03:05:20   Phase 1 started (1 second)
 <<Any number of lines inbetween>>
[YYYY-MM-DD 03:05:21] Standard Output for '/Task: UBA':
Mangolia Alien 03:05:21   Phase 1 ended (2 seconds)
 <<Any number of lines inbetween>>
--------------------------------------------------------------------------------
[YYYY-MM-DD 03:05:23] Standard Output for '/Task: CIE':
--------------------------------------------------------------------------------
Mangolia Alien 03:05:23   Phase 3 started (4 seconds)
 <<Any number of lines inbetween>>
--------------------------------------------------------------------------------
[YYYY-MM-DD 03:05:25] Standard Output for '/Task: CIE':
Mangolia Alien 03:05:25   Phase 3 ended (6 seconds)
 <<Any number of lines inbetween>>
--------------------------------------------------------------------------------
[YYYY-MM-DD 03:05:27] Standard Output for '/Task: CIE':
--------------------------------------------------------------------------------
Mangolia Alien 03:05:27   Phase 4 started (1 second)
 <<Any number of lines inbetween>>
--------------------------------------------------------------------------------
[YYYY-MM-DD 03:05:29] Standard Output for '/Task: UBA':
--------------------------------------------------------------------------------
Mangolia Alien 03:05:29   Phase 2 started (6 seconds)
 <<Any number of lines inbetween>>
--------------------------------------------------------------------------------
[YYYY-MM-DD 03:05:31] Standard Output for '/Task: CIE':
Mangolia Alien 03:05:31   Phase 4 ended (4 seconds)
 <<Any number of lines in between>>
--------------------------------------------------------------------------------
[YYYY-MM-DD 03:05:32] Standard Output for '/Task: UBA':
Mangolia Alien 03:05:32   Phase 2 ended (8 seconds)
 <<Any number of lines inbetween>>

Very urgent... please lets discuss and find a good solution.

Note:
I tried grep -B3 and grep -A3 to first separate UBA and CIE into two separate files and then fetch respective start and end time.

But -B and -A option is not there in AIX


Thanks,
TechGyaann

---------- Post updated at 06:30 PM ---------- Previous update was at 04:55 PM ----------

Can someone look into and assist!!

@Don Cragun or any other Mods please comment.
# 2  
Old 03-20-2016
How about this:

Code:
awk -F "[ \\\]]*" '
function difftime(st,et) {
   split(st,sa,":")
   split(et,ea,":")
   return "(" ea[1]*3600-sa[1]*3600 + ea[2]*60-sa[2]*60 + ea[3]-sa[3] " seconds)"
}
/Task: / { T=$NF; gsub(/['\'':]*$/,"",T) }
/Phase .*started/ { TL[T];TS[T,$5]=$3}
/Phase .*ended/ { TE[T,$5]=$3}
END{
  for(task in TL) {
      print task
      for(phase=0; (task SUBSEP phase) in TS; phase++) {
         if((task SUBSEP phase) in TE)
         print "Phase " phase, TS[task,phase], TE[task,phase],\
               difftime(TS[task,phase],TE[task,phase])
         else print "Phase " phase, TS[task,phase], "Unfinished"
      }
  }
} ' OFS=" " infile


Output:
Code:
CIE
Phase 0 03:04:56 Unfinished
Phase 1 03:04:58 Unfinished
Phase 2 03:05:07 03:05:16 (9 seconds)
Phase 3 03:05:23 03:05:25 (2 seconds)
Phase 4 03:05:27 03:05:31 (4 seconds)
UBA
Phase 0 03:05:12 03:05:19 (7 seconds)
Phase 1 03:05:20 03:05:21 (1 seconds)
Phase 2 03:05:29 03:05:32 (3 seconds)

This User Gave Thanks to Chubler_XL For This Post:
TechGyaann
# 3  
Old 03-29-2016
Thanks @Chubler_XL.
I will go through the code. try to execute it and comment more on it. Smilie
# 4  
Old 03-29-2016
May I doubt that making a problem "very urgent" (although this is highly deprecated here!), bumping it up as well, and then not coming back for a week should be considered good style?
# 5  
Old 03-29-2016
Chubler_XL, Can you please explain me so that I will understand it.

Eg: https://www.unix.com/302963081-post9.html
# 6  
Old 03-29-2016
Code:
function difftime(st,et) {
   split(st,sa,":")
   split(et,ea,":")
   return "(" ea[1]*3600-sa[1]*3600 + ea[2]*60-sa[2]*60 + ea[3]-sa[3] " seconds)"
}

difftime(starttime, endtime) - function takes two times in format "hh:mm:ss" and returns "(Y seconds)" where Y is the number of seconds endtime is ahead of starttime.

Code:
/Task: / { T=$NF; gsub(/['\'':]*$/,"",T) }

Matches lines that contain "Task: " eg:
[YYYY-MM-DD 03:04:58] Standard Output for '/Task: CIE':

T is assigned to last field of line ("CIE':" for example)
gsub() call replaces ' and : chars with nothing in T (giving "CIE").

Note as the awk program was quoted with ' we have to close the quoted string then escape a single ' with '\ and then start a new quoted string, hence '\'' is required to get a single ' in the code.

Code:
/Phase .*started/ { TL[T];TS[T,$5]=$3}

Matches line containing "Phase " followed by "started" eg:
Code:
Mangolia Alien 03:05:23   Phase 3 started (4 seconds)

TL[T] build an array TL[] with all task strings
TS[T, $5] build a 2 dimensional array TS with Task,phase# as the index and time as the value

Code:
/Phase .*ended/ { TE[T,$5]=$3}

build a 2nd 2 dimensional array TE with Task,phase# as index and as the index and time as the value

Code:
END{
  for(task in TL) {
      print task
      for(phase=0; (task SUBSEP phase) in TS; phase++) {
         if((task SUBSEP phase) in TE)
         print "Phase " phase, TS[task,phase], TE[task,phase],\
               difftime(TS[task,phase],TE[task,phase])
         else print "Phase " phase, TS[task,phase], "Unfinished"
      }
  }
}

After the file has been passed go through each task stored in the TL array

for(phase=0; (task SUBSEP phase) in TS; phase++)
starting with phase=0 loop while a phase exists in the TS[] (task start) array for our current task, increment the phase number at the end of each loop.

if their is an entry in TE[] (task end) for this task,phase then print start and end time and call difftime() to display seconds.
otherwise display start time and "Unfinished"
This User Gave Thanks to Chubler_XL For This Post:
TechGyaann
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Expect pattern matching in the command output

This is the command output need to be matched: Telnet console listening to port 42365. (the port number changes every time) Code to test it: ======================================= #!/tools/AGRtools/bin/expect exp_internal 1 set timeout 10 spawn bash set bashId $spawn_id ... (4 Replies)
Discussion started by: marsala
4 Replies

2. UNIX for Dummies Questions & Answers

Grep -v lines starting with pattern 1 and not matching pattern 2

Hi all! Thanks for taking the time to view this! I want to grep out all lines of a file that starts with pattern 1 but also does not match with the second pattern. Example: Drink a soda Eat a banana Eat multiple bananas Drink an apple juice Eat an apple Eat multiple apples I... (8 Replies)
Discussion started by: demmel
8 Replies

3. Shell Programming and Scripting

PHP - Regex for matching string containing pattern but without pattern itself

The sample file: dept1: user1,user2,user3 dept2: user4,user5,user6 dept3: user7,user8,user9 I want to match by '/^dept2.*/' but don't want to have substring 'dept2:' in output. How to compose such regex? (8 Replies)
Discussion started by: urello
8 Replies

4. Shell Programming and Scripting

Sed: printing lines AFTER pattern matching EXCLUDING the line containing the pattern

'Hi I'm using the following code to extract the lines(and redirect them to a txt file) after the pattern match. But the output is inclusive of the line with pattern match. Which option is to be used to exclude the line containing the pattern? sed -n '/Conn.*User/,$p' > consumers.txt (11 Replies)
Discussion started by: essem
11 Replies

5. UNIX for Dummies Questions & Answers

Find pattern suffix matching pattern

Hi, I am trying to get a result out of this but fails please help. Have two files /tmp/1 & /tmp/hosts. /tmp/1 IP=123.456.789.01 WAS_HOSTNAME=abcdefgh.was.tb.dsdc /tmp/hosts 123.456.789.01 I want this result in /tmp/hosts if hostname is already there dont want duplicate entry. ... (5 Replies)
Discussion started by: rajeshwebspere
5 Replies

6. Shell Programming and Scripting

Creating single pattern for matching multiple files.

Hi friends, I have a some files in a directory. for example 856-abc 856-def 851-abc 945-def 956-abc 852-abc i want to display only those files whose name starts with 856* 945* and 851* using a single pattern. i.e 856-abc 856-def 851-abc 945-def the rest of the two files... (2 Replies)
Discussion started by: Little
2 Replies

7. Shell Programming and Scripting

Creating number by pattern matching

I have a certain mnemonic string from which I want to calculate a number The pattern follows three letters s, v and d. If a letter is by its own, the number assigned to the letter is assumed to be one. Else it takes the value preceeding it. I then need to add the numbers together. Example ... (5 Replies)
Discussion started by: kristinu
5 Replies

8. Shell Programming and Scripting

sed - matching pattern one but not pattern two

All, I have the following file: -------------------------------------- # # /etc/pam.d/common-password - password-related modules common to all services # # This file is included from other service-specific PAM config files, # and should contain a list of modules that define the services... (2 Replies)
Discussion started by: RobertBerrie
2 Replies

9. Shell Programming and Scripting

counting the lines matching a pattern, in between two pattern, and generate a tab

Hi all, I'm looking for some help. I have a file (very long) that is organized like below: >Cluster 0 0 283nt, >01_FRYJ6ZM12HMXZS... at +/99% 1 279nt, >01_FRYJ6ZM12HN12A... at +/99% 2 281nt, >01_FRYJ6ZM12HM4TS... at +/99% 3 283nt, >01_FRYJ6ZM12HM946... at +/99% 4 279nt,... (4 Replies)
Discussion started by: d.chauliac
4 Replies

10. Shell Programming and Scripting

comment/delete a particular pattern starting from second line of the matching pattern

Hi, I have file 1.txt with following entries as shown: 0152364|134444|10.20.30.40|015236433 0233654|122555|10.20.30.50|023365433 ** ** ** In file 2.txt I have the following entries as shown: 0152364|134444|10.20.30.40|015236433 0233654|122555|10.20.30.50|023365433... (4 Replies)
Discussion started by: imas
4 Replies
Login or Register to Ask a Question