Extract fields before search pattern


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Extract fields before search pattern
# 1  
Old 08-24-2015
Extract fields before search pattern

Hi,

I have below file structure and need to display hours, minutes and seconds as different fields.

Incase hour or minute field is not there it should default to zero.

Code:
*** Total elapsed time was 2 hours, 54 minutes and 40 seconds.
*** Total elapsed time was 42 minutes and 36 seconds.
*** Total elapsed time was 42 minutes and 57 seconds.
*** Total elapsed time was 42 minutes and 55 seconds.
*** Total elapsed time was one hour, one minute and 44 seconds.
*** Total elapsed time was 1 second.
*** Total elapsed time was 8 seconds.
*** Total elapsed time was 3 minutes and 0 second.

Every line may have different formatting as mentioned above.

Basically what i am planning to do is to put hour/hours in $HR, minute/minutes in $MIN and second/seconds in $SEC

There could be singular/plural form of hour, minute and second as shown above.

Thanks

Last edited by Don Cragun; 08-24-2015 at 07:28 PM.. Reason: Add CODE and ICODE tags.
# 2  
Old 08-24-2015
Is this a homework assignment? If so, please refile you question in the Homework and coursework forum and include a completely filled out homework template as described here.

It is isn't homework, please give us much more details about the environment you're using. What operating system are you using? What shell are you using? What programming language are you using for this assignment?

First you say you want to display hours minutes and seconds (which I assumed to be numeric values (although I'm not sure what you would want from the input one hour, one minute and 44 seconds). Then you say you want to set $HR to "hour" or "hours", etc.

What code have you tried to solve this problem?

What output are you trying to produce from the sample input you provided?
# 3  
Old 08-24-2015
Nope its not a home assignment. I have a log file which has details for various elapsed time for different SQL. Want to convert these into seconds and insert it into a table along with their elapsed time details in seconds.

The OS is: RedHat/6.3/x86_64 . Making use of bash shell. As i am not so great in shell programming posted this question. Also before this tired to look into various others threads on this forum to see for close match in search pattern

Used something like this but it did not help

Code:
cat Canary1.log | grep -o -P '.{0,4}second' | sed -e 's/ seconds//g' -e 's/ second//g' -e 's/ //g'

Code:
grep -i "elapsed time" Canary.log | tail -1 | sed 's/.*\(.\) second\(.\).*/\1/'

---------- Post updated at 06:31 PM ---------- Previous update was at 05:54 PM ----------

I have finally came up with this. Which i think suffices my requirement. Dont know if this is the best possible solution though.

Second:
Code:
cat Canary1.log | grep -o -P '.{0,3}second' | sed -e 's/[a-z]//g' -e 's/ //g'

Hour:
Code:
cat Canary1.log | grep -o -P '.{0,4}hour' | sed -e 's/one/1/g' -e 's/[a-z]//g' -e 's/ //g'

minute:
Code:
cat Canary1.log | grep -o -P '.{0,4}minute' | sed -e 's/one/1/g' -e 's/[a-z]//g' -e 's/ //g'

For your thoughs

Last edited by Corona688; 08-25-2015 at 01:58 PM..
# 4  
Old 08-25-2015
I don't see that the three command lines you have produced the output you said you wanted. (There is nothing there that fills in 0 hours or 0 minutes when those values do not appear in the corresponding input lines.)

With a standards conforming implementation of the sed utility, the following command works:
Code:
sed 's/.*was /0 0 /
s/ and//
s/one/1/g
s/second.*//
s/minute[^[:space:]]* //
s/hour[^[:space:]]* //
s/.*\([0-9][0-9]* [0-9][0-9]* [0-9][0-9]*\)/\1/' Canary1.log

producing the output:
Code:
2 54 40 
0 42 36 
0 42 57 
0 42 55 
1 1 44 
0 0 1 
0 0 8 
0 3 0

from your sample input.

On Linux systems (or if you're using the the GNU utilities version of sed on another system), you might need to add a --posix long option to get the final substitute command to work properly (i.e.:
Code:
sed --posix 's/.*was /0 0 /
s/ and//
s/one/1/g
s/second.*//
s/minute[^[:space:]]* //
s/hour[^[:space:]]* //
s/.*\([0-9][0-9]* [0-9][0-9]* [0-9][0-9]*\)/\1/' Canary1.log

).
# 5  
Old 08-25-2015
Try also
Code:
sed -r 's/^/0 0 /; s/one/1/g; s/.*([0-9]+)[^0-9]+([0-9]+)[^0-9]+([0-9]+)[^0-9]+$/\1 \2 \3/' file
2 54 40
0 42 36
0 42 57
0 42 55
1 1 44
0 0 1
0 0 8
0 3 0

To work on the respective variables, (in a recent shell) try sth like
Code:
 while read HR MIN SEC REST; do echo $HR $MIN $SEC; done  <<< "$(sed -r 's/^/0 0 /; s/one/1/g; s/.*([0-9]+)[^0-9]+([0-9]+)[^0-9]+([0-9]+)[^0-9]+$/\1 \2 \3/' file)"

# 6  
Old 08-25-2015
Thanks for all the help on this...
# 7  
Old 08-30-2015
Quote:
Originally Posted by Don Cragun
With a standards conforming implementation of the sed utility, the following command works:
Code:
sed 's/.*was /0 0 /
s/ and//
s/one/1/g
s/second.*//
s/minute[^[:space:]]* //
s/hour[^[:space:]]* //
s/.*\([0-9][0-9]* [0-9][0-9]* [0-9][0-9]*\)/\1/' Canary1.log

This will not work if the hours and seconds are given but the minutes are missing. For instance, from the sample given:

Code:
echo "*** Total elapsed time was 42 minutes and 57 seconds." | yoursed
0 42 57

works as expected, but:

Code:
echo "*** Total elapsed time was 42 hours and 57 seconds." | yoursed
0 42 57

will get it wrong. If this could be the case in the logfile is everybodies guess, but it would be within the given specification.

Here is a script which will cover this:

Code:
sed --posix -n '
h
:hours
 /hour/ ! {
            s/.*/0 /
            b minutes
          }
s/hour.*/hour/
s/^.*[^0-9]\([0-9][0-9]*\)[     ]*hour$/\1 /
:minutes
 G
 /minute/ ! {
                s/^\([0-9][0-9]*\).*/\1 0/
                b seconds
            }
 s/minute.*/minute/
 s/^\([0-9][0-9]* \).*[^0-9]\([0-9][0-9]*\) minute$/\1 \2/
:seconds
 G
 /second/ ! {
                s/^\([0-9]* [0-9]*\).*/\1 0/
                b end
            }
 s/second.*/second/
 s/^\([0-9][0-9]* [0-9][0-9]*\).*[^0-9]\([0-9][0-9]*\) second$/\1 \2/
:end
p'

The inner workings is:

First, the line is copied to hold space. Then, if the line has "hour" in it, the amount of hours is extracted by removing everythng else from the line, otherwise the line is reduced to "0 ". At the next step (minutes), the content from the hold space is appended to this and the process repeated (with minimal changes to cover for the hour-number already on the line), the same a third time for seconds.

I hope this helps.

bakunin
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extract whole word preceding a specific character pattern with first occurence of the pattern

Hello. Here is a file contents : declare -Ax NEW_FORCE_IGNORE_ARRAY=(="§" ="§" ="§" ="§" ="§" .................. ="§"Here is a pattern =I want to extract 'NEW_FORCE_IGNORE_ARRAY' which is the whole word before the first occurrence of pattern '=' Is there a better solution than mine :... (3 Replies)
Discussion started by: jcdole
3 Replies

2. UNIX for Beginners Questions & Answers

How to extract fields from a CSV i.e comma separated where some of the fields having comma as value?

can anyone help me!!!! How to I parse the CSV file file name : abc.csv (csv file) The above file containing data like abv,sfs,,hju,',',jkk wff,fst,,rgr,',',rgr ere,edf,erg,',',rgr,rgr I have a requirement like i have to extract different field and assign them into different... (4 Replies)
Discussion started by: J.Jena
4 Replies

3. UNIX for Beginners Questions & Answers

Grep/awk using a begin search pattern and end search pattern

I have this fileA TEST FILE ABC this file contains ABC; TEST FILE DGHT this file contains DGHT; TEST FILE 123 this file contains ABC, this file contains DEF, this file contains XYZ, this file contains KLM ; I want to have a fileZ that has only (begin search pattern for will be... (2 Replies)
Discussion started by: vbabz
2 Replies

4. Shell Programming and Scripting

Search for a pattern in a String file and count the occurance of each pattern

I am trying to search a file for a patterns ERR- in a file and return a count for each of the error reported Input file is a free flowing file without any format example of output ERR-00001=5 .... ERR-01010=10 ..... ERR-99999=10 (4 Replies)
Discussion started by: swayam123
4 Replies

5. Shell Programming and Scripting

Search for a pattern,extract value(s) from next line, extract lines having those extracted value(s)

I have hundreds of files to process. In each file I need to look for a pattern then extract value(s) from next line and then search for value(s) selected from point (2) in the same file at a specific position. HEADER ELECTRON TRANSPORT 18-MAR-98 1A7V TITLE CYTOCHROME... (7 Replies)
Discussion started by: AshwaniSharma09
7 Replies

6. Shell Programming and Scripting

Need one liner to search pattern and print everything expect 6 lines from where pattern match made

i need to search for a pattern from a big file and print everything expect the next 6 lines from where the pattern match was made. (8 Replies)
Discussion started by: chidori
8 Replies

7. Shell Programming and Scripting

Print a pattern between the xml tags based on a search pattern

Hi all, I am trying to extract the values ( text between the xml tags) based on the Order Number. here is the sample input <?xml version="1.0" encoding="UTF-8"?> <NJCustomer> <Header> <MessageIdentifier>Y504173382</MessageIdentifier> ... (13 Replies)
Discussion started by: oky
13 Replies

8. Shell Programming and Scripting

extract specific line if the search pattern is found

Hi, I need to extract <APPNUMBER> tag alone, if the <college> haas IIT Chennai value. college tag value will have spaces embedded. Those spaces should not be suppresses. My Source file <Record><sno>1</sno><empid>E0001</empid><name>Rejsh suderam</name><college>IIT ... (3 Replies)
Discussion started by: Sekar1
3 Replies

9. Shell Programming and Scripting

sed: Find start of pattern and extract text to end of line, including the pattern

This is my first post, please be nice. I have tried to google and read different tutorials. The task at hand is: Input file input.txt (example) abc123defhij-E-1234jslo 456ujs-W-abXjklp From this file the task is to grep the -E- and -W- strings that are unique and write a new file... (5 Replies)
Discussion started by: TestTomas
5 Replies

10. Shell Programming and Scripting

search a pattern and if pattern found insert new pattern at the begining

I am trying to do some thing like this .. In a file , if pattern found insert new pattern at the begining of the line containing the pattern. example: in a file I have this. gtrow0unit1/gctunit_crrownorth_stage5_outnet_feedthru_pin if i find feedthru_pin want to insert !! at the... (7 Replies)
Discussion started by: pitagi
7 Replies
Login or Register to Ask a Question