Pattern Matching and printing

Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Pattern Matching and printing
# 1  
Old 07-29-2011
Pattern Matching and printing

Dear All,
I have a log file like below
13:26:31 |152.22
13:27:31 |154.25
13:28:31 |154.78
13:29:31 |151.23
13:30:31 |145.63
13:31:31 |142.10
13:32:31 |145.45

where values will be there from 00:00 hrs to 23:59 hrs. I'm matching for last occurance of 23:59 and printing 1440 lines (grep -B1440 "23:59").. But if values are missing from 00:00 hrs to 2:00 hrs... previous day values also gets printed.

How to avoid this prob???

Last edited by Scott; 07-29-2011 at 06:49 AM.. Reason: Code tags
# 2  
Old 07-29-2011
Well, you have an unreliable data set, so you need to analyze it without any speific time. You could use awk or shell to find each pair of lines when time folds back, an apparent day boundary. (Some like to log the date like "YYYY-MM-DD_HH:MM:SS ($$) ...." in every line, so this situation is prevented.) This finds the boundary lines:
while read tm pval
 ntm=${tm%%:*}${xtm%:*}${xtm#*:}   # make time tm look like an integer ntm
 if (( lntm > ntm ))   # is last numeric time greater (fold back)?
  echo $ltm $lpval
  echo ----
  echo $tm $pval
  echo . . .
 lntm=ntm ltm=tm lpval=pval
done <$in_file

Last edited by DGPickett; 07-29-2011 at 02:51 PM..
This User Gave Thanks to DGPickett For This Post:
# 3  
Old 07-29-2011
Little more explanation will be helpful.... whether pval is a command?
# 4  
Old 08-01-2011
pval is a shell variable, payload value. read puts values in variables, using the shell's usual $IFS characters as separators, and putting excess values, at least with ksh but it varies, in the last variable, so "read a b" is read first field into $a and the rest into $b. My personal mental nomenclature is that record fields are, at any given oment, either keys (needed to address this record) or payload (stuff that serves the application but not this support function, along for the ride. The key is part of the delivery mechanism, like the space shuttle, and the rest is cargo, goes in the bay to follow the key to the destination.

Now, an alphanumeric compare can compare times without removing the :, but I was lazy and had not gotten into researching string compare, a sin in the Internet Age, so now we both learn. It seems "[[ $a < $b ]]" works fine, especially with such bland strings. Depending on shell and maybe locale, '[[ "A" < "A" ]]' and '[[ "3" < "c" ]]' might not be what yoou think. The shell has several parallel comparison suites:
  • '[...]' is 'man test' -- while often built-in, for old sh, '[' is /bin/[, a link to /bin/test,
  • '[[ ... ]]' is 'man ksh - Conditional Expressions' does string comparison,
  • (( ... )) is 'man ksh - Arithmetic evaluation' with C-like operators, double precision floating point arithmetic or long double precision floating point, supports nice things like variable without '$' and '+=' but unless bash not '++'.
So, the narrative flow is:
  1. Prime the last numeric time value lntm
  2. Start a 'while read' loop, reads lines until EOF into variable(s), sometimes losing some white space, but a very useful shell bit for low latency, no-upper-list-size-limit, one line at a time processing.
  3. xtm is an intermediate variable, as the ksh $var slicers #, ##, %, %% are a bit crude.
  4. I manke numeric time ntm by concatenation of substrings of tm and xtm. Maybe I could have used a numeric substring slicer, like bash and ksh93 have '${var:start:len}', since the time strings are well behaved. If they were 1-2 digit variable width, I would need to multiply and add them, not concatenate.
  5. If the last numeric time is greater, this is a day boundary.
  6. then spit out a reconstituted last line and new line with divider and elipsis.
  7. load the current line variables as last line variables
  8. loop back. Redirect stdin to input file.
Revised as noted for string compare:
while read tm pval
if [[ $ltm > $ntm ]]   # is last time greater (fold back)?
  echo $ltm $lpval
  echo ----
  echo $tm $pval
  echo . . .
ltm=tm lpval=pval
done <$in_file

Last edited by DGPickett; 08-01-2011 at 11:13 AM..
These 2 Users Gave Thanks to DGPickett For This Post:
# 5  
Old 08-01-2011
Thanks a lot !!!!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need help on pattern matching and printing the same

Hi, I need to match for the pattern '.py' in my file and print the word which contains. For example: cat testfile a b 3 5 6 a 4 5 6 7 8 3 4 5 6 Expected output: (3 Replies)
Discussion started by: Sumanthsv
3 Replies

2. Shell Programming and Scripting

UNIX awk pattern matching and printing lines

I have the below plain text file where i have some result, in order to mail that result in html table format I have written the below script and its working well. cat result.txt Page 2015-01-01 2000 Colors 2015-02-01 3000 Landing 2015-03-02 4000 #!/bin/sh LOG=/tmp/maillog.txt... (1 Reply)
Discussion started by: close2jay
1 Replies

3. UNIX for Dummies Questions & Answers

Grep -v lines starting with pattern 1 and not matching pattern 2

Hi all! Thanks for taking the time to view this! I want to grep out all lines of a file that starts with pattern 1 but also does not match with the second pattern. Example: Drink a soda Eat a banana Eat multiple bananas Drink an apple juice Eat an apple Eat multiple apples I... (8 Replies)
Discussion started by: demmel
8 Replies

4. Shell Programming and Scripting

PHP - Regex for matching string containing pattern but without pattern itself

The sample file: dept1: user1,user2,user3 dept2: user4,user5,user6 dept3: user7,user8,user9 I want to match by '/^dept2.*/' but don't want to have substring 'dept2:' in output. How to compose such regex? (8 Replies)
Discussion started by: urello
8 Replies

5. Shell Programming and Scripting

Sed: printing lines AFTER pattern matching EXCLUDING the line containing the pattern

'Hi I'm using the following code to extract the lines(and redirect them to a txt file) after the pattern match. But the output is inclusive of the line with pattern match. Which option is to be used to exclude the line containing the pattern? sed -n '/Conn.*User/,$p' > consumers.txt (11 Replies)
Discussion started by: essem
11 Replies

6. UNIX for Dummies Questions & Answers

Find pattern suffix matching pattern

Hi, I am trying to get a result out of this but fails please help. Have two files /tmp/1 & /tmp/hosts. /tmp/1 IP=123.456.789.01 WAS_HOSTNAME=abcdefgh.was.tb.dsdc /tmp/hosts 123.456.789.01 I want this result in /tmp/hosts if hostname is already there dont want duplicate entry. ... (5 Replies)
Discussion started by: rajeshwebspere
5 Replies

7. Shell Programming and Scripting

Help With AWK Matching and Re-printing Lines

Hi All, I'm looking to use AWK to pattern match lines in XML file - Example patten for below sample would be /^<apple>/ The sample I wrote out is very basic compared to what I am actually working with but it will get me started I would like to keep the matched line(s) unchanged but have them... (4 Replies)
Discussion started by: rhoderidge
4 Replies

8. UNIX for Dummies Questions & Answers

Pattern matching and Printing Filename

Hi, My requirement is to search for a paritcular string from a group of .gz files and to print the lines containing that string and the name of the files in which that string is present. Daily 500 odd .gz files will be generated in a directory(directory name will be in the form of... (4 Replies)
Discussion started by: krao
4 Replies

9. Shell Programming and Scripting

counting the lines matching a pattern, in between two pattern, and generate a tab

Hi all, I'm looking for some help. I have a file (very long) that is organized like below: >Cluster 0 0 283nt, >01_FRYJ6ZM12HMXZS... at +/99% 1 279nt, >01_FRYJ6ZM12HN12A... at +/99% 2 281nt, >01_FRYJ6ZM12HM4TS... at +/99% 3 283nt, >01_FRYJ6ZM12HM946... at +/99% 4 279nt,... (4 Replies)
Discussion started by: d.chauliac
4 Replies

10. Shell Programming and Scripting

comment/delete a particular pattern starting from second line of the matching pattern

Hi, I have file 1.txt with following entries as shown: 0152364|134444||015236433 0233654|122555||023365433 ** ** ** In file 2.txt I have the following entries as shown: 0152364|134444||015236433 0233654|122555||023365433... (4 Replies)
Discussion started by: imas
4 Replies
Login or Register to Ask a Question