Parsing file, reading each line to variable, evaluating date/time stamp of each line


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Parsing file, reading each line to variable, evaluating date/time stamp of each line
# 1  
Old 12-05-2011
Parsing file, reading each line to variable, evaluating date/time stamp of each line

So, the beginning of my script will cat & grep a file with the output directed to a new file. The data I have in this file needs to be parsed, read and evaluated.

Basically, I need to identify the latest date/time stamp and then calculate whether or not it is within 15 minutes of the current date/time at run time.

My logic:

-read each line into an array using a while loop
-create a variable for each date/time stamp per line
-compare the date/time stamp from one line to the next to determine the most recent
-keep evaluating dat/time stamps until EOF
-once most recent date/time stamp has been identified, check to see that it is within 15 minutes of current date/time
- print message to screen "check complete, time is within 15 minutes" .. OR .. "check failed, time exceeds 15 minutes"


here is the test data set:
Code:
20111205-06:00:16.968|Info|Other|ALLOCPRODSTG|31659|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(94)|8=FIX.4.09=7235=A34=149=FPCASH0152=20111205-11:00:1656=ALLOCCASHSTG98=0108=60010=227
20111205-06:10:17.939|Info|Other|ALLOCPRODSTG|31657|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(81)|8=FIX.4.09=5935=034=249=FPCASH0152=20111205-11:10:1756=ALLOCCASHSTG10=142
20111205-06:20:18.937|Info|Other|ALLOCPRODSTG|31659|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(81)|8=FIX.4.09=5935=034=349=FPCASH0152=20111205-11:20:1856=ALLOCCASHSTG10=145
20111205-06:30:18.938|Info|Other|ALLOCPRODSTG|31658|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(81)|8=FIX.4.09=5935=034=449=FPCASH0152=20111205-11:30:1856=ALLOCCASHSTG10=147
20111205-06:40:18.944|Info|Other|ALLOCPRODSTG|31658|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(81)|8=FIX.4.09=5935=034=549=FPCASH0152=20111205-11:40:1856=ALLOCCASHSTG10=149
20111205-06:50:19.945|Info|Other|ALLOCPRODSTG|31656|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(81)|8=FIX.4.09=5935=034=649=FPCASH0152=20111205-11:50:1956=ALLOCCASHSTG10=152
20111205-07:00:19.948|Info|Other|ALLOCPRODSTG|31656|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(81)|8=FIX.4.09=5935=034=749=FPCASH0152=20111205-12:00:1956=ALLOCCASHSTG10=149
20111205-07:10:19.949|Info|Other|ALLOCPRODSTG|31656|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(81)|8=FIX.4.09=5935=034=849=FPCASH0152=20111205-12:10:1956=ALLOCCASHSTG10=151
20111205-07:20:20.944|Info|Other|ALLOCPRODSTG|31657|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(81)|8=FIX.4.09=5935=034=949=FPCASH0152=20111205-12:20:2056=ALLOCCASHSTG10=145
20111205-07:30:21.941|Info|Other|ALLOCPRODSTG|31659|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=1049=FPCASH0152=20111205-12:30:2156=ALLOCCASHSTG10=179
20111205-07:40:22.941|Info|Other|ALLOCPRODSTG|31657|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=1149=FPCASH0152=20111205-12:40:2256=ALLOCCASHSTG10=182
20111205-07:50:23.943|Info|Other|ALLOCPRODSTG|31657|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=1249=FPCASH0152=20111205-12:50:2356=ALLOCCASHSTG10=185
20111205-08:00:23.942|Info|Other|ALLOCPRODSTG|31656|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=1349=FPCASH0152=20111205-13:00:2356=ALLOCCASHSTG10=182
20111205-08:10:23.938|Info|Other|ALLOCPRODSTG|31658|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=1449=FPCASH0152=20111205-13:10:2356=ALLOCCASHSTG10=184
20111205-08:20:24.939|Info|Other|ALLOCPRODSTG|31656|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=1549=FPCASH0152=20111205-13:20:2456=ALLOCCASHSTG10=187
20111205-08:30:25.939|Info|Other|ALLOCPRODSTG|31658|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=1649=FPCASH0152=20111205-13:30:2556=ALLOCCASHSTG10=190
20111205-08:40:26.938|Info|Other|ALLOCPRODSTG|31657|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=1749=FPCASH0152=20111205-13:40:2656=ALLOCCASHSTG10=193
20111205-08:50:26.938|Info|Other|ALLOCPRODSTG|31656|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=1849=FPCASH0152=20111205-13:50:2656=ALLOCCASHSTG10=195
20111205-09:00:26.940|Info|Other|ALLOCPRODSTG|31659|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=1949=FPCASH0152=20111205-14:00:2656=ALLOCCASHSTG10=192
20111205-09:10:26.935|Info|Other|ALLOCPRODSTG|31656|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=2049=FPCASH0152=20111205-14:10:2656=ALLOCCASHSTG10=185
20111205-09:20:26.936|Info|Other|ALLOCPRODSTG|31656|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=2149=FPCASH0152=20111205-14:20:2656=ALLOCCASHSTG10=187
20111205-09:30:26.934|Info|Other|ALLOCPRODSTG|31656|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=2249=FPCASH0152=20111205-14:30:2656=ALLOCCASHSTG10=189
20111205-09:40:26.934|Info|Other|ALLOCPRODSTG|31659|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=2349=FPCASH0152=20111205-14:40:2656=ALLOCCASHSTG10=191
20111205-09:50:27.934|Info|Other|ALLOCPRODSTG|31658|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=2449=FPCASH0152=20111205-14:50:2756=ALLOCCASHSTG10=194
20111205-10:00:28.934|Info|Other|ALLOCPRODSTG|31659|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=2549=FPCASH0152=20111205-15:00:2856=ALLOCCASHSTG10=192
20111205-10:10:28.932|Info|Other|ALLOCPRODSTG|31659|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=2649=FPCASH0152=20111205-15:10:2856=ALLOCCASHSTG10=194
20111205-10:20:28.925|Info|Other|ALLOCPRODSTG|31658|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=2749=FPCASH0152=20111205-15:20:2856=ALLOCCASHSTG10=196
20111205-10:30:29.924|Info|Other|ALLOCPRODSTG|31657|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=2849=FPCASH0152=20111205-15:30:2956=ALLOCCASHSTG10=199
20111205-10:40:29.924|Info|Other|ALLOCPRODSTG|31658|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=2949=FPCASH0152=20111205-15:40:2956=ALLOCCASHSTG10=201
20111205-10:50:30.925|Info|Other|ALLOCPRODSTG|31657|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=3049=FPCASH0152=20111205-15:50:3056=ALLOCCASHSTG10=186
20111205-11:00:31.920|Info|Other|ALLOCPRODSTG|31656|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=3149=FPCASH0152=20111205-16:00:3156=ALLOCCASHSTG10=184
20111205-11:10:32.918|Info|Other|ALLOCPRODSTG|31658|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=3249=FPCASH0152=20111205-16:10:3256=ALLOCCASHSTG10=187
20111205-11:20:33.917|Info|Other|ALLOCPRODSTG|31657|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=3349=FPCASH0152=20111205-16:20:3356=ALLOCCASHSTG10=190
20111205-11:30:33.915|Info|Other|ALLOCPRODSTG|31659|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=3449=FPCASH0152=20111205-16:30:3356=ALLOCCASHSTG10=192
20111205-11:40:34.913|Info|Other|ALLOCPRODSTG|31659|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=3549=FPCASH0152=20111205-16:40:3456=ALLOCCASHSTG10=195
20111205-11:50:34.911|Info|Other|ALLOCPRODSTG|31658|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=3649=FPCASH0152=20111205-16:50:3456=ALLOCCASHSTG10=197
20111205-12:00:34.918|Info|Other|ALLOCPRODSTG|31657|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=3749=FPCASH0152=20111205-17:00:3456=ALLOCCASHSTG10=194
20111205-12:10:35.919|Info|Other|ALLOCPRODSTG|31656|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=3849=FPCASH0152=20111205-17:10:3556=ALLOCCASHSTG10=197
20111205-12:20:35.925|Info|Other|ALLOCPRODSTG|31659|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=3949=FPCASH0152=20111205-17:20:3556=ALLOCCASHSTG10=199
20111205-12:30:36.930|Info|Other|ALLOCPRODSTG|31656|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=4049=FPCASH0152=20111205-17:30:3656=ALLOCCASHSTG10=193
20111205-12:40:36.932|Info|Other|ALLOCPRODSTG|31659|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=4149=FPCASH0152=20111205-17:40:3656=ALLOCCASHSTG10=195
20111205-12:50:37.934|Info|Other|ALLOCPRODSTG|31659|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=4249=FPCASH0152=20111205-17:50:3756=ALLOCCASHSTG10=198
20111205-13:00:37.937|Info|Other|ALLOCPRODSTG|31659|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=4349=FPCASH0152=20111205-18:00:3756=ALLOCCASHSTG10=195
20111205-13:10:38.935|Info|Other|ALLOCPRODSTG|31659|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=4449=FPCASH0152=20111205-18:10:3856=ALLOCCASHSTG10=198
20111205-13:20:38.939|Info|Other|ALLOCPRODSTG|31658|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=4549=FPCASH0152=20111205-18:20:3856=ALLOCCASHSTG10=200
20111205-13:30:39.944|Info|Other|ALLOCPRODSTG|31657|FileLogger|0|JobProcessor.cpp::validateMsg()|319|Receiving: len(82)|8=FIX.4.09=6035=034=4649=FPCASH0152=20111205-18:30:3956=ALLOCCASHSTG10=203



I appreciate any help that can be provided. I am new to Unix and have only done some light coding in C+ and Java ... VERY light coding at that.

Moderator's Comments:
Mod Comment Video tutorial on how to use code tags in The UNIX and Linux Forums.
# 2  
Old 12-05-2011
Transforming and calculating timestamps isn't trivial and different systems have different utilities for it. If you don't have Linux you might have to do at least some bits in Perl. What's your system?
 
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Ksh: Read line parse characters into variable and remove the line if the date is older than 50 days

I have a test file with the following format, It contains the username_date when the user was locked from the database. $ cat lockedusers.txt TEST1_21062016 TEST2_02122015 TEST3_01032016 TEST4_01042016 I'm writing a ksh script and faced with this difficult scenario for my... (11 Replies)
Discussion started by: humble_learner
11 Replies

2. Shell Programming and Scripting

Reading text file, comparing a value in a line, and placing only part of the line in a variable?

I need some help. I would like to read in a text file. Take a variable such as ROW-D-01, compare it to what's in one line in the text file such as PROD/VM/ROW-D-01 and only input PROD/VM into a variable without the /ROW-D-01. Is this possible? any help is appreciated. (2 Replies)
Discussion started by: xChristopher
2 Replies

3. Shell Programming and Scripting

EXPECT: Assign variable by reading a line of text from a file

Hi All, I have been using a program on windows called AutoKey. My environment at work is Linux and I have been experimenting with expect. Very powerful. I can move my AutoKey scripts to Linux using Expect once I am educated on how to read from a file using Expect. My application would be... (1 Reply)
Discussion started by: quemalr
1 Replies

4. Shell Programming and Scripting

add string and time stamp on each line of file

I have file A.txt A 1023 B 123 C 1223 I want output Hello_12PM_A 1023 Hello_12PM_B 123 Helll_12PM_C 1223 Add Hello and time stamp in AM and PM. (4 Replies)
Discussion started by: asavaliya
4 Replies

5. Shell Programming and Scripting

Set date and time stamp of one file to another

Hi I use "touch -t xxxxxxxx" command to set date/time stamp of a file. My requirement is to read the date/time stamp of a file and apply it to another file. Is there anyway to do it simple instead of manually taking date/stamp of first file? TIA Prvn (2 Replies)
Discussion started by: prvnrk
2 Replies

6. Shell Programming and Scripting

[Solved] Problem in reading a file line by line till it reaches a white line

So, I want to read line-by-line a text file with unknown number of files.... So: a=1 b=1 while ; do b=`sed -n '$ap' test` a=`expr $a + 1` $here do something with b etc done the problem is that sed does not seem to recognise the $a, even when trying sed -n ' $a p' So, I cannot read... (3 Replies)
Discussion started by: hakermania
3 Replies

7. Shell Programming and Scripting

reading the whole line from a file into a variable

Hi, I am doing : while read line do printf "%s\n" ${line} done <datafile.txt but I am not getting each single line from the data file assigned to the variable line (but only tokens/fields at a time). I also tried while IFS= read -r lineI want the whole line assigned or read into the... (2 Replies)
Discussion started by: shri_nath
2 Replies

8. UNIX for Dummies Questions & Answers

Reading a file one line at a time

I have a file that I want to read each record, do some stuff and write a new file. I get as far as: file=`cat data.txt` for data in "$file" do echo "Record is: $data" done The output of this is not what I was expecting: Record is: This is record one This is record two This is... (4 Replies)
Discussion started by: app4dxh
4 Replies

9. UNIX for Dummies Questions & Answers

File date and time stamp

I have to capture the creation date and time stamp for a file. The ls command doesn't list all the required information. I need year, month, day, hour, minute and second. Any ideas... (1 Reply)
Discussion started by: Xenon
1 Replies
Login or Register to Ask a Question