Extract Specific pattern - log file


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract Specific pattern - log file
# 1  
Extract Specific pattern - log file

Hello everyone,

I am on AIX (6.1). I can only use shell (ksh) script.

I cant do this on my own, so will do my best to explain my needs.I also do not know what is the best idea to make it work, so here is what I am thinking, but I may wrong.


I need help to extract info on a log. The info needed will only concern the day today. So the script must be able to know the date of the day TODAY, and extract the scpecific pattern

Code:
tar: xxxxxxxxxxxxxxxxxxx: No such file or directory

The idea is to create shell script that could run on crontab 3 times a day. The script must be able to extract specific pattern for today day. Means if today is 2011/11/09 it wont get the pattern of yesterday for exemple. If the pattern doesnt not exist for today, it will exit.

Here is the date return format on the AIX :

Code:
Wed Nov  9 10:36:17 CUT 2011

As you can see, here is the date pattern written on the log file :

Code:
2011/10/21

Here is a sample of the log file :


Code:
  0+2011/10/20-06:10:06+ARCHIVER_FICHIERS+713+recup du SIZE_MAX_STAF OK 5000 5242880000
  load staf 3.2.4 : OK
  0+2011/10/21-06:10:12+ARCHIVER_FICHIERS+713+recup du SIZE_MAX_STAF OK 5000 5242880000
  load staf 3.2.4 : OK
  tar: GO_CONS_STR_QUA_2C_20100228T233231_20100301T010215_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T010215_20100301T023158_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T023158_20100301T040142_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T040142_20100301T053125_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T053125_20100301T070108_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T070108_20100301T083052_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T083052_20100301T100035_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T100035_20100301T113018_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T113018_20100301T130002_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T130002_20100301T142945_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T142945_20100301T155929_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T155929_20100301T172912_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T172912_20100301T185855_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T185855_20100301T202839_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T202839_20100301T215822_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T215822_20100301T232805_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T232805_20100302T005751_0001.IDF: No such file or directory
  0+2011/10/22-06:10:17+ARCHIVER_FICHIERS+713+recup du SIZE_MAX_STAF OK 5000 5242880000
  load staf 3.2.4 : OK
  0+2011/10/23-06:10:11+ARCHIVER_FICHIERS+713+recup du SIZE_MAX_STAF OK 5000 5242880000
  load staf 3.2.4 : OK
  0+2011/11/08-23:00:39+MONITEUR+1713+Erreur SQL.
  9+2011/11/08-23:00:39+MONITEUR+1718+ORA-03113: end-of-file on communication channel
  +

the result needed must be for exemple :

Code:
tar: GO_CONS_STR_QUA_2C_20100228T233231_20100301T010215_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T010215_20100301T023158_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T023158_20100301T040142_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T040142_20100301T053125_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T053125_20100301T070108_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T070108_20100301T083052_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T083052_20100301T100035_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T100035_20100301T113018_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T113018_20100301T130002_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T130002_20100301T142945_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T142945_20100301T155929_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T155929_20100301T172912_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T172912_20100301T185855_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T185855_20100301T202839_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T202839_20100301T215822_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T215822_20100301T232805_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T232805_20100302T005751_0001.IDF: No such file or directory

I hope you will be able to help me.

Thank you much as I completly lost on this.
# 2  
Code:
grep -E "tar.*$(date +%Y%m%d).*: No such file or directory" infile

(may be egrep rather than grep -E on AIX, I forget).
# 3  
Thanks CarloM, but I can't make it work.

I fact, I was looking for something like getting the date of the day,
Code:
Wed Nov  9 10:36:17 CUT 2011

then format the date to match the date used in the log file
Code:
2011/11/09

then start to read the log file from the first line that match the date of the day.

and find if there is or not any pattern that will be :

Code:
tar: xxxxxxxxxxxxxxxxxxx: No such file or directory

.

If no math found the script will exit, if match found it will send me a mail.

Thanks for your help, please.
# 4  
Ah, I thought the tar lines had today's date in them.

Try this:
Code:
awk -vADATE=$(date +%Y/%m/%d) '/ARCHIVER/ {inside=0} $0 ~ ADATE {inside=1} /tar:.*: No such file or directory/ { if (inside==1) {print}}' infile

This User Gave Thanks to CarloM For This Post:
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #316
Difficulty: Easy
RAM stands for Registered Access Memory.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extract whole word preceding a specific character pattern with first occurence of the pattern

Hello. Here is a file contents : declare -Ax NEW_FORCE_IGNORE_ARRAY=(="§" ="§" ="§" ="§" ="§" .................. ="§"Here is a pattern =I want to extract 'NEW_FORCE_IGNORE_ARRAY' which is the whole word before the first occurrence of pattern '=' Is there a better solution than mine :... (3 Replies)
Discussion started by: jcdole
3 Replies

2. Shell Programming and Scripting

Extract specific line in an html file starting and ending with specific pattern to a text file

Hi This is my first post and I'm just a beginner. So please be nice to me. I have a couple of html files where a pattern beginning with "http://www.site.com" and ending with "/resource.dat" is present on every 241st line. How do I extract this to a new text file? I have tried sed -n 241,241p... (13 Replies)
Discussion started by: dejavo
13 Replies

3. Shell Programming and Scripting

Extract portion of log info based on specific word

Hi Gurus, I'm using HP-UX B.11.23 operating system. I've been trying to extract a specific wording for example: "A tool used by tp produced warnings" from my below log data, but could not find a way to solve it. My intention is, if the log contain the word: "A tool used by tp produced... (9 Replies)
Discussion started by: superHonda123
9 Replies

4. Shell Programming and Scripting

extract specific line if the search pattern is found

Hi, I need to extract <APPNUMBER> tag alone, if the <college> haas IIT Chennai value. college tag value will have spaces embedded. Those spaces should not be suppresses. My Source file <Record><sno>1</sno><empid>E0001</empid><name>Rejsh suderam</name><college>IIT ... (3 Replies)
Discussion started by: Sekar1
3 Replies

5. Shell Programming and Scripting

to extract specific value in a file

Hi Friends, I have a file with the following values.. xyz.txt,12345.xml abc.txt,04567.xml cde.txt,12134.xml I would like to extract all the 2nd column values like 12345 04567 12134 Please advice!! (3 Replies)
Discussion started by: techmoris
3 Replies

6. Shell Programming and Scripting

Extract data from log file from or after the specific date

Hi , I am having a script which will start a process and appends the process related logs to a log file. The log file writes logs with every line starting with date in the format of: date +"%Y %b %d %H:%M:%S". So, in the script, before I start the process, I am storing the date as DATE=`date +"%Y... (5 Replies)
Discussion started by: chiru_h
5 Replies

7. Shell Programming and Scripting

Need to extract specific pattern from logfile

Log File: Attempting to contact (DESCRIPTION=(SOURCE_ROUTE=OFF)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=hostname1.com)(PORT=1521)))(CONNECT_DATA=(SID=database1)(SRVR=DEDICATED))) Attempting to contact... (2 Replies)
Discussion started by: techychap
2 Replies

8. Shell Programming and Scripting

Extract specific pattern from a file

Hi All, I am a newbie to Shell Scripting. I have a File The Server Name XXX002 ------------------------- 2.1 LAPD Iface Id Link MTU Side ecc_3_1 4 Up 512 User ecc_3_2 5 Up 512 User The Server Name XXX003 ------------------------- 2.1 LAPD (4 Replies)
Discussion started by: athreyavc
4 Replies

9. HP-UX

extract field of characters after a specific pattern - using UNIX shell script

Hello, Below is my input file's content ( in HP-UX platform ): ABCD120672-B21 1 ABCD142257-002 1 ABCD142257-003 1 ABCD142257-006 1 From the above, I just want to get the field of 13 characters that comes after 'ABCD' i.e '120672-B21'... . Could... (2 Replies)
Discussion started by: jansat
2 Replies

10. UNIX for Dummies Questions & Answers

extract specific lines from file

hi, how would i extract a range of lines in a file by using the line number? ex: file contains: 1 title 2 i want 3 this part 4 to be taken out 5 from this file 6 and sent to 7 another file 8 not needed 9 end of file In this case, i want to copy line number 2 to 7 on a new... (2 Replies)
Discussion started by: apalex
2 Replies

Featured Tech Videos