Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Extract Specific pattern - log file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract Specific pattern - log file
# 1  
Old 11-09-2011
Extract Specific pattern - log file
Hello everyone,

I am on AIX (6.1). I can only use shell (ksh) script.

I can't do this on my own, so will do my best to explain my needs.I also do not know what is the best idea to make it work, so here is what I am thinking, but I may wrong.


I need help to extract info on a log. The info needed will only concern the day today. So the script must be able to know the date of the day TODAY, and extract the scpecific pattern

Code:
tar: xxxxxxxxxxxxxxxxxxx: No such file or directory

The idea is to create shell script that could run on crontab 3 times a day. The script must be able to extract specific pattern for today day. Means if today is 2011/11/09 it won't get the pattern of yesterday for exemple. If the pattern doesn't not exist for today, it will exit.

Here is the date return format on the AIX :

Code:
Wed Nov  9 10:36:17 CUT 2011

As you can see, here is the date pattern written on the log file :

Code:
2011/10/21

Here is a sample of the log file :


Code:
  0+2011/10/20-06:10:06+ARCHIVER_FICHIERS+713+recup du SIZE_MAX_STAF OK 5000 5242880000
  load staf 3.2.4 : OK
  0+2011/10/21-06:10:12+ARCHIVER_FICHIERS+713+recup du SIZE_MAX_STAF OK 5000 5242880000
  load staf 3.2.4 : OK
  tar: GO_CONS_STR_QUA_2C_20100228T233231_20100301T010215_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T010215_20100301T023158_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T023158_20100301T040142_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T040142_20100301T053125_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T053125_20100301T070108_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T070108_20100301T083052_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T083052_20100301T100035_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T100035_20100301T113018_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T113018_20100301T130002_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T130002_20100301T142945_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T142945_20100301T155929_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T155929_20100301T172912_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T172912_20100301T185855_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T185855_20100301T202839_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T202839_20100301T215822_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T215822_20100301T232805_0001.IDF: No such file or directory
  tar: GO_CONS_STR_QUA_2C_20100301T232805_20100302T005751_0001.IDF: No such file or directory
  0+2011/10/22-06:10:17+ARCHIVER_FICHIERS+713+recup du SIZE_MAX_STAF OK 5000 5242880000
  load staf 3.2.4 : OK
  0+2011/10/23-06:10:11+ARCHIVER_FICHIERS+713+recup du SIZE_MAX_STAF OK 5000 5242880000
  load staf 3.2.4 : OK
  0+2011/11/08-23:00:39+MONITEUR+1713+Erreur SQL.
  9+2011/11/08-23:00:39+MONITEUR+1718+ORA-03113: end-of-file on communication channel
  +

the result needed must be for exemple :

Code:
tar: GO_CONS_STR_QUA_2C_20100228T233231_20100301T010215_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T010215_20100301T023158_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T023158_20100301T040142_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T040142_20100301T053125_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T053125_20100301T070108_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T070108_20100301T083052_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T083052_20100301T100035_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T100035_20100301T113018_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T113018_20100301T130002_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T130002_20100301T142945_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T142945_20100301T155929_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T155929_20100301T172912_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T172912_20100301T185855_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T185855_20100301T202839_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T202839_20100301T215822_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T215822_20100301T232805_0001.IDF: No such file or directory
tar: GO_CONS_STR_QUA_2C_20100301T232805_20100302T005751_0001.IDF: No such file or directory

I hope you will be able to help me.

Thank you much as I completly lost on this.
# 2  
Old 11-09-2011
Code:
grep -E "tar.*$(date +%Y%m%d).*: No such file or directory" infile

(may be egrep rather than grep -E on AIX, I forget).
# 3  
Old 11-09-2011
Thanks CarloM, but I can't make it work.

I fact, I was looking for something like getting the date of the day,
Code:
Wed Nov  9 10:36:17 CUT 2011

then format the date to match the date used in the log file
Code:
2011/11/09

then start to read the log file from the first line that match the date of the day.

and find if there is or not any pattern that will be :

Code:
tar: xxxxxxxxxxxxxxxxxxx: No such file or directory

.

If no math found the script will exit, if match found it will send me a mail.

Thanks for your help, please.
# 4  
Old 11-09-2011
Ah, I thought the tar lines had today's date in them.

Try this:
Code:
awk -vADATE=$(date +%Y/%m/%d) '/ARCHIVER/ {inside=0} $0 ~ ADATE {inside=1} /tar:.*: No such file or directory/ { if (inside==1) {print}}' infile

This User Gave Thanks to CarloM For This Post:
Aswex
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extract whole word preceding a specific character pattern with first occurence of the pattern

Hello. Here is a file contents : declare -Ax NEW_FORCE_IGNORE_ARRAY=(="§" ="§" ="§" ="§" ="§" .................. ="§"Here is a pattern =I want to extract 'NEW_FORCE_IGNORE_ARRAY' which is the whole word before the first occurrence of pattern '=' Is there a better solution than mine :... (3 Replies)
Discussion started by: jcdole
3 Replies

2. Shell Programming and Scripting

Extract specific line in an html file starting and ending with specific pattern to a text file

Hi This is my first post and I'm just a beginner. So please be nice to me. I have a couple of html files where a pattern beginning with "http://www.site.com" and ending with "/resource.dat" is present on every 241st line. How do I extract this to a new text file? I have tried sed -n 241,241p... (13 Replies)
Discussion started by: dejavo
13 Replies

3. Shell Programming and Scripting

Extract portion of log info based on specific word

Hi Gurus, I'm using HP-UX B.11.23 operating system. I've been trying to extract a specific wording for example: "A tool used by tp produced warnings" from my below log data, but could not find a way to solve it. My intention is, if the log contain the word: "A tool used by tp produced... (9 Replies)
Discussion started by: superHonda123
9 Replies

4. Shell Programming and Scripting

extract specific line if the search pattern is found

Hi, I need to extract <APPNUMBER> tag alone, if the <college> haas IIT Chennai value. college tag value will have spaces embedded. Those spaces should not be suppresses. My Source file <Record><sno>1</sno><empid>E0001</empid><name>Rejsh suderam</name><college>IIT ... (3 Replies)
Discussion started by: Sekar1
3 Replies

5. Shell Programming and Scripting

to extract specific value in a file

Hi Friends, I have a file with the following values.. xyz.txt,12345.xml abc.txt,04567.xml cde.txt,12134.xml I would like to extract all the 2nd column values like 12345 04567 12134 Please advice!! (3 Replies)
Discussion started by: techmoris
3 Replies

6. Shell Programming and Scripting

Extract data from log file from or after the specific date

Hi , I am having a script which will start a process and appends the process related logs to a log file. The log file writes logs with every line starting with date in the format of: date +"%Y %b %d %H:%M:%S". So, in the script, before I start the process, I am storing the date as DATE=`date +"%Y... (5 Replies)
Discussion started by: chiru_h
5 Replies

7. Shell Programming and Scripting

Need to extract specific pattern from logfile

Log File: Attempting to contact (DESCRIPTION=(SOURCE_ROUTE=OFF)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=hostname1.com)(PORT=1521)))(CONNECT_DATA=(SID=database1)(SRVR=DEDICATED))) Attempting to contact... (2 Replies)
Discussion started by: techychap
2 Replies

8. Shell Programming and Scripting

Extract specific pattern from a file

Hi All, I am a newbie to Shell Scripting. I have a File The Server Name XXX002 ------------------------- 2.1 LAPD Iface Id Link MTU Side ecc_3_1 4 Up 512 User ecc_3_2 5 Up 512 User The Server Name XXX003 ------------------------- 2.1 LAPD (4 Replies)
Discussion started by: athreyavc
4 Replies

9. HP-UX

extract field of characters after a specific pattern - using UNIX shell script

Hello, Below is my input file's content ( in HP-UX platform ): ABCD120672-B21 1 ABCD142257-002 1 ABCD142257-003 1 ABCD142257-006 1 From the above, I just want to get the field of 13 characters that comes after 'ABCD' i.e '120672-B21'... . Could... (2 Replies)
Discussion started by: jansat
2 Replies

10. UNIX for Dummies Questions & Answers

extract specific lines from file

hi, how would i extract a range of lines in a file by using the line number? ex: file contains: 1 title 2 i want 3 this part 4 to be taken out 5 from this file 6 and sent to 7 another file 8 not needed 9 end of file In this case, i want to copy line number 2 to 7 on a new... (2 Replies)
Discussion started by: apalex
2 Replies
Login or Register to Ask a Question