Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Extract three substrings from a logfile

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract three substrings from a logfile
# 1  
Old 09-12-2012
Extract three substrings from a logfile
I have a log file like below.

Code:
66.249.73.11 - - [13/Aug/2012:06:08:45 +0000] "UCiZ7QocVqYAABgwfP8AAHAA" "US" "Mediapartners-Google" "-" www.mahashwetha.com.sg "GET /site/travel/flights/transport-results.html?skin=ensg.mahashwetha.com.sg&mondial_auto=true&DPNOC=1&DPNOA=2&DPNOI=1&DPLOC=en_SG&DPNOS=0&DPHSI=0&date_in_month=9&DPXCP=F&DPFSI=0&DPCID=0&DPXAL=F&DPTRT=flights&DPSFT=&DPXTR=F&DPTTT=R&DPBFT=False&DPXCH=F&DPXMG=F&DPHEI=2&failureURL=http%3A%2F%2Fwww.mahashwetha.com.sg%2Fsite%2Ftravel%2Fflights%2Fflights-search-error.html&DPDAP=Singapore%2C+Singapore+(SIN)&DPBFE=False&DPXIN=T&DPCLS=Y&DPPID=5003&date_out_month=9&DPFEI=9&TRANS=ViewTransportationSearchBox%2CdoSearchForTransportation%2C&IMBOSS=Z05&date_out_day=7&DPDCC=SGD&DPIDT=0001&DP1WF=0&DPFIT=flights&DPODT=0001&DPAAP=Koh+Samui%2C+Thailand+(USM)&DPAIR=NONE&date_in_day=11 HTTP/1.1" 200 12593 2670985 "81.104.178.164,66.249.73.11"

I want to extract three substrings from this and the o/p should be like this.

66.249.73.11 IMBOSS=Z05 81.104.178.164

The first one is the IP address and the second is a string as "IMBOSS=some value" (this can be anywahere in the log file, the script should just search and extract) and then finally an ip address which is normally at the second last position. I have marked the required strings in the o/p in red color.

It would be a great help if someone can help me here.
# 2  
Old 09-12-2012
Hi Tuxidow

i'm not a scripting guru but maybe this is helpful for you:

Code:
#!/bin/bash

# save in the variable FILE the path to your "log" file
FILE=text

cat $FILE|while read line
do
 IP=`cat $FILE | awk '{print $1}'`
 IMBOSS=`cat $FILE|sed 's+.*IMBOSS=++g'|sed 's/&/ /g'|awk '{print $1}'`
 LASTIP=`cat $FILE |awk '{print $NF}' |sed 's/"//g'|sed 's/,/ /g'|awk '{print $1}'`
echo $IP "IMBOSS="$IMBOSS $LASTIP
done

there is certainly antother way to get the value from IMBOSS, in this case the scripts works only if the first character after IMBOSS=<some value> is a $ !!
sorry i'm not a sed expert ;-)
Code:
[root@saxmgt01 tmp]# ./script.sh
66.249.73.11 IMBOSS=Z05 81.104.178.164

try it and let me know
# 3  
Old 09-12-2012
Try something this..

Code:
awk -F '[ ",&]' '{for (i=1;i<=NF;i++) {if ( $i ~ /^IMBOSS/ ) { print $1,$i,$(NF-2) }}}' file

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Logfile monitoring with logfile replacement

Bonjour, I've wrote a script to monitor a logfile in realtime. It is working almost perfeclty except for two things. The script use the following technique : tail -fn0 $logfile | \ while read line ; do ... some stuff done First one, I'd like a way to end the monitoring script if a... (3 Replies)
Discussion started by: Warluck
3 Replies

2. Shell Programming and Scripting

Extract the last 10 minutes from logfile

Any better way to extract the last 10 minutes from logfile? Dec 18 09:41:18 aaa Dec 18 09:46:29 aa Dec 18 09:48:39 vvv Dec 18 09:48:54 bbb Dec 18 09:54:47 bbb Dec 18 09:55:33 fcf Dec 18 09:55:38 ssdf Dec 18 09:57:58 sdsds Dec 18 09:58:10 sdsd Dec 18 10:00:50 sdsd Dec 18... (5 Replies)
Discussion started by: timmywong
5 Replies

3. UNIX for Dummies Questions & Answers

Replace substrings in awk

Hi ! my input looks like that: --AAA-AAAAAAA---------AA- AAA------AAAAAAAAAAAAAA ------A----AAAA-----A------- Using awk, I would need to replace only the "-" located between the last letter and the end of the string by "~" in order to get: --AAA-AAAAAAA---------AA~... (7 Replies)
Discussion started by: beca123456
7 Replies

4. Shell Programming and Scripting

extracting substrings from variables

Hello Everyone, I am looking for a way to extract substrings to local variables. Here is the format of the string variable i am using : /var/x/www && /usr/x/share/doc && /etc/x/logs where the substrings i must extract are the "/var/x/www" and such. I was originally thinking of using... (15 Replies)
Discussion started by: jimmy75_13
15 Replies

5. Shell Programming and Scripting

Script to extract line from logfile

Hi , Can someone help me,I am not well versed with scripting,it is very urjent. Need a script in perl or shell for the following task. The logfile contains several entries of which one particular string to be searched and that complete line has to be removed and copied to another file to... (25 Replies)
Discussion started by: garryt21@rediff
25 Replies

6. Shell Programming and Scripting

Need to extract specific pattern from logfile

Log File: Attempting to contact (DESCRIPTION=(SOURCE_ROUTE=OFF)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=hostname1.com)(PORT=1521)))(CONNECT_DATA=(SID=database1)(SRVR=DEDICATED))) Attempting to contact... (2 Replies)
Discussion started by: techychap
2 Replies

7. Shell Programming and Scripting

extracting substrings

Hi guys, I am stuck in this problem. Please help. I have two files. FILE1 (with records starting from '>' ) >TC1723_3 similar to Scific_A7Q9Q3 EMSPSQDYCDDYFKLTYPCTAGAQYYGRGALPVYWNYNYGAIGEALKLDLLNHPEYIEQN ATMAFQAAIWRWMNPMKKGQPSAHDAFVGNWKP >TC214_2 similar to Quiet_Ref100_Q8W2B2 Cluster;... (1 Reply)
Discussion started by: smriti_shridhar
1 Replies

8. AIX

Substrings and the likes in AIX 4.2 ?

In AIX 4.2, are there any shell commands to do substrings and the text like manipulation commands ? I want to take an error log where errors are multi-ligned and convert them into single lines to ease tracking/monitoring. I may need to shorten them out too. If I can manage to put them into an... (2 Replies)
Discussion started by: Browser_ice
2 Replies

9. Shell Programming and Scripting

Extract large list of substrings

I have a very long string (millions of characters). I have a file with start location and length that is thousands of rows long: Start Length 5 10 16 21 44 100 215 37 ... I'd like to extract the substring that corresponds to the start and length from each row of the list: I tried... (7 Replies)
Discussion started by: dcfargo
7 Replies

10. Shell Programming and Scripting

Breaking strings into Substrings

I'm only new to shell programming and have been given a task to do a program in .sh, however I've come to a point where I'm not sure what to do. This is my code so far: # process all arguments (i.e. loop while $1 is present) while ; do # echo "Arg is $1" case $1 in -h*|-H*) echo "help... (4 Replies)
Discussion started by: switch
4 Replies
Login or Register to Ask a Question