Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Format a log file using sed

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Format a log file using sed
# 1  
Old 09-10-2012
Format a log file using sed
I am looking to extract something like this from the below sample log file.

10.28.76.15 SSLC=Z42 71.19.175.130
10.28.76.15 SSLC=Z42 71.19.175.130
10.28.76.15 SSLC=Z42 71.19.175.130

for that I tried something like below and I have no clue how to extract the IP address marked red in the log file to the o/p.

Code:
egrep "10.28.76.15|10.28.86.15" log.file | sed 's/\([^ ]*\).*\(SSLC=...\).*/\1 \2/'

Sample log file below.
Code:
10.28.76.15 SSLC=Z42" www.crazynet.com "GET /site/transportationScripts.js?skin=enhk.muttusoochi.com.hk HTTP/1.1" 200 10351 8245 "71.19.175.130, 10.28.76.15"

10.28.76.15 SSLC=Z42" www.crazynet.com "GET /site/carrier_AS.gif?skin=default HTTP/1.1" 200 1962 2717 "71.19.175.130, 10.28.76.15"
10.28.76.15 SSLC=Z42" www.crazynet.com "GET /site/footer-logo-truste-100x28.png?skin=enhk.muttusoochi.com.hk HTTP/1.1" 200 1096 2027 "71.19.175.130, 10.28.76.15"

It would be a great help if some one here can help me to solve this.
# 2  
Old 09-10-2012
Must it be sed?

Code:
$ awk -F "[ ,\"]" '/(10.28.76.15|10.28.86.15)/ {print $1,$2,$(NF-3)}' infile
10.28.76.15 SSLC=Z42 71.19.175.130
10.28.76.15 SSLC=Z42 71.19.175.130
10.28.76.15 SSLC=Z42 71.19.175.130

This User Gave Thanks to zaxxon For This Post:
Tuxidow
# 3  
Old 09-10-2012
Or:
Code:
# awk '/10.28.76.15|10.28.86.15/{gsub(/\"|,/,"");print $1,$2,$(NF-1)}' infile

This User Gave Thanks to Klashxx For This Post:
Tuxidow
# 4  
Old 09-10-2012
like this.....?

Code:
awk -F '[" ]' '{for (i=1;i<=NF;i++){ if ((! s) && $i ~ /[0-9][0-9].[0-9][0-9].[0-9][0-9][0-9].[0-9][0-9][0-9]/) {print $1,$2,$i}}}' file

This User Gave Thanks to pamu For This Post:
Tuxidow
# 5  
Old 09-10-2012
Code:
sed -n '/10\.28\.[78]6\.15/{
s/\("[^"]*\)\{3\}"/ /
s/,.*//
p
}' file

This User Gave Thanks to elixir_sinari For This Post:
Tuxidow
# 6  
Old 09-10-2012
awesome... I was looking for an answer and got 4 different working answers.. This is super cool.. Thank you all. Smilie

---------- Post updated at 07:30 AM ---------- Previous update was at 07:00 AM ----------

Code:
10.28.76.15 2Fflights-search-error.html%26successURL%3D%2Ftravel%2Fflights%2Ftransport-results.html%26intcmp%3Dhome1_flights%26DPAIR%3DNONE%26isEurostarSearch%3DFalse%26DPODT%3D0001%26DPNOS%3D0%26DPIDT%3D0001%26DPDAP%3DLON%26DPAAP%3DYTO%26DPNOA%3D1%26DPNOC%3D0%26DPNOI%3D0%26date_out_day%3D25%26date_out_month%3D12%26date_in_day%3D28%26date_in_month%3D12%26DPCLS%3DY%26DPTTT%3DR%26DP1WF%3D0&source=aff-td1377904&tduid=bb69e501d40cf8b5dbd3a3a2a398d4f8SSLC=Z42" www.crazynet.com "GET /site/transportationScripts.js?skin=enhk.muttusoochi.com.hk HTTP/1.1" 200 10351
 8245 "71.19.175.130, 10.28.76.15"

How can I extract if the sample is like the above?
# 7  
Old 09-10-2012
Quote:
Originally Posted by Tuxidow
awesome... I was looking for an answer and got 4 different working answers.. This is super cool.. Thank you all. Smilie

---------- Post updated at 07:30 AM ---------- Previous update was at 07:00 AM ----------

Code:
10.28.76.15 2Fflights-search-error.html%26successURL%3D%2Ftravel%2Fflights%2Ftransport-results.html%26intcmp%3Dhome1_flights%26DPAIR%3DNONE%26isEurostarSearch%3DFalse%26DPODT%3D0001%26DPNOS%3D0%26DPIDT%3D0001%26DPDAP%3DLON%26DPAAP%3DYTO%26DPNOA%3D1%26DPNOC%3D0%26DPNOI%3D0%26date_out_day%3D25%26date_out_month%3D12%26date_in_day%3D28%26date_in_month%3D12%26DPCLS%3DY%26DPTTT%3DR%26DP1WF%3D0&source=aff-td1377904&tduid=bb69e501d40cf8b5dbd3a3a2a398d4f8SSLC=Z42" www.crazynet.com "GET /site/transportationScripts.js?skin=enhk.muttusoochi.com.hk HTTP/1.1" 200 10351
 8245 "71.19.175.130, 10.28.76.15"

How can I extract if the sample is like the above?
I guess now you have enough answers to work/tweet the commands yourself Smilie Smilie
This User Gave Thanks to vidyadhar85 For This Post:
pamu
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Datestamp format 2nd change in csv file (awk or sed)

I have a csv file formatted like this: 2014-08-21 18:06:26,A,B,12345,123,C,1232,26/08/14 18:07and I'm trying to change it to MM/DD/YYYY HH:MM for both occurances. I have got this: awk -F, 'NR <=1 {print;next}{"date +%d/%m/%Y\" \"%H:%m -d\""$1 "\""| getline dte;$1=dte}1' OFS="," test.csvThis... (6 Replies)
Discussion started by: say170
6 Replies

2. Shell Programming and Scripting

Help on Log File format using sed or awk

Hello Gurus, First, i would like to know is there any way to solve my problem. i have a log file like this: INFO - ABCDRequest :: processing started for the record <0> TransactionNo <Txn#1> recordID <recID#1> INFO - ABCDRequest :: processing started for the record <0> TransactionNo... (9 Replies)
Discussion started by: VasuKukkapalli
9 Replies

3. Shell Programming and Scripting

Need awk/sed to format a file

My content of source file is as below scr1 a1 scr2 a2 b2 scr3 a3 b3 c3 I need a awk/sed command (to be used in C shell)to format it to something like below scr1 $a1 >file1 scr2 $a2 $b2 >file2 scr3 $a3 $b3 $c3 >file3 (12 Replies)
Discussion started by: animesharma
12 Replies

4. Shell Programming and Scripting

Format log file - SED

Hello, I need help to format a log file...again ; )) Here is what I have : FILE='/OPERATIONNEL/SATURNE/MASTER/SATURNE_MASTER_PRE_R20110119.TAR.GZ ... (5 Replies)
Discussion started by: Aswex
5 Replies

5. Shell Programming and Scripting

Log File - Format

Dear all, I need help to format a log file : Here is the output (Go to the end of the Wrap Code to see caracter at the end of line): FILE='/OPERATIONNEL/SATURNE/PHYS_MOD/NOWMOD/SATURNE_1DAV_20110201_20110202_ICEMOD_R20110202.NC ... (5 Replies)
Discussion started by: Aswex
5 Replies

6. Shell Programming and Scripting

awk or sed to format text file

hi all, i have a text file which looks like the below 01 02 abc Top 40 music Kidz Only! MC 851 MC 852 MC 853 7NOW Arch_Diac xyz2 abc h211 Commacc1 Commacc2 Commacc3 (4 Replies)
Discussion started by: posner
4 Replies

7. Shell Programming and Scripting

SED - log file format

Hello everyone, I have a log file : \data\file\script\command_0001 \data\file\script\command_0002 \data\file\script\command_0003 I need to make all lines on this way \data\file\script\command_0001 \data\file\script\command_0002 \data\file\script\command_0003 I know this could be done... (11 Replies)
Discussion started by: Aswex
11 Replies

8. Shell Programming and Scripting

convert date format to mysql date format in log file

I have a comma delimited log file which has the date as MM/DD/YY in the 2nd column, and HH:MM:SS in the 3rd column. I need to change the date format to YYYY-MM-DD and merge it with the the time HH:MM:SS. How will I got about this? Sample input 02/27/09,23:52:31 02/27/09,23:52:52... (3 Replies)
Discussion started by: hazno
3 Replies

9. UNIX for Advanced & Expert Users

Sed to format data in a file

Hi , i need help with formatting a file i am generating which is to be used in mainframe app so the file length has to be 80 for each rows. The file that m able to generate looks like this (consists of two rows only) E 1006756 1006756 Active T E 0551055 0551055 Active T I... (2 Replies)
Discussion started by: cnilashis
2 Replies

10. UNIX for Dummies Questions & Answers

how to number format a data file without using SED?

Hi I have a file which contains data (list of data) and I want to put a number with bracket 1) 2) 3) etc at the beginning of every successive line I can do it with SED and I can also do it using the nl route but am looking for a different method. I'm guessing I would need some sort of loop... (3 Replies)
Discussion started by: Cactus Jack
3 Replies
Login or Register to Ask a Question