Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Extract key words and print their values

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract key words and print their values
# 1  
Old 07-19-2012
Bug Extract key words and print their values
Input file (HTTP request log file):
Code:
GET /dynamic_branding_playlist.fmil?domain=915oGLbNZhb&pluginVersion=3.2.7_2.6&pubchannel=usa&sdk_ver=2.4.6.3&width=680&height=290&embeddedIn=http%3A%2F%2Fviewster.com%2Fsplash%2FOscar-Videos-1.aspx%3Futm_source%3Dadon_272024_113535_24905_24905%26utm_medium%3Dcpc%26utm_campaign%3DUSYME%26adv%3D573900%26req%3D5006e9ce1ca8b26347b88a7.1.825&sdk_url=http%3A%2F%2Fdivaag.vo.llnwd.net%2Fo42%2Fhttp_only%2Fviewster_com%2Fv25%2Fyume%2F&viewport=42

Out put file:
Code:
domain          sdk_version
915oGLbNZhb   2.4.6.3

Thousands of logs similar to the example above, so I need to find a way to extract the value of domain&sdk_version. And the positions of domain and sdk_version are not fixed. sometimes appear in the 2 field, sometimes apprear in the last field (if split by &).

Could anyone help me in this problem? Thanks so much in advanceSmilie
Last edited by Scrutinizer; 07-19-2012 at 04:47 AM.. Reason: code tags
# 2  
Old 07-19-2012
GNU sed version 4.2.1

Code:
$less samplelog 
GET /dynamic_branding_playlist.fmil?domain=915oGLbNZhb&pluginVersion=3.2.7_2.6&pubchannel=usa&sdk_ver=2.4.6.3&width=680&height=290&embeddedIn=http%3A%2F%2Fviewster.com%2Fsplash%2FOscar-Videos-1.aspx%3Futm_source%3Dadon_272024_113535_24905_24905%26utm_medium%3Dcpc%26utm_campaign%3DUSYME%26adv %3D573900%26req%3D5006e9ce1ca8b26347b88a7.1.825&sdk_url=http%3A%2F%2Fdivaag.vo.llnwd.net%2Fo42%2Fhtt p_only%2Fviewster_com%2Fv25%2Fyume%2F&viewport=42
GET /dynamic_branding_playlist.fmil?domain=915oGLbNZhb&pluginVersion=3.2.7_2.6&pubchannel=usa&sdk_ver=2.4.6.3&width=680&height=290&embeddedIn=http%3A%2F%2Fviewster.com%2Fsplash%2FOscar-Videos-1.aspx%3Futm_source%3Dadon_272024_113535_24905_24905%26utm_medium%3Dcpc%26utm_campaign%3DUSYME%26adv %3D573900%26req%3D5006e9ce1ca8b26347b88a7.1.825&sdk_url=http%3A%2F%2Fdivaag.vo.llnwd.net%2Fo42%2Fhtt p_only%2Fviewster_com%2Fv25%2Fyume%2F&viewport=42
GET /dynamic_branding_playlist.fmil?domain=915oGLbNZhb&pluginVersion=3.2.7_2.6&pubchannel=usa&sdk_ver=2.4.6.3&width=680&height=290&embeddedIn=http%3A%2F%2Fviewster.com%2Fsplash%2FOscar-Videos-1.aspx%3Futm_source%3Dadon_272024_113535_24905_24905%26utm_medium%3Dcpc%26utm_campaign%3DUSYME%26adv %3D573900%26req%3D5006e9ce1ca8b26347b88a7.1.825&sdk_url=http%3A%2F%2Fdivaag.vo.llnwd.net%2Fo42%2Fhtt p_only%2Fviewster_com%2Fv25%2Fyume%2F&viewport=42

$printf "domain sdk_version\n"; sed -e 's/.*domain=\([0-9a-zA-Z]\+\)&.*sdk_ver=\([.0-9]\+\).*/\1 \2/' -e 's/ /\n/2' samplelog

domain sdk_version
915oGLbNZhb 2.4.6.3
915oGLbNZhb 2.4.6.3
915oGLbNZhb 2.4.6.3

Correction:
This "-e 's/ /\n/2'" was left in by mistake. As noted by Scrutinizer. It is not needed.

Code:
$printf "domain sdk_version\n"; sed 's/.*domain=\([0-9a-zA-Z]\+\)&.*sdk_ver=\([.0-9]\+\).*/\1 \2/' samplelog

Last edited by ni2; 07-19-2012 at 07:04 AM.. Reason: Added sed version; Added Correction.
# 3  
Old 07-19-2012
Code:
sed 's/.*domain=\([^&]*\).*sdk_ver=\([^&]*\).*/\1 \2/' infile


--
Note: \+ is GNU sed only (but it is not needed here, as * would suffice because of greedy matching) and \n cannot be used in the replacement part in standard sed (nor is it needed in this case), you would need an actual linefeed preceded with an escape character ( \ )..
This User Gave Thanks to Scrutinizer For This Post:
ni2
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Perl to extract values and print at end of each line

In the below perl I am trying to extract and print the values AF1=, the GT value, and F or QUAL diveded by 33 (rounded to the nearest whole #). The GT value is at the end after the GT:PL so all the possibilities are read into a hash h, then depending on the value that is in the line the... (1 Reply)
Discussion started by: cmccabe
1 Replies

2. Red Hat

How to block some key words in my url for apache config?

Hi Folks, I am running a website and that needs to be tightened with security in terms of hacking... Whereas, In my URL, when i click on certain links the entire link as contains some words like below:/control_panel /controlpanel /admin /cms Whereas, i need to block those words in apache... (1 Reply)
Discussion started by: gsiva
1 Replies

3. Shell Programming and Scripting

Delete lines containing key words dynamically

Hi Frens, I have a requirement where I need to delete lines having key words and am using the below command to do that sed '/UNIX/d' inputfile > output But now I have one more requirement where in there will be one reference file which has the ID's to be deleted from the master file. ... (3 Replies)
Discussion started by: weknowd
3 Replies

4. Shell Programming and Scripting

Using key words or symbols to determine parser

There are multiple conditions in which the user can input data. 1. > 2. del 3. ins 4. delins The user input is saved in a specific directory using the code below: Code: cd 'C:' C:/Users/cmccabe/Desktop/Python27/python.exe C:/Users/cmccabe/Desktop/Python27/run_batch_job.py... (12 Replies)
Discussion started by: cmccabe
12 Replies

5. UNIX for Dummies Questions & Answers

Delete lines according to a key words in that line

HI, I have a file A like this: c 1 length 14432 width 3434 temp 34 c 2 length 3343 width 0923 height 9383 hm 902 temp34 c 3 length 938 height 982 hm 9292 temp 23 ... (2 Replies)
Discussion started by: the_simpsons
2 Replies

6. UNIX for Dummies Questions & Answers

Filter logs for key words and create an index

well, im a total novice of unix commands and shell scripting, i never made anything. But yesterday in my work i was working with a lot of log files and i was wondering if there is a command for doing something like this: i have a bunch of files with text like this: blablabla errorcode1... (11 Replies)
Discussion started by: matius_88
11 Replies

7. Shell Programming and Scripting

grep - Extracting multiple key words from stdout

Hello. From command line, the command zypper info nxclient return a bloc of data : linux local # zypper info nxclient Loading repository data... Reading installed packages... Information for package nxclient: Repository: zypper_local Name: nxclient Version: 3.5.0-7 Arch: x86_64... (7 Replies)
Discussion started by: jcdole
7 Replies

8. Shell Programming and Scripting

extract key values

I am parsing a log with key values spread all over in the following fashion: TEST 1 SCHEME 12 SET EMPTY VARLEN SET TEST 1201 PARAM1 EMTY PARAM2 SET SCHEME 12 REFRESH TEST 8 I need to extract test number, my result should be 1 1201 8 I use awk for processing this log and use... (4 Replies)
Discussion started by: migurus
4 Replies

9. Shell Programming and Scripting

Print a key with its all values using awk/others

input COL1 a1 b1 c1 d1 e1 f1 C1 10 10 10 100 100 1000 C2 20 20 200 200 200 2000 output C1 a1 10 1 C1 b1 10 1 C1 c1 10 1 C1 d1 100 2 C1 e1 100 2 C1 f1 1000 3 C2 ... (12 Replies)
Discussion started by: ruby_sgp
12 Replies

10. Shell Programming and Scripting

How to pick values from column based on key values by usin AWK

Dear Guyz:) I have 2 different input files like this. I would like to pick the values or letters from the inputfile2 based on inputfile1 keys (A,F,N,X,Z). I have done similar task by using awk but in that case the inputfiles are similar like in inputfile2 (all keys in 1st column and values in... (16 Replies)
Discussion started by: repinementer
16 Replies
Login or Register to Ask a Question