Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: How to use GREP to extract URL from file
Top Forums Shell Programming and Scripting How to use GREP to extract URL from file Post 302698185 by durden_tyler on Saturday 8th of September 2012 07:25:40 PM
Old 09-08-2012
Quote:
Originally Posted by Naks_Sh10
...3. Also check is each component in the URL is greater than 200 characters

For eg. Is "/liveupdate-aka.symantec.com/1340071490jtun_nav2k8enn09m25.m25" greater than 800 characters and

is "liveupdate-aka.symantec.com" geater than 200 characters

is "1340071490jtun_nav2k8enn09m25.m25" greater than 200 characters.


It would be good if we can do all of the above in one command if possible.
Code:

$
$ cat f10
98.70.217.222 - - [08/Jul/2012:09:14:29 +0000] "GET /liveupdate-aka.symantec.com/1340071490jtun_nav2k8enn09m25.m25?h=abcdefgh HTTP/1.1" 200 159229484 "-" "hBU1OhDsPXknMepDBJNScBj4BQcmUz5TwAAAAA" "-"
$
$
$ perl -lne 's/^.*GET (.*?)\?.*/$1/; print $_, " is ",length($_)<=800?"not":"", " greater than 800 char";
             s/^\///; map {print $_, " is ", length($_)<=200?"not":"", " greater than 200 char"} split/\//
            ' f10
/liveupdate-aka.symantec.com/1340071490jtun_nav2k8enn09m25.m25 is not greater than 800 char
liveupdate-aka.symantec.com is not greater than 200 char
1340071490jtun_nav2k8enn09m25.m25 is not greater than 200 char
$
$

tyler_durden
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

SED extract url - please help a lamer

Hello everybody. I have lines that looks something like this: <done16=""118"" done18=""$ title=""thisisatitle"" href=""/JoeBanana" alt=""Joe""><done16=""118"" done18=""$ title=""thisisatitle"" href=""/GeraldGiraffe" alt=""Gerald""> What kind of SED command would I need to use to extract... (4 Replies)
Discussion started by: digi
4 Replies

2. Shell Programming and Scripting

How to open an url and grep for a word

Hi All, I am new to shell scripting,Could any of you help me on this below :confused: -------------------- I need to write a shell script where i have open an url and grep for a particular word in the url and want to display it in a flatfile. Eg: Want to open yahoo.com and grep for yahoo... (1 Reply)
Discussion started by: sumithra
1 Replies

3. UNIX for Advanced & Expert Users

how to grep/read a file inside compressed tgz without extract?

Hi all, I would like to ask whether in Unix shell/perl have any functions or command to allow grep/cat/read a file inside compressed .tgz without extract it? I know we can tar tvf a compressed tgz but this only allow we read the path/filename contained inside the tarball. If we want to read... (3 Replies)
Discussion started by: mayshy
3 Replies

4. Shell Programming and Scripting

Extract URL from RSS Feed in AWK

Hi, I have following data file; <outline title="Matt Cutts" type="rss" version="RSS" xmlUrl="http://www.mattcutts.com/blog/feed/" htmlUrl="http://www.mattcutts.com/blog"/> <outline title="Stone" text="Stone" type="rss" version="RSS" xmlUrl="http://feeds.feedburner.com/STC-Art"... (8 Replies)
Discussion started by: fahdmirza
8 Replies

5. Shell Programming and Scripting

How to extract url from html page?

for example, I have an html file, contain <a href="http://awebsite" id="awebsite" class="first">website</a>and sometime a line contains more then one link, for example <a href="http://awebsite" id="awebsite" class="first">website</a><a href="http://bwebsite" id="bwebsite"... (36 Replies)
Discussion started by: 14th
36 Replies

6. Shell Programming and Scripting

Need help please with Grep/Sed command to extract text and numbers from a file

Hello All, I need to extract lines from a file that contains ALPHANUMERIC and the length of Alphanumeric is set to 16. I have pasted the sample of the lines from the text file that I have created. My problem is that sometimes 16 appears in other part of the line. I'm only interested to... (14 Replies)
Discussion started by: mnassiri
14 Replies

7. Shell Programming and Scripting

Need help in writing a script that do internal grep on a log file..and extract acct no's from it..

I need to write a script, which takes the input a log file and create output file with acct no's line by line from selected records with text like (in red) : 18:51:18 | 217863|Acct 0110855565|RC 17608| 16 Subs| 1596 UsgRecs| 2 Secs| 430 CPUms| prmis2:26213 <MoveUsage d aemon needs to run... (7 Replies)
Discussion started by: rkrish
7 Replies

8. Shell Programming and Scripting

Extract values from multi lined url source

Hello, I want extract multi values from multi url source to a csv text. Thank you very much for help. my curl code : curl "http://www.web.com/cities//city.html Source code: div class="clear"></div> <table class="listing-details"> <tr> ... (1 Reply)
Discussion started by: hoo
1 Replies

9. Shell Programming and Scripting

Reading URL using Mechanize and dump all the contents of the URL to a file

Hello, Am very new to perl , please help me here !! I need help in reading a URL from command line using PERL:: Mechanize and needs all the contents from the URL to get into a file. below is the script which i have written so far , #!/usr/bin/perl use LWP::UserAgent; use... (2 Replies)
Discussion started by: scott_cog
2 Replies

10. Shell Programming and Scripting

Script extract text from txt file with grep

All, I require a script that grabs some text from the gitHub API and will grep (or other function) for a string a characters that starts with (") quotes followed by two letters, may contain a pipe |, and ending with ) . What i have so far is below but it's not returning anything. ... (4 Replies)
Discussion started by: ChocoTaco
4 Replies

LEARN ABOUT PHP

truncate

TRUNCATE(1)							   User Commands						       TRUNCATE(1)

NAME

       truncate - shrink or extend the size of a file to the specified size

SYNOPSIS

       truncate OPTION... FILE...

DESCRIPTION

       Shrink or extend the size of each FILE to the specified size

       A FILE argument that does not exist is created.

       If  a  FILE  is	larger than the specified size, the extra data is lost.  If a FILE is shorter, it is extended and the extended part (hole)
       reads as zero bytes.

       Mandatory arguments to long options are mandatory for short options too.

       -c, --no-create
	      do not create any files

       -o, --io-blocks
	      treat SIZE as number of IO blocks instead of bytes

       -r, --reference=RFILE
	      base size on RFILE

       -s, --size=SIZE
	      set or adjust the file size by SIZE bytes

       --help display this help and exit

       --version
	      output version information and exit

       The SIZE argument is an integer and optional unit (example: 10K is 10*1024).  Units are K,M,G,T,P,E,Z,Y (powers of 1024) or KB,MB,... (pow-
       ers of 1000).

       SIZE  may also be prefixed by one of the following modifying characters: '+' extend by, '-' reduce by, '<' at most, '>' at least, '/' round
       down to multiple of, '%' round up to multiple of.

AUTHOR

       Written by Padraig Brady.

REPORTING BUGS

       GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
       Report truncate translation bugs to <http://translationproject.org/team/>

COPYRIGHT

       Copyright (C) 2017 Free Software Foundation, Inc.  License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
       This is free software: you are free to change and redistribute it.  There is NO WARRANTY, to the extent permitted by law.

SEE ALSO

       dd(1), truncate(2), ftruncate(2)

       Full documentation at: <http://www.gnu.org/software/coreutils/truncate>
       or available locally via: info '(coreutils) truncate invocation'

GNU coreutils 8.28						   January 2018 						       TRUNCATE(1)
All times are GMT -4. The time now is 06:13 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy