11-22-2008
Extract a substring.
I have a shell script that uses wget to grab a bunch of html from a url.
URL_DATA=`wget -qO - "$URL1"`
I now have a string $URL_DATA that I need to pull a substring out of..say I had the following in my string
<p><a href="/scooby/929011567.html">Dog pictures check them out! -</a><font size="-1"> (Silly)</font></p> <p><a href="/shaggey/928861647.html">Vacation -</a><font size="-1"> (boating)</font></p> <p><a href="/gopher/928782568.html">Garden -</a><font size="-1"> (winter)</font></p>
I want to extract the URL, Title and Description throughout the string...like the following
/scooby/929011567.html
Dog pictures check them out!
(silly)
/shaggey/928861647.html
Vacation
(boating)
/gopher/928782568.html
Garden
(winter)
and keep going with that pattern as many times as it's in the string. How would I do this?
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
On OS 10.4.11
I have filenames like:
670711 SA T2 v1-1_DS_EF.doc
CT_670520 AM T1 v1-2_DS_EF.doc
CT_670716 - 2 SA T4 v1-2_DS_EF.doc
CT_670713 SA T3 v1-1_DS_EF.doc
670421 PA DYP1 v1-1_DS_EF.doc
CT_670425 PA DYP2 v1-1_DS_EF.doc
CT_670107 RA T3 v1-2_DS_EF.doc
CT_670521 AM T2 v1-2_DS_EF.doc... (3 Replies)
Discussion started by: mlommel
3 Replies
2. Shell Programming and Scripting
> tnsping $TWO_TASK | grep HOST
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = 10.12.10.212)(PORT = 1540)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = OMTST15)))
I want to extract like this
HOST = 10.12.10.212
PORT = 1540
SERVICE_NAME = OMTST15
I... (4 Replies)
Discussion started by: dashok.83
4 Replies
3. Solaris
i have srtring i.e. "NAME,CLASS,AGE" (length of string is not constant) and from this string i've extract each word delimited by "," (comma).
INPUT: "NAME,CLASS,AGE"
OUTPUT: NAME
CLASS
AGE
how can i do that?
i have tried some string manipulation function like... (5 Replies)
Discussion started by: jadoo_c2
5 Replies
4. Shell Programming and Scripting
Hi,
I'm a newbie to shell scripting and have searched the forum but couldn't find what i was looking for.
Basically I have a list of filenames like...
123-fileone.txt
I want to be able to extract the prefix up to the first '-'. So I'd end up with 123. I have attempted it using a pretty... (2 Replies)
Discussion started by: kirkg
2 Replies
5. Shell Programming and Scripting
Hi,
I have an input string say for example:
ABC,DEF,IJK,LMN,...,XYZ
The above string is comma delimited. Now I have to extract the last part after the comma i.e. XYZ.
:b: (3 Replies)
Discussion started by: bghosh
3 Replies
6. Shell Programming and Scripting
Hi All,
I have a log file in which name and version of applications are coming in the following format
name
It may look like following, based on the name of the application and version:
XYZ OR xyz OR XyZ OR xyz
I want to separate out the name and version and store them into variables.... (4 Replies)
Discussion started by: bhaskar_m
4 Replies
7. Shell Programming and Scripting
Hi
Input:
aa-bb-cc-dd.ee.ff.gg
Output:
dd
I want to get the word after the last '-' until the first dot
I have tried with regex lookbehind and lookahead like this:
(?<=-).*(?=\.)
but his returns too much
bb-cc-dd.ee.ff (7 Replies)
Discussion started by: chitech
7 Replies
8. Shell Programming and Scripting
Hello,
A question please.
A have a file that contains a string. Ex:
AAAABBCCCCCDDEEEEEEEEEEFF
I'd want to recover 2 substrings, 'BB' and 'FF' and then leave them in a new file.
Could anoyone help me please?
Thanks in advance (3 Replies)
Discussion started by: nolo41
3 Replies
9. Shell Programming and Scripting
Hello,
A question please.
A have a file that contains a string. Ex:
AAAABBCCCCCDDEEEEEEEEEEFF
I'd want to recover 2 substrings, 'BB' and 'FF' and then leave them in a new file.
From position 5, 2 caracters (ex:"BB") and from position 25, 2 caracters (ex:"FF") in a file.
Could anoyone help me... (3 Replies)
Discussion started by: nolo41
3 Replies
10. Shell Programming and Scripting
Hi all,
Please, i'd like to extract string just before '.fr'.
Here is some lines of my file:
g-82.text.text1.fr.worker1
g-xx.yyyyyy.zzzz.fr.worker2
i'd like to extract this text:
g-82.text.text1
g-xx.yyyyyy.zzzz
Please, which command i have to use in my script shell ?
... (16 Replies)
Discussion started by: chercheur111
16 Replies
LEARN ABOUT DEBIAN
httpindex
httpindex(1) General Commands Manual httpindex(1)
NAME
httpindex - HTTP front-end for SWISH++ indexer
SYNOPSIS
wget [ options ] URL... 2>&1 | httpindex [ options ]
DESCRIPTION
httpindex is a front-end for index++(1) to index files copied from remote servers using wget(1). The files (in a copy of the remote direc-
tory structure) can be kept, deleted, or replaced with their descriptions after indexing.
OPTIONS
wget Options
The wget(1) options that are required are: -A, -nv, -r, and -x; the ones that are highly recommended are: -l, -nh, -t, and -w. (See the
EXAMPLE.)
httpindex Options
httpindex accepts the same short options as index++(1) except for -H, -I, -l, -r, -S, and -V.
The following options are unique to httpindex:
-d Replace the text of local copies of retrieved files with their descriptions after they have been indexed. This is useful to display
file descriptions in search results without having to have complete copies of the remote files thus saving filesystem space. (See
the extract_description() function in WWW(3) for details about how descriptions are extracted.)
-D Delete the local copies of retrieved files after they have been indexed. This prevents your local filesystem from filling up with
copies of remote files.
EXAMPLE
To index all HTML and text files on a remote web server keeping descriptions locally:
wget -A html,txt -linf -t2 -rxnv -nh -w2 http://www.foo.com 2>&1 |
httpindex -d -e'html:*.html,text:*.txt'
Note that you need to redirect wget(1)'s output from standard error to standard output in order to pipe it to httpindex.
EXIT STATUS
Exits with a value of zero only if indexing completed sucessfully; non-zero otherwise.
CAVEATS
In addition to those for index++(1), httpindex does not correctly handle the use of multiple -e, -E, -m, or -M options (because the Perl
script uses the standard GetOpt::Std package for processing command-line options that doesn't). The last of any of those options ``wins.''
The work-around is to use multiple values for those options seperated by commas to a single one of those options. For example, if you want
to do:
httpindex -e'html:*.html' -e'text:*.txt'
do this instead:
httpindex -e'html:*.html,text:*.txt'
SEE ALSO
index++(1), wget(1), WWW(3)
AUTHOR
Paul J. Lucas <pauljlucas@mac.com>
SWISH++ August 2, 2005 httpindex(1)