Sponsored Content
Top Forums Shell Programming and Scripting Extracting urls from curl output Post 302964630 by yazu on Saturday 16th of January 2016 08:59:26 PM
Old 01-16-2016
If you can use lynx then
Code:
lynx -dump URL

produces a good text output of a page. Every link on the page goes to References section in the end of the output.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extracting fields from an output 8-)

I am getting a variable as x=2006/01/18 now I have to extract each field from it. Like x1=2006, x2=01 and x3=18. Any idea how? Thanks a lot for help. Thanks CSaha (6 Replies)
Discussion started by: csaha
6 Replies

2. Shell Programming and Scripting

let curl output to stdout AND save to a file

hello hackers. i have a curl process running as cgi directly pushing stdout to the client. but i want to additionally save that stream to a file at the same time. any directions madly welcome. thanks in advance (3 Replies)
Discussion started by: scarfake
3 Replies

3. Shell Programming and Scripting

Pattern matching extracting urls from rss, shell scripts

Hi all, how could i do ? I have a Rss file, i want to extract only the Urls (many) matching http://www.xxx.com/trailers/ from that file and copy into another file. like " <pubDate>Wed, 29 Apr 2009 00:00:00 PST</pubDate> <content:encoded><!Apple - Movie Trailers - The Hangover"><img... (3 Replies)
Discussion started by: BremboloIV
3 Replies

4. Shell Programming and Scripting

script to output curl result as html

hi, new to scripting and would like to know how can I have a script which will curl a few URLs and have the results such as the URLs being curled, dns lookup time, connection time, total time, etc save in a html format in a form of table with column and rows. thank you. (4 Replies)
Discussion started by: squidusr
4 Replies

5. Shell Programming and Scripting

Getting cURL to output verbose to a file

This is about to drive me crazy. What I want to do is simple: output ALL the verbose information from curl to a file I have read the manual, tried several options and searched this forum but no salvation... I'm using curl -k -Q "command" --user user:passwd --ftp-pasv --ftp-ssl -v... (1 Reply)
Discussion started by: caramandi
1 Replies

6. Shell Programming and Scripting

web service call: curl output to xsltproc input

I need to invoke a web service and extract what I need from the response using a combination of curl and xsltproc. However, any file-based parameters that must be supplied to both these programs must be from stdin and not actual files. At least with curl, it seems to think that I am supplying a... (3 Replies)
Discussion started by: webuser
3 Replies

7. Shell Programming and Scripting

ery weird wget/curl output - what should I do?

Hi, I'm trying to write a script to download RedHat's errata digest. It comes in a txt.gz format, and i can get it easily with firefox. HOWEVER: output is VERY strange when donwloading it in a script. It seems I'm getting a file of the same size - but partially text and partly binary! It... (5 Replies)
Discussion started by: jstilby
5 Replies

8. Shell Programming and Scripting

Encapsulating output of CURL and/or WGET

i use curl and wget quite often. i set up alarms on their output. for instance, i would run a "wget" on a url and then search for certain strings within the output given by the "wget". the problem is, i cant get the entire output or response of my wget/curl command to show up correctly in... (3 Replies)
Discussion started by: SkySmart
3 Replies

9. Shell Programming and Scripting

Filter output in curl

Hello guys, I'm writing a little script which sends me sms with my shell script via api of a sms provider. problem is I can't filter my curl output for this site: site url:... (1 Reply)
Discussion started by: genius90
1 Replies

10. Web Development

Filename output in curl

How can I get the name of the default output filename from curl using the argument -O? Using -o one can choose a filename. I want to get the name of the original file, but don't understand how to get it. curl -o filename http://www.website.com curl -O http://www.website.com The... (3 Replies)
Discussion started by: locoroco
3 Replies
DPKG-WWW(1)						      General Commands Manual						       DPKG-WWW(1)

NAME
dpkg-www - program to remotely open a WWW Debian package browser SYNOPSIS
dpkg-www [-s|--stdout] [-h|--host host] [query] DESCRIPTION
dpkg-www is used to remotely control a WEB browser and open a dpkg URL on the local host from an interactive shell or script. If the command is invoked while running under X-window the script will try to find an installed X browser to open the specified URL. If not running under X it will try to use a text browser instead. If Netscape is found and a Netscape instance is already running it will be asked to open the dpkg URL on localhost with the optional query supplied on the command line. If there is no browser running, it will start automatically a new one. The functionality provided by this program is identical to running a web browser with the -remote openURL(http://localhost/cgi-bin/dpkg) argument or opening the same URL from within the browser. dpkg-www-installer is an helper application which can configured in the WWW browser for web installation. It should never be invoked directly by the user. OPTIONS
-s, --stdout Redirect output to stdout. Requires one of the text browsers (lynx, lynx-ssl or links) installed. -h, --host host Send the query to a remote host, where dpkg-www must be installed. query Specifies an optional package name, an absolute pathname or a query argument which is passed to the dpkg cgi-bin. See dpkg-www(8) for more information about the use of the cgi. FILES
/etc/dpkg-www.conf ~/.dpkg-www Configuration files for dpkg-www. It is not necessary for these files to exist, there are sensible defaults for everything, but you can specify your preferred www browser with the DPKG_WWW_BROWSER variable, for example: DPKG_WWW_BROWSER=mozilla EXAMPLES
dpkg-www This would open a dpkg URL on localhost listing all the installed packages. dpkg-www bash This would open a dpkg URL asking info on the bash package. dpkg-www -h pisolo bash This would open a dpkg URL asking info on the bash package on host pisolo. dpkg-www 'dpkg*' This would open a dpkg URL listing all packages matching dpkg*. dpkg-www /bin/bash This would open a dpkg URL asking info on the package(s) owning the file /bin/bash . dpkg-www depends=svgalib This would open a dpkg URL listing all packages depending on svgalib. dpkg-www --stdout depends=awk | grep ^ii This would list on stdout all packages depending on awk and grep all lines of installed packages. SEE ALSO
dpkg(8), dpkg-www(8) AUTHOR
Massimo Dal Zotto <dz@debian.org>. Bugs should be reported via the normal Debian bug reporting system. LICENCE
dpkg-www is licensed under the GNU General Public License version 2. September 1, 2004 DPKG-WWW(1)
All times are GMT -4. The time now is 05:00 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy