12-18-2009
Getting source code of a page
I want to download a particular page from the internet and get the source code of the page in html format.
I want to parse the source code to find a specific parameters using grep command.
could someone tell me the linux command to download a specific page and parse the source code of it.
Thanks
Ahamed
9 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi folks!
I am using MacOsX that runs freeBSD. Could you tell me what comand to type on the Unix Terminal to display on the terminal the source code of a certain web page?
I think something like
#<comand> http://www.apple.com
will display on the terminal's window the html source code... (11 Replies)
Discussion started by: fundidor
11 Replies
2. UNIX for Dummies Questions & Answers
is there a command that allows you to take a url and grab the source code from the page and output it to stdout?
i want to know because i want to grab a page and pass it thru another program to analyze the page.
any help would be appreciated
thanks (3 Replies)
Discussion started by: jaymzlee
3 Replies
3. Shell Programming and Scripting
I'm new to PERL, but I want to take the page source and write it to a file or standard output. I used perl.org as a test website. Here is the script:
use strict;
use warnings;
use LWP::Simple;
getprint('http://www.perl.org') or die 'Unable to get page';
exit 0;
... (1 Reply)
Discussion started by: wxornot
1 Replies
4. Shell Programming and Scripting
is it possible to pass webpages to remove all tag style information, but leave the tag...
say I have
<h1 style='font-size: xxx; color: xxxxxx'>headline 1</h1>
i want to get
<h1>headline 1</h1>
BTW, i got an oneliner here to remove all tags:
sed -n '/^$/!{s/<*>//g;p;
Thanks a... (4 Replies)
Discussion started by: dtdt
4 Replies
5. Shell Programming and Scripting
I have downloaded a web source page to a file. I then egrep a single word to extract a line containing it to another file.
I then cat the second file and remove everything before a word and after a second word to capture the phrase desired.
This did not work. I used vi to validate that the 2... (1 Reply)
Discussion started by: slak0
1 Replies
6. Shell Programming and Scripting
I need to get the source code of a webpage. I have tried to use wget and curl, but it doesn't show the necessary javascript part of the source. I don't have to execute it, only to view the source.
How do I do that? (1 Reply)
Discussion started by: locoroco
1 Replies
7. Shell Programming and Scripting
Hi,
I want to remove the following code from Source files (or replace the code with empty.) from all the source files in given directory.
finally {
if (null != hibernateSession && hibernateSession.isOpen()) {
//hibernateSession.close();
}
}
It would be great if the script has... (2 Replies)
Discussion started by: hareeshram
2 Replies
8. Shell Programming and Scripting
Hi Friends,
I have a bunch of URLs.
Each URL will open up an abstract page.
But, the source contains a link to the main PDF article.
I am looking for a script to do the following task
1. Read input file with URLs.
2. Parse the source and grab all the lines that has the word 'PDF'.... (1 Reply)
Discussion started by: jacobs.smith
1 Replies
9. Shell Programming and Scripting
Hi guys|
I need to retrieve a specific .m3u8 link from a web page, which makes use of iframes and JavaScript
I tried to get the full source with "wget", "lynx", "w3m" and "phantomjs", but they can't dump all the source, with the part containing the link that i need, which seems to be inside... (0 Replies)
Discussion started by: Marmz
0 Replies
LEARN ABOUT DEBIAN
pull-lp-source
PULL-LP-SOURCE(1) General Commands Manual PULL-LP-SOURCE(1)
NAME
pull-lp-source - download a source package from Launchpad
SYNOPSIS
pull-lp-source [options] source package [release|version]
DESCRIPTION
pull-lp-source downloads and extracts the specified version of <source package> from Launchpad, or the latest version of the specified
release. To request a version from a particular pocket say release-pocket (with a magic -release for only the release pocket). If no ver-
sion or release is specified, the latest version in the development release will be downloaded.
OPTIONS
Listed below are the command line options for pull-lp-source:
source package
This is the source package that you would like to be downloaded from Launchpad.
version
This is the version of the source package to be downloaded.
release
This is the release that you would like the source package to be downloaded from. This value defaults to the current development
release.
-h, --help
Display a help message and exit.
-d, --download-only
Do not extract the source package.
-m UBUNTU_MIRROR, --mirror=UBUNTU_MIRROR
Use the specified Ubuntu mirror. Should be in the form http://archive.ubuntu.com/ubuntu. If the package isn't found on this mir-
ror, pull-lp-source will fall back to Launchpad, as its name implies.
--no-conf
Do not read any configuration files, or configuration from environment variables.
ENVIRONMENT
All of the CONFIGURATION VARIABLES below are also supported as environment variables. Variables in the environment take precedence to
those in configuration files.
DIST Specifies the default target.
CONFIGURATION VARIABLES
The following variables can be set in the environment or in ubuntu-dev-tools(5) configuration files. In each case, the script-specific
variable takes precedence over the package-wide variable.
PULL_LP_SOURCE_UBUNTU_MIRROR, UBUNTUTOOLS_UBUNTU_MIRROR
The default value for --mirror.
SEE ALSO
dget(1), pull-debian-source(1), pull-debian-debdiff(1), ubuntu-dev-tools(5)
AUTHOR
pull-lp-source and this manual page were written by Iain Lane <iain@orangesquash.org.uk>. Both are released under the GNU General Public
License, version 3 or later.
ubuntu-dev-tools 4 August 2008 PULL-LP-SOURCE(1)