|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
Dump web page source as rendered by browser
Post 302960855 by Marmz on Friday 20th of November 2015 09:16:57 AM
11-20-2015
Dump web page source as rendered by browser
Hi guys|
I need to retrieve a specific .m3u8 link from a web page, which makes use of iframes and JavaScript
I tried to get the full source with "wget", "lynx", "w3m" and "phantomjs", but they can't dump all the source, with the part containing the link that i need, which seems to be inside an iframe..
Also the browser "QT web", which has an option to dump to stdout everything it renders of a page, can't output all the source i need...
I tried to "wget" that specific iframe but it's not accessible, if not from the site itself (i get a "forbidden" message
)
Only with the function “inspect element“ in Chrome (right-clicking the area where the stream is loaded) i can see that part of the code (and the.m3u8 link too!
)...
If i use the "Save page" option, instead, that code is missing..
I was wondering if maybe the same Chrome (or something similar) has an option that, from the command line, allows you to write everything it renders as a “dump“.
Sice that part of the source is accessible with “inspect element“, i can't believe there's no way to get it to console stdout (or to a dump fle).
Thanks in advance to all of you.. maybe this discussion could be useful for other users
[EDIT]
I hope this is the right forum section for my question ..
I need to retrieve a specific .m3u8 link from a web page, which makes use of iframes and JavaScript
I tried to get the full source with "wget", "lynx", "w3m" and "phantomjs", but they can't dump all the source, with the part containing the link that i need, which seems to be inside an iframe..
Also the browser "QT web", which has an option to dump to stdout everything it renders of a page, can't output all the source i need...
I tried to "wget" that specific iframe but it's not accessible, if not from the site itself (i get a "forbidden" message
)Only with the function “inspect element“ in Chrome (right-clicking the area where the stream is loaded) i can see that part of the code (and the.m3u8 link too!
)...If i use the "Save page" option, instead, that code is missing..
I was wondering if maybe the same Chrome (or something similar) has an option that, from the command line, allows you to write everything it renders as a “dump“.
Sice that part of the source is accessible with “inspect element“, i can't believe there's no way to get it to console stdout (or to a dump fle).
Thanks in advance to all of you.. maybe this discussion could be useful for other users
[EDIT]
I hope this is the right forum section for my question ..
Last edited by Marmz; 11-20-2015 at 10:24 AM..
|
|
8 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi there all
I am looking for both a web browser as well as a web server (Ie. Netscape Fasttrack) that will run on HPUX 10.
If you know where I can get these free via FTP, please contact me as soon as possible!!
Thanks,
Mark (1 Reply)
Discussion started by: mleathers
1 Replies
2. UNIX for Dummies Questions & Answers
Hi folks!
I am using MacOsX that runs freeBSD. Could you tell me what comand to type on the Unix Terminal to display on the terminal the source code of a certain web page?
I think something like
#<comand> http://www.apple.com
will display on the terminal's window the html source code... (11 Replies)
Discussion started by: fundidor
11 Replies
3. UNIX for Dummies Questions & Answers
is there a command that allows you to take a url and grab the source code from the page and output it to stdout?
i want to know because i want to grab a page and pass it thru another program to analyze the page.
any help would be appreciated
thanks (3 Replies)
Discussion started by: jaymzlee
3 Replies
4. Programming
Hello dear...i'm need web browser source code just the simple web browser..for my development site porn detected..thanks,,, (3 Replies)
Discussion started by: demhyt
3 Replies
5. Solaris
On clicking on my WEB BROWSER . no window is appearing , only a dialog box appearing with named "ALERT" any nothing else.
What is browser package name , which is by default in Solaris 10 (3 Replies)
Discussion started by: kush singh
3 Replies
6. Shell Programming and Scripting
is it possible to pass webpages to remove all tag style information, but leave the tag...
say I have
<h1 style='font-size: xxx; color: xxxxxx'>headline 1</h1>
i want to get
<h1>headline 1</h1>
BTW, i got an oneliner here to remove all tags:
sed -n '/^$/!{s/<*>//g;p;
Thanks a... (4 Replies)
Discussion started by: dtdt
4 Replies
7. Shell Programming and Scripting
I have downloaded a web source page to a file. I then egrep a single word to extract a line containing it to another file.
I then cat the second file and remove everything before a word and after a second word to capture the phrase desired.
This did not work. I used vi to validate that the 2... (1 Reply)
Discussion started by: slak0
1 Replies
8. Web Development
OK, please vote on your favorite web browser and comment why!
Only currently supported browers please!
Thanks.
PS: I am a big FireFox fan. FireFox add-ons rock! (61 Replies)
Discussion started by: Neo
61 Replies
LEARN ABOUT NETBSD
hprop
HPROP(8) BSD System Manager's Manual HPROP(8) NAME
hprop -- propagate the KDC database SYNOPSIS
hprop [-m file | --master-key=file] [-d file | --database=file] [--source=heimdal|mit-dump] [-r string | --v4-realm=string] [-c cell | --cell=cell] [-k keytab | --keytab=keytab] [-R string | --v5-realm=string] [-D | --decrypt] [-E | --encrypt] [-n | --stdout] [-v | --verbose] [--version] [-h | --help] [host[:port]] ... DESCRIPTION
hprop takes a principal database in a specified format and converts it into a stream of Heimdal database records. This stream can either be written to standard out, or (more commonly) be propagated to a hpropd(8) server running on a different machine. If propagating, it connects to all hosts specified on the command by opening a TCP connection to port 754 (service hprop) and sends the data- base in encrypted form. Supported options: -m file, --master-key=file Where to find the master key to encrypt or decrypt keys with. -d file, --database=file The database to be propagated. --source=heimdal|mit-dump|krb4-dump|kaserver Specifies the type of the source database. Alternatives include: heimdal a Heimdal database mit-dump a MIT Kerberos 5 dump file +.It Fl k Ar keytab , Fl -keytab= Ns Ar keytab The keytab to use for fetching the key to be used for authenticating to the propaga- tion daemon(s). The key hprop/hostname is used from this keytab. The default is to fetch the key from the KDC database. -R string, --v5-realm=string Local realm override. -D, --decrypt The encryption keys in the database can either be in clear, or encrypted with a master key. This option transmits the database with unencrypted keys. -E, --encrypt This option transmits the database with encrypted keys. -n, --stdout Dump the database on stdout, in a format that can be fed to hpropd. EXAMPLES
The following will propagate a database to another machine (which should run hpropd(8)): $ hprop slave-1 slave-2 SEE ALSO
hpropd(8) BSD
December 8, 2004 BSD