Sponsored Content
Top Forums Shell Programming and Scripting Dump web page source as rendered by browser Post 302960855 by Marmz on Friday 20th of November 2015 09:16:57 AM
Old 11-20-2015
Tools Dump web page source as rendered by browser

Hi guys|

I need to retrieve a specific .m3u8 link from a web page, which makes use of iframes and JavaScript

I tried to get the full source with "wget", "lynx", "w3m" and "phantomjs", but they can't dump all the source, with the part containing the link that i need, which seems to be inside an iframe..

Also the browser "QT web", which has an option to dump to stdout everything it renders of a page, can't output all the source i need...

I tried to "wget" that specific iframe but it's not accessible, if not from the site itself (i get a "forbidden" message Smilie)

Only with the function “inspect element“ in Chrome (right-clicking the area where the stream is loaded) i can see that part of the code (and the.m3u8 link too! Smilie)...
If i use the "Save page" option, instead, that code is missing..

I was wondering if maybe the same Chrome (or something similar) has an option that, from the command line, allows you to write everything it renders as a “dump“.

Sice that part of the source is accessible with “inspect element“, i can't believe there's no way to get it to console stdout (or to a dump fle).

Thanks in advance to all of you.. maybe this discussion could be useful for other users Smilie


[EDIT]
I hope this is the right forum section for my question ..

Last edited by Marmz; 11-20-2015 at 10:24 AM..
 

8 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Web browser and web server for Unix

Hi there all I am looking for both a web browser as well as a web server (Ie. Netscape Fasttrack) that will run on HPUX 10. If you know where I can get these free via FTP, please contact me as soon as possible!! Thanks, Mark (1 Reply)
Discussion started by: mleathers
1 Replies

2. UNIX for Dummies Questions & Answers

Which comand to show the source code on a web page?

Hi folks! I am using MacOsX that runs freeBSD. Could you tell me what comand to type on the Unix Terminal to display on the terminal the source code of a certain web page? I think something like #<comand> http://www.apple.com will display on the terminal's window the html source code... (11 Replies)
Discussion started by: fundidor
11 Replies

3. UNIX for Dummies Questions & Answers

reading web page source in unix

is there a command that allows you to take a url and grab the source code from the page and output it to stdout? i want to know because i want to grab a page and pass it thru another program to analyze the page. any help would be appreciated thanks (3 Replies)
Discussion started by: jaymzlee
3 Replies

4. Programming

Need simple web browser source code

Hello dear...i'm need web browser source code just the simple web browser..for my development site porn detected..thanks,,, (3 Replies)
Discussion started by: demhyt
3 Replies

5. Solaris

Web browser not opening

On clicking on my WEB BROWSER . no window is appearing , only a dialog box appearing with named "ALERT" any nothing else. What is browser package name , which is by default in Solaris 10 (3 Replies)
Discussion started by: kush singh
3 Replies

6. Shell Programming and Scripting

web page source cleanup

is it possible to pass webpages to remove all tag style information, but leave the tag... say I have <h1 style='font-size: xxx; color: xxxxxx'>headline 1</h1> i want to get <h1>headline 1</h1> BTW, i got an oneliner here to remove all tags: sed -n '/^$/!{s/<*>//g;p; Thanks a... (4 Replies)
Discussion started by: dtdt
4 Replies

7. Shell Programming and Scripting

Performing extractions on web source page

I have downloaded a web source page to a file. I then egrep a single word to extract a line containing it to another file. I then cat the second file and remove everything before a word and after a second word to capture the phrase desired. This did not work. I used vi to validate that the 2... (1 Reply)
Discussion started by: slak0
1 Replies

8. Web Development

What is Your Favorite Web Browser?

OK, please vote on your favorite web browser and comment why! Only currently supported browers please! Thanks. PS: I am a big FireFox fan. FireFox add-ons rock! (61 Replies)
Discussion started by: Neo
61 Replies
GIT-WEB--BROWSE(1)						    Git Manual							GIT-WEB--BROWSE(1)

NAME
git-web--browse - Git helper script to launch a web browser SYNOPSIS
git web--browse [OPTIONS] URL/FILE ... DESCRIPTION
This script tries, as much as possible, to display the URLs and FILEs that are passed as arguments, as HTML pages in new tabs on an already opened web browser. The following browsers (or commands) are currently supported: o firefox (this is the default under X Window when not using KDE) o iceweasel o seamonkey o iceape o chromium (also supported as chromium-browser) o google-chrome (also supported as chrome) o konqueror (this is the default under KDE, see Note about konqueror below) o opera o w3m (this is the default outside graphical environments) o elinks o links o lynx o dillo o open (this is the default under Mac OS X GUI) o start (this is the default under MinGW) o cygstart (this is the default under Cygwin) o xdg-open Custom commands may also be specified. OPTIONS
-b <browser>, --browser=<browser> Use the specified browser. It must be in the list of supported browsers. -t <browser>, --tool=<browser> Same as above. -c <conf.var>, --config=<conf.var> CONF.VAR is looked up in the Git config files. If it's set, then its value specifies the browser that should be used. CONFIGURATION VARIABLES
CONF.VAR (from -c option) and web.browser The web browser can be specified using a configuration variable passed with the -c (or --config) command-line option, or the web.browser configuration variable if the former is not used. browser.<tool>.path You can explicitly provide a full path to your preferred browser by setting the configuration variable browser.<tool>.path. For example, you can configure the absolute path to firefox by setting browser.firefox.path. Otherwise, git web--browse assumes the tool is available in PATH. browser.<tool>.cmd When the browser, specified by options or configuration variables, is not among the supported ones, then the corresponding browser.<tool>.cmd configuration variable will be looked up. If this variable exists then git web--browse will treat the specified tool as a custom command and will use a shell eval to run the command with the URLs passed as arguments. NOTE ABOUT KONQUEROR
When konqueror is specified by a command-line option or a configuration variable, we launch kfmclient to try to open the HTML man page on an already opened konqueror in a new tab if possible. For consistency, we also try such a trick if browser.konqueror.path is set to something like A_PATH_TO/konqueror. That means we will try to launch A_PATH_TO/kfmclient instead. If you really want to use konqueror, then you can use something like the following: [web] browser = konq [browser "konq"] cmd = A_PATH_TO/konqueror Note about git-config --global Note that these configuration variables should probably be set using the --global flag, for example like this: $ git config --global web.browser firefox as they are probably more user specific than repository specific. See git-config(1) for more information about this. GIT
Part of the git(1) suite Git 2.17.1 10/05/2018 GIT-WEB--BROWSE(1)
All times are GMT -4. The time now is 05:24 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy