Sponsored Content
Top Forums Shell Programming and Scripting Dump web page source as rendered by browser Post 302960855 by Marmz on Friday 20th of November 2015 09:16:57 AM
Old 11-20-2015
Tools Dump web page source as rendered by browser

Hi guys|

I need to retrieve a specific .m3u8 link from a web page, which makes use of iframes and JavaScript

I tried to get the full source with "wget", "lynx", "w3m" and "phantomjs", but they can't dump all the source, with the part containing the link that i need, which seems to be inside an iframe..

Also the browser "QT web", which has an option to dump to stdout everything it renders of a page, can't output all the source i need...

I tried to "wget" that specific iframe but it's not accessible, if not from the site itself (i get a "forbidden" message Smilie)

Only with the function “inspect element“ in Chrome (right-clicking the area where the stream is loaded) i can see that part of the code (and the.m3u8 link too! Smilie)...
If i use the "Save page" option, instead, that code is missing..

I was wondering if maybe the same Chrome (or something similar) has an option that, from the command line, allows you to write everything it renders as a “dump“.

Sice that part of the source is accessible with “inspect element“, i can't believe there's no way to get it to console stdout (or to a dump fle).

Thanks in advance to all of you.. maybe this discussion could be useful for other users Smilie


[EDIT]
I hope this is the right forum section for my question ..

Last edited by Marmz; 11-20-2015 at 10:24 AM..
 

8 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Web browser and web server for Unix

Hi there all I am looking for both a web browser as well as a web server (Ie. Netscape Fasttrack) that will run on HPUX 10. If you know where I can get these free via FTP, please contact me as soon as possible!! Thanks, Mark (1 Reply)
Discussion started by: mleathers
1 Replies

2. UNIX for Dummies Questions & Answers

Which comand to show the source code on a web page?

Hi folks! I am using MacOsX that runs freeBSD. Could you tell me what comand to type on the Unix Terminal to display on the terminal the source code of a certain web page? I think something like #<comand> http://www.apple.com will display on the terminal's window the html source code... (11 Replies)
Discussion started by: fundidor
11 Replies

3. UNIX for Dummies Questions & Answers

reading web page source in unix

is there a command that allows you to take a url and grab the source code from the page and output it to stdout? i want to know because i want to grab a page and pass it thru another program to analyze the page. any help would be appreciated thanks (3 Replies)
Discussion started by: jaymzlee
3 Replies

4. Programming

Need simple web browser source code

Hello dear...i'm need web browser source code just the simple web browser..for my development site porn detected..thanks,,, (3 Replies)
Discussion started by: demhyt
3 Replies

5. Solaris

Web browser not opening

On clicking on my WEB BROWSER . no window is appearing , only a dialog box appearing with named "ALERT" any nothing else. What is browser package name , which is by default in Solaris 10 (3 Replies)
Discussion started by: kush singh
3 Replies

6. Shell Programming and Scripting

web page source cleanup

is it possible to pass webpages to remove all tag style information, but leave the tag... say I have <h1 style='font-size: xxx; color: xxxxxx'>headline 1</h1> i want to get <h1>headline 1</h1> BTW, i got an oneliner here to remove all tags: sed -n '/^$/!{s/<*>//g;p; Thanks a... (4 Replies)
Discussion started by: dtdt
4 Replies

7. Shell Programming and Scripting

Performing extractions on web source page

I have downloaded a web source page to a file. I then egrep a single word to extract a line containing it to another file. I then cat the second file and remove everything before a word and after a second word to capture the phrase desired. This did not work. I used vi to validate that the 2... (1 Reply)
Discussion started by: slak0
1 Replies

8. Web Development

What is Your Favorite Web Browser?

OK, please vote on your favorite web browser and comment why! Only currently supported browers please! Thanks. PS: I am a big FireFox fan. FireFox add-ons rock! (61 Replies)
Discussion started by: Neo
61 Replies
XDG-SETTINGS(1) 						xdg-settings Manual						   XDG-SETTINGS(1)

NAME
xdg-settings - get various settings from the desktop environment SYNOPSIS
xdg-settings {get | check | set} {property} [subproperty] [value] xdg-settings {--help | --list | --manual | --version} DESCRIPTION
xdg-settings gets various settings from the desktop environment. For instance, desktop environments often provide proxy configuration and default web browser settings. Using xdg-settings these parameters can be extracted for use by applications that do not use the desktop environment's libraries (which would use the settings natively). xdg-settings is for use inside a desktop session only. It is not recommended to use xdg-settings as root. OPTIONS
--help Show command synopsis. --list List all properties xdg-settings knows about. --manual Show this manual page. --version Show the xdg-utils version information. PROPERTIES
When using xdg-settings to get, check or set a destkop setting, properties and possibly sub-properties are used to specify the setting to be changed. Some properties (such as default-web-browser) fully describe the setting to be changed. Other properties (such as default-url-scheme-handler) require more information (in this case the actual scheme to set the default handler for) which must be provided in a sub-property. EXIT CODES
An exit code of 0 indicates success while a non-zero exit code indicates failure. The following failure codes can be returned: 1 Error in command line syntax. 2 One of the files passed on the command line did not exist. 3 A required tool could not be found. 4 The action failed. EXAMPLES
Get the desktop file name of the current default web browser xdg-settings get default-web-browser Check whether the default web browser is firefox.desktop, which can be false even if "get default-web-browser" says that is the current value (if only some of the underlying settings actually reflect that value) xdg-settings check default-web-browser firefox.desktop Set the default web browser to google-chrome.desktop xdg-settings set default-web-browser google-chrome.desktop Set the default mailto URL scheme handler to be evolution.desktop xdg-settings set default-url-scheme-handler mailto evolution.desktop AUTHOR
Mike Mammarella Author. COPYRIGHT
Copyright (C) 2009-2011 [FIXME: source] 06/10/2014 XDG-SETTINGS(1)
All times are GMT -4. The time now is 06:54 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy