Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Reading URL using Mechanize and dump all the contents of the URL to a file
Top Forums Shell Programming and Scripting Reading URL using Mechanize and dump all the contents of the URL to a file Post 302888400 by ctsgnb on Friday 14th of February 2014 05:43:59 AM
Old 02-14-2014
Check the wget utility, i guess it does what you need ... without having to reinvent the wheel Smilie
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Replacing URL in a file with space

Hi, I have a file with a URL text written in it within double quotes e.g. "http://abcd.xyz.com/mno/somefile.dtd" I want the above text to get replaced by a single space character. I tried cat File1.txt | sed -e 's/("http)*(dtd")/ /g' > File2.txt But it didnt work out. Can someone... (5 Replies)
Discussion started by: dsrookie
5 Replies

2. UNIX for Advanced & Expert Users

Posting a file from Unix to URl

HI Can you please help me,how to post a xml file from Unix to URL. Basically,i want to map contents of my file at an url Regards Pooja (1 Reply)
Discussion started by: PoojaM
1 Replies

3. UNIX for Advanced & Expert Users

url calling and parameter passing to url in script

Hi all, I need to write a unix script in which need to call a url. Then need to pass parameters to that url. please help. Regards, gander_ss (1 Reply)
Discussion started by: gander_ss
1 Replies

4. Shell Programming and Scripting

url calling and parameter passing to url in script

Hi all, I need to write a unix script in which need to call a url. Then need to pass parameters to that url. please help. Regards, gander_ss (1 Reply)
Discussion started by: gander_ss
1 Replies

5. UNIX for Dummies Questions & Answers

ReDirecting a URL to another URL - Linux

Hello, I need to redirect an existing URL, how can i do that? There's a current web address to a GUI that I have to redirect to another webaddress. Does anyone know how to do this? This is on Unix boxes Linux. example: https://m45.testing.address.net/host.php make it so the... (3 Replies)
Discussion started by: SkySmart
3 Replies

6. Shell Programming and Scripting

[lynx dump] Order (by name/URL)

Hi :) How to use dump in lynx. $ lynx -dump http://www.google.com So, this is an example of a lynx dump: txt1 blabla Other txt some text 1. http://url_of_txt1 2. http://url_of_blabla 3. http://url_of_Other_txt 4. http://url_of_some_text ... How can i obtain this output? ... (12 Replies)
Discussion started by: aspire
12 Replies

7. Web Development

Regex to rewrite URL to another URL based on HTTP_HOST?

I am trying to find a way to test some code, but I need to rewrite a specific URL only from a specific HTTP_HOST The call goes out to http://SUB.DOMAIN.COM/showAssignment/7bde10b45efdd7a97629ef2fe01f7303/jsmodule/Nevow.Athena The ID in the middle is always random due to the cookie. I... (5 Replies)
Discussion started by: EXT3FSCK
5 Replies

8. Shell Programming and Scripting

Given a url how to get last part of file

HI, I have a URL that points to a file: LINK= "http://www.webpage.org/project/team2/file.tar" However when I try to use wget on this variable I receive the following error. wget $LINK line 4: http://www.webpage.org/project/team2/file.tar: No such file or directory wget:... (1 Reply)
Discussion started by: bashnewbee
1 Replies

9. UNIX for Dummies Questions & Answers

Awk: print all URL addresses between iframe tags without repeating an already printed URL

Here is what I have so far: find . -name "*php*" -or -name "*htm*" | xargs grep -i iframe | awk -F'"' '/<iframe*/{gsub(/.\*iframe>/,"\"");print $2}' Here is an example content of a PHP or HTM(HTML) file: <iframe src="http://ADDRESS_1/?click=5BBB08\" width=1 height=1... (18 Replies)
Discussion started by: striker4o
18 Replies

10. Shell Programming and Scripting

Extracting the file name from the specified URL

Hello Everyone, I am trying to write a shell script(or Perl Script) that would do the following: I have a file that contains the following lines: File: https://ims-svnus.com/dev/DB/trunk/feeds/templates/shell_script.txt -r860... (5 Replies)
Discussion started by: filter
5 Replies

LEARN ABOUT CENTOS

httppower

httppower(8)							     powerman							      httppower(8)

NAME

       httppower - communicate with HTTP based power distribution units

SYNOPSIS

       httppower [--url URL]

DESCRIPTION

       httppower  is  a  helper program for powerman which enables it to communicate with HTTP based power distribution units.	It is run interac-
       tively by the powerman daemon.

OPTIONS

       -u, --url URL
	      Set the base URL.

INTERACTIVE COMMANDS

       The following commands are accepted at the httppower> prompt:

       auth user:pass
	      Authenticate to the base URL with specified user and password, using ``basic'' HTTP authentication which sends the user and password
	      over the network in plain text.

       seturl URL
	      Set the base URL.  Overrides the command line option.

       get [URL-suffix]
	      Send an HTTP GET to the base URL with the optional URL-suffix appended.

       post [URL-suffix] key=val[&key=val]...
	      Send an HTTP POST to the base URL with the optional URL-suffix appended, and key-value pairs as argument.

FILES

       /usr/sbin/httppower
       /etc/powerman/powerman.conf

ORIGIN

       PowerMan was originally developed by Andrew Uselton on LLNL's Linux clusters.  This software is open source and distributed under the terms
       of the GNU GPL.

SEE ALSO

       powerman(1), powermand(8), httppower(8), plmpower(8), vpcd(8), powerman.conf(5), powerman.dev(5), powerman-devices(7).

       http://sourceforge.net/projects/powerman

powerman-2.3.5							    2009-02-09							      httppower(8)
All times are GMT -4. The time now is 02:37 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy