My script needs to crawl the data from a third party site. Currently it is written in wget. The third party site is of shared interface with different IP addresses.
My wget works with all the IP address but not with one. Whereas the curl is able to hit that IP address and comes out with 200 status.
Hence I would like to know why my wget is not working whereas the curl succeeds for the same IP address.
How can i resolve this Problem, i don't want my script to get rewritten with curl. Your help at this moment is so huge, thanks in advance.
It hungs and comes out reaching the timeout.
PS: I have added the Proxy in my ~/.wgetrc file
---------- Post updated at 09:14 AM ---------- Previous update was at 04:57 AM ----------
We are trying to invoke a https service from our unix script using curl command. The service is not getting invoked because it is SSL configured. Bypassing certification (using curl –k) does not work.
curl -k https://site
curl -k -x IP:Port https://site
curl -k -x IP:443 https://id:pwd@site
... (0 Replies)
Hello,
I am wondering does anyone know of a method using curl/wget or other where by I could specify the IP address of the server I wish to query for a website.
Something similar to editing /etc/hosts but that can be done directly from the command line. I have looked through the man pages... (4 Replies)
Hi
I need a Shell script that will download a zip file every second from a http server but i can't use neither curl nor wget.
Can anyone will help me go about this task ???
Thanks!! (1 Reply)
i use curl and wget quite often.
i set up alarms on their output. for instance, i would run a "wget" on a url and then search for certain strings within the output given by the "wget".
the problem is, i cant get the entire output or response of my wget/curl command to show up correctly in... (3 Replies)
Experts,
I login to a 3rd party and pull some valuable information with my credentials. I pass my credentials via --post-data in wget.
Now my Account is locked. I want my wget to alert that the Account is locked. How can i achieve this.
My idea is, get the Source page html from the... (2 Replies)
Hello,
What I am trying to do is to get html data of a website automatically.
Firstly I decided to do it manually and via terminal I entered below code:
$ wget http://www.***.*** -q -O code.html
Unfortunately code.html file was empty.
When I enter below code it gave Error 303-304
$... (1 Reply)
i'm using this command to post data to a remote host:
wget --post-data="My Data" http://<my-ip>:80 -O /dev/null -q
and
curl --data "My Data" http://<my-ip>:80
however, when i run the above, i see the following in my access log on the remote host:
Wget:
10.10.10.10 - - "POST /... (1 Reply)
What can I use instead of wget/curl when I need to log into websites that use javascript?
Wget and curl don't handle javascript. (6 Replies)
Discussion started by: locoroco
6 Replies
LEARN ABOUT MOJAVE
curlopt_proxytype
CURLOPT_PROXYTYPE(3) curl_easy_setopt options CURLOPT_PROXYTYPE(3)NAME
CURLOPT_PROXYTYPE - proxy protocol type
SYNOPSIS
#include <curl/curl.h>
CURLcode curl_easy_setopt(CURL *handle, CURLOPT_PROXYTYPE, long type);
DESCRIPTION
Pass one of the values below to set the type of the proxy.
CURLPROXY_HTTP
HTTP Proxy. Default.
CURLPROXY_HTTPS
HTTPS Proxy. (Added in 7.52.0 for OpenSSL, GnuTLS and NSS)
CURLPROXY_HTTP_1_0
HTTP 1.0 Proxy. This is very similar to CURLPROXY_HTTP except it uses HTTP/1.0 for any CONNECT tunnelling. It does not change
the HTTP version of the actual HTTP requests, controlled by CURLOPT_HTTP_VERSION(3).
CURLPROXY_SOCKS4
SOCKS4 Proxy.
CURLPROXY_SOCKS4A
SOCKS4a Proxy. Proxy resolves URL hostname.
CURLPROXY_SOCKS5
SOCKS5 Proxy.
CURLPROXY_SOCKS5_HOSTNAME
SOCKS5 Proxy. Proxy resolves URL hostname.
Often it is more convenient to specify the proxy type with the scheme part of the CURLOPT_PROXY(3) string.
DEFAULT
CURLPROXY_HTTP
PROTOCOLS
Most
EXAMPLE
TODO
AVAILABILITY
Always
RETURN VALUE
Returns CURLE_OK
SEE ALSO CURLOPT_PROXY(3), CURLOPT_PROXYPORT(3),
libcurl 7.54.0 February 06, 2017 CURLOPT_PROXYTYPE(3)