Sponsored Content
Full Discussion: Website crawler
Top Forums Shell Programming and Scripting Website crawler Post 302563633 by yaylol on Tuesday 11th of October 2011 07:09:37 PM
Old 10-11-2011
Might be a foolish question, but how do I install curl? I downloaded it (curl-7.19.7). The instructions from curl itself don't work. Says can't find make. Configuration also fails. And then, how to run your script? Is this all code I need? Thank you!
 

7 More Discussions You Might Find Interesting

1. Filesystems, Disks and Memory

website

HELLO FELLOW GEEKS. PLZ CHECK OUT MY FRIENDS SITE AT http://isunshine.dhs.org or u can also join the message board at http://isunshine.dhs.org/scripts/ikonboard.cgi wixifer (1 Reply)
Discussion started by: wixifer
1 Replies

2. Web Development

sitemap.xml from crawler or CMS?

Hello, I have question. Is it better to use site Crawler or scripts from CMS/blog to do that? (2 Replies)
Discussion started by: gstoychev
2 Replies

3. UNIX for Dummies Questions & Answers

Website

Hey guys I know you probably get this question a lot but, I want to make a website, and I don't have any experience doing this. I have a iMac and i was wondering if there is someone you could refer me to or a site that will show me how to do it. Thanks. (2 Replies)
Discussion started by: mmecca21
2 Replies

4. Web Development

my website.please. help me.

hello!! well, i am planning to make my own virtual pet site like that of a neopets. unfortunately i don't have any idea on how to do it.. i've tried searching in the net, but the result is really complicated. i don't know where to begin.i have already drawn some that i think would help... (2 Replies)
Discussion started by: ackiemae
2 Replies

5. Homework & Coursework Questions

crawler in bash

hello i'm doing a assignment in bash building a crawler.. i downloaded a url into a file and i need to check if the url is an html format and also to save in a different file all the html's links.. please help me thx (1 Reply)
Discussion started by: yanivlug
1 Replies

6. Web Development

Which technology is used on this website?

Hi, Home ~ InfoBeans I want to know technology framework using which we can achieve that kind of look and feel Thanks (6 Replies)
Discussion started by: ezee
6 Replies

7. Shell Programming and Scripting

Monitor Website

Hi all, i need to do the following, when i connect to my website it prints out "Welcome User", but sometimes there are errors like "dictionary not loaded" or "wrong user name or password" so i wanted to make a script that checks that login page, and if i get the Welcome massage do nothing,... (6 Replies)
Discussion started by: charli1
6 Replies
CURLOPT_MAXREDIRS(3)					     curl_easy_setopt options					      CURLOPT_MAXREDIRS(3)

NAME
CURLOPT_MAXREDIRS - maximum number of redirects allowed SYNOPSIS
#include <curl/curl.h> CURLcode curl_easy_setopt(CURL *handle, CURLOPT_MAXREDIRS, long amount); DESCRIPTION
Pass a long. The set number will be the redirection limit amount. If that many redirections have been followed, the next redirect will cause an error (CURLE_TOO_MANY_REDIRECTS). This option only makes sense if the CURLOPT_FOLLOWLOCATION(3) is used at the same time. Setting the limit to 0 will make libcurl refuse any redirect. Set it to -1 for an infinite number of redirects. DEFAULT
-1, unlimited PROTOCOLS
HTTP(S) EXAMPLE
curl = curl_easy_init(); if(curl) { curl_easy_setopt(curl, CURLOPT_URL, "http://example.com/"); /* enable redirect following */ curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L); /* allow three redirects */ curl_easy_setopt(curl, CURLOPT_MAXREDIRS, 3L); /* Perform the request */ curl_easy_perform(curl); } AVAILABILITY
Along with HTTP RETURN VALUE
Returns CURLE_OK if HTTP is supported, and CURLE_UNKNOWN_OPTION if not. SEE ALSO
CURLOPT_FOLLOWLOCATION(3), libcurl 7.54.0 February 03, 2016 CURLOPT_MAXREDIRS(3)
All times are GMT -4. The time now is 06:06 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy