Change the letters from 0000 to 99ZZ so the first two digits should be all numbers from 00 to 99 and the last two digits every letter and every number in every combination.
If you have a mac, you'll have to use curl. It's actually fairly good at this -- you can tell it whole lists of things to fetch, and split on --_curl_-- to tell when each new page begins and ends when you pipe it into something else. Unfortunately, it'll take [a-z] and [0-9], but not [a-z0-9] or [0123456789abcdefghijklmnopqrstuvwxyz], so you have to give it four blocks of stuff to fetch:
Something like:
Code:
BASE="http://website/xxxxx[00-99]"
TAIL="yyyyy"
# Fetch all pages with curl, feed them through awk, print all pages containing 'searchstr'
curl "${BASE}[0-9][0-9]${TAIL}" "${BASE}[0-9][a-z]${TAIL}" "${BASE}[a-z][0-9]${TAIL}" "${BASE}[a-z][a-z]${TAIL}" 2>/dev/null |
# Split on curl header of --_curl_--. $1 is the URL following it. Print all URLs for pages containing 'searchstr'
awk -v RS="^--_curl_--" -v FS="\n" '/searchstr/ { print $1 }'
Last edited by Corona688; 10-11-2011 at 07:57 PM..
Might be a foolish question, but how do I install curl? I downloaded it (curl-7.19.7). The instructions from curl itself don't work. Says can't find make. Configuration also fails. And then, how to run your script? Is this all code I need? Thank you!
Hi all,
i need to do the following,
when i connect to my website it prints out "Welcome User",
but sometimes there are errors like "dictionary not loaded" or "wrong user name or password"
so i wanted to make a script that checks that login page,
and if i get the Welcome massage do nothing,... (6 Replies)
hello
i'm doing a assignment in bash building a crawler..
i downloaded a url into a file and i need to check if the url is an html format and also to save in a different file all the html's links..
please help me
thx (1 Reply)
hello!!
well, i am planning to make my own virtual pet site like that of a neopets.
unfortunately i don't have any idea on how to do it..
i've tried searching in the net, but the result is really complicated. i don't know where to begin.i have already drawn some that i think would help... (2 Replies)
Hey guys I know you probably get this question a lot but, I want to make a website, and I don't have any experience doing this. I have a iMac and i was wondering if there is someone you could refer me to or a site that will show me how to do it. Thanks. (2 Replies)
HELLO FELLOW GEEKS. PLZ CHECK OUT MY FRIENDS SITE AT http://isunshine.dhs.org or u can also join the message board at http://isunshine.dhs.org/scripts/ikonboard.cgi
wixifer (1 Reply)
