Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

lynx --dump on site that needs username and password??

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting lynx --dump on site that needs username and password??
# 8  
Old 01-13-2010
That would seem to be the correct location and correct parameters. So far so good.

Next I'd suggest following up on my repeated suggestions of saving cookies, including session cookies, so wget can remember the login. wget has a manual page. If you need me to lead you keypress by keypress I will need to know what file you wish to download as well.
# 9  
Old 01-14-2010
Quote:
Originally Posted by Corona688
That would seem to be the correct location and correct parameters. So far so good.

Next I'd suggest following up on my repeated suggestions of saving cookies, including session cookies, so wget can remember the login. wget has a manual page. If you need me to lead you keypress by keypress I will need to know what file you wish to download as well.
Thank you very much for your prolonged help.

Firstly, I think by not using quotes it set variables as opposed to what it was supposed to do. So I had to put double quotes around the post data bit. Also I tried to save the session cookies by adding --keep-session cookies.

Code:
wget --keep-session-cookies "--post-data=username=*****password=*****" "http://www.torrentday.com/takelogin.php"

I am not trying to download a specific file but I want to set up a cronjob to download a file when it comes available.

I tried to get wget to check for new Big Brother releases by typing
Code:
 wget "http://www.torrentday.com/browse.php#/&search=big+brother&incldead=0&cata=yes"

but I don't think it loaded the session cookie.

I did find a --load-cookies option but I'm not sure how to use it correctly.
# 10  
Old 01-14-2010
It would probably help to actually save the cookies, too. Smilie

You dropped an ampersand, but you need it.

Code:
wget --save-cookies cookiefile --keep-session-cookies "--post-data=username=*****&password=*****" "http://www.torrentday.com/takelogin.php"
wget --load-cookies cookiefile --keep-session-cookies "http://www.torrentday.com/browse.php#/&search=big+brother&incldead=0&cata=yes"

Does this work? I hope so, but these things can be finicky sometimes. That # in the URL might need to go, but I'm not sure.
# 11  
Old 01-14-2010
Quote:
Originally Posted by Corona688
It would probably help to actually save the cookies, too. Smilie

You dropped an ampersand, but you need it.

Code:
wget --save-cookies cookiefile --keep-session-cookies "--post-data=username=*****&password=*****" "http://www.torrentday.com/takelogin.php"
wget --load-cookies cookiefile --keep-session-cookies "http://www.torrentday.com/browse.php#/&search=big+brother&incldead=0&cata=yes"

Does this work? I hope so, but these things can be finicky sometimes. That # in the URL might need to go, but I'm not sure.
Thanks. It works!! ..apart from the search term not coming up but now I know how to save and load cookies and how to post data I can work on that.

Thank you very much!! Smilie
# 12  
Old 01-14-2010
Here's a tip I just remembered: Some sites don't like to return proper results to anything they think isn't a web browser. wget reports that it's not a web browser because, well, it isn't. wget also reports no referrer for the same reason, which breaks some other web sites.

So the options -U netscape --referer=http://www.torrentday.com might help it cooperate if it's being persnickety.
# 13  
Old 01-14-2010
Quote:
Originally Posted by Corona688
Here's a tip I just remembered: Some sites don't like to return proper results to anything they think isn't a web browser. wget reports that it's not a web browser because, well, it isn't. wget also reports no referrer for the same reason, which breaks some other web sites.

So the options -U netscape --referer=http://www.torrentday.com might help it cooperate if it's being persnickety.
Thanks for your help, will do Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Red Hat

Need to crack username and password on laptop

I am not a computer geek but I recently was given a Dell Latitude c400 laptop which has Red Hat Linux 2.4.20-6 GRUB Version 0.93. First of all I have no clue how to even use this operating system and I was never given the username or password. Is there anyone out there who could possibly help me... (4 Replies)
Discussion started by: missfixit74
4 Replies

2. Shell Programming and Scripting

Steps after username and password is entered !

Hi, I know this sounds crazy question.. but I am just curious to know what happens next when I enter username and password and hit enter on a new Unix session (using Putty)? I mean which file gets executed, how the default login shell is determined etc... regards juzz4fun (5 Replies)
Discussion started by: juzz4fun
5 Replies

3. UNIX for Dummies Questions & Answers

How do you reset username/password

Picked up a 3b2 running System V. Works fine, but it requires a username and password. Is the username "root" or "sysadm"? How do I find out and how to I reset it or bypass it? Thanks. (2 Replies)
Discussion started by: TanRuNomad
2 Replies

4. Shell Programming and Scripting

Username and password

Hi I am new to using unix and am struggling with a script i am writing. What i am trying to do is get a user to enter a username, check the original file i created with username and pin to see if their is a corresponding entry. Next ask the user to enter the pin and see if this matches... (5 Replies)
Discussion started by: somersetdan
5 Replies

5. Shell Programming and Scripting

input username and password interactively

Hi guys, I am using Solaris 10 and we install IBM OnDemand server there. there's a command called arsload that we use to load document and query document too. I need to write a script to query document but when querying document, this command prompt for username then password. My question is... (3 Replies)
Discussion started by: markthien
3 Replies

6. UNIX for Dummies Questions & Answers

How can i hide username/password

hi all, i run sqlplus command on unix(HP-UX) like "sqlplus username/password@serverA @deneme.sql" but when someone run "ps -ef | grep sqlplus", it can see my username and password :( How can i hide username and password. thanx. (1 Reply)
Discussion started by: temhem
1 Replies

7. UNIX for Dummies Questions & Answers

Trying to make fixtures table with lynx --dump and pipe filters

Hey, I'm trying to make a nice clear table of fixtures. lynx --dump Fixtures & Reports | Fixtures | Arsenal.com | tail -n+360 | less #tail to remove 1st 360 line I'm trying to remove the 'Add to Calendar' bit next I tried pipping through sed but not sure if I did it right sed 's/\Add... (3 Replies)
Discussion started by: 64mb
3 Replies

8. UNIX and Linux Applications

Take all db dump with username and passowrd

Hi All I have mysql server,That have many Database with different username and password for each,How can i dump all database collectively,And then restore to an other mysql db sever . (1 Reply)
Discussion started by: aliahsan81
1 Replies

9. Shell Programming and Scripting

[lynx dump] Order (by name/URL)

Hi :) How to use dump in lynx. $ lynx -dump http://www.google.com So, this is an example of a lynx dump: txt1 blabla Other txt some text 1. http://url_of_txt1 2. http://url_of_blabla 3. http://url_of_Other_txt 4. http://url_of_some_text ... How can i obtain this output? ... (12 Replies)
Discussion started by: aspire
12 Replies

10. Shell Programming and Scripting

username password in script

Can we write a script to telnet to a unix server from unix with the username and password hardcoded in the script?? something like ssh a@b -p password ??? (5 Replies)
Discussion started by: roshanjain2
5 Replies
Login or Register to Ask a Question