When downloading a webpage by curl, what happen?


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers When downloading a webpage by curl, what happen?
# 1  
Old 03-29-2014
When downloading a webpage by curl, what happen?

Hello,

when im downloading an webpage from command line (CLI) by curl or wget the target website is loaded like i load it from browser? meaning target server connect to database and render data from mysql? Or only static content is downloaded?
# 2  
Old 03-29-2014
Did you try the --post-data or --post-file options to wget?
# 3  
Old 03-31-2014
Quote:
Originally Posted by postcd
when im downloading an webpage from command line (CLI) by curl or wget the target website is loaded like i load it from browser?
In principle: yes. A web page is served similar to a multipart/mime-message over e-mail. You have some header information (encoded into the URL), some "framework information" and optionally one or several additional data sources (pictures, videos, programs like flash, ...) embedded in this "framework". A browser receives such messages, displays them (quite like a mail reader would do for mails) and - driven by user actions - replies with a similar sort of messages which can range from very simple to equally complicated and similarly structured (quite like you answer a mail with a reply of your own).

Quote:
Originally Posted by postcd
meaning target server connect to database and render data from mysql? Or only static content is downloaded?
Some of the elements i talked about above only work interactively. Both cURL and wget have (limited) provisions to deal with that. That means, some aspects of a user sitting there and interactively clicking/etc. something can be emulated, some cannot.

I hope this helps.

bakunin
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Curl - upload multiple attachment arrays to server webpage

The html page of the form data is as below <form name="uploadform" id="uploadform" action="htmlupload.php" enctype="multipart/form-data" method="post"> <table class="tborder" cellpadding="6" cellspacing="1" border="0" width="100%" align="center"> <tr> <td class="tcat"> Upload Files ... (0 Replies)
Discussion started by: jaango123
0 Replies

2. Shell Programming and Scripting

How to get content of a webpage Curl vs Wget?

Hello, What I am trying to do is to get html data of a website automatically. Firstly I decided to do it manually and via terminal I entered below code: $ wget http://www.***.*** -q -O code.html Unfortunately code.html file was empty. When I enter below code it gave Error 303-304 $... (1 Reply)
Discussion started by: baris35
1 Replies

3. Shell Programming and Scripting

Downloading of dynamically generated URL using curl and sed

I've been attempting to use curl and sed to allow for downloading a file from a dynamically generated URL. I've been able to retrieve and save the HTML of the page that does the dynamic generation of the download URL using curl but I'm very new to sed and I seem to be stuck at this part. HTML: ... (1 Reply)
Discussion started by: schwein
1 Replies

4. Homework & Coursework Questions

songs downloading using curl

Hi , I need to download songs from website using shell script.I have written script for that. The code is #!/bin/ksh curl "http://www.songs.pk/$1.html" >tmp_page 2>tmp_error grep songid tmp_page|cut -d'"' -f2 > song_links while read link; do echo $link curl "$link" >tmp_song 2>tmp_err... (1 Reply)
Discussion started by: pandu25
1 Replies

5. AIX

How does ITIL processing happen in AIX?

How does ITIL process is implemened in AIX? (6 Replies)
Discussion started by: AIXlearner
6 Replies

6. Programming

what would happen if a process wrote to its own stdin?

what would happen if a process wrote to its own stdin? #include<unistd.h> #include<fcntl.h> int main() { if((write(STDIN_FILENO,"arrgh!",6))==-1) { perror("error writing to file"); } } output: $ gcc temp.c $ ./a.out arrgh!$ (9 Replies)
Discussion started by: c_d
9 Replies

7. UNIX for Dummies Questions & Answers

How Do I Set a Task to Happen in the Future?

Is it possible to set a task to happen in the future? Say I want to log-off only after 10 hours of being logged on with out doing any activity in between? (2 Replies)
Discussion started by: Slick
2 Replies

8. UNIX for Dummies Questions & Answers

What would happen if. . .

Hi, Could someone please tell me what would happen if the following were entered into the command line: rm -i /books/*.* rm /books/* Many thanks! (3 Replies)
Discussion started by: crispy
3 Replies

9. Programming

Any one can tell me how this happen?

The #1 Online Store for Louis Vuitton Replicas is: http://www.opichina.com.cn. We offer Louis Vuitton Replicas and more! Whatever you call it: LV Bags, LV Replicas, Louis Vuitton Fake, Louis Vuitton Knockoffs, Louis Vuitton Bag, Louis Vuitton Purse, Louis Vuitton Wallet, Louis Vuitton Shoes,... (10 Replies)
Discussion started by: jiangyanna
10 Replies

10. UNIX for Dummies Questions & Answers

what happen when changing Hostname?

I 'm using RH 7.2 Genome in the Network Configuration I change therer are two places one for static hostname for my machine and in DNS hostname I don't know what happen when restarting my PC when connecting using dialer I can't browse the Internet also I can't use sendmail .......Server timeout... (2 Replies)
Discussion started by: atiato
2 Replies
Login or Register to Ask a Question