socket function to read a webpage (socket.h)


 
Thread Tools Search this Thread
Top Forums Programming socket function to read a webpage (socket.h)
# 1  
Old 03-31-2010
socket function to read a webpage (socket.h)

Why does this socket function only read the first 1440 chars of the stream. Why not the whole stream ? I checked it with gdm and valgrind and everything seems correct...



Code:
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <string.h>
#include <sys/socket.h>
#include <netdb.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <sys/time.h>

int main(void)
{
    // set family and socket type
    struct addrinfo hints, *result;
    memset(&hints, 0, sizeof hints);
    hints.ai_family = AF_INET;
    hints.ai_socktype = SOCK_STREAM;

    // resolve hostname to IP address: specify host and port number (in char not int)
    if(getaddrinfo("www.unix.com", "80", &hints, &result) != 0)
    {
        freeaddrinfo(result);
        puts("Could not resolve hostname.");
        exit(1);
    }

    // create socket and free addrinfo memory
    int newsocket = socket(result->ai_family, result->ai_socktype, 0);
    if(newsocket == -1)
    {
        puts("Could not create socket.");
        freeaddrinfo(result); // free addrinfo memory
        close(newsocket);
        exit(1);
    }

    // set socket timeouts
	struct timeval timeout;
	memset(&timeout, 0, sizeof(timeout)); // zero timeout struct before use
	timeout.tv_sec = 5;
	timeout.tv_usec = 0;
	setsockopt(newsocket, SOL_SOCKET, SO_SNDTIMEO, &timeout, sizeof(timeout)); // send timeout
	setsockopt(newsocket, SOL_SOCKET, SO_RCVTIMEO, &timeout, sizeof(timeout)); // receive timeout

    // connect to website
    if(connect(newsocket, result->ai_addr, result->ai_addrlen) == -1)
    {
        puts("Could not connect.");
        freeaddrinfo(result); // free addrinfo memory
        close(newsocket);
        exit(1);
    }

    // send headers to connection
    char headers[256] = "GET /index HTTP/1.1\r\nHost: www.unix.com\r\nUser-Agent: Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 6.0)\r\nReferer: \r\nConnection: Close\r\n\r\n";
    if( (send(newsocket, headers, strlen(headers), 0)) == -1 )
    {
        puts("Could not send data.");
        freeaddrinfo(result); // free addrinfo memory
        close(newsocket);
        exit(1);
    }

    // receive webpage from connection
    char response[51201] = "";
    ssize_t length = recv(newsocket, response, 51200, 0);
    response[strlen(response)] = '\0'; // terminate the string
    if(length == -1)
    {
        puts("Could not receive data.");
        freeaddrinfo(result); // free addrinfo memory
        close(newsocket);
        exit(1);
    }

    // free addrinfo memory
    freeaddrinfo(result);

    // close socket
    close(newsocket);

printf("%s\n%i\n", response, length);

    return 0;
}


Last edited by cyler; 04-03-2010 at 08:05 AM..
# 2  
Old 04-01-2010
It should be because of socket receive buffer limit. You can check that using set getsockopt();
You can change the buffer limit using setsockopt(). for more details "man setsockopt"
It is recommended that you should read with in a loop, instead of trying to read whole buffer in single shot.
# 3  
Old 04-01-2010
Because you just read the contents of the first TCP packet that you received.

Basically, you need to keep reading from the socket until you get nothing more back. If the server you connect to closes the connection, that's easy to find. If not, you have to figure out how you're going to define when the server is done sending data.
# 4  
Old 04-01-2010
Thank you. Suggestions worked.

Last edited by cyler; 04-02-2010 at 06:00 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Programming

read from socket

hello I have a C program and I want to read data from a socket ,and especially I want a client to send a .jpg to the server.The problem I face is that no matter what the size of the photo is,my read command seems to read an amount of data and then does nothing. while(1){ ... (4 Replies)
Discussion started by: vlm
4 Replies

2. Shell Programming and Scripting

Read and write to tcp socket

Hello all, I have a requirement to read and write to a tcp socket from an HP-UX shell script. I see a /dev/tcp character device on my servers: crw-rw-rw- 1 root root 72 0x00004f Mar 28 18:37 /dev/tcp So I believe this is what I should use. The problem is that all the... (2 Replies)
Discussion started by: lupin..the..3rd
2 Replies

3. IP Networking

Clarification - Setting socket options at the same time when socket is listening

I need clarification on whether it is okay to set socket options on a listening socket simultaneously when it is being used in an accept() call? Following is the scenario:- -- Task 1 - is executing in a loop - polling a listen socket, lets call it 'fd', (whose file descriptor is global)... (2 Replies)
Discussion started by: jake24
2 Replies

4. Programming

Error with socket operation on non-socket

Dear Experts, i am compiling my code in suse 4.1 which is compiling fine, but at runtime it is showing me for socket programming error no 88 as i searched in errno.h it is telling me socket operation on non socket, what is the meaning of this , how to deal with this error , please... (1 Reply)
Discussion started by: vin_pll
1 Replies

5. Programming

which socket should socket option on be set

Hi all, On the server side, one socket is used for listening, the others are used for communicating with the client. My question is: if i want to set option for socket, which socket should be set on? If either can be set, what's the different? Again, what's the different if set option... (1 Reply)
Discussion started by: blademan100
1 Replies

6. UNIX for Advanced & Expert Users

connect problem for sctp socket (ipv6 socket) - Runtime fail Invalid Arguments

Hi, I was porting ipv4 application to ipv6; i was done with TCP transports. Now i am facing problem with SCTp transport at runtime. To test SCTP transport I am using following server and client socket programs. Server program runs fine, but client program fails giving Invalid Arguments for... (0 Replies)
Discussion started by: chandrutiptur
0 Replies

7. Programming

Please help! accept function problems in Socket programming

Hi, I have a client-server socket program. It has been working fine for over a year, but recently it started to show strange behavior.:confused: After the server program runs for a while, it will show in the top command saying it is using lots of CPU, MEM. I assume it means the server code is... (1 Reply)
Discussion started by: natxie
1 Replies

8. Programming

connect() function in C++ socket programming

Hello All, I have a problem using connect(...) function in C++. I am using SSH from my windows system to connect it to linux server. The program works fine if I run it directly in Linux machine but I need it to run through windows machine. The function returns -1 and so my program terminates. ... (3 Replies)
Discussion started by: smdhd3
3 Replies

9. Programming

Anyone know how to use socket select() function?

hello socket programming expert, I having difficulties in understanding how select() function in socket programming work.... I'm trying to create my own peer-to-peer chat or file transfer program by using the select() function.... Therefore does anyone had any tutorial or source code that... (4 Replies)
Discussion started by: draggy
4 Replies

10. Shell Programming and Scripting

send function in socket

Hi All, I encountered a stange problem while doing a perl script to use socket. i need to transfer a file from client to sever. but error came as argument missing in send function.........Plz tell me the wt r the arguments in send and recv functions....... (0 Replies)
Discussion started by: trupti_rinku
0 Replies
Login or Register to Ask a Question