Socket Keep-Alive


 
Thread Tools Search this Thread
Top Forums Programming Socket Keep-Alive
# 1  
Old 04-27-2016
Socket Keep-Alive

Hi

I'm adding http 1.1 GET to my project and trying to use “Keep-Alive” HTTP connections to the host, The problem is when I recv() the first page, it succeeds. However, the 2nd consecutive recv() will receive zero bytes, for which I really have no idea. As per HTTP 1.1 I have Connection: “Keep-Alive” in the http header I send.

If anyone is familar with this would you please take a look at the code and explain what was wrong with it?

here is the extracted socket-related code in a small test program Smilie

Code:
#include <sys/types.h>
#include <sys/socket.h>
#include <netdb.h>
#include <netinet/in.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>

#include <string>
#include <iostream>

using namespace std;

// ------------------------------------------------------
const short SOCKET_ERROR = -1;

const short RECV_BUFFER_SIZE = 20;
const short REQUEST_BUFFER_SIZE = 255;

int sock;
char recv_buf [RECV_BUFFER_SIZE + 1];
char req_buf [REQUEST_BUFFER_SIZE + 1];

const static char REQUEST_TEMPLATE [] = 
{
    "GET %s HTTP/1.1\r\n"
    "Host: xxx.xxx.xxx\r\n" // should be replaced with a really host
    "Connection: Keep-Alive\r\n"
    "\r\n"
};

// ------------------------------------------------------
void create_socket ();
void download (const string& path, string& response);

// ------------------------------------------------------

int main (void)
{
    string first_addr = "/~pdu/index.html"; // should be replaced with a really URL
    string second_addr = "/~pdu/a.html"; // should be replaced with a really URL
    string response;

    create_socket ();
    download (first_addr, response);
    cout << response << endl << endl;
    response = "";
    download (second_addr, response);
    if (response.size() > 0)
    {
        cout << response << endl;
    }
    else
    {
        cout << "### The 2nd recv() failed to receive any bytes from the socket!" << endl << endl;
    }
    
    close (sock);

    return 0;
}

void create_socket ()
{
    struct sockaddr_in addr;

    // ------------------------------------------------------
    sock = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
    if(sock == SOCKET_ERROR)
    {
        perror ("Could not make a socket.\n");
        exit (-1);
    }

    cout << ">>> Socket created!" << endl;

    // ------------------------------------------------------
    struct hostent* host_info = gethostbyname("cse.unl.edu");    
    
    cout << ">>> DNS done!" << endl;

    long host_addr;

    /* copy address into long */
    memcpy(&host_addr, host_info->h_addr,
        host_info->h_length);

    /* fill address struct */
    addr.sin_addr.s_addr = host_addr;
    addr.sin_port = htons(80);
    addr.sin_family = AF_INET;

    // ------------------------------------------------------
    if( connect(sock, (struct sockaddr*)(&addr),
        sizeof(addr)) == SOCKET_ERROR )
    {
        perror("Could not connect to HTTP server.\n");
        exit (-1);
    }

    cout << ">>> Connection established!" << endl;
}

void download (const string& path, string& response) 
{
    size_t nBytes = snprintf(
        req_buf, 
        REQUEST_BUFFER_SIZE, 
        REQUEST_TEMPLATE, 
        path.c_str());

    if (nBytes >= REQUEST_BUFFER_SIZE)
    {
        cerr << "Buffer is too small for making a request message" << endl;
        exit (-1);
    }

    if (send(sock, req_buf, nBytes, 0) != nBytes)
    {
        perror("Could not send request to the HTTP server.\n");
        exit (-1);
    }
    cout << ">>> Request sent! -> " << path << endl << req_buf << endl;
    
    ssize_t size = 0;
    
    while ((size = recv(sock, recv_buf, 
        RECV_BUFFER_SIZE, 0)) > 0) 
    {
        recv_buf[size] = '\0';
        response.append(recv_buf);
    }

    cout << ">>> Response received!" << endl;
}

# 2  
Old 04-27-2016
Looking at it.

Add stdio.h and stdlib.h to your headers, btw.
This User Gave Thanks to Corona688 For This Post:
# 3  
Old 04-27-2016
Something fishy is going on with the way the connection lags before delivering the content of the first page...
# 4  
Old 04-27-2016
All right, this loop is suspect:

Code:
    while ((size = recv(sock, recv_buf,
        RECV_BUFFER_SIZE, 0)) > 0)
    {
        recv_buf[size] = '\0';
        response.append(recv_buf);
    }

From man recv:

Code:
       When a stream socket peer has performed an orderly shutdown, the return
       value will be 0 (the traditional "end-of-file" return).

So, your program downloads the entire page, but doesn't stop there -- it calls recv() one more time when the transfer is done, which hangs until the web server becomes impatient and kicks you. Which is mecifully much less than the many minutes TCP usually defaults to. Then your program declares the download finished, writes another request to the dead socket, and "receives" another EOF in reply.

Reading until EOF might make sense without keepalives, but obviously won't do for persistent connections -- TCP/IP doesn't have an EOF signal, just an end of connection signal. This is why the web server must warn you of the content's length somehow -- as a content-length header, as chunked sections preprended with lengths in hexadecimal, etc.

Last edited by Corona688; 04-27-2016 at 01:47 PM..
This User Gave Thanks to Corona688 For This Post:
# 5  
Old 04-28-2016
thanks for looking & solving. Yes I see now the recv loop should read a stream and work out when to stop pulling more bytes from recv(), In my full code I extract the chucked hex later for decompressing gzip, I'll have to read the header & chunk in the while(recv loop.

When i read about keep-alive I believe the Apache server is meant to wait for timout of at least 5 seconds (after handling a request), but my recv loop finishes pretty instantly after recieving 3-4 HTTP packets for the entire gzip webpage (in my full code).

I'm guessing somehow the server sees a request for more information and Apache has code to detect clients with while(recv() != 0).

Last edited by Projecteer; 04-28-2016 at 03:18 PM..
# 6  
Old 05-03-2016
It's also possible they've decreased the timeouts to help deal with congestion, I seem to get the entire five.
# 7  
Old 05-31-2016
I do similar operations where the reply contains the size in the metadata in FIX data.
I use the following:

MSG_PEEK
This flag causes the receive operation to return data from the
beginning of the receive queue without removing that data from
the queue. Thus, a subsequent receive call will return the same
data.

Then you can get the content-length and work out how much to read, then use

MSG_WAITALL(since Linux 2.2)
This flag requests that the operation block until the full
request is satisfied. However, the call may still return less
data than requested if a signal is caught, an error or disconā€
nect occurs, or the next data to be received is of a different
type than that returned.
This User Gave Thanks to bigearsbilly For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. What is on Your Mind?

Is this forum alive?

Odd thing. I posted in the Solaris forum, new user, just asking for a bit of advice. Nothing too complicated. As of this post there have been 140 views and zero replies. So that got me thinking, is this normal? I had a look around, and I see the same thing on many other threads, and in other... (2 Replies)
Discussion started by: _JonB_
2 Replies

2. IP Networking

Clarification - Setting socket options at the same time when socket is listening

I need clarification on whether it is okay to set socket options on a listening socket simultaneously when it is being used in an accept() call? Following is the scenario:- -- Task 1 - is executing in a loop - polling a listen socket, lets call it 'fd', (whose file descriptor is global)... (2 Replies)
Discussion started by: jake24
2 Replies

3. Programming

Error with socket operation on non-socket

Dear Experts, i am compiling my code in suse 4.1 which is compiling fine, but at runtime it is showing me for socket programming error no 88 as i searched in errno.h it is telling me socket operation on non socket, what is the meaning of this , how to deal with this error , please... (1 Reply)
Discussion started by: vin_pll
1 Replies

4. Programming

socket function to read a webpage (socket.h)

Why does this socket function only read the first 1440 chars of the stream. Why not the whole stream ? I checked it with gdm and valgrind and everything seems correct... #include <stdio.h> #include <stdlib.h> #include <sys/types.h> #include <sys/stat.h> #include <string.h> #include... (3 Replies)
Discussion started by: cyler
3 Replies

5. Programming

which socket should socket option on be set

Hi all, On the server side, one socket is used for listening, the others are used for communicating with the client. My question is: if i want to set option for socket, which socket should be set on? If either can be set, what's the different? Again, what's the different if set option... (1 Reply)
Discussion started by: blademan100
1 Replies

6. UNIX for Advanced & Expert Users

connect problem for sctp socket (ipv6 socket) - Runtime fail Invalid Arguments

Hi, I was porting ipv4 application to ipv6; i was done with TCP transports. Now i am facing problem with SCTp transport at runtime. To test SCTP transport I am using following server and client socket programs. Server program runs fine, but client program fails giving Invalid Arguments for... (0 Replies)
Discussion started by: chandrutiptur
0 Replies

7. Shell Programming and Scripting

Can we keep our script alive while logging out.

Hi I want to keep my script running even when i am logged off from the box. can we run the script in background which can automatically run every hour? Please advise. Thank you (1 Reply)
Discussion started by: Prateek007
1 Replies

8. UNIX for Dummies Questions & Answers

Socket still alive!!

Hi, I'm investigate about a problem regarding a connection between two server (S1 and S2). A client software on S1 made a pool of connections on S2; for some reason some connections end but sockets still alive on S2 and not on S1. I always knew about sockets as a pair of processes; is it... (1 Reply)
Discussion started by: grado
1 Replies

9. Programming

HTTP Keep-Alive socket problem

Hello everyone, I am a newbie in UNIX/Linux socket programming. This is a class project that I had trouble with. ================================================== I was trying to make “Keep-Alive” HTTP connections to the server in a tiny web crawler project. Here is the problem: when I tried... (0 Replies)
Discussion started by: imdupeng
0 Replies
Login or Register to Ask a Question