HTTP Keep-Alive socket problem


 
Thread Tools Search this Thread
Top Forums Programming HTTP Keep-Alive socket problem
# 1  
Old 03-29-2008
Question HTTP Keep-Alive socket problem

Hello everyone, I am a newbie in UNIX/Linux socket programming. This is a class project that I had trouble with.

==================================================

I was trying to make “Keep-Alive” HTTP connections to the server in a tiny web crawler project. Here is the problem: when I tried to recv() the first page, it succeeded. However, the 2nd consecutive recv() will receive zero bytes, for which I really have no idea. I did put a “Keep-Alive” field in the request message when I called send().

I don't quite know how to implement HTTP persistent connection by using Keep-Alive. I extracted the socket-related code into a small program as attached.

Would you please take a look at the code and explain what was wrong with it to me? That would be of great help since I have stuck on this problem for days.

By the way, the program was compiled under SunOS using g++ and -lsocket.
==================================================

Code:
#include <sys/types.h>
#include <sys/socket.h>
#include <netdb.h>
#include <netinet/in.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>

#include <string>
#include <iostream>

using namespace std;

// ------------------------------------------------------
const short SOCKET_ERROR = -1;

const short RECV_BUFFER_SIZE = 20;
const short REQUEST_BUFFER_SIZE = 255;

int sock;
char recv_buf [RECV_BUFFER_SIZE + 1];
char req_buf [REQUEST_BUFFER_SIZE + 1];

const static char REQUEST_TEMPLATE [] = 
{
	"GET %s HTTP/1.1\r\n"
	"Host: xxx.xxx.xxx\r\n" // should be replaced with a really host
	"Connection: Keep-Alive\r\n"
	"\r\n"
};

// ------------------------------------------------------
void create_socket ();
void download (const string& path, string& response);

// ------------------------------------------------------

int main (void)
{
	string first_addr = "/~pdu/index.html"; // should be replaced with a really URL
	string second_addr = "/~pdu/a.html"; // should be replaced with a really URL
	string response;

	create_socket ();
	download (first_addr, response);
	cout << response << endl << endl;
	response = "";
	download (second_addr, response);
	if (response.size() > 0)
	{
		cout << response << endl;
	}
	else
	{
		cout << "### The 2nd recv() failed to receive any bytes from the socket!" << endl << endl;
	}
	
	close (sock);

	return 0;
}

void create_socket ()
{
	struct sockaddr_in addr;

	// ------------------------------------------------------
	sock = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
    if(sock == SOCKET_ERROR)
    {
        perror ("Could not make a socket.\n");
		exit (-1);
    }

	cout << ">>> Socket created!" << endl;

	// ------------------------------------------------------
	struct hostent* host_info = gethostbyname("cse.unl.edu");	
	
	cout << ">>> DNS done!" << endl;

	long host_addr;

	/* copy address into long */
	memcpy(&host_addr, host_info->h_addr,
		host_info->h_length);

	/* fill address struct */
	addr.sin_addr.s_addr = host_addr;
	addr.sin_port = htons(80);
	addr.sin_family = AF_INET;

	// ------------------------------------------------------
	if( connect(sock, (struct sockaddr*)(&addr),
		sizeof(addr)) == SOCKET_ERROR )
    {
        perror("Could not connect to HTTP server.\n");
		exit (-1);
    }

	cout << ">>> Connection established!" << endl;
}

void download (const string& path, string& response) 
{
	size_t nBytes = snprintf(
		req_buf, 
		REQUEST_BUFFER_SIZE, 
		REQUEST_TEMPLATE, 
		path.c_str());

	if (nBytes >= REQUEST_BUFFER_SIZE)
	{
        cerr << "Buffer is too small for making a request message" << endl;
		exit (-1);
	}

	if (send(sock, req_buf, nBytes, 0) != nBytes)
	{
        perror("Could not send request to the HTTP server.\n");
		exit (-1);
	}
	cout << ">>> Request sent! -> " << path << endl << req_buf << endl;
	
	ssize_t size = 0;
	
	while ((size = recv(sock, recv_buf, 
		RECV_BUFFER_SIZE, 0)) > 0) 
	{
		recv_buf[size] = '\0';
		response.append(recv_buf);
    }

	cout << ">>> Response received!" << endl;
}

Thank you!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Programming

Socket Keep-Alive

Hi I'm adding http 1.1 GET to my project and trying to use “Keep-Alive” HTTP connections to the host, The problem is when I recv() the first page, it succeeds. However, the 2nd consecutive recv() will receive zero bytes, for which I really have no idea. As per HTTP 1.1 I have Connection: ... (6 Replies)
Discussion started by: Projecteer
6 Replies

2. Programming

http sock problem

hi am senthil am doing my server-client prog in C++ to send and receive sms through the API .. in which server is a webserver(remote server) and my prog act as a client.. i will form a url based on the HTTP 1.0 protocol.. and hit the server through socket send funtion.. by hitting the... (1 Reply)
Discussion started by: senkerth
1 Replies

3. Shell Programming and Scripting

sending http url through http socket programming..

hi am senthil am developing a software to send and receive SMS using HTTP connection first of all am forming a URL and sending that URL to a remote server using my Client Program i send that url through Socket(using Send() Function) if i send more than one URL one by one using the same... (4 Replies)
Discussion started by: senkerth
4 Replies

4. Programming

sending http url through http socket programming..

hi am senthil am developing a software to send and receive SMS using HTTP connection first of all am forming a URL and sending that URL to a remote server using my Client Program i send that url through Socket(using Send() Function) if i send more than one URL one by one using the same... (0 Replies)
Discussion started by: senkerth
0 Replies

5. Programming

Socket++ library problem.

Hi, My name is Daniel and I'm spanish, so I'm sorry if you can't undertand something becouse of my low-level english. Something stranger is happening to me with socket++ library and I don't know how to work on it. I has a library called commands.so and the sslclient is and object of that... (4 Replies)
Discussion started by: lock.cda
4 Replies

6. Programming

C - HTTP Socket Programming

Hello everybody, I learning socket programming in C and was wondering if anybody here could help me out. I have two .c programs namely server.c and client.c ....... The client sends a message and the server in turn returns the value. Basically I want to send in a request to a webserver... (24 Replies)
Discussion started by: kev_1234
24 Replies

7. UNIX for Advanced & Expert Users

connect problem for sctp socket (ipv6 socket) - Runtime fail Invalid Arguments

Hi, I was porting ipv4 application to ipv6; i was done with TCP transports. Now i am facing problem with SCTp transport at runtime. To test SCTP transport I am using following server and client socket programs. Server program runs fine, but client program fails giving Invalid Arguments for... (0 Replies)
Discussion started by: chandrutiptur
0 Replies

8. Programming

unable to get end of file while reading HTTP data from socket

I am trying to read HTTP data from a socket. However, for the final set of data being read using read(), read blocks and the control doesnt come back for further processing. I tried using select, but it didn't work... Any help would be greatly acknowledged.:) (2 Replies)
Discussion started by: Harish.joshi
2 Replies

9. UNIX for Dummies Questions & Answers

Socket still alive!!

Hi, I'm investigate about a problem regarding a connection between two server (S1 and S2). A client software on S1 made a pool of connections on S2; for some reason some connections end but sockets still alive on S2 and not on S1. I always knew about sockets as a pair of processes; is it... (1 Reply)
Discussion started by: grado
1 Replies

10. Programming

Socket Problem

Hi all, I have developed server/client application (using C) and tested it on the same machine .. but when I deploy them on different machines I get connection timeout. Well .. server machine and client machine exists on different network segments, so there is a linux firewall box to route... (3 Replies)
Discussion started by: Agent007
3 Replies
Login or Register to Ask a Question