Programming

Hi, I'm wondering how one typically uses epoll for sending out data on a socket?

I have a server that listens on a socket in a thread that calls epoll_wait to receive data. Each time a new connection is opened, the new FD gets added to epoll. I am using EPOLLET by the way.

I have other threads that may want to send data on the connected clients. So when they do, they add their message to a thread-safe ring-buffer.

But when a message gets added in the ring-buffer, I need to wake up the epolling thread so that it can start emptying the buffer.

So what is the proper way that people usually do this?
1) upon adding data to the ringbuffer, add EPOLLOUT in the event, and remove it when the ringbuffer is empty (and if EAGAIN happens, then things will work out naturally) ?

2) set EPOLLOUT upon adding the FD, when adding data to the ringbuffer, send a signal to the other thread to wake him up and flush the ringbuffer (and if EAGAIN happens, then things will work out naturally)?

3) should I NOT use EPOLLET?

Any advice would be great. Thanks.

How about a semaphore, initialized to -1? The reader tries to wait on the semaphore, and gets frozen... Then when anything succeeds in writing to the ring buffer, it should post to the semaphore to wake the thread.

No that wont work. The thread can only block on epoll. I will have a lot of connections I need to handle in that thread.

If you return from epoll() every time any socket is ready to accept data for write() (EPOLLOUT bit set), you'll effectively never block and spend a lot of CPU time spinning through sockets you don't have outgoing data for.

Just send the data out the socket in the thread that has the data using a single write() call - a single write() call is atomic.

Atomic, yes, guaranteed to send everything in one go, not quite. It might return having written less than you asked for. (Though this situation usually just comes up for very large writes.)

I am not returning everytime a socket is ready. I mentionned I was using EPOLLET.

and as Corona688 pointed out, send() could return EAGAIN. so I need to handle that in the reactor thread.

If you set the sockets as nonblocking, I think you can write to them until their buffers are full, and at least know whether you need to defer or not.