11-20-2012
The separate write and read pointers do make the array into a ring buffer with simultaneous read and write. I like to use a modulo-two size array and just mask for the low bits. You can write if write_cnt - read_cnt < size, and you can read if write_cnt > read_cnt. The writer can zero the read count and then the write count if the buffer is empty, but due to cache issues it is sometimes safer to just use long long and never zero the counters. Works for bytes, pointers, and arrays of whatever. You can even have the writer double the buffer size on full, copying the bottom half to the top half, and then adjust the mask up a bit. I have a version that recorded the last three relative maxima in usage and adjusted the size down by halves if space was underutilized. It was a structure with a long long write counter, read counter, size-1 mask, a pointer to current space (realloc()able), and stats.
Now if you have a manager thread for writing and another for reading, you can use multiple input buffers, one per writer, and have the manager consolidate them in the central container. A read manager thread can fill output buffers to multiple consumers. If service times are highly variable, output buffers should be shallow (2) so one thread does not end up lagging due to bad luck when all others are done. Excess output threads is more effective than deep buffers dedicated to one thread.
However, one global lock can work, as long as dry reads lead to sufficient sleep to allow writers access. If the readers check the depth before locking to read, even that is not a problem. Writers should check available space before locking to write. It is OK if readers underestimate how much is to be read, and it is OK for writers to underestimate how much space there is to write, for a moment until variables are updated and cache catches up. If they check before the lock, they need not sleep, but a little sleep is nice so the CPUs are shared around.
Last edited by DGPickett; 11-20-2012 at 04:28 PM..
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hello,
I just tried to run this command: /usr/lib/sendmail -d0.1 -bt < /dev/null | grep -i version
Its doing what I want: writing out the sendmail version.
But on some machines it writing the version and then exit to the prompt but on others its writing the version but then hangs, I need to... (0 Replies)
Discussion started by: jOOc
0 Replies
2. Solaris
hi all,
i had a program which created a socket on port 7113, but for some reason the program was hunbg and I had to Ctrl+c it. I ran the program for a couple of times and now when I do a netstat -a I see that these sockets are lingering and that might be the reason why my program is not... (1 Reply)
Discussion started by: Naanu
1 Replies
3. UNIX for Dummies Questions & Answers
Ok, this question my be different. I can ping our unix box, but when we I to access the webpage I cant. To access the webpage I type http://ipaddress:some port. How do I check if a port is hanging and how would I un hang it. Sorry if question doesnt make sense. (1 Reply)
Discussion started by: NycUnxer
1 Replies
4. Shell Programming and Scripting
Hello,
I have the following shell script and when i execute, it keeps hanging and nothing happens
Please let me know.
Requirement is to read data from file and pass it to the sql and create files as shown.
code
/********
#!/bin/sh
while read user.dat
do
echo "user = $1 email =... (1 Reply)
Discussion started by: rakeshsr12
1 Replies
5. Programming
Hi Experts,
I have a programming assignment that asks us to implement a pipegrep function. it basically has 5 stages and each stage has a thread and buffers are used between stages.
am currently implementing stage 1 . In stage 1 am suppose to read directory and store the filenames in buffer1... (15 Replies)
Discussion started by: amejoish
15 Replies
6. Solaris
Hi Folks,
When i execute the command df -kh in my system the o/p hangs..
The command runs fine but takes a lot of time before coming back to the # prompt.
Can anyone please suggest the possible cause and solution?. (10 Replies)
Discussion started by: vivek.goel.piet
10 Replies
7. Shell Programming and Scripting
Hey folks. Long time lurker, first time poster. I'm a bit of a newbie at "coding" (obviously, scripting is a teensy bit different than coding) and I've run into a problem that I just can't seem to get around.
I'm going through a list of servers to check their name, IP, reverse-NSLOOKUP name and... (2 Replies)
Discussion started by: Bearwhale
2 Replies
8. Shell Programming and Scripting
/bin/sed -n '$q;5633653,0 p' lfile
lfile is a log file that is being updated several times a second.
so, the command above figures out the line number it wants in the lfile and then proceeds to output all lines from that line number to the end of the file.
the problem is, the end of the... (2 Replies)
Discussion started by: SkySmart
2 Replies
9. Shell Programming and Scripting
Hi All,
I am transfering a file through sftp. But the script is hanging at exit occasionally.
I am suspecting that sftp is hanging due to buffer size issue.
If that is the case can any body suggest a solution.
Please find the code.
echo "cd /${CUST_ID}/inbound/${SAFET_ID}" >... (0 Replies)
Discussion started by: Girish19
0 Replies
10. Shell Programming and Scripting
Could you please give your inputs on the below issue:
source.xml
<?xml version="1.0" encoding="UTF-16"?>
<P1 >
<C1 type="i"><2></C1>
<V1 type="string"><6.2></V1>
<D1 type="string">
<D2><1.0></D2>
<D2><2.0></D2>
</D1>
......................
......................
many more... (7 Replies)
Discussion started by: unme
7 Replies