Does sync(2) block writes until completed? - UNIX for Advanced & Expert Users

mschwage · #1 09-24-2008

Gentle readers,
I am trying to observe system behavior on our RHEL 5.2 machines. I notice that, it appears to me, based on random iterations of

Code:

dd if=/dev/zero of=/var/tmp/bigfile bs=1024 count=20000

...that if the pdflush is flushing buffers at the time of my write, my write will take about as long as

Code:

dd if=/dev/zero of=/var/tmp/bigfile bs=1024 count=20000 ; sync

This indicates to me that pdflush blocks all write()'s until the flush to disk is complete. The question is: am I correct?

Most of the time, the first dd (without the sync) is much faster than the second. By watching /proc/meminfo, I am able to see approximately when the sync initiated by pdflush takes place. This would be every 30 seconds, as my machine is not filling the disk buffers very quickly. We're talking only a few hundred kilobytes every 30 seconds.

Thanks.
-Mike

jim mcnamara · #2 09-24-2008

When sync is called - maybe by the syncer daemon, it requests the kernel to write all of it's buffers from all buffers for all file descriptors to disk. The kernel does not have to do it exactly at that point, it is a scheduled request.

sync returns before the completion of all the kernel I/O it asked to schedule. It can return before any I/O starts in fact.

On systems with no buffer cache, sync may just be a dummy call that does nothing, because no 'syncing' is required. On a multiprocessor system, processes may continue to call the write() system call when you call sync -- it will not block writes.

I don't know what you are seeing, but sync is not necessarily the cause. try

Code:

 strace -rT sync

and see what you get.

mschwage · #3 09-24-2008

I should mention that I'm not really concerned so much about sync but rather on pdflush itself. ...The mechanisms they use may be different. Anyway, I notice that if I modify the pdflush behavior with these kernel parameters:

Code:

vm.dirty_writeback_centisecs=100
vm.dirty_expire_centisecs=100

...it appears to affect my dd writes negatively; that is, if I loop the dd and run it over and over again, the time for dd to complete jumps on an erratic basis.

Here's what I'm executing:

Code:

while true; do
    echo -n ">>> New File >>>"; date +%r
    /usr/bin/time -f "%E" /tmp/doit
done

and /tmp/doit contains:
dd if=/dev/zero of=/var/tmp/bigfile$i bs=1024 count=40000 oflag=nonblock 2>/dev/null

Then in another window, I am modifying sysctl parameters while the while loop is running. First, the default settings:

Code:

sysctl -w vm.dirty_writeback_centisecs=500 ; sysctl -w vm.dirty_expire_centisecs=3000
sysctl -w vm.dirty_writeback_centisecs=100 ; sysctl -w vm.dirty_expire_centisecs=100
...observe times from the while loop.  These should be more "choppy"

The fact that the dd's more frequently take longer when the centisecs parameter is shortened lead me to believe that my app is blocking when it bumps into the disk flushing. The machine is otherwise quiescent, and the size of the write (40 Meg) should not be enough to trigger pdflush. It should happen every second, based on the sysctl settings.

Note that dd is not doing direct i/o or synchronous i/o. If I explicitly set those flags, it slows the write enormously. Note too that it doesn't matter if I use the "nonblock" flag or not... i/o behavior is the same.
-Mike
P.S. We don't have a syncer daemon. Also, while running my tests above, it appears to me that the effect of sync is nearly instantaneous...

jim mcnamara · #4 09-24-2008

Here is a discussion on pdflush and how a sync call can bog the system - read down about 5 paragraphs in the Tuning... section

The Linux Page Cache and pdflush

mschwage · #5 09-24-2008

Thanks, Jim, but I've been all over that page. The one part that gave me hope is

Quote:

processes are forced to write dirty buffers themselves...Note that all processes are blocked for writes when this happens, not just the one that filled the write buffers.

but then I realized that it didn't apply. It says that the processes will write during this anomalous period, but what I'm trying to get at is normal everyday pdflush behavior. I wonder if pdflush blocks other processes during their write(2)'ing until the blocks are flushed to disk.

...Oh, I see... well, I've been down to the Tuning section, and it seems to imply that pdflush blocks, but I'm still not sure...
-Mike

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
Perl: FH and multiple writes	gctaylor	Shell Programming and Scripting	2	08-21-2008 06:14 AM
FQO: V1 Completed	iBot	Software Releases - RSS News	0	07-12-2008 02:30 PM
which process writes to file	hachik	UNIX for Dummies Questions & Answers	2	07-12-2008 12:28 AM
Sync to Green vs. Separate Sync	psantinello	SUN Solaris	0	06-25-2008 07:39 PM
mowyw writes your websites 0.5.0 (Default branch)	iBot	Software Releases - RSS News	0	04-12-2008 10:30 PM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode