semaphore access speed - Page 6

migurus · #1 (**permalink**) 10-01-2008

Thanks for the code, when I ran it I got this:
2.6.9/Xeon/3.2 MHz
(3 runs)
129186.76 semop/s (8784700/68) [100]
129367.65 semop/s (8797000/68) [100]
129257.35 semop/s (8789500/68) [100]

SCO/Xeon/3.2 MHz
(3 runs)
527641.18 semop/s (8969900/17) [100]
564212.50 semop/s (9027400/16) [100]
535800.00 semop/s (9108600/17) [100]

I see SCO is capable of doing roughly 3 times more semop/s than Linux.

BTW, the h/w is identical, I believe I have it posted in this looong thread.

For Linux, I will re-post here s/w versions:
Fedora 2.6.9
gcc - ver.3.4.3
ldd - ver. 2.3.5
glibc - ver 2.3.5

migurus · #2 (**permalink**) 10-01-2008

This is a little off-topic, but I am afraid I do not fully understand the code.

My understanding of the purpose of the initial loop

Quote:

Originally Posted by otheus

Code:

    for (i = 0; i < 1000; ++i) {
        usleep(10);
        last=time(NULL);
        if (last > start) break;
    }

is to start the whole measurement at the point of change of the second. Right?

Next,
we have two loops: first run 8750000 times, given the maxloop value of 1000000,

Then we run loop up to 1250000 times where every 100th iteration we check for time and if second changed, we break out

Quote:

Originally Posted by otheus

Code:

    stop=time(NULL);
    for (i = maxloop - maxloop/8; i < maxloop; ++i) {
      if ( !(i % estimate) ) {
        last=time(NULL);
        if (last > stop) break;
        stop=last;
      }
 
      /* repeat semaphore opts */
      if ((sid = semget(key, NSEMS, IPC_CREAT | 0777)) == -1) {
          perror("Can Not Get Semaphore ID");
      }
      if (semctl(sid, NSEMS, GETALL, vals) == -1) {
          perror("Can Not Get Semaphore Values");
      }
    }
    stop=last;

What is this technique of checking every Nth iteration for? why not check time every iteration? is it because this would add to many extra time calls and muddy the measurements?

Your clarification would be appreciated.

otheus · #3 (**permalink**) 10-01-2008

Quote:

Originally Posted by migurus

This is a little off-topic, but I am afraid I do not fully understand the code.

That's quite ON topic and one reason I posted the code.

Quote:

is to start the whole measurement at the point of change of the second. Right?

right.

Quote:

What is this technique of checking every Nth iteration for? why not check time every iteration? is it because this would add to many extra time calls and muddy the measurements?

You answered it. AFAIK each time() call involves a system call. I think there is a better way of handling this, but this is the first that came to mind.

I do appreciate you checking the code. That I'm prone to failure may be a modest understatement.

otheus · #4 (**permalink**) 10-06-2008

I checked the kernels. There were major changes in 1995 (before version 2.0), 1998 (before version 2.4), and then again with the introduction of the "O(1) scheduler" (not sure). Here's the code -- unchanged since 2.4 -- that does semctl(GETALL):

Code:

        sma = sem_lock(semid);
/* check condition omitted */
        nsems = sma->sem_nsems;
        err=-EIDRM;
        if (sem_checkid(sma,semid)) goto out_unlock;
        err = -EACCES;
        if (ipcperms (&sma->sem_perm, (cmd==SETVAL||cmd==SETALL)?S_IWUGO:S_IRUGO))
                goto out_unlock;

	case GETALL:	{
		ushort __user *array = arg.array;
		int i;

		if(nsems > SEMMSL_FAST) {
/* omitted code -- relevant only when nsems > 256 */
		}

		for (i = 0; i < sma->sem_nsems; i++)
			sem_io[i] = sma->sem_base[i].semval;
		sem_unlock(sma);
		err = 0;
		if(copy_to_user(array, sem_io, nsems*sizeof(ushort)))
			err = -EFAULT;
		goto out_free;

Here's the code from 2.0:

Code:

        case GETALL:
                if (ipcperms (ipcp, S_IRUGO)) return -EACCES;
                switch (cmd) {
/* ommitted irrelevant code */
                case GETALL:
                        array = arg.array;
                        i = verify_area (VERIFY_WRITE, array, nsems*sizeof(ushort));
                        if (i)
                                return i;
                }
                break;
/* skipping case statements */

        if (semary[id] == IPC_UNUSED || semary[id] == IPC_NOID)
                return -EIDRM;
/* the next line provides the sem_checkid() call from 2.4/2.6 code */
        if (sma->sem_perm.seq != (unsigned int) semid / SEMMNI)
                return -EIDRM;

        switch (cmd) {
        case GETALL:
                if (ipcperms (ipcp, S_IRUGO))
                        return -EACCES;
                for (i = 0; i < sma->sem_nsems; i++)
                        sem_io[i] = sma->sem_base[i].semval;
                memcpy_tofs (array, sem_io, nsems*sizeof(ushort));
                break;

When you break it down, the only difference is in the semlock() call, which is needed on multi-CPU systems. It could be SCO is also similarly limited. Why don't you try a linux 2.0 distribution, like RedHat 5.2, which uses Linux 2.0.36 (and uses the same sem code as above). Install it and benchmark the same code. That would be a great help to all of us, I think.

otheus · #5 (**permalink**) 10-06-2008

Migurus,

I emailed the maintainers of the code. This is a response I got back from Alan "Maddog" Cox (with permission to post here):

Quote:

Perhaps you can contribute to this discussion (concerning SCO vs linux performance differences with semget):

No but if you've got a good test case using gettimeofday() rather than time
so you get high precision time data file a bug in bugzilla.kernel.org as I
imagine Ingo Molnar and a few others might be interested.

High performance Linux code uses futex locks rather than sys5 locks but it
would still be nice to know if there really is such a big difference and why

Alan

So here is yet another version using gettimeofday(). Don't bother posting the benchmarks here, unless they are significantly different. But prepare them for bugzilla:

Code:

#include <stdio.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/sem.h>
#include <time.h>
#define NSEMS   2

const static long maxseconds = 1000;
main(int argc, char *argv[])
{
    struct timeval tod_start,tod_stop;
    long int start,stop;
    long int maxloop = 5000000;
    long int i;
    int sid;
    key_t key;
    ushort vals[NSEMS] = { 0, 0 };

    if (argc > 1)
      maxloop = atol(argv[1]);
    i = maxloop;

    key = ftok("/tmp",99);

    gettimeofday(&tod_start, NULL);
    while (--i) {
      if ((sid = semget(key, NSEMS, IPC_CREAT | 0777)) == -1) {
          perror("Can Not Get Semaphore ID");
      }
      if (semctl(sid, NSEMS, GETALL, vals) == -1) {
          perror("Can Not Get Semaphore Values");
      }
    }
    gettimeofday(&tod_stop,NULL);

    start = 1000*1000*(tod_start.tv_sec - maxseconds) + tod_start.tv_usec;
    stop  = 1000*1000*(tod_stop.tv_sec  - maxseconds) + tod_stop.tv_usec;

    printf("%.2f semop/s (%ld/%ld)\n",
      (double)maxloop/(stop-start)*1000*1000, maxloop, stop-start);
}

migurus · #6 (**permalink**) 10-06-2008

Otheus, I never dealt with reporting on bugzilla, it seems ti me I don't quite qualify and I'm afraid I will post something not pertinent, so if you would agree to do it yourself, here are the results from 2nd version of your 'gettimeofday-based' code, which I re-run 4 times as to get average:

SCO
$ tmx2
575108.94 semop/s (5000000/8694005)
$ tmx2
575215.00 semop/s (5000000/8692402)
$ tmx2
575183.63 semop/s (5000000/8692876)
$ tmx2
559832.49 semop/s (5000000/8931243)

Linux:
$ ./tmx2
129363.77 semop/s (5000000/38650699)
$ ./tmx2
129428.22 semop/s (5000000/38631452)
$ ./tmx2
129601.55 semop/s (5000000/38579786)
$ ./tmx2
129511.76 semop/s (5000000/38606534)

As you see, no difference from the former tests.

Just for my clarification: somehow people who participated in this discussion on unix.com formu as well as on kerneltrap and other places were very concerned with 2 things, that in my perspective are irrelevant to the subject of this thread: time measurement accuracy and involvement of running multiple processes/shells etc that would muddy the results. I would wholeheartedly agree to that IF SCO vs Linux results were comparable. But this is not the case here. SCO is three times faster, no matter we used my method or gettimeofday very accurate method. With all the overhead being roughly similar, I would think we were kind of beating around the bush when we were trying to achieve have accuracy in measurement (which does not hurt, of cource!), but it made the thread bloated.

Again, thanks to Otheus for your suggestions, at this point I the kernel code you pinpoint shows that multi-cpu capability adds a layer of complexity and it does not help the speed. Your and strcmp's (kerneltrap forum) recomendations to upgrade Linux kernel are well taken.
migurus

ramen_noodle · #7 (**permalink**) 10-06-2008

Just for contrast on a modern intel SMP processor running linux 2.6.18 I get more
than a million operations/sec.
On that basis I don't believe this qualifies as a bug..more of a growing pain.
Linux has always been a project in forward motion. You either catch up or suffer.
Linus is short on apologies for this model that I've seen.

Code:

1402465.14 semop/s (5000000/3565151)
1409187.85 semop/s (5000000/3548143)
1448182.72 semop/s (5000000/3452603)

On a FreeBSD 6.0 machine (uniprocessor) the time is interesting...
Intel(R) Celeron(R) CPU 2.00GHz (1999.94-MHz 686-class CPU)

Code:

501281.02 semop/s (5000000/9974445)
500868.00 semop/s (5000000/9982670)
450727.95 semop/s (5000000/11093166)

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
semaphore	raguramtgr	UNIX for Dummies Questions & Answers	7	06-15-2009 09:39 AM
Semaphore	Jaken	Shell Programming and Scripting	2	04-04-2009 05:10 PM
dmidecode, RAM speed = "Current Speed: Unknown"	Santi	Filesystems, Disks and Memory	0	02-16-2006 06:16 AM
Semaphore	vjsony	UNIX for Dummies Questions & Answers	3	04-07-2003 02:06 PM
semaphore	yls177	UNIX for Dummies Questions & Answers	1	10-08-2002 11:18 PM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Switch to Linear Mode Hybrid Mode Switch to Threaded Mode	Rate This Thread: