I would concentrate on finding the difference between the two systems. And be sure you have checked all of the shared memory kernel settings. Especially shmmni(5) which is a system-wide limitation.

Run
ipcs -mb

on both systems to see how much shared memeory is currently in use. Has something used it all up? And what about free memory? You need free memory to make shared memory. Run
vmstat 1 5
to see how that looks.