The key is not exactly secret. The ipcs command will display it. Think of it as unique.
One way of handling the key, is for a creating process to use IPC_PRIVATE as you suggest. The kernel will then pick a key. Now the creating process needs to tell the other processes about the key.
In many cases, the creating process will now fork and create those processes. The key is still in the data segment somewhere. Data base packages often work like this. Note that we discussing System V shared memory here (there are other kinds) and this was developed prior to threads. These days, anyone about to write something like this might want to consider separate threads rather than separate processes.
If the other processes are not direct desendents of the creating process, then it must use another form of IPC to transmit the key. Or maybe even write it in a file or something. This is why unrelated processes do not usually go with IPC_PRIVATE.
The other way is for the creating process to use a key (not IPC_PRIVATE) that is not currently in use. It is supposed to use ftok() to create this key. One input to ftok is a file. This can be any file. As long as all processes use the same file, and the same 2nd arg to ftok() they will get the same key. The file is not opened, ftok() gets the
inode number of the file and device number of the filesystem that contains it. It conbines these two numbers together along with that second argument to get a unique key. Some people bypass ftok and pick a number out of thin air. I would go with ftok().