Real time I/O processing usually uses the aio calls. These let you make a write call while you are doing something else. Normally these processes run at special priority, so they can get the CPU whenever they need it.

As for IPC - interprocess communication - shared memory is very fast. The idea of message queues is also very good because you can send messages without regard to how fast they are being read. As long as they do get read....

However there is a limit to I/O throughput.

It actually sounds like your system is I/O bound, if you know that term. If you have 900ms waits on I/O completion, you are going to have a problem no matter what design you implement. That is about 13 x 30 x 30ms "sleep" cycles. Which will create another condition requiring a disk write by the kernel as soon as it comes back from a write. Every one of your writes (assuming an 8 byte double and all 10000 array elements are written) is 100 X 100 X 8 bytes = 80 thousand bytes every 30ms.

13 requests ~ 1MB of data every 900ms. This is on top of all the other system I/O.
You may want to have a sysadmin look at monitoring I/O for you. You could write to several different low I/O volume filesystems simultaneously, for example.

The kernel normally manages I/O requests -it lets them pile up for a while, then it writes all of the pending requests to disk. There are ways to reconfigure the kernel for real time I/O considerations, but it alters the system for everybody else. You can't do this to a multiuser production box without causing problems.

You might want to get a PC with a really high throughput disk I/O subsystem, maybe one that supports realtime streaming, put Linux on it, and go from there. You could also consider doing something to reduce I/O like compressing the data first. If compression gets you a lot.