Linux

I am trying to add support for hugepages into an existing application. I have done a lot of online research, but I still have a few questions that I haven't been able get answers to. Hopefully someone here can help!

(1) My application requires that all mapped memory segments be shared. The existing application, which does not use hugepages, uses shm_open() and mmap() to map the shared memory. Since shm_open() automatically maps into the /dev/shm filepath and hugepages must be mapped into the mount filepath (/mnt on my system), it seems that I cannot use shm_open() for hugepages. Is this correct?

(2) If I cannot use shm_open() for hugepages, is there any way to use open() and mmap() and still get shared memory? Are objects mapped to the hugepage mount filepath shared or can they be?

(3) If I use open() instead of shm_open() but use the shared memory filepath (/dev/shm) does that make it shared?

That is correct. Since the use of hugepages can in some cases change memory semantics drastically(getpagesize() becomes no longer accurate for all memory!), general-purpose system calls won't create them by default. There's some modified, experimental systems out there that create hugepages by default in some places, google published a paper on it but it currently eludes me; but you'd have to know exactly what you were doing with your system to avoid potential problems. If you map them as MAP_SHARED they should be. If independent processes map the same areas of any mappable file with mmap's MAP_SHARED flag, the address space is effectively shared, as specified by man mmap:

Code:

       MAP_SHARED Share this mapping.  Updates to the mapping are  visible  to
                  other  processes that map this file, and are carried through
                  to the underlying  file.   The  file  may  not  actually  be
                  updated until msync(2) or munmap() is called.

/dev/shm(aka tmpfs) and hugetlbfs are just kernel extensions to the filesystem that allow mmap to map and share new kinds of memory without otherwise changing the behavior of the mmap call or requiring new and strange system calls; instead of some special new create_explicit_memory_segment system call, just create ordinary files in a special place... I suspect shm_open ultimately just calls open() and mmap() on Linux.

That said, I am not intimately familiar with the hugepage system yet, and there may be corner cases/gotchas I'm not yet aware of. You might try using libhugetlbfs or at least examining its operation.

Thanks. That pretty much confirmed what I thought, but it's nice to have some assurance. I am fairly new to Linux and kind of had to jump right into it.