Programming

If x*y > SIZE_MAX, because of that overflow, malloc will silently attempt to allocate an amount of storage less than what was intended. This won't happen with a properly implemented calloc(x, y). If the overhead of zeroing the allocation is undesirable, then explicitly checking for overflow before using malloc would be wise.


I have, but that was a while ago. These days, the case of the absent header, that you mentioned, is a much more likely scenario.

Regards,
Alister

If you're allocating enough memory to approach SIZE_MAX, you really, really don't want to zero it first

Please bring in 2 GB worth of pages kernel; I'll wait, lol.

Re-reading my post (#11), I see how it can be misunderstood. I did not intend for the first paragraph to have anything to do with the rest of the post. I should have indicated that clearly (either with language or formatting) or I should have made it a separate post.

When the second paragraph begins, Even so, memset() is largely irrelevant in this scenario, the scenario I'm referring to has nothing to do with the memset "benchmark" in the immediately preceding, opening paragraph. I was referring instead to what had been the topic of the thread at that point, a singularly large allocation, and how it's handled by calloc() in today's open source systems. (I'm curious if the proprietary unices behave similarly. I assume so, but I have no specific information.)

The memset benchmark isn't intended to prove anything except that memset-ing 2 GB takes on the order of a fraction of a second rather than a minute or an hour. Nothing more. As far as benchmarks go, it wasn't a particularly ambitious one.

Regards and apologies for the confusion,
Alister

Thanks for the explanation, and sorry for being dense in my initial reading of it. Memsetting one gig of RAM takes 1.4 seconds for me. Imagine the amount of actual work that computer could've done in that time instead. Congratulations on your fast computer though.

Yeah man, honestly I don't know what he's talking about with his benchmark. I just ran my own really quick test comapring a calloc'ing vs the malloc -> memset sequence and they are darn near identical. Which makes sense because calloc has to bring in pages and once you touch the malloc'd page it has to be pulled in too. Worse, my laptop only has a gig of ram, so deity forbid I attempt to calloc or malloc and memset a gig, I'll start swapping! But, unsurprisingly, I can call malloc for a gig of ram and it'll return immediately. So long as I don't touch the pages, it'll never slow down.

Point is, calloc is different than malloc and shouldn't be used unless you need all your memory zero'd. And really, what application does? Most malloc's would be followed by something "useful" like a memcpy or filling in the malloc'd memory with useful data. Further, you most certainly wouldn't use calloc for a sparse array, that'd just be crazy.

P.S. here's uname -a

Code:

Linux laptop 2.6.32-34-generic #77-Ubuntu SMP Tue Sep 13 19:40:53 UTC 2011 i686 GNU/Linux

edit: I just ran it on a work machine, memcpy followed by memset for 1 GB and calloc for 1 GB were also identical and about 2 seconds. This is on an P570 frame with 12 GB of memory and a 2 CPU's allocated. So...I'd love to know what computer does it in "fractions of a second".

edit2: I suppose I also stirred this up, by saying "I'll wait" as if to imply it'd be ages. But, in computer terms, 2 seconds is "ages". Plus, if you had to swap, it'd really be "I'll wait" because on my poor laptop with 1 GB of RAM, asking it to memset (or calloc) a gig started it swapping; my music in the background was starting to skip and the harddrive started to spin as memory was being paged to disc. It was BAD, lol. After I killed the process, it still took about 10 seconds for the poor thing to normalize, and my music player hung and wouldn't come back, so I had to kill it, lol. Fortunately, the same program without the memset (and just the malloc) ran and ended immediately, because it just pulled in address space to the process, never a physical page, and so never did any actual work. Hence the BIG difference between malloc and calloc that started this whole off topic thread of communication.

alister explained why this is irrelevant for large amounts of memory: 1) it doesn't bother, because 2) it maps it in with mmap instead, meaning 3) the kernel does it for you at the time of paging in and not before.