No. It does not. It may, but nothing requires it. Small allocations may be handled by already resident pages. Large allocations are mmap'd and since those pages will be zeroed by the kernel, calloc doesn't need to touch them. None of the callocs in in any of the standard c libraries used by the popular open source unix flavors (I looked at Linux/glibc, FreeBSD, NetBSD, and OpenBSD) will call memset to zero a page which will already be zeroed by the kernel before being made available to the process.
And that's fine. It still seems to, on my systems, require the pages to be backed immediately and that's time consuming. It certainly still performs very similarly to malloc + memset. In fact, allocating 700 MB (all my poor laptop can handle without swapping) it is only 40ms faster than malloc + memset. While this is an eternity in computer time, given the total time for the calloc is about 700 ms, that meager 40 ms savings tells me that the bulk of the time is spent backing the pages, a job which both the first memset after a malloc and, apparently on my system, calloc need to do.
Quote:
Originally Posted by alister
The most obvious explanation for why your system shows no difference between malloc+memset and calloc is that your c library's calloc is naive. Or perhaps your code is flawed. Or perhaps your kernel vm subsystem is prefaulting for some reason. Or perhaps your system's environment has enabled malloc/calloc options which affect their behavior (such as filling the allocation with "junk" or zeroes). Perhaps one of the bazillion linux kernel compile options is to blame. If it were my system, I'd look into it just to satisfy my curiosity.
I'm sure there are many reasons, but at work I'm not the sys admin, so I don't configure the systems. I just code effectively for the systems as configured. Further, my personal system is a stock Ubuntu system, an arguably popular *nix choice. So, any code I wrote for that would fall victim to calloc's performance.
My point is, to some extent, you can code either the system you're running on or an expected worse case. In my case, it happens to be I'm running on the worse case. That means, possibly, you may consider turning away from calloc.
Quote:
Originally Posted by alister
Obviously. My point is only that under certain conditions malloc and calloc are practically identical (both will return zeroed memory without calling memset). See for yourself in the malloc.c source links I provided in an earlier post. You'll find that both are implemented using the same internal routines. Further, if you follow the code path for a large allocation, you'll see that a calloc never memsets (unless certain options which are disabled by default are enabled).
Splendid, but you're 0 for 3 on systems I have available to me. Those being two AIX machines at various O/S levels (5.3 and 6) and Ubuntu for which I provided a uname for prior and here's the libc version:
While I know it's not you're job, nor am I asking you, to figure out why these systems don't perform as you say, I'm simply supplying information to show you that it's not like I'm running some obscure setup. If I wrote code that used calloc for large allocations on any of these I'd be doing myself a disservice over regular malloc, if I didn't need zero'd memory. Time and time again I've been making that point -- is it falling on deaf ears?
Quote:
Originally Posted by alister
I must retract my earlier quarter of a second figure. I cannot reproduce it. I must have misread the value. Perhaps it was 2.50s instead of 0.25s.
Here's some code and timings from OS X running on a 2.16 GHz Core2Duo Macbook with 2 GB of 667 MHz DDR2 (similar results were observed using NetBSD on similar hardware). Without any command line arguments, the executable will attempt to calloc 1 GiB. With command line arguments, it will malloc and memset 1 GiB:
Ok, well here's some code I wrote:
So, let's see, the first calloc took FOREVER (but, for the record, the first malloc+memset would have too for the O/S to put some pages together). Subsequent calloc and malloc+memset perform within 40-50ms as I alternate testing them.
How about a straight malloc, though represented and timed above, here's the program just doing that:
Better:
I don't think anyone's surprised though, right?
Now, let's see after we have all those pages put together, backed, and ready to go; what's a "memset" really take:
Humm...so, about a third the time of the original allocation. Surprised? Well, I mean, it still sucks, so I certainly wouldn't go ahead and advocate calloc followed by a memset for 0, that'd just be dumb...oh wait, I never did!
Quote:
Originally Posted by alister
That's your prerogative, but, for a large allocation with a reasonably recent C library, you're choosing to use memset to zero malloc'd memory that is probably already zeroed, instead of using calloc, which knows whether the memory is already zeroed and can avoid the overhead of a redundant memset.
So long as they're not aimed at my code, use your guns as you see fit.
Regards,
Alister
HAHA, my friend, I believe I can code just fine. I know what I'm talking about, and I believe you do as well.
At the end of the day, I'm the one developing for my systems whose integrity I have to maintain. A 2 second calloc call would be disastrous in my environment. I can spread that over the duration of the code's runtime by calling malloc and gracefully filling in the memory as needed with useful values (which are almost never all zeros) and allow the O/S to more gracefully back the pages as each one is touched rather than immediately (which appears to be what calloc is doing on my systems).
Furthermore, if I haven't already driven home my point that calloc isn't good, we'll run the final test I've created in the suite. It simulates only touching part of each allocated page, something akin to what a sparse array may do. I'm mallocing some memory and then only using some of it. The key is, to do it on a machine that is overcome, in my case that's to allocate a gig when that's all the RAM I have. Here are some "normal" results:
What you'll notice here is that the entire program time is consistently longer for calloc than malloc. If you figure that I'm using the system page size (and I am) I should be touching each page once, forcing each page to be physically created.
So, now lets thrash my poor laptop to death. It only has a gig, so this will do it, lots of swapping, and disc work....
Now...wait for it.... I'll run the same code with the only difference being calloc was called to allocate the memory (you can see the code, right):
Hummm...204 seconds, 10 times the malloc version! WHAT?! Now, what say you? Oh, and not to mention the fact that the calloc itself took about as long as the entire longest program run above!
Still convinced? Maybe your machines are different, I'm sure they are, but mine (and probably the reset of the Linux world running stock Ubuntu) work badly with calloc.
Oh, and the one that should arguably do the absolute worst, malloc + memset + the subsequent reset:
Not even 2 seconds behind, and there's about that variation run to run with this stuff. Yep, calloc...certainly worth it on my box .
Hey guys, need some help. Running AIX Version 5.2 and one of our cron jobs is writing errors to a log file. Any ideas on the following error message.
Error: Internal system error: Unable to initialize standard output file
I'm guessing more info might be needed, so let me know.
Thanks (2 Replies)
Hi
Having a lil trouble with a rather simple application I'm writing. It so happens that I have to copy some data using memcpy() and so far I've been doing just fine compiling it with VC.Net and running it on Windows XP. Now I'm trying to port the thing to Solaris (which shouldn't really be too... (3 Replies)
hi there
i write one awk script file in shell programing
the code is related to dd/mm/yy to month, day year format
but i get an error
please can anybody help me out in this problem ??????
i give my code here including error
awk `
# date-month -- convert mm/dd/yy to month day,... (2 Replies)
Hi ,
I am having records in a file like
00412772784705041008FRUITFUL STRWBRRY
00412772784703041008FRUITFUL STRWBERE
00000570632801448078 X
i have declared a structure like
typedef struct {
char Uname;
char Pname;
... (4 Replies)
--------------------------------------------------------------------------------
Hello, help me please.
I am trying to create a mksysb bakup using nim. I am geting this error, how to correct it ? :
Command : failed stdout: yes stderr: no... (9 Replies)
Hello everybody,
I'm coding a test program for ARP protocol, and i don't know why i'm getting a SIGSEGV, i traced it with gdb and it says it's due to the memcpy function from /lib/libc.so.6.
Program received signal SIGSEGV, Segmentation fault.
0xb7e9e327 in memcpy () from /lib/libc.so.6
This... (5 Replies)
im kinda new to shell scripting so i need some help
i try to run this script and get the error code
> 5 ")syntax error: operand expected (error token is "
the code for the script is
#!/bin/sh
#
# script to see if the given value is correct
#
# Define errors
ER_AF=86 # Var is... (4 Replies)
I have two servers with a fresh install of Solaris 11, and having problems when doing rpcinfo between them. There is no firewall involved, so everything should theoretically be getting through. Does anyone have any ideas? I did a lot of Google searches, and haven't found a working solution yet.
... (2 Replies)
Dear All,
I am trying to compile OpenFOAM-1.7.x in RHEL. I could not able to compile some of the applications because of libc version issues.
It is saying
undefined reference to memcpy@GLIBC_2.14
Can anybody look into it?
Thanks & Regards,
linuxUser_ (3 Replies)
I am writing a shell script with 2 run time arguments. During the execution if i got any error, then it needs to redirected to a error file and in console. Also both error and output to be redirected to a log file. But i am facing the below error.
#! /bin/sh
errExit ()
{
errMsg=`cat... (1 Reply)