Sponsored Content
Full Discussion: memcpy error
Top Forums Programming memcpy error Post 302566750 by alister on Friday 21st of October 2011 04:28:11 AM
Old 10-21-2011
Quote:
Originally Posted by DreamWarrior
...calloc has to bring in pages...
No. It does not. It may, but nothing requires it. Small allocations may be handled by already resident pages. Large allocations are mmap'd and since those pages will be zeroed by the kernel, calloc doesn't need to touch them. None of the callocs in in any of the standard c libraries used by the popular open source unix flavors (I looked at Linux/glibc, FreeBSD, NetBSD, and OpenBSD) will call memset to zero a page which will already be zeroed by the kernel before being made available to the process.

The most obvious explanation for why your system shows no difference between malloc+memset and calloc is that your c library's calloc is naive. Or perhaps your code is flawed. Or perhaps your kernel vm subsystem is prefaulting for some reason. Or perhaps your system's environment has enabled malloc/calloc options which affect their behavior (such as filling the allocation with "junk" or zeroes). Perhaps one of the bazillion linux kernel compile options is to blame. If it were my system, I'd look into it just to satisfy my curiosity.

Quote:
Point is, calloc is different than malloc and shouldn't be used unless you need all your memory zero'd.
Obviously. My point is only that under certain conditions malloc and calloc are practically identical (both will return zeroed memory without calling memset). See for yourself in the malloc.c source links I provided in an earlier post. You'll find that both are implemented using the same internal routines. Further, if you follow the code path for a large allocation, you'll see that a calloc never memsets (unless certain options which are disabled by default are enabled).

Quote:
I just ran it on a work machine, memcpy followed by memset for 1 GB and calloc for 1 GB were also identical and about 2 seconds. This is on an P570 frame with 12 GB of memory and a 2 CPU's allocated. So...I'd love to know what computer does it in "fractions of a second".
I must retract my earlier quarter of a second figure. I cannot reproduce it. I must have misread the value. Perhaps it was 2.50s instead of 0.25s.

Here's some code and timings from OS X running on a 2.16 GHz Core2Duo Macbook with 2 GB of 667 MHz DDR2 (similar results were observed using NetBSD on similar hardware). Without any command line arguments, the executable will attempt to calloc 1 GiB. With command line arguments, it will malloc and memset 1 GiB:

Code:
$ cat large-calloc.c
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define ALLOCSIZE 1073741824U

int
main (int argc, char **argv) {
	void *vp;

	if (argc == 1) {
		if ((vp = calloc(1, ALLOCSIZE)) == NULL)
			return 1;
	}
	else {
		if ((vp = malloc(ALLOCSIZE)) == NULL)
			return 1;
		memset(vp, 0, ALLOCSIZE);
	}
	printf("%p\n", vp);
	return 0;
}
$ cc -Wall -pedantic large-calloc.c 
$ time ./a.out
0x2008000

real    0m0.005s    # calloc
user    0m0.001s
sys     0m0.004s
$ time ./a.out with-memset
0x2008000

real    0m2.124s    # malloc + memset
user    0m0.946s
sys     0m1.168s

Quote:
I'll stick to my guns, calloc is pointless; use malloc and initialize the memory in-situ as appropriate afterwards.
That's your prerogative, but, for a large allocation with a reasonably recent C library, you're choosing to use memset to zero malloc'd memory that is probably already zeroed, instead of using calloc, which knows whether the memory is already zeroed and can avoid the overhead of a redundant memset.

So long as they're not aimed at my code, use your guns as you see fit.

Regards,
Alister
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Error: Internal system error: Unable to initialize standard output file

Hey guys, need some help. Running AIX Version 5.2 and one of our cron jobs is writing errors to a log file. Any ideas on the following error message. Error: Internal system error: Unable to initialize standard output file I'm guessing more info might be needed, so let me know. Thanks (2 Replies)
Discussion started by: firkus
2 Replies

2. Programming

memcpy segfaults, but not in windows

Hi Having a lil trouble with a rather simple application I'm writing. It so happens that I have to copy some data using memcpy() and so far I've been doing just fine compiling it with VC.Net and running it on Windows XP. Now I'm trying to port the thing to Solaris (which shouldn't really be too... (3 Replies)
Discussion started by: khoma
3 Replies

3. UNIX for Dummies Questions & Answers

awk Shell Script error : "Syntax Error : `Split' unexpected

hi there i write one awk script file in shell programing the code is related to dd/mm/yy to month, day year format but i get an error please can anybody help me out in this problem ?????? i give my code here including error awk ` # date-month -- convert mm/dd/yy to month day,... (2 Replies)
Discussion started by: Herry
2 Replies

4. Programming

Problem with memcpy

Hi , I am having records in a file like 00412772784705041008FRUITFUL STRWBRRY 00412772784703041008FRUITFUL STRWBERE 00000570632801448078 X i have declared a structure like typedef struct { char Uname; char Pname; ... (4 Replies)
Discussion started by: arunkumar_mca
4 Replies

5. AIX

nim mksysb error :/usr/bin/savevg[33]: 1016,07: syntax error

-------------------------------------------------------------------------------- Hello, help me please. I am trying to create a mksysb bakup using nim. I am geting this error, how to correct it ? : Command : failed stdout: yes stderr: no... (9 Replies)
Discussion started by: astjen
9 Replies

6. Programming

Segmentation Fault by memcpy

Hello everybody, I'm coding a test program for ARP protocol, and i don't know why i'm getting a SIGSEGV, i traced it with gdb and it says it's due to the memcpy function from /lib/libc.so.6. Program received signal SIGSEGV, Segmentation fault. 0xb7e9e327 in memcpy () from /lib/libc.so.6 This... (5 Replies)
Discussion started by: Zykl0n-B
5 Replies

7. UNIX for Dummies Questions & Answers

> 5 ")syntax error: operand expected (error token is " error

im kinda new to shell scripting so i need some help i try to run this script and get the error code > 5 ")syntax error: operand expected (error token is " the code for the script is #!/bin/sh # # script to see if the given value is correct # # Define errors ER_AF=86 # Var is... (4 Replies)
Discussion started by: metal005
4 Replies

8. Solaris

Rpcinfo: can't contact portmapper: RPC: Authentication error; why = Failed (unspecified error)

I have two servers with a fresh install of Solaris 11, and having problems when doing rpcinfo between them. There is no firewall involved, so everything should theoretically be getting through. Does anyone have any ideas? I did a lot of Google searches, and haven't found a working solution yet. ... (2 Replies)
Discussion started by: christr
2 Replies

9. Shell Programming and Scripting

Undefined reference to memcpy@GLIBC_2.14

Dear All, I am trying to compile OpenFOAM-1.7.x in RHEL. I could not able to compile some of the applications because of libc version issues. It is saying undefined reference to memcpy@GLIBC_2.14 Can anybody look into it? Thanks & Regards, linuxUser_ (3 Replies)
Discussion started by: linuxUser_
3 Replies

10. UNIX for Beginners Questions & Answers

Print Error in Console and both Error & Output in Log file - UNIX

I am writing a shell script with 2 run time arguments. During the execution if i got any error, then it needs to redirected to a error file and in console. Also both error and output to be redirected to a log file. But i am facing the below error. #! /bin/sh errExit () { errMsg=`cat... (1 Reply)
Discussion started by: sarathy_a35
1 Replies
All times are GMT -4. The time now is 08:37 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy