Can't create file bigger than 2GB with my application


 
Thread Tools Search this Thread
Top Forums Programming Can't create file bigger than 2GB with my application
# 1  
Old 05-28-2009
Can't create file bigger than 2GB with my application

Hi,

I've created a simple application that is supposed to fill up a file with messages up
to the size I pass as parameter.

The problem is that once the file reaches the 2GB size, it stops growing.

The flow of the application, for what is worth, is as follows:
while ( bytes written < bytes wanted )
create message;
fwrite message;
free message;

At first I thought it was something related to some default limit of the file system (perhaps some obscure ubuntu configuration), so I tried with dd but I managed to create a 7GB file.

I then realized that I was passing the size as a parameter in bytes `expr 1024 * 1024 * 1024 * 4` which was converted with atoi and assigned to an int variable, so I thought that was problem. I decided to manually set the size internally using an unit64_t variable set to 4GB, but the result is the same.

I also tried to attach to the process with gdb once the size of 2147483647 bytes was reached but the flow was just normal (althought I don't check the return value of fwrite).

Can somebody shine some light on this one?

Thanks,
S.
# 2  
Old 05-28-2009
you can use
Code:
ulimit -a

and check if your filesize is limited.
Regards
# 3  
Old 05-28-2009
Quote:
Originally Posted by thanhdat
you can use
Code:
ulimit -a

and check if your filesize is limited.
Regards
Yes, I did that already and it shows not limit.
Thanks.

Code:
salvatore@salvatore-desktop:~$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 20
file size               (blocks, -f) unlimited
pending signals                 (-i) 16382
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) unlimited
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

# 4  
Old 05-28-2009
you can use mkfile .Check the man pages for details.
# 5  
Old 05-28-2009
Quote:
Originally Posted by amitranjansahu
you can use mkfile .Check the man pages for details.
I'm sorry, but what do I need mkfile for ?

My problem here is not only to be able to write with my application to a file until it reaches a certain size, but also to understand why it does not work as it is coded right now.

Thanks anyway for your answer.
# 6  
Old 05-28-2009
Try compiling your code with the macro -D_FILE_OFFSET_BITS=64.

Look at _FILE_OFFSET_BITS under info libc 'Feature Test Macros'.
# 7  
Old 05-28-2009
Quote:
Originally Posted by vino
Try compiling your code with the macro -D_FILE_OFFSET_BITS=64.

Look at _FILE_OFFSET_BITS under info libc 'Feature Test Macros'.
Yeah! That worked. Thank you very much.
Now I want to understand what the problem was.
Unfortunately info libc says "No menu item `libc' in node `(dir)Top'".
and I don't know how to get it in Ubuntu.

Care to explain with few words or point me somewhere else beside info?
Otherwise I'll just google! ;-)

S.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Delete the file which crossed 2GB

Hi , I wants to create the bash script for deleting the specified 2gb file and wants to take the backup before doing that. please help me how to do the same,I use RHEL5 server (22 Replies)
Discussion started by: Rahulne25
22 Replies

2. HP-UX

2GB file size limit

Greetings, I'm attempting to dump a filesystem from a RHEL5 Linux server to a VXFS filesystem on an HP-UX server. The VXFS filesystem is large file enabled and I've confirmed that I can copy/scp a file >2GB to the filesystem. # fsadm -F vxfs /os_dumps largefiles # mkfs -F vxfs -m... (12 Replies)
Discussion started by: bkimura
12 Replies

3. UNIX for Advanced & Expert Users

How to create a file more than 2GB

Hi, I am executing a SQL query and the output is more than 2GB. Hence the process is failing. How can I have a file created more than 2GB ? Thanks, Risshanth (1 Reply)
Discussion started by: risshanth
1 Replies

4. Linux

unzipping file > 2gb

I am not able to unzip file greater then 2gb, Any suggestions how to do that in linux? Regards, Manoj (5 Replies)
Discussion started by: manoj.solaris
5 Replies

5. AIX

Creating > 2GB file

I am trying to execute a database dump to a file, but can't seem to get around the 2GB file size. I have tried setting the user limit to -1, but no luck. (4 Replies)
Discussion started by: markper
4 Replies

6. UNIX for Dummies Questions & Answers

MAX file size limited to 2GB

Hi All, We are running HP rp7400 box with hpux 11iv1. Recently, we changed 3 kernel parameters a) msgseg from 32560 to 32767 b) msgmnb from 65536 to 65535 c) msgssz from 128 to 256 Then we noticed that all application debug file size increase upto 2GB then it stops. So far we did not... (1 Reply)
Discussion started by: mhbd
1 Replies

7. Shell Programming and Scripting

efficiently split a 2GB text file into two

Can an expert kindly write an efficient Linux ksh script that will split a large 2 GB text file into two? Here is a couple of sample record from that text file: "field1","field2","field3",11,22,33,44 "TG","field2b","field3b",1,2,3,4 The above rows are delimited by commas. This script is to... (2 Replies)
Discussion started by: ihot
2 Replies

8. Solaris

SUN Solaris 9 - Is there a 2GB file size limit?

Hi I am using SUN/Solaris 9 and I was told that some unix versions have 2GB size limit. Does this applies to SUN/Solaris 9? Thanks. (2 Replies)
Discussion started by: GMMike
2 Replies

9. Programming

C++ Problem, managing >2Gb file

My C++ program returns 'Disk Full' Message when I tried to manage a file larger than 2Gb. the process is very simple: based on a TXT file, the process combine the information generating another temporary file (generating the error) to fillup a database. My FS, during the process, reaches 40%...... (4 Replies)
Discussion started by: ASOliveira
4 Replies

10. UNIX for Dummies Questions & Answers

File size exceeding 2GB

I am working on HP-Unix. I have a 600 MB file in compressed form. During decompression, when file size reaches 2GB, decompression aborts. What should be done? (3 Replies)
Discussion started by: Nadeem Mistry
3 Replies
Login or Register to Ask a Question