File size limit exceeded


 
Thread Tools Search this Thread
Top Forums Programming File size limit exceeded
# 8  
Old 02-27-2006
I don't know. I do know that v 8.0 of RH has a version of perl that supports large files, so the rest of the OS must also comply.

Try man fopen64
# 9  
Old 02-28-2006
Thanks a lot.
I also don't know whether my system supports or not.

If i assume my system does not support can you suggest me some way to change my code according to the system otherthan using a different linux filesystem?
How do i make the pointers support to open a big file?
# 10  
Old 02-28-2006
Because you didn't expect to do this at the outset, put a few global
variables at the top of the code, then add two small functions:
Code:
#include <stdlib.h>
/* global variables */
unsigned long byte_count=0;
int debug_file_count=0;
char *debug_file_name="/path/debug";
FILE *dbg=NULL;
/* ..........................  further on --- */
void dbg_file_open(void)
{
    char tmp[256]={0x0};

    sprintf(tmp,"%s%d",debug_file_name,debug_file_count++);
    dbg=fopen(tmp,"w");
    if(dbg==NULL)
    {
         perror("Error opening debug file");
         exit(EXIT_FAILURE);
    }
    byte_count=0;
}


void write_debug_file(char *string)
{
    char tmp[256]={0x0};
    size_t len=strlen(string);
    if(dbg==NULL  || byte_count + len > 0x7fffffff)
    {
        dbg_file_open();
        
    }
    byte_count+=len;
    fprintf(dbg,"%s\n",string);
}

# 11  
Old 03-02-2006
Thanks.
All the answers were very helpful.
Thank you all.
# 12  
Old 03-02-2006
Quote:
void write_debug_file(char *string)
{
char tmp[256]={0x0};
size_t len=strlen(string);
if(dbg==NULL || byte_count + len > 0x7fffffff)
{
dbg_file_open();
}
byte_count+=len;
fprintf(dbg,"%s\n",string);
}
jim,

I would like to add few points,
prior opening to new file through dbg_file_open function, close the previously opened file, else the last batch of writes done to the previous file is not guaranteed to be flushed.

the need for char tmp[256] in write_debug_file ???
# 13  
Old 03-02-2006
You're right about closing the file, my bad. However - 256 appears arbitrary. It is not.
Whatever file record length he/she expects, best practice dictates using a variable that is decalred to be significantly larger.

However - and don't get me going on this - picking lengths that are just one char longer than the record leads to trouble in production systems, if the program deals with data derived from any external source. And the memory savings is not worth the cost of debugging and fixing it later on, because some users did not follow procedures.


If you're worried about memory, which, within reason, is basically not merited except in embedded systems or realtime processing, check your requirements. I seldom see - "must run in less than 10MB of memory" - as a stipulated requirement. It's the same issue as using floats instead of doubles because it "saves memory" or "is faster".
The "is faster" is debatable, and is hardware dependant, plus floats are often promoted to doubles. The "saves memory" is usually correct except that what you gain is not worth what you lose - 9 digits of precision.
# 14  
Old 03-03-2006
my question abt tmp[] is not abt the allocation of 256.
simple as that; where the variable tmp is made use of in the program,
if thats for use in later point of the program.. then it is self-answered,

besides the effort spent in debugging for a mistake where the developer had deliberately missed out sentinel byte allocation for a variable is cumbersome
and pathetic. Pt noted.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

httpd count exceeded threshold limit

Hello Everyone, I am new to this forum and also unix/linux. Our application today threw an alert whcih read as "The users active count on host has crossed the threshold limit of 50 and is standing at 65." This was although cleared when I restarted tomcat. But I am not sure why this count... (0 Replies)
Discussion started by: ykhati
0 Replies

2. Shell Programming and Scripting

Problem with awk awk: program limit exceeded: sprintf buffer size=1020

Hi I have many problems with a script. I have a script that formats a text file but always prints the same error when i try to execute it The code is that: { if (NF==17){ print $0 }else{ fields=NF; all=$0; while... (2 Replies)
Discussion started by: fate
2 Replies

3. UNIX for Advanced & Expert Users

file size limit?

hi, how can I find out what the limit of a file size is on unix? thanks (6 Replies)
Discussion started by: JamesByars
6 Replies

4. Shell Programming and Scripting

Logfile Size exceeded ????

Hi, How to write a script which checks the size of a log file? I want that the log file contents to get cleared as soon as it increases 1 KB. Thanks (3 Replies)
Discussion started by: skyineyes
3 Replies

5. UNIX for Dummies Questions & Answers

File size limit exceeded... SCO ulimit?

Hello - O/S is UnixWare 7.1.4 My prefered method of copying files between servers is 'rcp', which does not recognize symbolic links; therefore, files are duplicated many times over. To avoid this duplication, I would like to use 'tar' and/or 'cpio' and pipe them through 'rcp', but... (1 Reply)
Discussion started by: rm -r *
1 Replies

6. Solaris

/tmp: File system full, swap space limit exceeded

Can you help. My server sunning solaris 9 on x86 platform pretty much hung for a few hours... I could not use telnet or ssh to the box - it kept refusing connection. A few hours later - I was able to log in again. The server has not rebooted but here are the first errors in the messages log... (5 Replies)
Discussion started by: frustrated1
5 Replies

7. UNIX for Dummies Questions & Answers

Error, Login Limit Exceeded by 1 user

Would appreciate some help, system was displaying an error regarding the kernal when a "sar" was run, after a reboot we get "WARNING user login limit exceeded by 1 user". We have plenty of licences. any ideas? (1 Reply)
Discussion started by: nchrocc
1 Replies

8. UNIX for Advanced & Expert Users

File Size Limit

Hi, I have a problem writing or copying a file 2GB or larger to either the second or third disk on my C8000. I've searched this forum and found some good information on this but still nothing to solve the problem. I'm running hpux 11i, JFS3.3 and disk version 4 (from fstyp) on all 3 disks. ... (2 Replies)
Discussion started by: HaidoodFaulkauf
2 Replies

9. AIX

file size limit

Can anybody help me? How to increase file size limit in aix 5.2? I have already specified in /etc/security/limits file : default: fsize = -1 core = 2097151 cpu = -1 data = -1 rss = -1 stack = -1 nofiles = 2000 (2 Replies)
Discussion started by: vjm
2 Replies

10. Solaris

File size limit

I want to have a permanent file created - and limit the size that this file can grow.. I want a circular file.. ie max size of file is 10 mb.. and if any new data written to file the oldest data removed.. How can I do this? I am on solaris 9 x86 (3 Replies)
Discussion started by: frustrated1
3 Replies
Login or Register to Ask a Question