Logging in shared file


 
Thread Tools Search this Thread
Top Forums Programming Logging in shared file
# 1  
Old 09-28-2013
Logging in shared file

Hi.
The problem is to write logs in a shared file from several processes.
The cooperate call of fprintf() leads to messing the content even in one call

example:
Code:
fprintf(f,"a1 \n a2");
fprintf(f,"a3 \n a4");

out:
Code:
a1
a3
a2
a4

Ofcourse this is possible to implement the file as a critical section with semaphores. But there is a significant chanse to crash for one of the processes, so there is not any garanty that the crashed prosess would not be in the critical section at the moment of crash. So it can cause locking of all processes around the file.
Is there another solution for this problem?

Last edited by Don Cragun; 09-28-2013 at 02:53 AM.. Reason: Use CODE tags for sample input, output, and code segments.
# 2  
Old 09-28-2013
The two fprintf() statements you listed will never produce the output you listed for the file named out no matter what order the fprintf() statements were executed in and no matter whether both fprintf()s were in the same process or different processes. Depending on the mode argument(s) supplied to your call(s) to fopen() to create the STDIO stream(s) f and any calls to setbuf() or setvbuf() on the stream(s) f to set the output buffering mode, there are several possible outcomes, but none of them will result in "a2" or "a4" appearing at the start of a line and none of them will have a <newline> immediately following "a2" or "a4".

However, if all of your calls to fopen() use the mode "a+" and you use setbuf() or setvbuf() to unbuffered (or line buffered if you have each fprintf() format string end output fields result in the last character written being a <newline> character) you could guarantee that the output from each fprintf() would appear in the output as a contiguos string and that the output from one call to fprintf() would not overwrite the output written by another call to fprintf() whether or not those calls occurred in the same thread, same process, or multiple processes as long as no single call to fprintf() produces more than BUFSIZ bytes of output. As long as you use append mode in all cases where you open the stream, you don't need critical sections surrounding the fprintf() calls.
# 3  
Old 09-28-2013
Thanks for detailed answer.
Indeed I've got the similar situation in my log after two vfprintf calls. I'll check my situation and try to formulate my question more accurate.
# 4  
Old 09-28-2013
Whether you use fprintf() or vfprintf() (or putc(), putchar(), printf(), dprintf(), etc.) doesn't matter. What matters is:
  1. using append mode in all of your fopen() calls and
  2. using unbuffered output, or writing complete lines in every output request and using line-buffered output.
This User Gave Thanks to Don Cragun For This Post:
# 5  
Old 09-28-2013
Some details:
logger.c
Code:
FILE * file;// file = fopen(stampedPath,"a");

void writeLog(char * format, ...)
{
  va_list args;
      va_start (args, format);
      vprintf(format, args);
      va_end(args);
      
      va_start(args, format);
      vfprintf(file, format, args);
      va_end(args);
}

function call
Code:
char buff[2048];
char upd ="x10000";
//...
for(i=0; i< 500; ++i)
{
  getMessage(buff,2048);//writing message in the buffer
  writeLog("Got 'isAuthorized' call\n");
  writeLog("Retreiving data from\n");
  writeLog("UPD=%s\n",upd);
  writeLog(buff);
}
//...

LogFragments
Normal:
Code:
...
UPD = x10000
<xdata created_by="MA" created_at="">
    <error number="-3123" message="ERROR OPENING IDF FILE [Cannot open IDF file IDF_901234432.xml.]"/>
</xdata>
Got 'isAuthorized' call
...

Corrupted:
Code:
...
UPD = x100000
<?xml version="1.0" encoding="UTF-8"?>
<xdata created_by="MA" created_at="">
    <error number="-3123" message="ERROR OPENING IDF FILE [Cannot open IDF file IDF_901234432.xml.]"/>
</xdata>
Got 'isAuthorized' call
RetreivingFILE [Cannot open IDF file IDF_901234432.xml.]"/>
</xdata>
Got 'isAuthorized' call
...

Legend:
Message 1
Start of message 2
End of message 3


May be the reason of mixing up in the vfprintf() using?

---------- Post updated at 03:37 AM ---------- Previous update was at 03:33 AM ----------

Some details:
logger.c
Code:
FILE * file;//initialization is ommited

void writeLog(char * format, ...)
{
  va_list args;
      va_start (args, format);
      vprintf(format, args);
      va_end(args);
      
      va_start(args, format);
      vfprintf(file, format, args);
      va_end(args);
}

function call
Code:
char buff[2048];
char upd ="x10000";
//...
for(i=0; i< 500; ++i)
{
  getMessage(buff,2048);//writing message in the buffer
  writeLog("Got 'isAuthorized' call\n");
  writeLog("Retreiving data from\n");
  writeLog("UPD=%s\n",upd);
  writeLog(buff);
}
//...

LogFragments
Normal:
Code:
...
UPD = x10000
<xdata created_by="MA" created_at="">
    <error number="-3123" message="ERROR OPENING IDF FILE [Cannot open IDF file IDF_901234432.xml.]"/>
</xdata>
Got 'isAuthorized' call
...

Corrupted:
Code:
...
UPD = x100000
<?xml version="1.0" encoding="UTF-8"?>
<xdata created_by="MA" created_at="">
    <error number="-3123" message="ERROR OPENING IDF FILE [Cannot open IDF file IDF_901234432.xml.]"/>
</xdata>
Got 'isAuthorized' call
RetreivingFILE [Cannot open IDF file IDF_901234432.xml.]"/>
</xdata>
Got 'isAuthorized' call
...

Legend:
Message 1
Start of message 2
End of message 3


---------- Post updated at 03:42 AM ---------- Previous update was at 03:37 AM ----------

Thanks. I'll try to set larger buffer.

---------- Post updated at 03:51 AM ---------- Previous update was at 03:42 AM ----------

Thanks. I'll try to experiment with buffer
# 6  
Old 09-28-2013
A larger buffer won't help. Setting the buffer size to zero won't help. The problem is that each vfprintf() call can generate multiple calls to write() - the calls that actually write the data to the file. Each individual write() call is guaranteed to be atomic, but a series of multiple write() calls from different processes and/or threads will wind up as you're seeing - all interleaved together.

You need to make certain each call to your writeLog() function results in one and only one underlying write() call.

write(). Not fwrite() or fprintf().

Code:
#define BUF_LEN 4096
void writeLog( const char *format, ... )
{
    char buffer[ BUF_LEN ];
    char *ptr;
    int length;
    va_list args;
    
    va_start( args, format );

    // expand the format and args into a single string
    length = vsnprintf( buffer, BUF_LEN, format, args );

    // if the buffer isn't big enough, allocate a temporary
    // one on the stack (if strings can be really long and
    // blow up the stack, use malloc() here - but then you
    // have to add complexity to the code to make sure
    // any malloc()'d buffer is freed.)
    if ( length >= BUF_LEN )
    {
        // I don't recall offhand if the  "+ 1" is needed, but
        // it doesn't hurt to have it
        ptr = ( char * ) alloca( length + 1 );
        length = vsnprintf( ptr, length + 1, format, args );
    }
    else
    {
        ptr = buffer;
    }

    write( logFd, ptr, length );

    va_end( args );
}

In theory, you could pass a zero length to the first call to vsnprintf() and get back the known length of the final string. In practice there are still vsnprintf() implementations that don't do that per current standards. (The old SUSv2 standard specified different return values.)
This User Gave Thanks to achenle For This Post:
# 7  
Old 09-30-2013
Thanks. I did not know about the fact what 'write' is an atomic function. I'll use it. I am using pretty old compiller version on the target system. And with high possibility there are no implementation of vsnprintf.

---------- Post updated at 10:49 PM ---------- Previous update was at 10:39 PM ----------

I've checked this and I found the implementation. The system was updated recently. So the problem with determination of buffer length was solved.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Linux

Cannot open shared object file: No such file or directory

Hi, While running tcpdump command on my Fedora 16 machine I am get shared library issue. # tcpdump tcpdump: error while loading shared libraries: libcrypto.so.6: cannot open shared object file: No such file or directory # which tcpdump /usr/software/sbin/tcpdump I have tried... (3 Replies)
Discussion started by: muzaffar.k
3 Replies

2. Linux

Syslog not logging successful logging while unlocking server's console

When unlocking a Linux server's console there's no event indicating successful logging Is there a way I can fix this ? I have the following in my rsyslog.conf auth.info /var/log/secure authpriv.info /var/log/secure (1 Reply)
Discussion started by: walterthered
1 Replies

3. Shell Programming and Scripting

Logging success event into file

Hi, I've the following code to log the errors any after the command is executed. # Ksh 88 Version log_path=/home/etc/fls/fls_log.log del_path=/home/etc/fls/to_day rm $del_path/* >> $log_path 2>&1 But I even want to log if the rm command is success without any error along with... (1 Reply)
Discussion started by: smile689
1 Replies

4. Programming

Shared library with acces to shared memory.

Hello. I am new to this forum and I would like to ask for advice about low level POSIX programming. I have to implement a POSIX compliant C shared library. A file will have some variables and the shared library will have some functions which need those variables. There is one special... (5 Replies)
Discussion started by: iamjag
5 Replies

5. Red Hat

libodbc.so.1: cannot open shared object file: No such file or directory

We are trying to install third party software on this unix server... Here is the error message we are getting... error while loading shared libraries: libodbc.so.1: cannot open shared object file: No such file or directory It seems like odbc driver is not installed... >rpm -q unixODBC... (1 Reply)
Discussion started by: govindts
1 Replies

6. Shell Programming and Scripting

logging to file

I am trying to figure a way to have a log file and still keep the output in the terminal in a script. The example below logs to a file nicely but i still want the output in the terminal as well #!/bin/bash #Create a log exec >> /path/to/my/logfile echo "hello world" Any help would be... (3 Replies)
Discussion started by: dave100
3 Replies

7. Programming

libRmath.so: cannot open shared object file: No such file or directory

% locate Rmath /m/backup/backup/lib/R/include/Rmath.h /usr/lib/R/include/Rmath.h % gcc -g -o stand stand.c -I/usr/lib/R/include/ -lRmath -lm % ./stand ./stand: error while loading shared libraries: libRmath.so: cannot open shared object file: No such file or directory What's the trouble... (6 Replies)
Discussion started by: cdbug
6 Replies

8. Programming

Shared memory for shared library

I am writing a shared library in Linux (but compatible with other UNIXes) and I want to allow multiple instances to share a piece of memory -- 1 byte is enough. What's the "best" way to do this? I want to optimize for speed and portability. Obviously, I'll have to worry about mutual exclusion. (0 Replies)
Discussion started by: otheus
0 Replies

9. UNIX for Dummies Questions & Answers

Logging all console activity to a file - how?

Hi all, Well I've had a bit more experience with Unix-like environments since my last post, now that I have started working on my website in earnest and am doing much of the file manipulation via the command line through SSH. The thing is, I want to be able to log all console activity,... (4 Replies)
Discussion started by: patwa
4 Replies

10. Programming

Shared memory in shared library

I need to create a shared library to access an in memory DB. The DB is not huge, but big enough to make it cumbersome to carry around in every single process using the shared library. Luckily, it is pretty static information, so I don't need to worry much about synchronizing the data between... (12 Replies)
Discussion started by: DreamWarrior
12 Replies
Login or Register to Ask a Question