Sponsored Content
Top Forums Programming C program in Unix / Linux - Time differences Post 82871 by jim mcnamara on Wednesday 7th of September 2005 10:45:15 AM
Old 09-07-2005
It looks like you are trying to do random access on a file, because you're spending a lot of time repositioning the disk heads (__lseek_sys).

This is probably okay on desktop Linux boxes. This is a no-no on a multiuser system.

Here's why. Suppose you want to make 20 separate deposit transactions at the bank.
If you go to the teller and do all 20 at one time, it's fast. If you are limited to one transaction, and then you have to go to the back of the line and wait your turn again, it takes a LOT longer. Your code is in line and having to wait for everybody else's I/O requests to complete everytime you ask the disk to reposition the head. Plus you end up re-reading a lot more off disk (assuming a disk hardware cache) than exists in the file.

50% of your time is disk I/O. You're I/O bound. Big time.

Read the entire file sequentially into memory with one call. Put the file into an array of binary records, then access them using random methods. try something like this:
Code:
#include <stddef.h>
#include <sys/types.h>
#include <unistd.h>
#include <fcntl.h>
#include <stdlib.h>
#include <stdio.h>
#include <sys/stat.h>
#include <errno.h>
#include <assert.h>

#define ck(x) \
if( (x) == (-1) ){ perror("");exit(EXIT_FAILURE);}

/* read a buffer from a file */
ssize_t readall(int fd, void *buf, size_t *bytes){
     ssize_t nread = 0, n=0;
     size_t nbyte = *bytes;

     do {
         if ((n = read(fd, &((char *)buf)[nread], nbyte - nread)) == -1) {
             if (errno == EINTR)
                 continue;
             else
                 return (-1);
         }
         if (n == 0)
             return nread;
         nread += n;
     } while (nread < nbyte);
     return nread;
}

/* read control */
void readfile(char *fname, char *buffer, size_t *size, mode_t *mode)
{
   int fd=0;   
   struct stat st;
   
   ck(fd=open(fname,O_RDONLY) );
   ck(fstat(fd,&st) );
   *size=st.st_size;   
   *mode=st.st_mode;
   buffer=calloc(1,*size+1);
   ck(readall(fd, buffer, size) );    
   ck(close(fd) );
}


int main(int argc, char *argv[])
{
   char *buffer=NULL;
   size_t size;
   mode_t mode;
 
   readfile(argv[1],buffer,&size,&mode);
   /* play with buffer here  */
   free(buffer);
   return 0;  
}

 

8 More Discussions You Might Find Interesting

1. Programming

Program in linux for all unix os

Hello there, I have the following mission for my internship: - Take an existing program which uses OpenGL writen in Delphi, for Windows. - Write this program in C++ for all operating systems from which the name end on a "X" So, what kind of features does this program need. - There... (1 Reply)
Discussion started by: lmnt22
1 Replies

2. Shell Programming and Scripting

Working out time differences

Hi everyone, I need to be able to write into a ksh script, a function that can look at 2 24 hour time variables and work out the difference between them. e.g job1 runs at 21:00 job2 runs at 01:00 diff = 04:00 hours I would also need negative numbers i.e where job1 runs after job2 ... (1 Reply)
Discussion started by: rik1551
1 Replies

3. Shell Programming and Scripting

Date/time differences

A thanks to all ahead of time. I've checked previous posts about this subject and can't find any that quite fit what I need. If I've missed the post could you point me there. When I do an ls -al I get the following output: -rw-r--r-- 1 staff staff 855 July 24 20:05 ... (4 Replies)
Discussion started by: gillr
4 Replies

4. Solaris

difficult time differences

:rolleyes: Hi, How to take the time diffence between start and finish time from a log file? It is like ..... started at Jun 20 23:20 . . ..... finished at Jun 21 01:40 Tryed so many ways but failed to ger exact way. :confused: Your help will be honoured. Ta........Lokesha (1 Reply)
Discussion started by: Lokesha
1 Replies

5. UNIX for Dummies Questions & Answers

Socket Handling Differences Between Linux & Unix?

Sorry if this is a stupid question! I have been developing a Java application that I am deploying on both Unix and Linux servers, which uses lots of socket handling. When the server side connection is dropped by the server un-gracefully I have been seeing close_waits and null connections. ... (0 Replies)
Discussion started by: Vinnie
0 Replies

6. Windows & DOS: Issues & Discussions

XP as a program on Unix/Linux system?

I am sure this question has been asked and answered before, also, if it is the wrong catergory, please let me know. I would like to know how, if possible to run Unix/Linux as my operating system, and then load XP as a program? (I am open to other operating systems, but it has to be able to... (3 Replies)
Discussion started by: mountainwolf
3 Replies

7. UNIX for Dummies Questions & Answers

Differences between time command and usr/bin/time

I wondered if someone could point out the differences between the time commmand and usr/bin/time and the accuracy one might have over another. Also, is there a website or two a person could maybe link for me to describe the differences? Thank you for your time. (2 Replies)
Discussion started by: icedrake
2 Replies

8. Shell Programming and Scripting

differences in linux and unix shell

hi all, can any one plz tell me that what is the difference between linux shell scripting and unix shell scripting. is there any difference at all?? if yes that what are the differences and how could it be combatted thanks in advance (1 Reply)
Discussion started by: nasir_khan
1 Replies
FOPENCOOKIE(3)						     Linux Programmer's Manual						    FOPENCOOKIE(3)

NAME
fopencookie - opening a custom stream SYNOPSIS
#define _GNU_SOURCE /* See feature_test_macros(7) */ #include <stdio.h> FILE *fopencookie(void *cookie, const char *mode, cookie_io_functions_t io_funcs); DESCRIPTION
The fopencookie() function allows the programmer to create a custom implementation for a standard I/O stream. This implementation can store the stream's data at a location of its own choosing; for example, fopencookie() is used to implement fmemopen(3), which provides a stream interface to data that is stored in a buffer in memory. In order to create a custom stream the programmer must: * Implement four "hook" functions that are used internally by the standard I/O library when performing I/O on the stream. * Define a "cookie" data type, a structure that provides bookkeeping information (e.g., where to store data) used by the aforementioned hook functions. The standard I/O package knows nothing about the contents of this cookie (thus it is typed as void * when passed to fopencookie()), but automatically supplies the cookie as the first argument when calling the hook functions. * Call fopencookie() to open a new stream and associate the cookie and hook functions with that stream. The fopencookie() function serves a purpose similar to fopen(3): it opens a new stream and returns a pointer to a FILE object that is used to operate on that stream. The cookie argument is a pointer to the caller's cookie structure that is to be associated with the new stream. This pointer is supplied as the first argument when the standard I/O library invokes any of the hook functions described below. The mode argument serves the same purpose as for fopen(3). The following modes are supported: r, w, a, r+, w+, and a+. See fopen(3) for details. The io_funcs argument is a structure that contains four fields pointing to the programmer-defined hook functions that are used to implement this stream. The structure is defined as follows typedef struct { cookie_read_function_t *read; cookie_write_function_t *write; cookie_seek_function_t *seek; cookie_close_function_t *close; } cookie_io_functions_t; The four fields are as follows: cookie_read_function_t *read This function implements read operations for the stream. When called, it receives three arguments: ssize_t read(void *cookie, char *buf, size_t size); The buf and size arguments are, respectively, a buffer into which input data can be placed and the size of that buffer. As its function result, the read function should return the number of bytes copied into buf, 0 on end of file, or -1 on error. The read function should update the stream offset appropriately. If *read is a NULL pointer, then reads from the custom stream always return end of file. cookie_write_function_t *write This function implements write operations for the stream. When called, it receives three arguments: ssize_t write(void *cookie, const char *buf, size_t size); The buf and size arguments are, respectively, a buffer of data to be output to the stream and the size of that buffer. As its func- tion result, the write function should return the number of bytes copied from buf, or 0 on error. (The function must not return a negative value.) The write function should update the stream offset appropriately. If *write is a NULL pointer, then output to the stream is discarded. cookie_seek_function_t *seek This function implements seek operations on the stream. When called, it receives three arguments: int seek(void *cookie, off64_t *offset, int whence); The *offset argument specifies the new file offset depending on which of the following three values is supplied in whence: SEEK_SET The stream offset should be set *offset bytes from the start of the stream. SEEK_CUR *offset should be added to the current stream offset. SEEK_END The stream offset should be set to the size of the stream plus *offset. Before returning, the seek function should update *offset to indicate the new stream offset. As its function result, the seek function should return 0 on success, and -1 on error. If *seek is a NULL pointer, then it is not possible to perform seek operations on the stream. cookie_close_function_t *close This function closes the stream. The hook function can do things such as freeing buffers allocated for the stream. When called, it receives one argument: int close(void *cookie); The cookie argument is the cookie that the programmer supplied when calling fopencookie(). As its function result, the close function should return 0 on success, and EOF on error. If *close is NULL, then no special action is performed when the stream is closed. RETURN VALUE
On success fopencookie() returns a pointer to the new stream. On error, NULL is returned. CONFORMING TO
This function is a nonstandard GNU extension. EXAMPLE
The program below implements a custom stream whose functionality is similar (but not identical) to that available via fmemopen(3). It implements a stream whose data is stored in a memory buffer. The program writes its command-line arguments to the stream, and then seeks through the stream reading two out of every five characters and writing them to standard output. The following shell session demonstrates the use of the program: $ ./a.out 'hello world' /he/ / w/ /d/ Reached end of file Note that a more general version of the program below could be improved to more robustly handle various error situations (e.g., opening a stream with a cookie that already has an open stream; closing a stream that has already been closed). Program source #define _GNU_SOURCE #include <sys/types.h> #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <string.h> #define INIT_BUF_SIZE 4 struct memfile_cookie { char *buf; /* Dynamically sized buffer for data */ size_t allocated; /* Size of buf */ size_t endpos; /* Number of characters in buf */ off_t offset; /* Current file offset in buf */ }; ssize_t memfile_write(void *c, const char *buf, size_t size) { char *new_buff; struct memfile_cookie *cookie = c; /* Buffer too small? Keep doubling size until big enough */ while (size + cookie->offset > cookie->allocated) { new_buff = realloc(cookie->buf, cookie->allocated * 2); if (new_buff == NULL) { return -1; } else { cookie->allocated *= 2; cookie->buf = new_buff; } } memcpy(cookie->buf + cookie->offset, buf, size); cookie->offset += size; if (cookie->offset > cookie->endpos) cookie->endpos = cookie->offset; return size; } ssize_t memfile_read(void *c, char *buf, size_t size) { ssize_t xbytes; struct memfile_cookie *cookie = c; /* Fetch minimum of bytes requested and bytes available */ xbytes = size; if (cookie->offset + size > cookie->endpos) xbytes = cookie->endpos - cookie->offset; if (xbytes < 0) /* offset may be past endpos */ xbytes = 0; memcpy(buf, cookie->buf + cookie->offset, xbytes); cookie->offset += xbytes; return xbytes; } int memfile_seek(void *c, off64_t *offset, int whence) { off64_t new_offset; struct memfile_cookie *cookie = c; if (whence == SEEK_SET) new_offset = *offset; else if (whence == SEEK_END) new_offset = cookie->endpos + *offset; else if (whence == SEEK_CUR) new_offset = cookie->offset + *offset; else return -1; if (new_offset < 0) return -1; cookie->offset = new_offset; *offset = new_offset; return 0; } int memfile_close(void *c) { struct memfile_cookie *cookie = c; free(cookie->buf); cookie->allocated = 0; cookie->buf = NULL; return 0; } int main(int argc, char *argv[]) { cookie_io_functions_t memfile_func = { .read = memfile_read, .write = memfile_write, .seek = memfile_seek, .close = memfile_close }; FILE *fp; struct memfile_cookie mycookie; ssize_t nread; long p; int j; char buf[1000]; /* Set up the cookie before calling fopencookie() */ mycookie.buf = malloc(INIT_BUF_SIZE); if (mycookie.buf == NULL) { perror("malloc"); exit(EXIT_FAILURE); } mycookie.allocated = INIT_BUF_SIZE; mycookie.offset = 0; mycookie.endpos = 0; fp = fopencookie(&mycookie,"w+", memfile_func); if (fp == NULL) { perror("fopencookie"); exit(EXIT_FAILURE); } /* Write command-line arguments to our file */ for (j = 1; j < argc; j++) if (fputs(argv[j], fp) == EOF) { perror("fputs"); exit(EXIT_FAILURE); } /* Read two bytes out of every five, until EOF */ for (p = 0; ; p += 5) { if (fseek(fp, p, SEEK_SET) == -1) { perror("fseek"); exit(EXIT_FAILURE); } nread = fread(buf, 1, 2, fp); if (nread == -1) { perror("fread"); exit(EXIT_FAILURE); } if (nread == 0) { printf("Reached end of file "); break; } printf("/%.*s/ ", nread, buf); } exit(EXIT_SUCCESS); } SEE ALSO
fclose(3), fmemopen(3), fopen(3), fseek(3) COLOPHON
This page is part of release 3.53 of the Linux man-pages project. A description of the project, and information about reporting bugs, can be found at http://www.kernel.org/doc/man-pages/. Linux 2013-03-17 FOPENCOOKIE(3)
All times are GMT -4. The time now is 08:27 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy