Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: overhead of fopen/freopen
Top Forums Programming overhead of fopen/freopen Post 302408127 by jim mcnamara on Sunday 28th of March 2010 08:03:05 AM
Old 03-28-2010
One problem is - you are changing disk metadata everytime you close a file.
ie., each close may require direct I/O to the disk to update data then direct I/O to update file metadata - the stuff you see with stat: mtime, atime, #bytes in file. Disk I/O is usually several orders of magnitude slower than memory..

So -
1.consider using two large blocks of shared memory - have your process write directly to memory.

2. When your block is nearly full create several threads, one for each filename you need, to do file writes and cleanup the memory block. One file open/close per thread.

3. While the worker threads are busy have the main process write to the second memory block.
 

10 More Discussions You Might Find Interesting

1. Programming

difference between fdopen() and freopen()

hi , I came acroos two functions fdopen() and freopen(). what is the difference between these two functions and where can they be used. Is it that fdopen() is used to write freopen(). Advance Thanks for your co-operation. :) (1 Reply)
Discussion started by: kinnaree
1 Replies

2. Programming

.cc fopen failed - Broken Pipe

hello.. i make some code with C in freebsd 5.4 and compile it in solaris somehow i succeed compile the program. but when i run it, i got error message "Broken Pipe" i looked out the syntax that that caused this, fp = fopen("file.tmp","r"); does anyone know why, and how to solve this... (3 Replies)
Discussion started by: kuampang
3 Replies

3. Web Development

CAN TCPDF USE fopen() or Convert URL To PDF?

Dear all, I'm a newbie for PHP and TCPDF ,I have to change the URL to PDF, so I used FPDF , But it cannot convert most of the advanced HTML tags. So explored again and found TCPDF , it can do most of the tag but I cannot found to change URL to PDF. So Does anyone can point the example... (0 Replies)
Discussion started by: athae
0 Replies

4. UNIX for Advanced & Expert Users

overhead in the archive

Hi everyone, I am currently trying to work out the size overhead in the library archive. The total size of all my objects file is about 100KB. However, when I package them into the archive (libXX.a), the size gets boosted up to 200KB. I want to know what exact is that 100KB overhead. I tried... (1 Reply)
Discussion started by: jasoncrab
1 Replies

5. UNIX for Advanced & Expert Users

Linux fopen() mistery. Help required.

Hello! I'm having problems with fopen() call in Linux. I have shared library (created by myself) that implements some file operations: int lib_func(char* file_name) { ... fd = fopen(file_name, "r"); if(!fd) {... exit with error ...} ... do something useful using fd ... ... (2 Replies)
Discussion started by: kalbi
2 Replies

6. Programming

fopen and open

what is the difference between fopen and open fread and read fwrite and write open and create why this much of functions for the i/o when everything does the same...? What is their major difference? In which case, which is the best to use. :confused:'ed Collins (2 Replies)
Discussion started by: collins
2 Replies

7. Programming

fopen() - don't know what I'm doing wrong

This code works fine when I use a command line argument for fopen()'s parameter, but when I change it to a filename, the program freezes upon compilation. input.txt is definitely there, so I can't figure it out. Thanks. #include <stdlib.h> #include <stdio.h> #include <ctype.h> int... (3 Replies)
Discussion started by: lazypeterson
3 Replies

8. UNIX for Dummies Questions & Answers

Overhead of using a shared library

Hi, I found a very strange thing when I linked my executable with a shared library. That is the executable only references a small function of the shared library, and the size of this function is only hundred bytes, but when I check the /proc/pid/smaps, I found that the 'Rss' of this shared... (8 Replies)
Discussion started by: Dongping84
8 Replies

9. Web Development

Java overhead

Hey Guys and girls,can anybody with a experience in java since i am pretty new in it, tell me why a java or java enabled web program is eating up so much system resources like CPU,Ram......ect and how to go by finding what is causing the overhead.;) Thanks a mill (3 Replies)
Discussion started by: techcreeb
3 Replies

10. Programming

help plz - fopen()

Hello, I have a problem here, I want to write a function called"myfopen()" instead of "fopen()" for writing this function I must not use the <stdio.h> library, Can you help me? thanks a lot (2 Replies)
Discussion started by: hamed.samie
2 Replies

LEARN ABOUT REDHAT

vmstat

VMSTAT(8)						   Linux Administrator's Manual 						 VMSTAT(8)

NAME

       vmstat - Report virtual memory statistics

SYNOPSIS

       vmstat [-n] [delay [ count]]
       vmstat[-V]

DESCRIPTION

       vmstat reports information about processes, memory, paging, block IO, traps, and cpu activity.

       The  first report produced gives averages since the last reboot.  Additional reports give information on a sampling period of length delay.
       The process and memory reports are instantaneous in either case.

   Options
       The -n switch  causes the header to be displayed only once rather than periodically.

       delay is the delay between updates in seconds.  If no delay is specified, only one report is printed with the average values since boot.

       count is the number of updates.	If no count is specified and delay is defined, count defaults to infinity.

       The -V switch results in displaying version information.

FIELD DESCRIPTIONS

   Procs
       r: The number of processes waiting for run time.
       b: The number of processes in uninterruptable sleep.
       w: The number of processes swapped out but otherwise runnable.  This
	  field is calculated, but Linux never desperation swaps.

   Memory
       swpd: the amount of virtual memory used (kB).
       free: the amount of idle memory (kB).
       buff: the amount of memory used as buffers (kB).

   Swap
       si: Amount of memory swapped in from disk (kB/s).
       so: Amount of memory swapped to disk (kB/s).

   IO
       bi: Blocks sent to a block device (blocks/s).
       bo: Blocks received from a block device (blocks/s).

   System
       in: The number of interrupts per second, including the clock.
       cs: The number of context switches per second.

   CPU
       These are percentages of total CPU time.
       us: user time
       sy: system time
       id: idle time

NOTES

       vmstat does not require special permissions.

       These reports are intended to help identify system bottlenecks.	Linux vmstat does not count itself as a running process.

       All linux blocks are currently 1k, except for CD-ROM blocks which are 2k.

FILES

       /proc/meminfo
       /proc/stat
       /proc/*/stat

SEE ALSO

       ps(1), top(1), free(1)

BUGS

       Does not tabulate the block io per device or count the number of system calls.

AUTHOR

       Written by Henry Ware <al172@yfn.ysu.edu>.

Throatwobbler Ginkgo Labs					   27 July 1994 							 VMSTAT(8)
All times are GMT -4. The time now is 05:21 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy