Sponsored Content
Special Forums Hardware Filesystems, Disks and Memory Write Speed into a big file (in Gb's) Post 302278412 by pludi on Tuesday 20th of January 2009 07:12:25 AM
Old 01-20-2009
Let's assume an average write speed of 50MB/s. A file with 10KB will take about 0.0002 seconds to write, whereas a file with 50 GB will need about 17 minutes. So yes, it will take more time to write a large file than a small one.
But if you mean if it takes more time to change a large file than a smaller one, that depends more on the structure and the program accessing it than the filesystem/OS. As an example, mmap()-ing a small file is very fast, but with large files you'll run into problems quickly. On the other hand, accessing data that's stored as a binary tree, with an application optimized for it, is almost always quicker than a linear search through a smaller file (O(log n) vs O(n))
 

8 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to view a big file(143M big)

1 . Thanks everyone who read the post first. 2 . I have a log file which size is 143M , I can not use vi open it .I can not use xedit open it too. How to view it ? If I want to view 200-300 ,how can I implement it 3 . Thanks (3 Replies)
Discussion started by: chenhao_no1
3 Replies

2. Filesystems, Disks and Memory

dmidecode, RAM speed = "Current Speed: Unknown"

Hello, I have a Supermicro server with a P4SCI mother board running Debian Sarge 3.1. This is the "dmidecode" output related to RAM info: RAM speed information is incomplete.. "Current Speed: Unknown", is there anyway/soft to get the speed of installed RAM modules? thanks!! Regards :)... (0 Replies)
Discussion started by: Santi
0 Replies

3. Filesystems, Disks and Memory

data from blktrace: read speed V.S. write speed

I analysed disk performance with blktrace and get some data: read: 8,3 4 2141 2.882115217 3342 Q R 195732187 + 32 8,3 4 2142 2.882116411 3342 G R 195732187 + 32 8,3 4 2144 2.882117647 3342 I R 195732187 + 32 8,3 4 2145 ... (1 Reply)
Discussion started by: W.C.C
1 Replies

4. UNIX for Dummies Questions & Answers

How big is too big a config.log file?

I have a 5000 line config.log file with several "maybe" errors. Any reccomendations on finding solvable problems? (2 Replies)
Discussion started by: NeedLotsofHelp
2 Replies

5. Filesystems, Disks and Memory

Disk read and write speed.

Being a novice user to linux i m little unaware of how would i check disk read write speed. One of my mate is suggesting to create a file using dd command and check how much time it takes to create a 30 gb file . I think this has a little sense however i would also like to take your reviews... (5 Replies)
Discussion started by: pinga123
5 Replies

6. UNIX for Dummies Questions & Answers

Find the file which is growing at high speed

Hi, A log file which is growing at high speed, don't know the name of it. How to find the respective file? Many thanks. (2 Replies)
Discussion started by: venkatesht
2 Replies

7. Hardware

Hardware Correction: How to change DVD write speed

I am now on Kernel 2.6.32-26 For me 16x CD write speed is okay. I have old hardware which was able to write DVDs at 1x, back in previous linux version. Now, I dont get speed of less than 4x. Tested on k3b, xfburn, and brasero. But all start at bottom 4x write speed. k3b forced back to... (0 Replies)
Discussion started by: makh
0 Replies

8. Shell Programming and Scripting

Speed : awk command to count the occurrences of fields from one file present in the other file

Hi, file1.txt AAA BBB CCC DDD file2.txt abc|AAA|AAAabcbcs|fnwufnq bca|nwruqf|AAA|fwfwwefwef fmimwe|BBB|fnqwufw|wufbqw wcdbi|CCC|wefnwin|wfwwf DDD|wabvfav|wqef|fwbwqfwfe i need the count of rows of file1.txt present in the file2.txt required output: AAA 2 (10 Replies)
Discussion started by: mdkm
10 Replies
directio(3C)						   Standard C Library Functions 					      directio(3C)

NAME
directio - provide advice to file system SYNOPSIS
#include <sys/types.h> #include <sys/fcntl.h> int directio(int fildes, int advice); DESCRIPTION
The directio() function provides advice to the system about the expected behavior of the application when accessing the data in the file associated with the open file descriptor fildes. The system uses this information to help optimize accesses to the file's data. The directio() function has no effect on the semantics of the other operations on the data, though it may affect the performance of other operations. The advice argument is kept per file; the last caller of directio() sets the advice for all applications using the file associated with fildes. Values for advice are defined in <sys/fcntl.h>. DIRECTIO_OFF Applications get the default system behavior when accessing file data. When an application reads data from a file, the data is first cached in system memory and then copied into the applica- tion's buffer (see read(2)). If the system detects that the application is reading sequentially from a file, the system will asynchronously "read ahead" from the file into system memory so the data is immediately available for the next read(2) operation. When an application writes data into a file, the data is first cached in system memory and is written to the device at a later time (see write(2)). When possible, the system increases the performance of write(2) operations by cacheing the data in memory pages. The data is copied into system memory and the write(2) operation returns immediately to the application. The data is later written asynchronously to the device. When possible, the cached data is "clustered" into large chunks and written to the device in a single write operation. The system behavior for DIRECTIO_OFF can change without notice. DIRECTIO_ON The system behaves as though the application is not going to reuse the file data in the near future. In other words, the file data is not cached in the system's memory pages. When possible, data is read or written directly between the application's memory and the device when the data is accessed with read(2) and write(2) operations. When such transfers are not possible, the system switches back to the default behav- ior, but just for that operation. In general, the transfer is possible when the application's buffer is aligned on a two- byte (short) boundary, the offset into the file is on a device sector boundary, and the size of the operation is a multiple of device sectors. This advisory is ignored while the file associated with fildes is mapped (see mmap(2)). The system behavior for DIRECTIO_ON can change without notice. RETURN VALUES
Upon successful completion, directio() returns 0. Otherwise, it returns -1 and sets errno to indicate the error. ERRORS
The directio() function will fail if: EBADF The fildes argument is not a valid open file descriptor. ENOTTY The fildes argument is not associated with a file system that accepts advisory functions. EINVAL The value in advice is invalid. USAGE
Small sequential I/O generally performs best with DIRECTIO_OFF. Large sequential I/O generally performs best with DIRECTIO_ON, except when a file is sparse or is being extended and is opened with O_SYNC or O_DSYNC (see open(2)). The directio() function is supported for the NFS and UFS file system types (see fstyp(1M)). ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |MT-Level |MT-Safe | +-----------------------------+-----------------------------+ SEE ALSO
fstyp(1M), mmap(2), open(2), read(2), write(2), fcntl.h(3HEAD), attributes(5) WARNINGS
Switching between DIRECTIO_OFF and DIRECTIO_ON can slow the system because each switch to DIRECTIO_ON might entail flushing the file's data from the system's memory. SunOS 5.10 9 Apr 2003 directio(3C)
All times are GMT -4. The time now is 02:31 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy