Sponsored Content
Special Forums Hardware Filesystems, Disks and Memory Write Speed into a big file (in Gb's) Post 302278412 by pludi on Tuesday 20th of January 2009 07:12:25 AM
Old 01-20-2009
Let's assume an average write speed of 50MB/s. A file with 10KB will take about 0.0002 seconds to write, whereas a file with 50 GB will need about 17 minutes. So yes, it will take more time to write a large file than a small one.
But if you mean if it takes more time to change a large file than a smaller one, that depends more on the structure and the program accessing it than the filesystem/OS. As an example, mmap()-ing a small file is very fast, but with large files you'll run into problems quickly. On the other hand, accessing data that's stored as a binary tree, with an application optimized for it, is almost always quicker than a linear search through a smaller file (O(log n) vs O(n))
 

8 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to view a big file(143M big)

1 . Thanks everyone who read the post first. 2 . I have a log file which size is 143M , I can not use vi open it .I can not use xedit open it too. How to view it ? If I want to view 200-300 ,how can I implement it 3 . Thanks (3 Replies)
Discussion started by: chenhao_no1
3 Replies

2. Filesystems, Disks and Memory

dmidecode, RAM speed = "Current Speed: Unknown"

Hello, I have a Supermicro server with a P4SCI mother board running Debian Sarge 3.1. This is the "dmidecode" output related to RAM info: RAM speed information is incomplete.. "Current Speed: Unknown", is there anyway/soft to get the speed of installed RAM modules? thanks!! Regards :)... (0 Replies)
Discussion started by: Santi
0 Replies

3. Filesystems, Disks and Memory

data from blktrace: read speed V.S. write speed

I analysed disk performance with blktrace and get some data: read: 8,3 4 2141 2.882115217 3342 Q R 195732187 + 32 8,3 4 2142 2.882116411 3342 G R 195732187 + 32 8,3 4 2144 2.882117647 3342 I R 195732187 + 32 8,3 4 2145 ... (1 Reply)
Discussion started by: W.C.C
1 Replies

4. UNIX for Dummies Questions & Answers

How big is too big a config.log file?

I have a 5000 line config.log file with several "maybe" errors. Any reccomendations on finding solvable problems? (2 Replies)
Discussion started by: NeedLotsofHelp
2 Replies

5. Filesystems, Disks and Memory

Disk read and write speed.

Being a novice user to linux i m little unaware of how would i check disk read write speed. One of my mate is suggesting to create a file using dd command and check how much time it takes to create a 30 gb file . I think this has a little sense however i would also like to take your reviews... (5 Replies)
Discussion started by: pinga123
5 Replies

6. UNIX for Dummies Questions & Answers

Find the file which is growing at high speed

Hi, A log file which is growing at high speed, don't know the name of it. How to find the respective file? Many thanks. (2 Replies)
Discussion started by: venkatesht
2 Replies

7. Hardware

Hardware Correction: How to change DVD write speed

I am now on Kernel 2.6.32-26 For me 16x CD write speed is okay. I have old hardware which was able to write DVDs at 1x, back in previous linux version. Now, I dont get speed of less than 4x. Tested on k3b, xfburn, and brasero. But all start at bottom 4x write speed. k3b forced back to... (0 Replies)
Discussion started by: makh
0 Replies

8. Shell Programming and Scripting

Speed : awk command to count the occurrences of fields from one file present in the other file

Hi, file1.txt AAA BBB CCC DDD file2.txt abc|AAA|AAAabcbcs|fnwufnq bca|nwruqf|AAA|fwfwwefwef fmimwe|BBB|fnqwufw|wufbqw wcdbi|CCC|wefnwin|wfwwf DDD|wabvfav|wqef|fwbwqfwfe i need the count of rows of file1.txt present in the file2.txt required output: AAA 2 (10 Replies)
Discussion started by: mdkm
10 Replies
xmemfs(7FS)							   File Systems 						       xmemfs(7FS)

NAME
xmemfs - extended memory file system SYNOPSIS
#include <sys/mount.h> mount(special_file, directory, MS_DATA, "xmemfs", dataptr, datalen); DESCRIPTION
The xmemfs file system is an extended memory file system that provides an efficient mechanism for managing and accessing physical memory that exceeds 4 Gbytes in size. Currently, the xmemfs file system is supported on IA32 architecture systems only. The Physical Address Extension (PAE) is the xmemfs internal processor feature that enables a 36-bit physical memory address that supports up to 64 Gbytes of physical memory. Once mounted, the xmemfs file system provides standard file operations and semantics on directories and regular files only. Because xmemfs does not allow execute permissions to be set on regular files, execution of object files is pre- vented. With xmemfs, the special_file argument, (typically the device on which file systems reside), is ignored and serves only as a placeholder. File data and metadata in xmemfs are always memory-resident. The dataptr argument must (at a minimum) contain the required size specific option. See mount_xmemfs(1M) for more information. Because xmemfs is a memory-based file system, files and directories that are created are not persistent across reboots or unmounts. EXTENDED DESCRIPTION
To mount the xmemfs file system, do the following: mount -F xmemfs -osize=4g xmem directory You can also mount a xmemfs file system on /xmem at multi-user startup time prior to physical memory becoming fragmented. To do this, add the following line to your /etc/vfstab file: xmem - /xmem xmemfs - yes largebsize,size=4g The xmemfs file system is expressly designed for performance-driven applications (for example, RDBMS) that require large amounts of physi- cal memory. The xmemfs file system provides file system semantics to manage and access extended memory spaces that exceed 4 Gbytes. From an application perspective, extended memory under the control of a mounted xmemfs file system is viewed as a single, large memory pool that can be partitioned as needed through file creation. You can obtain windows into each memory partition by using mmap(2). Memory controlled by xmemfs can be partitioned by creating files of the required size in the file system. The xmemfs file system allocates sufficient block-sized memory pages for a file based on the file's size. Files can be created using any standard file utility, including mkfile(1M) and dd(1M). The xmemfs file system optimizes the creation of large files that initially contain all zeroes by allocating memory pages for the file 'hole' that is created by writing beyond the end of file. If sufficient xmemfs extended memory is available, an application can quickly create an 8 Gbyte file in the xmemfs file system by using llseek(2) to offset 8GB-1 and then write(2) a one-byte buffer containing zero. With xmemfs, you can share and protect partitioned memory by setting appropriate file permissions. To avoid wasting memory resources, (especially with the -largebsize option specified), newly cre- ated option-specified files should be a multiple of the block size of the xmemfs file system. Creation of many small files is strongly dis- couraged. See statvfs(2) for information on determining file system block sizes. The xmemfs file system should only be used with performance-driven applications that require quick access to large amounts of physical mem- ory. Using xmemfs for other applications may result in non-optimal use of system resources and possible system performance degradation. To maximize xmemfs ability to access a file's extended memory partition, use mmap(2). The initial mmap(2) call enables the system to assign a map size containing as much memory as an application may actively access at any time. The map size is constrained by the application's virtual address space, (usually a maximum of 3 Gbytes on machines with more than 4 Gbytes of physical memory). To access extended memory that is not contained in the existing mapping, use mmap(2) with the -MAP_FIXED flag to remap a window within the address range returned by the initial mmap call. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ |ATTRIBUTE TYPE |ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Architecture |i386 | +-----------------------------+-----------------------------+ |Interface Stability |Evolving | +-----------------------------+-----------------------------+ SEE ALSO
df(1M), mount(1M), mount_xmemfs(1M), mmap(2), mount(2), statvfs(2), umount(2), vfstab(4) DIAGNOSTICS
If the xmemfs file system runs out of space, the following message is displayed in the console indicating that there is insufficient mem- ory to satisfy a write(2) request: directory: File system full, no memory WARNINGS
Files and directories on an xmemfs file system are not preserved across reboots or unmounts. SunOS 5.10 25 May 1999 xmemfs(7FS)
All times are GMT -4. The time now is 04:42 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy