Sponsored Content
Top Forums Shell Programming and Scripting Inserting Header to another file Post 302924908 by Don Cragun on Thursday 13th of November 2014 02:57:41 AM
Old 11-13-2014
Quote:
Originally Posted by gvkumar25
Can you please share me the method without using temp files. The problem I had is my data files are in GB's (20GB). My process takes more time to move 20GB of data into tmp file instead what I taught is adding a single line to 20GB data is much faster.
As I said before, there is no way to add data to the start of a file without copying the data at least once. If the file you're modifying has more than one link and you don't want to break the links or the temp file is on a different filesystem than the file you're modifying, you'll have to copy the data twice.

Even if you have more than 20B of memory you can allocate to a program to edit your file, you still have to read the entire 20GB into memory and write the entire 20+GB of data back into your file.

Of course you could create a new filesystem type that allows you to add data to either end of a file without moving existing data and create a new system call to write data into a file at negative offsets. (But, before you ask, I'll warn you that I'm not going to volunteer to design either of those for you for free in a forum like this!)
 

10 More Discussions You Might Find Interesting

1. Linux

Reading the header of a tar file(posix header)

say i have these many file in a directory named exam. 1)/exam/newfolder/link.txt. 2)/exam/newfolder1/ and i create a tar say exam.tar well the problem is, when i read the tar file i dont find any metadata about the directories,as you cannot create a tar containig empty directories. on the... (2 Replies)
Discussion started by: Tanvirk
2 Replies

2. Shell Programming and Scripting

Inserting a String in a file header.

Dear all, I have a file created in the name sample.txt in UNIX with header and footer. How to insert a required string (for example "FILE1") in the header part after the file has been created. What kind of command can i use to do the same. Thanks in advance Hari (3 Replies)
Discussion started by: Hari123
3 Replies

3. Shell Programming and Scripting

Inserting Header and footer

Hi All, I have several txt files i need to enter specific header and footer (both are separate) to all these files how can i do this? plz help.. Regards, Raghav (4 Replies)
Discussion started by: digitalrg
4 Replies

4. Ubuntu

Inserting a header with column number to a 1.6 GB file with special spacing

Hi; I've been searching posts to find a solution to what I'm trying to do, but I've have NOT found anything yet. I have a file (file1) with 300K columns and 1411 rows, the columns don't have a column no. header (No header at all) and I'm trying to fetch the information from specific columns.... (3 Replies)
Discussion started by: sogi
3 Replies

5. Shell Programming and Scripting

Comparing one file header with another file header

Hi Experts, In our project we have requirement where in we have to compare header of one file with header in the parameter file. There are 20 files which we ftp from one site. All this files have different header. We are comapring this file with our parameter file(which is having the header... (2 Replies)
Discussion started by: Amey Joshi
2 Replies

6. UNIX for Dummies Questions & Answers

Merge all csv files in one folder considering only 1 header row and ignoring header of all others

Friends, I need help with the following in UNIX. Merge all csv files in one folder considering only 1 header row and ignoring header of all other files. FYI - All files are in same format and contains same headers. Thank you (4 Replies)
Discussion started by: Shiny_Roy
4 Replies

7. Shell Programming and Scripting

Inserting header after every nth line

Hi Experts, I have a file which contain hundreds of records/lines. I want to insert the below header in the file after every 60 lines. #Header FirstName LastName Address --------- ---------- --------- Let say I saved the... (6 Replies)
Discussion started by: brichigo
6 Replies

8. UNIX for Beginners Questions & Answers

Inserting Header at different position in a file

I would like to hear your directions on how to Insert theses tag </TITLE> and <TEXT> at a given position in 1000 of text files. My Files look like as samplefile1.txt <DOC> <DOCNO>3_September_2012</DOCNO> <TITLE> ... ... ... .... ... .. .. .. ... .. .. .... </TITLE> <TEXT> .... (1 Reply)
Discussion started by: imranrasheedamu
1 Replies

9. Shell Programming and Scripting

Inserting a header with special character

Hi, I am trying to insert header row with a special character delimiter with Unicode u0109 into a file with ‘echo’, header looks like below echo –e “header1\u0109header\u0109header3\u0109header4” It just inserting as it is in the quotes but not the special character, Please suggest if am... (2 Replies)
Discussion started by: oom
2 Replies

10. Shell Programming and Scripting

Find header in a text file and prepend it to all lines until another header is found

I've been struggling with this one for quite a while and cannot seem to find a solution for this find/replace scenario. Perhaps I'm getting rusty. I have a file that contains a number of metrics (exactly 3 fields per line) from a few appliances that are collected in parallel. To identify the... (3 Replies)
Discussion started by: verdepollo
3 Replies
xfs_fsr(8)						      System Manager's Manual							xfs_fsr(8)

NAME
xfs_fsr - filesystem reorganizer for XFS SYNOPSIS
xfs_fsr [-vdg] [-t seconds] [-p passes] [-f leftoff] [-m mtab] xfs_fsr [-vdg] [xfsdev | file] ... xfs_fsr -V DESCRIPTION
xfs_fsr is applicable only to XFS filesystems. xfs_fsr improves the organization of mounted filesystems. The reorganization algorithm operates on one file at a time, compacting or oth- erwise improving the layout of the file extents (contiguous blocks of file data). The following options are accepted by xfs_fsr. The -m, -t, and -f options have no meaning if any filesystems or files are specified on the command line. -m mtab Use this file for the list of filesystems to reorganize. The default is to use /etc/mtab. -t seconds How long to reorganize. The default is 7200 seconds (2 hours). -p passes Number of passes before terminating global re-org. The default is 10 passes. -f leftoff Use this file instead of /var/tmp/.fsrlast to read the state of where to start and as the file to store the state of where reorganization left off. -v Verbose. Print cryptic information about each file being reorganized. -d Debug. Print even more cryptic information. -g Print to syslog (default if stdout not a tty). -V Prints the version number and exits. When invoked with no arguments xfs_fsr reorganizes all regular files in all mounted filesystems. xfs_fsr makes many cycles over /etc/mtab each time making a single pass over each XFS filesystem. Each pass goes through and selects files that have the largest number of extents. It attempts to defragment the top 10% of these files on each pass. It runs for up to two hours after which it records the filesystem where it left off, so it can start there the next time. This information is stored in the file /var/tmp/.fsrlast_xfs. If the information found here is somehow inconsistent or out of date it is ignored and reor- ganization starts at the beginning of the first filesystem found in /etc/mtab. xfs_fsr can be called with one or more arguments naming filesystems (block device name), and files to reorganize. In this mode xfs_fsr does not read or write /var/tmp/.fsrlast_xfs nor does it run for a fixed time interval. It makes one pass through each specified regular file and all regular files in each specified filesystem. A command line name referring to a symbolic link (except to a file system device), FIFO, or UNIX domain socket generates a warning message, but is otherwise ignored. While traversing the filesystem these types of files are silently skipped. FILES
/etc/mtab contains default list of filesystems to reorganize. /var/tmp/.fsrlast_xfs records the state where reorganization left off. SEE ALSO
xfs_fsr(8), mkfs.xfs(8), xfs_ncheck(8), xfs(5). NOTES
xfs_fsr improves the layout of extents for each file by copying the entire file to a temporary location and then interchanging the data extents of the target and temporary files in an atomic manner. This method requires that enough free disk space be available to copy any given file and that the space be less fragmented than the original file. It also requires the owner of the file to have enough remaining filespace quota to do the copy on systems running quotas. xfs_fsr generates a warning message if space is not sufficient to improve the target file. A temporary file used in improving a file given on the command line is created in the same parent directory of the target file and is pre- fixed by the string '.fsr'. The temporary files used in improving an entire XFS device are stored in a directory at the root of the target device and use the same naming scheme. The temporary files are unlinked upon creation so data will not be readable by any other process. xfs_fsr does not operate on files that are currently mapped in memory. A 'file busy' error can be seen for these files if the verbose flag (-v) is set. Files marked as no-defrag will be skipped. The xfs_io(8) chattr command with the f attribute can be used to set or clear this flag. Files and directories created in a directory with the no-defrag flag will inherit the attribute. An entry in /etc/mtab or the file specified using the -m option must have the rw option specified for read and write access. If this option is not present, then xfs_fsr skips the filesystem described by that line. See the fstab(5) reference page for more details. In general we do not foresee the need to run xfs_fsr on system partitions such as /, /boot and /usr as in general these will not suffer from fragmentation. There are also issues with defragmenting files lilo(8) uses to boot your system. It is recommended that these files should be flagged as no-defrag with the xfs_io(8) chattr command. Should these files be moved by xfs_fsr then you must rerun lilo before you reboot or you may have an unbootable system. xfs_fsr(8)
All times are GMT -4. The time now is 05:34 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy