Sponsored Content
Top Forums Shell Programming and Scripting insert a header in a huge data file without using an intermediate file Post 302289114 by ddreggors on Wednesday 18th of February 2009 07:03:39 PM
Old 02-18-2009
try this:
Code:
sed 1'i\HeaderText' ExtractDataFile.txt

If this outputs to the screen as expected (this does not update the file) then the inner 'i' (insert command) is working fine.

Next try:

Code:
sed -i 1'i\HeaderText' ExtractDataFile.txt

If either gives an error then you may have to upgrade sed to get the use of the 'inline' flag (-i) or the insert command ('i\text') to be able to do this without writing to another file first and specifying a line number.
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Insert a line as the first line into a very huge file

Hello, I need to insert a line (like a header) as the first line of a very huge file (about 3 ml rows). I am able to do it with sed, but redirecting the output and creating a new file takes quite some time. I was wondering if there was a more efficient way of doing it? Any help would be... (3 Replies)
Discussion started by: shriek
3 Replies

2. UNIX for Dummies Questions & Answers

search and grab data from a huge file

folks, In my working directory, there a multiple large files which only contain one line in the file. The line is too long to use "grep", so any help? For example, if I want to find if these files contain a string like "93849", what command I should use? Also, there is oder_id number... (1 Reply)
Discussion started by: ting123
1 Replies

3. Shell Programming and Scripting

How to extract data from a huge file?

Hi, I have a huge file of bibliographic records in some standard format.I need a script to do some repeatable task as follows: 1. Needs to create folders as the strings starts with "item_*" from the input file 2. Create a file "contents" in each folders having "license.txt(tab... (5 Replies)
Discussion started by: srsahu75
5 Replies

4. Programming

to find header in Mp3 file and retrieve data

hi all, In an mp3 file , data is arranged in sequence of header and data ,how to retrieve data between two headers. Is the data between two headers fixed? because as per theory it says 1152 samples will be there , but dont knw how many bits one sample correspond to? it would help if any c... (2 Replies)
Discussion started by: shashi
2 Replies

5. Shell Programming and Scripting

Three Difference File Huge Data Comparison Problem.

I got three different file: Part of File 1 ARTPHDFGAA . . Part of File 2 ARTGHHYESA . . Part of File 3 ARTPOLYWEA . . (4 Replies)
Discussion started by: patrick87
4 Replies

6. Shell Programming and Scripting

Help- counting delimiter in a huge file and split data into 2 files

I’m new to Linux script and not sure how to filter out bad records from huge flat files (over 1.3GB each). The delimiter is a semi colon “;” Here is the sample of 5 lines in the file: Name1;phone1;address1;city1;state1;zipcode1 Name2;phone2;address2;city2;state2;zipcode2;comment... (7 Replies)
Discussion started by: lv99
7 Replies

7. Shell Programming and Scripting

Extract header data from one file and combine it with data from another file

Hi, Great minds, I have some files, in fact header files, of CTD profiler, I tried a lot C programming, could not get output as I was expected, because my programming skills are very poor, finally, joined unix forum with the hope that, I may get what I want, from you people, Here I have attached... (17 Replies)
Discussion started by: nex_asp
17 Replies

8. Shell Programming and Scripting

Insert date/time header at top of file

I'm trying to take mrt output and put it at the top of a file along with the date and time. I was able to do it at the bottom of the file with the following printf "********** $(date) **********\n\n" >> $OUTPUT_PATH/$HOSTNAME mtr -r -w -c 10 $HOSTADDRESS >> $OUTPUT_PATH/$HOSTNAME printf... (2 Replies)
Discussion started by: kramer65
2 Replies

9. UNIX for Advanced & Expert Users

Need Optimization shell/awk script to aggreagte (sum) for all the columns of Huge data file

Optimization shell/awk script to aggregate (sum) for all the columns of Huge data file File delimiter "|" Need to have Sum of all columns, with column number : aggregation (summation) for each column File not having the header Like below - Column 1 "Total Column 2 : "Total ... ...... (2 Replies)
Discussion started by: kartikirans
2 Replies

10. UNIX for Advanced & Expert Users

File comaprsons for the Huge data files ( around 60G) - Need optimized and teh best way to do this

I have 2 large file (.dat) around 70 g, 12 columns but the data not sorted in both the files.. need your inputs in giving the best optimized method/command to achieve this and redirect the not macthing lines to the thrid file ( diff.dat) File 1 - 15 columns File 2 - 15 columns Data is... (9 Replies)
Discussion started by: kartikirans
9 Replies
edinplace(1)							Mail Avenger 0.8.3						      edinplace(1)

NAME
edinplace - edit a file in place SYNOPSIS
edinplace [--error=code] [[--file=file] command [arg ...]] DESCRIPTION
edinplace runs command with its input from file (or standard input by default), and then replaces the contents of file with the output of command. To the extent possible, edinplace attempts to exit with the same status as command. If edinplace is run on standard input (no --file option), it must inherit a file descriptor 0 that is open for both reading and writing. When processing standard input, if edinplace does not encounter a fatal error, it rewinds its standard input to offset 0 before exiting. Thus, a script can first run edinplace command, then run another filter command such as grep, and the resulting output will be the output of grep on command's output. If no command is specified, edinplace just rewinds its standard input to file offset 0. In this case, it is an error to supply the --file option. Of course, rewinding only works when standard input is a real file (as opposed to a pipe or device). There are two options: --error=code (-x code) Ordinarily, edinplace attempts to exit with the same status as command. However, if edinplace encounters some fatal error (such as being unable to execute command), it will exit with status code. The default value is 1. The range of valid exit codes is 1-255, inclusive. --file=file (-f file) Specifies that file should be edited. Otherwise, edinplace will edit its standard input (which must be opened for both reading and writing). --skipfrom Skip the first line of the file if it starts "From ". If edinplace is run without a command, positions the file offset at the start of the second line of the file. If edinplace is run with a command, then the first line of the file is neither fed to the command, nor overwritten. This option is useful for running edinplace over mail files, which sometimes start with a "From " line specifying the envelope sender of the message. Since "From " is not part of the message header, just a Unix convention, some programs are confused by the presence of that line. Note that if you specify a command, then edinplace resets the file offset to 0 upon exiting, even if the --skipfrom option was present. EXAMPLES
The following command prepends the string "ORIGINAL: " to the beginning of each line in text file message: edinplace -f message sed -e 's/^/ORIGINAL: /' The following command runs the spamassassin mail filter program on a mail message stored in file message, replacing the contents of message with spamassassin's annotated output, and exiting with code 100 if spamassassin thinks the message is spam. If edinplace encounters any fatal errors, it will exit with code 111. edinplace -x 111 -f message spamassassin -e 100 (spamassassin reads a mail message on standard input and outputs an annotated copy of the message including information about whether or not the message is likely to be spam and why. The -e option to spamassassin specifies what exit status spamassassin should use if the message appears to be spam; edinplace will use the same exit code as the program it has run.) To run spamassassin on incoming mail before accepting the mail from the remote client, place the following line in an appropriate Mail Avenger rcpt file as the last command executed: bodytest edinplace -x 111 spamassassin -e 100 SEE ALSO
avenger(1) The Mail Avenger home page: <http://www.mailavenger.org/>. BUGS
edinplace does not make a copy of the file being edited, but rather overwrites the file as it is being processed. At any point where command has produced more output than it has consumed input from the file, edinplace buffers the difference in memory. Thus, a command that outputs large amounts of data before reading the input file can run edinplace out of memory. (A program that outputs data as it reads even a very large file should be fine, however.) If command crashes or malfunctions for any reason, you will likely lose the input file, since edinplace will view this as a program that simply outputs the empty file. AUTHOR
David Mazieres Mail Avenger 0.8.3 2012-04-05 edinplace(1)
All times are GMT -4. The time now is 03:19 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy