insert a header in a huge data file without using an intermediate file Post: 302289103

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Insert a line as the first line into a very huge file

Hello, I need to insert a line (like a header) as the first line of a very huge file (about 3 ml rows). I am able to do it with sed, but redirecting the output and creating a new file takes quite some time. I was wondering if there was a more efficient way of doing it? Any help would be...

2. UNIX for Dummies Questions & Answers

search and grab data from a huge file

folks, In my working directory, there a multiple large files which only contain one line in the file. The line is too long to use "grep", so any help? For example, if I want to find if these files contain a string like "93849", what command I should use? Also, there is oder_id number...

3. Shell Programming and Scripting

How to extract data from a huge file?

Hi, I have a huge file of bibliographic records in some standard format.I need a script to do some repeatable task as follows: 1. Needs to create folders as the strings starts with "item_*" from the input file 2. Create a file "contents" in each folders having "license.txt(tab...

4. Programming

to find header in Mp3 file and retrieve data

hi all, In an mp3 file , data is arranged in sequence of header and data ,how to retrieve data between two headers. Is the data between two headers fixed? because as per theory it says 1152 samples will be there , but dont knw how many bits one sample correspond to? it would help if any c...

5. Shell Programming and Scripting

Three Difference File Huge Data Comparison Problem.

I got three different file: Part of File 1 ARTPHDFGAA . . Part of File 2 ARTGHHYESA . . Part of File 3 ARTPOLYWEA . .

6. Shell Programming and Scripting

Help- counting delimiter in a huge file and split data into 2 files

I’m new to Linux script and not sure how to filter out bad records from huge flat files (over 1.3GB each). The delimiter is a semi colon “;” Here is the sample of 5 lines in the file: Name1;phone1;address1;city1;state1;zipcode1 Name2;phone2;address2;city2;state2;zipcode2;comment...

7. Shell Programming and Scripting

Extract header data from one file and combine it with data from another file

Hi, Great minds, I have some files, in fact header files, of CTD profiler, I tried a lot C programming, could not get output as I was expected, because my programming skills are very poor, finally, joined unix forum with the hope that, I may get what I want, from you people, Here I have attached...

8. Shell Programming and Scripting

Insert date/time header at top of file

I'm trying to take mrt output and put it at the top of a file along with the date and time. I was able to do it at the bottom of the file with the following printf "********** $(date) **********\n\n" >> $OUTPUT_PATH/$HOSTNAME mtr -r -w -c 10 $HOSTADDRESS >> $OUTPUT_PATH/$HOSTNAME printf...

9. UNIX for Advanced & Expert Users

Need Optimization shell/awk script to aggreagte (sum) for all the columns of Huge data file

Optimization shell/awk script to aggregate (sum) for all the columns of Huge data file File delimiter "|" Need to have Sum of all columns, with column number : aggregation (summation) for each column File not having the header Like below - Column 1 "Total Column 2 : "Total ... ......

10. UNIX for Advanced & Expert Users

File comaprsons for the Huge data files ( around 60G) - Need optimized and teh best way to do this

I have 2 large file (.dat) around 70 g, 12 columns but the data not sorted in both the files.. need your inputs in giving the best optimized method/command to achieve this and redirect the not macthing lines to the thrid file ( diff.dat) File 1 - 15 columns File 2 - 15 columns Data is...

LEARN ABOUT LINUX

dh_movefiles

DH_MOVEFILES(1) 						     Debhelper							   DH_MOVEFILES(1)

NAME

       dh_movefiles - move files out of debian/tmp into subpackages

SYNOPSIS

       dh_movefiles [debhelperoptions] [--sourcedir=dir] [-Xitem] file...]

DESCRIPTION

       dh_movefiles is a debhelper program that is responsible for moving files out of debian/tmp or some other directory and into other package
       build directories. This may be useful if your package has a Makefile that installs everything into debian/tmp, and you need to break that
       up into subpackages.

       Note: dh_install is a much better program, and you are recommended to use it instead of dh_movefiles.

FILES

       debian/package.files
	   Lists the files to be moved into a package, separated by whitespace. The filenames listed should be relative to debian/tmp/. You can
	   also list directory names, and the whole directory will be moved.

OPTIONS

       --sourcedir=dir
	   Instead of moving files out of debian/tmp (the default), this option makes it move files out of some other directory. Since the entire
	   contents of the sourcedir is moved, specifying something like --sourcedir=/ is very unsafe, so to prevent mistakes, the sourcedir must
	   be a relative filename; it cannot begin with a `/'.

       -Xitem, --exclude=item
	   Exclude files that contain item anywhere in their filename from being installed.

       file ...
	   Lists files to move. The filenames listed should be relative to debian/tmp/. You can also list directory names, and the whole directory
	   will be moved. It is an error to list files here unless you use -p, -i, or -a to tell dh_movefiles which subpackage to put them in.

NOTES

       Note that files are always moved out of debian/tmp by default (even if you have instructed debhelper to use a compatibility level higher
       than one, which does not otherwise use debian/tmp for anything at all). The idea behind this is that the package that is being built can be
       told to install into debian/tmp, and then files can be moved by dh_movefiles from that directory. Any files or directories that remain are
       ignored, and get deleted by dh_clean later.

SEE ALSO

       debhelper(7)

       This program is a part of debhelper.

AUTHOR

       Joey Hess <joeyh@debian.org>

8.9.0ubuntu2.1							    2012-06-12							   DH_MOVEFILES(1)