Did sed win? Or did file caching speed up sed? Modern controllers and RAM cache - HPUX - can cache 100 MB of a single file without really using up system resources.
I vote for caching. The only fair test is two separate files.
BTW: programs like sed, awk, head, tail, grep are all highly optimized for their respective jobs. There are several of external factors like: caching, I/O load (I/O request queue length), SAN vs disk, that distort these kinds of tests. So, by the time you have runs some tests, any time differences between the commands will likely have been eaten up by testing.
Your best bet is to parallelize, use the cpu and disk I/O to the max. With a quad core maybe you want to consider 4 simultaneous child processes, for example:
Code:
cd /directory
cnt=1
for fname in $(find . -type f)
do
(awk 'FNR>1' $fname > tmp.${cnt}; mv tmp.${cnt} $fname) &
cnt=$(( $cnt + 1 ))
[ $(( $cnt % 4 )) -eq 0 ] && wait
done
wait
Hi
I am running a script (which compares two directory contents) for which I am getting an output of 70 pages in which few pages are blank so I was able to delete those blank lines.
But I also want to delete the headers present for each page. can any one help me by providing the code... (1 Reply)
I have a file with millions of records...Before I experiment, I would like to know which one is faster.
Both the commands work absolutely fine on a smaller set of records.
Please advice.
sed 's/^M//g' ${INPUT_FILE} > tmp.txt
mv tmp.txt ${INPUT_FILE}
tr -d "\15" < ${INPUT_FILE} > ... (11 Replies)
I have a data file with over 500,000 records/lines that has the header throughout the file.
SEQ_ID Name Start_Date Ins_date Add1 Add2
1 Harris 04/02/08 03/02/08 333 Main Suite 101
2 Smith 02/03/08 01/23/08 287 Jenkins
SEQ_ID Name ... (3 Replies)
Hello,
So i want to send mails in any way from a solaris 5.8 system, perhaps using mailx or sendmail. My purpose is to stay clear of systems name in head data. So i want to strip at least the "Message-Id" and the "Recieved" headers of the mail. Yet this seems to be a bit of a problem.
Now i... (2 Replies)
Hi,
I'm trying to strip all lines between two headers in a file:
### BEGIN ###
Text to remove, contains all kinds of characters
...
Antispyware-Downloadserver.com (Germany)=http://www.antispyware-downloadserver.c
om/updates/
Antispyware-Downloadserver.com #2... (3 Replies)
Hi ,
I have a typical situation. I have 4 files and with different headers (number of headers is varible ).
I need to make such a merged file which will have headers combined from all files (comman coluns should appear once only).
For example -
File 1
H1|H2|H3|H4
11|12|13|14
21|22|23|23... (1 Reply)
Hi All,
I have some 80,000 files in a directory which I need to rename. Below is the command which I am currently running and it seems, it is taking fore ever to run this command. This command seems too slow. Is there any way to speed up the command. I have have GNU Parallel installed on my... (6 Replies)
Hi,
I have catenated multiple output files (from a monte carlo run) into one big output file. Each individual file has it's own two line header. So when I catenate, there are multiple two line headers (of the same wording) within the big file. How do I use the sed command to search for the... (1 Reply)
Good evening
I need your help please, im new at Unix and i wanted to remove the first 5 headers for 100000 records files and then create a control file .ctl that contains the number of records and all seem to work out but when i tested at production it didnt wotk.
Here is the code:
#!... (6 Replies)
I have a file called "dsout" with empty rows and duplicate headers.
DATE TIME TOTAL_GB USED_GB %USED
--------- -------- ---------- ---------- ----------
03/05/013 12:34 PM 3151.24316 2331.56653 73.988785 ... (3 Replies)
Discussion started by: Daniel Gate
3 Replies
LEARN ABOUT LINUX
dh_installdeb
DH_INSTALLDEB(1) Debhelper DH_INSTALLDEB(1)NAME
dh_installdeb - install files into the DEBIAN directory
SYNOPSIS
dh_installdeb [debhelperoptions]
DESCRIPTION
dh_installdeb is a debhelper program that is responsible for installing files into the DEBIAN directories in package build directories with
the correct permissions.
FILES
package.postinst
package.preinst
package.postrm
package.prerm
These maintainer scripts are installed into the DEBIAN directory.
Inside the scripts, the token #DEBHELPER# is replaced with shell script snippets generated by other debhelper commands.
package.triggers
package.shlibs
These control files are installed into the DEBIAN directory.
package.conffiles
This control file will be installed into the DEBIAN directory.
In v3 compatibility mode and higher, all files in the etc/ directory in a package will automatically be flagged as conffiles by this
program, so there is no need to list them manually here.
package.maintscript
Lines in this file correspond to dpkg-maintscript-helper(1) commands and parameters. Any shell metacharacters will be escaped, so
arbitrary shell code cannot be inserted here. For example, a line such as "mv_conffile /etc/oldconffile /etc/newconffile" will insert
maintainer script snippets into all maintainer scripts sufficient to move that conffile.
A versioned Pre-Dependency on dpkg is needed to use dpkg-maintscript-helper(1). An appropriate Pre-Dependency is set in
${misc:Pre-Depends} ; you should make sure to put that token into an appropriate place in your debian/control file.
SEE ALSO debhelper(7)
This program is a part of debhelper.
AUTHOR
Joey Hess <joeyh@debian.org>
8.9.0ubuntu2.1 2012-06-12 DH_INSTALLDEB(1)