|
|
Sponsored Content
Top Forums
UNIX for Advanced & Expert Users
sed working slow on big files
Post 302498032 by methyl on Friday 18th of February 2011 06:28:04 PM
02-18-2011
|
|
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I am facing some strange problem.
I know, there is only one record in a file 'test.txt' which starts with 'X'
I ensure that with following command,
awk /^X/ test.txt | wc -l
This gives me output = '1'.
Now I take out this record out of the file, as follows :
awk /^X/ test.txt >... (1 Reply)
Discussion started by: videsh77
1 Replies
2. Shell Programming and Scripting
Hi,
I have one file stat.
Stat file contents are as follows: for example.
H50768020040913,00260100,507680,13,0000000643,0000000643,00000,0000
H50769520040808,00260100,507695,13,0000000000,0000000000,00000,0000 H50770620040611,00260100,507706,13,0000000000,0000000000,00000,0000
Now i... (1 Reply)
Discussion started by: davidpreml
1 Replies
3. Shell Programming and Scripting
hi all,
i am again surrounded by a big problem,,,
i have 2 files
file1.txt file2.txt
aaaa xxxx xxxxx
xxxxxxxxxxxxxxx
zzzz zzzz zzz bbb
aaaa xx xxxx xxxx
xxx zzzz zzzz zzz... (1 Reply)
Discussion started by: go4desperado
1 Replies
4. AIX
Hi,
I am trying to send oracle archives over WAN and it is taking hell a lot of time. To reduce the time, I tried to gzip the files and send over to the other side. That seems to reduce the time. Does anybody have experienced this kind of problem and any possible ways to reduce the time.
... (1 Reply)
Discussion started by: giribt
1 Replies
5. Shell Programming and Scripting
hi friends,
i have to cut a large html file between tag " <!-- DEFACEMENTS ROWS -->"
"<!-- DISCLAIMER FOOTER -->"
and store cut data in other file
please help me!!!! (2 Replies)
Discussion started by: praneshbmishra
2 Replies
6. Shell Programming and Scripting
Hello everyone,
suppose there is a very big text file (>800 mb) that each line contains an article from wikipedia. Each article begins with a tag (<..>) containing its url. Currently there are 10^6 articles in the file.
I want to take random N articles, eliminate all non-alpharithmetic... (14 Replies)
Discussion started by: fedonMan
14 Replies
7. Shell Programming and Scripting
I have many files which contain about two million lines.
Now I want to use sed to delete the 9th line and add a new
line behind the 8th line. I use the command as follows:
for((i=1;i<100;i++));
do
echo $i;
sed -i '9d' $i.dat;
sed -i '8a this is a new line' $i.dat;
done
But it is... (3 Replies)
Discussion started by: wxuyec
3 Replies
8. Shell Programming and Scripting
Hi
We are using sed to clean up a file of a pattern and its talking a lot of time on XML output file
The command that we are using is
sed -e "s/tns1://g" $OUTPUTFILENM > $TEMPFILE
Where $OUTPUTFILENM is the file to be cleaned and $TEMPFILE is the cleaned output
Can you... (3 Replies)
Discussion started by: jimmyb
3 Replies
9. Shell Programming and Scripting
Gents,
Please can u help me to improve this script to be more faster, it works perfectly but for big files take a lot time to end the job..
I see the problem is in the step (while) and in this part the script takes a lot time..
Please if you can find a best way to do will be great.
... (13 Replies)
Discussion started by: jiam912
13 Replies
10. Shell Programming and Scripting
I am trying to use the below sed command to replace all "|" to ^, in a folder had 50 dat files. when i tried with 1 file it worked but when i tried with wild card, is not working.
sed -i 's/"|"/\^/g' *.dat
Is this the proper way to use sed command thank you very much for help. (3 Replies)
Discussion started by: cplusplus1
3 Replies
LEARN ABOUT DEBIAN
shtool-subst
SHTOOL-SUBST.TMP(1) GNU Portable Shell Tool SHTOOL-SUBST.TMP(1) NAME
shtool-subst - GNU shtool sed(1) substitution operations SYNOPSIS
shtool subst [-v|--verbose] [-t|--trace] [-n|--nop] [-w|--warning] [-q|--quiet] [-s|--stealth] [-i|--interactive] [-b|--backup ext] [-e|--exec cmd] [-f|--file cmd-file] [file] [file ...] DESCRIPTION
This command applies one or more sed(1) substitution operations to stdin or any number of files. OPTIONS
The following command line options are available. -v, --verbose Display some processing information. -t, --trace Enable the output of the essential shell commands which are executed. -n, --nop No operation mode. Actual execution of the essential shell commands which would be executed is suppressed. -w, --warning Show warning on substitution operation resulting in no content change on every file. The default is to show a warning on substitution operations resulted in no content change on all files. -q, --quiet Suppress warning on substitution operation resulting in no content change. -s, --stealth Stealth operation. Preserve timestamp on file. -i, --interactive Enter interactive mode where the user has to approve each operation. -b, --backup ext Preserve backup of original file using file name extension ext. Default is to overwrite the original file. -e, --exec cmd Specify sed(1) command directly. -f, --file cmd-file Read sed(1) command from file. EXAMPLE
# shell script shtool subst -i -e 's;(c) ([0-9]*)-2000;(c) 1-2001;' *.[ch] # RPM spec-file %install shtool subst -v -n -e 's;^(prefix=).*;1 $RPM_BUILD_ROOT%{_prefix};g' -e 's;^(sysconfdir=).*;1 $RPM_BUILD_ROOT%{_prefix}/etc;g' `find . -name Makefile -print` make install HISTORY
The GNU shtool subst command was originally written by Ralf S. Engelschall <rse@engelschall.com> in 2001 for GNU shtool. It was prompted by the need to have a uniform and convenient patching frontend to sed(1) operations in the OpenPKG package specifications. SEE ALSO
shtool(1), sed(1). 18-Jul-2008 shtool 2.0.8 SHTOOL-SUBST.TMP(1)