Sponsored Content
Top Forums UNIX for Advanced & Expert Users sed working slow on big files Post 302496908 by Perderabo on Tuesday 15th of February 2011 04:53:42 PM
Old 02-15-2011
Not sure how much this buys you but: sed 's/ *$//' file > file1 seems like it should be a little faster.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk not working as expected with BIG files ...

I am facing some strange problem. I know, there is only one record in a file 'test.txt' which starts with 'X' I ensure that with following command, awk /^X/ test.txt | wc -l This gives me output = '1'. Now I take out this record out of the file, as follows : awk /^X/ test.txt >... (1 Reply)
Discussion started by: videsh77
1 Replies

2. Shell Programming and Scripting

bash script working for small size files but not for big size files.

Hi, I have one file stat. Stat file contents are as follows: for example. H50768020040913,00260100,507680,13,0000000643,0000000643,00000,0000 H50769520040808,00260100,507695,13,0000000000,0000000000,00000,0000 H50770620040611,00260100,507706,13,0000000000,0000000000,00000,0000 Now i... (1 Reply)
Discussion started by: davidpreml
1 Replies

3. Shell Programming and Scripting

Big (at least to me) sed proble

hi all, i am again surrounded by a big problem,,, i have 2 files file1.txt file2.txt aaaa xxxx xxxxx xxxxxxxxxxxxxxx zzzz zzzz zzz bbb aaaa xx xxxx xxxx xxx zzzz zzzz zzz... (1 Reply)
Discussion started by: go4desperado
1 Replies

4. AIX

How to send big files over slow network?

Hi, I am trying to send oracle archives over WAN and it is taking hell a lot of time. To reduce the time, I tried to gzip the files and send over to the other side. That seems to reduce the time. Does anybody have experienced this kind of problem and any possible ways to reduce the time. ... (1 Reply)
Discussion started by: giribt
1 Replies

5. Shell Programming and Scripting

sed of big html files

hi friends, i have to cut a large html file between tag " <!-- DEFACEMENTS ROWS -->" "<!-- DISCLAIMER FOOTER -->" and store cut data in other file please help me!!!! (2 Replies)
Discussion started by: praneshbmishra
2 Replies

6. Shell Programming and Scripting

Very big text file - Too slow!

Hello everyone, suppose there is a very big text file (>800 mb) that each line contains an article from wikipedia. Each article begins with a tag (<..>) containing its url. Currently there are 10^6 articles in the file. I want to take random N articles, eliminate all non-alpharithmetic... (14 Replies)
Discussion started by: fedonMan
14 Replies

7. Shell Programming and Scripting

Why is SED so slow?

I have many files which contain about two million lines. Now I want to use sed to delete the 9th line and add a new line behind the 8th line. I use the command as follows: for((i=1;i<100;i++)); do echo $i; sed -i '9d' $i.dat; sed -i '8a this is a new line' $i.dat; done But it is... (3 Replies)
Discussion started by: wxuyec
3 Replies

8. Shell Programming and Scripting

sed Very Slow

Hi We are using sed to clean up a file of a pattern and its talking a lot of time on XML output file The command that we are using is sed -e "s/tns1://g" $OUTPUTFILENM > $TEMPFILE Where $OUTPUTFILENM is the file to be cleaned and $TEMPFILE is the cleaned output Can you... (3 Replies)
Discussion started by: jimmyb
3 Replies

9. Shell Programming and Scripting

Improve script - slow process with big files

Gents, Please can u help me to improve this script to be more faster, it works perfectly but for big files take a lot time to end the job.. I see the problem is in the step (while) and in this part the script takes a lot time.. Please if you can find a best way to do will be great. ... (13 Replies)
Discussion started by: jiam912
13 Replies

10. Shell Programming and Scripting

Using sed command to replace "|" with ^ for all *.dat files in a folder not working

I am trying to use the below sed command to replace all "|" to ^, in a folder had 50 dat files. when i tried with 1 file it worked but when i tried with wild card, is not working. sed -i 's/"|"/\^/g' *.dat Is this the proper way to use sed command thank you very much for help. (3 Replies)
Discussion started by: cplusplus1
3 Replies
CH_LAB(1)						    BSD General Commands Manual 						 CH_LAB(1)

NAME
ch_lab -- change/copy label files SYNOPSIS
ch_lab [-S frame spacing] [-start time] [-end time] [-ext file extension] [-extract file] [-extend time] [-f sample frequency] [-lf sample frequency] [-itype file type] [-key key file] [-lablist list of labels] [-length time] [-map map file] [-name feature name] [-class class] [-o output file] [-otype file type] [-pad high | low] [-pos list of labels] [-q timestep] [-range range] [-sed sed file] [fl shift time delta] [-style output stype] [-vocab vocab file] [-verify] [-nopath] [-base] [-combine] [-divide] [-h] input files ... DESCRIPTION
ch_lab copies from one or more input label files to an output label file, optionally performing various operations along the way. The following option flags are recognized: -h Print a summary of usage to standard output. -S frame spacing frame spacing of output -start time start time, in seconds, for label extraction -end time end time, in seconds, for label extraction -ext file extension filename extension to use for multiple output files -extract file extract a single file from a list of files -extend time extend track file by time seconds beyond label file -f sample frequency sample frequency of label file -lf sample frequency sample frequency for labels -itype file type type of input label file: esps htk ogi -key key file key label file -lablist list list of labels to be considered as blank -length time length of track produced, in seconds -map map file label mapping file -name feature name eg. Fo Phoneme -class class name of class defined in op file -o output file output file name -otype file type output file type: xmg, ascii, esps, htk -pad high | low pad with high or low values -pos list list of labels to be regarded as 'pos' -q timestep quantize label timings to nearst value -range range difference between high and low values -sed sed file perform regex editing using sed file fl shift time delta shift the times of the labels -style output style output style e.g. track -vocab file file containing list of words in vocabulary -verify check that only labels in vocab file are in label file -nopath ignore pathnames when searching label lists -base use base filenames for lists of label files -combine -divide SEE ALSO
ch_wave(1) ch_track(1) Edinburgh Speech Tools April 5, 2001 Edinburgh Speech Tools
All times are GMT -4. The time now is 06:03 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy