Sponsored Content
Top Forums Shell Programming and Scripting Deletion of characters in every second line Post 302427468 by alister on Sunday 6th of June 2010 08:56:31 AM
Old 06-06-2010
You're quite welcome.

Yeah, it's much faster, heheheh. I concatenated 150,000 instances of that `head -n8` sample data to create a 52 MB file containing over a million lines:
Code:
$ du -sh data
 52M    data
$ wc data   
 1074672 1074672 54942606 data

Test runs show sed completes in a second and a half while your shell script, killed after a minute of real time has elapsed, has only processed a small fraction of the file. Roughly, we're talking about 1.5 secs versus an 1 hour, a 2400x improvement, with this data on my hardware:

Code:
$ time sed 'n;s/......//' data > dump

real    0m1.545s
user    0m1.123s
sys     0m0.407s
$ wc -l dump
 1074672 dump

$ time ./trim.sh data > dump
^C

real    1m1.610s
user    0m12.378s
sys     0m45.385s
$ wc -l dump
   16825 dump

Regards,
Alister

Last edited by alister; 06-06-2010 at 10:05 AM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Line deletion help using perl -ne

Hi all, I know the one liner to delete all the lines in a file which matches a pattern i.e perl -i.old -ne 'print unless /pattern/' file Now i need the perl onliner to delete all the lines which doesnt match the pattern. Also what is the difference between perl -i and perl -i.old. Does... (1 Reply)
Discussion started by: lijju.mathew
1 Replies

2. Shell Programming and Scripting

sed remove last 10 characters of a line start from 3rd line

hello experts, I need a sed command that remove last 10 characters of a line start from 3rd line. any suggestions? Thanks you (7 Replies)
Discussion started by: minifish
7 Replies

3. Shell Programming and Scripting

blank line deletion in a file using perl

Dear All, I am looking for an option in perl using which i could delete empty lines in a file. Or the alternative of sed '/^$/d' <filename> in perl. Sed is not working in my perl script :( Pls help me out . Thanks, VG (4 Replies)
Discussion started by: vguleria
4 Replies

4. Shell Programming and Scripting

Get the 1st 99 characters and add new line feed at the end of the line

I have a file with varying record length in it. I need to reformat this file so that each line will have a length of 100 characters (99 characters + the line feed). AU * A01 EXPENSE 6990370000 CWF SUBC TRAVEL & MISC MY * A02 RESALE 6990788000 Y... (3 Replies)
Discussion started by: udelalv
3 Replies

5. UNIX for Dummies Questions & Answers

deletion of duplicate characters and count

to delete the duplicate characters in a file I used this code cat file.txt|tr -s "" tell the other ways using sed command to count of duplicate characters thanks:) (0 Replies)
Discussion started by: tsurendra
0 Replies

6. UNIX for Dummies Questions & Answers

How to specify beginning-of-line/end-of-line characters inside a regex range

How can I specify special meaning characters like ^ or $ inside a regex range. e.g Suppose I want to search for a string that either starts with '|' character or begins with start-of-line character. I tried the following but it does not work: sed 's/\(\)/<do something here>/g' file1 ... (3 Replies)
Discussion started by: jawsnnn
3 Replies

7. UNIX for Dummies Questions & Answers

sed - combination of line deletion and pattern matching

I want to delete all the blank lines from a file before a certain line number. e.g. Input file (n: denotes line number) 1: a 2: 3: b 4: c 5: 6: d I want to delete all blank lines before line number 3, such that my output is: a b c d I see that sed '/^$/d' in_file works... (9 Replies)
Discussion started by: jawsnnn
9 Replies

8. Shell Programming and Scripting

[Solved] How to separate one line to mutiple line based on certain number of characters?

hi Gurus, I need separate a file which is one huge line to multiple lines based on certain number of charactors. for example: abcdefghi high abaddffdd I want to separate the line to multiple lines for every 4 charactors. the result should be abcd efgh i hi gh a badd ffdd Thanks in... (5 Replies)
Discussion started by: ken6503
5 Replies

9. Shell Programming and Scripting

Ksh: Read line parse characters into variable and remove the line if the date is older than 50 days

I have a test file with the following format, It contains the username_date when the user was locked from the database. $ cat lockedusers.txt TEST1_21062016 TEST2_02122015 TEST3_01032016 TEST4_01042016 I'm writing a ksh script and faced with this difficult scenario for my... (11 Replies)
Discussion started by: humble_learner
11 Replies

10. Shell Programming and Scripting

Remove first 2 characters and last two characters of each line

here's what im trying to do. i have a file containing lines similar to this: data.txt: 1hsRmRsbHRiSFZNTTA1dlEyMWFkbU5wUW5CSlIyeDFTVU5SYjJOSFRuWmpia0ZuWXpKV2FHTnRU 1lKUnpWMldrZFZaMG95V25oYQpSelEyWTBka2QyRklhSHBrUjA1b1kwUkJkd3BOVXpWM1lVaG5k... (5 Replies)
Discussion started by: SkySmart
5 Replies
set_color(1)							       fish							      set_color(1)

NAME
set_color - set_color - set the terminal color set_color - set the terminal color Synopsis set_color [-v --version] [-h --help] [-b --background COLOR] [COLOR] Description Change the foreground and/or background color of the terminal. COLOR is one of black, red, green, brown, yellow, blue, magenta, purple, cyan, white and normal. o -b, --background Set the background color o -c, --print-colors Prints a list of all valid color names o -h, --help Display help message and exit o -o, --bold Set bold or extra bright mode o -u, --underline Set underlined mode o -v, --version Display version and exit Calling set_color normal will set the terminal color to whatever is the default color of the terminal. Some terminals use the --bold escape sequence to switch to a brighter color set. On such terminals, set_color white will result in a grey font color, while set_color --bold white will result in a white font color. Not all terminal emulators support all these features. This is not a bug in set_color but a missing feature in the terminal emulator. set_color uses the terminfo database to look up how to change terminal colors on whatever terminal is in use. Some systems have old and incomplete terminfo databases, and may lack color information for terminals that support it. Download and install the latest version of ncurses and recompile fish against it in order to fix this issue. Version 1.23.1 Sun Jan 8 2012 set_color(1)
All times are GMT -4. The time now is 04:07 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy