Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Deletion of characters in every second line

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Deletion of characters in every second line
# 8  
Old 06-06-2010
You're quite welcome.

Yeah, it's much faster, heheheh. I concatenated 150,000 instances of that `head -n8` sample data to create a 52 MB file containing over a million lines:
Code:
$ du -sh data
 52M    data
$ wc data   
 1074672 1074672 54942606 data

Test runs show sed completes in a second and a half while your shell script, killed after a minute of real time has elapsed, has only processed a small fraction of the file. Roughly, we're talking about 1.5 secs versus an 1 hour, a 2400x improvement, with this data on my hardware:

Code:
$ time sed 'n;s/......//' data > dump

real    0m1.545s
user    0m1.123s
sys     0m0.407s
$ wc -l dump
 1074672 dump

$ time ./trim.sh data > dump
^C

real    1m1.610s
user    0m12.378s
sys     0m45.385s
$ wc -l dump
   16825 dump

Regards,
Alister
Last edited by alister; 06-06-2010 at 10:05 AM..
# 9  
Old 06-06-2010
What about the OP request ?
Quote:
Originally Posted by s052866
I want to remove the first 6 characters of every second line. This means in lines that do not start with "@HWUSI" or only contains "+".
# 10  
Old 06-06-2010
Hi, I am not sure what OP means?
# 11  
Old 06-07-2010
Quote:
Originally Posted by alister
You're quite welcome.

[code]$ time sed 'n;s/......//' data > dump

Alister
Alister,

Can you explain how "n;" means every 2nd line? I undertand the obvious expression.

Code:
 
sed '2~1 s/......//'
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove first 2 characters and last two characters of each line

here's what im trying to do. i have a file containing lines similar to this: data.txt: 1hsRmRsbHRiSFZNTTA1dlEyMWFkbU5wUW5CSlIyeDFTVU5SYjJOSFRuWmpia0ZuWXpKV2FHTnRU 1lKUnpWMldrZFZaMG95V25oYQpSelEyWTBka2QyRklhSHBrUjA1b1kwUkJkd3BOVXpWM1lVaG5k... (5 Replies)
Discussion started by: SkySmart
5 Replies

2. Shell Programming and Scripting

Ksh: Read line parse characters into variable and remove the line if the date is older than 50 days

I have a test file with the following format, It contains the username_date when the user was locked from the database. $ cat lockedusers.txt TEST1_21062016 TEST2_02122015 TEST3_01032016 TEST4_01042016 I'm writing a ksh script and faced with this difficult scenario for my... (11 Replies)
Discussion started by: humble_learner
11 Replies

3. Shell Programming and Scripting

[Solved] How to separate one line to mutiple line based on certain number of characters?

hi Gurus, I need separate a file which is one huge line to multiple lines based on certain number of charactors. for example: abcdefghi high abaddffdd I want to separate the line to multiple lines for every 4 charactors. the result should be abcd efgh i hi gh a badd ffdd Thanks in... (5 Replies)
Discussion started by: ken6503
5 Replies

4. UNIX for Dummies Questions & Answers

sed - combination of line deletion and pattern matching

I want to delete all the blank lines from a file before a certain line number. e.g. Input file (n: denotes line number) 1: a 2: 3: b 4: c 5: 6: d I want to delete all blank lines before line number 3, such that my output is: a b c d I see that sed '/^$/d' in_file works... (9 Replies)
Discussion started by: jawsnnn
9 Replies

5. UNIX for Dummies Questions & Answers

How to specify beginning-of-line/end-of-line characters inside a regex range

How can I specify special meaning characters like ^ or $ inside a regex range. e.g Suppose I want to search for a string that either starts with '|' character or begins with start-of-line character. I tried the following but it does not work: sed 's/\(\)/<do something here>/g' file1 ... (3 Replies)
Discussion started by: jawsnnn
3 Replies

6. UNIX for Dummies Questions & Answers

deletion of duplicate characters and count

to delete the duplicate characters in a file I used this code cat file.txt|tr -s "" tell the other ways using sed command to count of duplicate characters thanks:) (0 Replies)
Discussion started by: tsurendra
0 Replies

7. Shell Programming and Scripting

Get the 1st 99 characters and add new line feed at the end of the line

I have a file with varying record length in it. I need to reformat this file so that each line will have a length of 100 characters (99 characters + the line feed). AU * A01 EXPENSE 6990370000 CWF SUBC TRAVEL & MISC MY * A02 RESALE 6990788000 Y... (3 Replies)
Discussion started by: udelalv
3 Replies

8. Shell Programming and Scripting

blank line deletion in a file using perl

Dear All, I am looking for an option in perl using which i could delete empty lines in a file. Or the alternative of sed '/^$/d' <filename> in perl. Sed is not working in my perl script :( Pls help me out . Thanks, VG (4 Replies)
Discussion started by: vguleria
4 Replies

9. Shell Programming and Scripting

sed remove last 10 characters of a line start from 3rd line

hello experts, I need a sed command that remove last 10 characters of a line start from 3rd line. any suggestions? Thanks you (7 Replies)
Discussion started by: minifish
7 Replies

10. UNIX for Dummies Questions & Answers

Line deletion help using perl -ne

Hi all, I know the one liner to delete all the lines in a file which matches a pattern i.e perl -i.old -ne 'print unless /pattern/' file Now i need the perl onliner to delete all the lines which doesnt match the pattern. Also what is the difference between perl -i and perl -i.old. Does... (1 Reply)
Discussion started by: lijju.mathew
1 Replies
Login or Register to Ask a Question