How to replace all but the first 3 characters with sed?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to replace all but the first 3 characters with sed?
# 15  
Old 02-10-2015
I hate sed loops for one byte change per loop, so you can split the lines and then rejoin them:
Code:
sed '
  s/^.\{0,3\}/&\
/
' | sed '
  $q
  N
  P
  s/.*\n//
  s/./*/g
'|sed '
  $q
  N
  s/\n//
'

This User Gave Thanks to DGPickett For This Post:
# 16  
Old 02-10-2015
Point taken! I don't really like pipe organs, so tried a no loop no pipe solution:
Code:
sed 'h;s/\(...\).*/\1/;x;s/^...//;s/./*/g;H;x;s/\n//' file

# 17  
Old 02-10-2015
Some more awk variations:
Code:
awk -F'^...' '{a=$2; gsub(/./,"*",a); print substr($0,1,3) a}' file

GNU:
Code:
awk '{gsub(/./,"*",$2)}1' FIELDWIDTHS="3 99" OFS= file

GNU/mawk/BSD awk
Code:
awk '{for(i=4; i<=NF; i++) $i="*"}1' FS= OFS= file


Last edited by Scrutinizer; 02-10-2015 at 11:09 PM..
# 18  
Old 02-10-2015
Quote:
Originally Posted by DGPickett
I hate sed loops for one byte change per loop, so you can split the lines and then rejoin them[..]
Quote:
Originally Posted by RudiC
Point taken! I don't really like pipe organs, so tried a no loop no pipe solution[..]
Did a small test and indeed, those 1 character loops are expensive.
It turns the awk solutions appear to be fastest.
Code:
$ time sed -e :a -e 's/\(...\)[^*]/\1*/;ta' file > /dev/null

real	0m0.114s
user	0m0.110s
sys	0m0.003s

$ time { sed '
  s/^.\{0,3\}/&\
/
' file | sed '
  $q
  N
  P
  s/.*\n//
  s/./*/g
'|sed '
  $q
  N
  s/\n//
' ;} > /dev/null

real	0m0.015s
user	0m0.015s
sys	0m0.006s

$ time sed 'h;s/\(...\).*/\1/;x;s/^...//;s/./*/g;H;x;s/\n//' file > /dev/null

real	0m0.018s
user	0m0.013s
sys	0m0.002s

$ time perl -pe 's/(?<=...)./*/g' file >/dev/null

real	0m0.030s
user	0m0.022s
sys	0m0.005s

$ time awk '{x=substr($0,N+1); gsub(".","*",x); print substr($0,1,N) x}' N=3 file > /dev/null

real	0m0.010s
user	0m0.007s
sys	0m0.002s

$ time gsed "s/./*/4g" file > /dev/null

real	0m0.022s
user	0m0.018s
sys	0m0.002s

time awk -F'^...' '{a=$2; gsub(/./,"*",a); print substr($0,1,3) a}' file > /dev/null

real	0m0.009s
user	0m0.006s
sys	0m0.003s


Last edited by Scrutinizer; 02-10-2015 at 11:13 PM.. Reason: Some sed/awk times were GNU sed/awk, rather than BSD sed/awk. Correct it
# 19  
Old 02-12-2015
The '\(...\)' and '\1' feature has been known to be a bit slow relative to simpler choices.

Maybe awk could get the length of the line - 3 and substring a string of "***************" to produce the rest of the line, faster? Or do you need perl/python/ruby for that?
# 20  
Old 02-12-2015
Irritating,when i run my code using time i get:
Code:
+ ~/tmp $ cat ./leolson 
while read line
do
	len=${#line} ; [ $len -gt 3 ] && len=$(( $len - 3))
	tmp="$(printf '%*s' $len|sed s,\ ,*,g)"
	echo "${line:0:3}${tmp}"
done<file1

+ ~/tmp $ time bash ./leolson
aba*****
cdc*******
efe******
a*b****

real	0m0.007s
user	0m0.003s
sys	0m0.006s

I did a few runs, and felt tendency is more around 6-11 than 12+.
(had more 0.006 + 0.007 than everything else together)

Though, with MadeInGermany's 0.010 sec awk code i get 0.001 Smilie
Code:
time awk '{x=substr($0,N+1); gsub(".","*",x); print substr($0,1,N) x}' N=3 file1 > /dev/null

real	0m0.001s
user	0m0.000s
sys	0m0.001s

And your perl code behaves quite irrrational:
Code:
+ ~/tmp $ time perl -pe 's/(?<=...)./*/g' file1 >/dev/null

real	0m0.119s
user	0m0.003s
sys	0m0.002s
+ ~/tmp $ time perl -pe 's/(?<=...)./*/g' file1 >/dev/null

real	0m0.006s
user	0m0.002s
sys	0m0.004s

I find the time diffrence quite immmense and confusing.
Sure, some mili-secs diffrence can happen - but by factor 19.8:1?

Last edited by sea; 02-12-2015 at 01:01 PM..
# 21  
Old 02-12-2015
You should ignore the time it takes the first time, because that is used to read pieces of program and data into memory, which varies a lot, in addition to the time it takes to run the program. The second time, it is already in memory, so the time is largely the time to run the program. That's why it is a lot faster and also more consistent, which makes it a more reliable way to do comparisons relative to other solutions..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed command on AIX, replace specific characters

Hi, Im using sed on an AIX machine. I am trying to change the 137-139 characters if they are a ' 36'/'000' to a '036'. The positions that need to be changed are fixed. the source data that I have is$cat v.txt 4000422985400050462239065593606500000007422985707771046154054910075641MC0318AMWAY... (9 Replies)
Discussion started by: dsid
9 Replies

2. Shell Programming and Scripting

sed replace characters using a wildcard

Hello, I have some data that looks like the following, > <SALTDATA> (OVS0199262) HCl > <IDNUMBER> (OVS0199262) OVS0199262 > <SUPPLIER> (OVS0199262) TimTec > <EMAIL> (OVS0199262) info@timtec.net > <WEBSITE> (OVS0199262) http://www.timtec.net I need to remove the data in... (3 Replies)
Discussion started by: LMHmedchem
3 Replies

3. Shell Programming and Scripting

sed replace nth characters with string

Hi, I hope you can help me out please? I need to replace from character 8-16 with AAAAAAAA and the rest should stay the same after character 16 gtwrhtrd11111111rjytwyejtyjejetjyetgeaEHT wrehrhw22222222hytekutkyukrylryilruilrGEQTH hrwjyety33333333gtrhwrjrgkreglqeriugn;RUGNEURGU ... (4 Replies)
Discussion started by: stinkefisch
4 Replies

4. Shell Programming and Scripting

Replace characters infile with sed

I have several files in a directory that look like this: jacket-n r potential-n - outcome-n f reputation-n b I want to replace the characters in the second column with certain numbers. For instance, I want the letters 'f', 'r' and 'b' in the second column to replaced with 0 and I want the... (1 Reply)
Discussion started by: owwow14
1 Replies

5. Shell Programming and Scripting

sed replace range of characters in each line

Hi, I'm trying to replace a range of characters by their position in each line by spaces. I need to replace characters 95 to 145 by spaces in each line. i tried below but it doesn't work sed -r "s/^(.{94})(.{51})/\ /" inputfile.txt > outputfile.txt can someone please help me... (3 Replies)
Discussion started by: Kevin Tivoli
3 Replies

6. UNIX for Dummies Questions & Answers

Bash: using SED, trying to replace some characters except first or last line

Hi, I require to replace 2 items: 1. replace start of all lines in a file with ' except the first line 2. replace end of all lines in a file with '||chr( except last line I am able to do the entire file using sed -e s/^/\'/g -e s/$/\'\|\|chr\(/g "$file" > newfile.txt but am not yet... (3 Replies)
Discussion started by: Chella15
3 Replies

7. Shell Programming and Scripting

Bash: using SED, trying to replace some characters except first or last line

Hi, I require to replace 2 items: 1. replace start of all lines in a file with ' except the first line 2. replace end of all lines in a file with '||chr( except last line I am able to do the entire file using sed -e s/^/\'/g -e s/$/\'\|\|chr\(/g "$file" > newfile.txt but am not yet able... (0 Replies)
Discussion started by: Chella15
0 Replies

8. Shell Programming and Scripting

Using sed to replace special characters

Hi everyone I have file1 contains: '7832' ' 8765 6543 I want a sed command that will format as: '7832' , '8765' , '6543' I tried sed -e s/\'//g -e 's/^*//;s/*$//' file1 > file2 sed -e :a -e '$!N; s/\n/ /; ta' file2 which gives: 7832 8765 6543 I need some help to continue with... (5 Replies)
Discussion started by: nimo
5 Replies

9. Shell Programming and Scripting

Sed replace characters not equal to an expression

Hi all, Suppose I have a file with the contents below, and I only want to print words %S_ then | sort -u. ------------------------------ The %S_MSG that starts with '%.*s' is too long. Maximum length is %d. The %S_MSG name '%.*s' contains more than the maximum number of prefixes. The... (5 Replies)
Discussion started by: poldo
5 Replies

10. Shell Programming and Scripting

how to replace control characters using sed?

How can I use sed to replace a ctrl character such as 'new line' (\0a) to something else? Or any other good command can do this job? Thanks, Hillxy (5 Replies)
Discussion started by: hillxy
5 Replies
Login or Register to Ask a Question