Optimizing find with many replacements


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Optimizing find with many replacements
# 8  
Old 04-17-2018
Quote:
Originally Posted by MadeInGermany
You can still have an embedded sed script.
All shells but (t)csh can have a multiline string
Code:
echo 'two
lines'

So the following should work
Code:
gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r '
    1{/^#./! s/.*/'"$Longstring"'/}
    s/ts=4/ts=2/g
... 
    s/^(python.versions.*)$/python.versions 27 36/g
' {} \;

@Don, not true, -i outputs to file, given in all the sed invocations.
Hi MadeInGermany,
Yes, you're correct. Neither sed -i nor sed -r are included in the standards so I seldom use either of them. I confused the meanings of -i and -r in gsed. (I mistakenly thought gsed's -i performed case insensitive pattern matching and -r created backups. Smilie )

On the BSD-based sed that I use on macOS, there is no -r option and the commands:
Code:
Longstring='lots of stuff'
sed -i '1{/^#./! s/.*/'"$Longstring"'/}' filename

would be a request to use 1{/^#./! s/.*/lots of stuff/} as a filename extension on standard input as a backup filename, use filename as an editing command, and process text read from standard input, producing the diagnostic message:
Code:
sed: 1: "filename": invalid command code f

Most of the other commands would fail to perform as expected on macOS (after translating gsed to sed) because the -r option that you want would be interpreted as the extension to be used on the backup files created (not as a request to use EREs instead of BREs when interpreting substitution search patterns).

Looking more closely at the gsed man page and the 1st post in this thread, it would seem that the code presented there could be made to run much more quickly if that script was replaced by:
Code:
Longstring='lots of stuff'
spushd $HOME/somepath

gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r '1{/^#./! s/.*/'"$Longstring"'/}
    s/ts=4/ts=2/g
    s/sw=4/sw=2/g
    s/tab-width: 4/tab-width: 2/g
    s/mode: tcl/mode: _tcl/g'
    s/c-basic-offset: 4/c-basic-offset: 2/g
    s/^\s*(size.*)$/\1/g
    s/^\s*(md.*)$/\1/g
    s/^\s*(rmd.*)$/\1/g
    s/^\s*(sha.*)$/\1/g
    s/^(python.versions.*)$/python.versions 27 36/g' {} +

spopd

which would invoke find once and gsed just a few times (maybe only once depending on the number of pathnames to be processed and the lengths of those pathnames) instead of invoking find 11 times and invoking gsed 11 times for each pathname processed.

But, of course, since I don't have gsed on my system, this suggestion is totally untested.
# 9  
Old 04-17-2018
Some sed versions (BSD ?) have introduced option -E that does the same as the -r in GNU sed.
IMHO the -E is more intuitive because it is used for grep in the same way: use ERE instead of BRE.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help building a variable string from a keyword - character replacements!

Hello scripting geniusii! I come to kneel before the alter of your wisdom! I am looking to take a keyword and replace characters within that keyword and add them to a string variable. I would like this to only go through however many characters the word has, which may vary in size. ... (10 Replies)
Discussion started by: ghaniba
10 Replies

2. Web Development

Optimizing JS and CSS

Yes. Got few suggestions. - How about minifying resources - mod_expires - Service workers setup https://www.unix.com/attachments/web-programming/7709d1550557731-sneak-preview-new-unix-com-usercp-vuejs-demo-screenshot-png (8 Replies)
Discussion started by: Akshay Hegde
8 Replies

3. Shell Programming and Scripting

BASH script to read external file to perform text replacements?

Hi all, I have a moderate size (300 lines) BASH Shell script that performs various tasks on different source reports (CSV files). One of the tasks that it performs, is to use SED to replace 'non-conforming' titles with conformant ones. For example "How to format a RAW Report" needs to become... (3 Replies)
Discussion started by: richardsantink
3 Replies

4. Shell Programming and Scripting

awk delete newline after other replacements

Dear All, could you please help me to remove \n characters after all other replacements have been done as in the code below: { #remove punctuation and starting whitespaces gsub("]"," "); $1=$1; } { #print lines containing 'whatever' if ($1=="whatever") {print} #print... (3 Replies)
Discussion started by: shivacoder
3 Replies

5. UNIX for Dummies Questions & Answers

Selective Replacements: Using sed or awk to replace letters with numbers in a very specific way

Hello all. I am a beginner UNIX user who is using UNIX to work on a bioinformatics project for my university. I have a bit of a complicated issue in trying to use sed (or awk) to "find and replace" bases (letters) in a genetics data spreadsheet (converted to a text file, can be either... (3 Replies)
Discussion started by: Mince
3 Replies

6. Shell Programming and Scripting

How to get count of replacements done by sed?

Hi , How can i get count of replacements done by sed in a file. I know grep -c is a method. But say if sed had made 10 replacement in a file, can i get number 10 some how? (8 Replies)
Discussion started by: abhitanshu
8 Replies

7. UNIX for Dummies Questions & Answers

How to use sed to do multiple replacements all at once?

I have a text file where I want to use sed to do multiple replacements all at once (i.e. with a single command) . I want to convert all AA's to 0, all AG's to 1 and all GG's to 2. How do I go about doing that? Thanks! (2 Replies)
Discussion started by: evelibertine
2 Replies

8. Shell Programming and Scripting

Gen random char then sed replacements

Hey guys, I need to first generate some random characters, which I am already doing perfectly as follows: randomize=`cat /dev/urandom | tr -dc "a-z0-9" | fold -w 6 | head -n 1` This is where I am stuck...I need to sed replace some static values with those random characters, but I need each... (4 Replies)
Discussion started by: holyearth
4 Replies

9. Shell Programming and Scripting

Conditional replacements

Hi, I have a requirement as below Input Jacuzzi,"Jet Rings, Pillows",Accessory,Optional,,9230917,69094,,P556805,69094,FALSE,1,0,, Jacuzzi,"Jet Rings, Pillows, Skirt/Apron",Accessory,Optional,,9230917,69094,,P556805,69094,FALSE,1,0,, Output Jacuzzi,"Jet Rings!@%... (6 Replies)
Discussion started by: kewk
6 Replies

10. UNIX for Dummies Questions & Answers

optimizing - to find the number of occurrence

Hi, I need to find the number of occurrence of string in a file, for ex: >cat filename abc abc def ghi ghi ghi ghi abc abc >output would be abc 4 def 1 (10 Replies)
Discussion started by: matrixmadhan
10 Replies
Login or Register to Ask a Question