Optimizing find with many replacements Post: 303015926

Sponsored Content

Top Forums Shell Programming and Scripting Optimizing find with many replacements Post 303015926 by f77hack on Monday 16th of April 2018 03:22:15 PM

04-16-2018

Registered User

Optimizing find with many replacements

Hello,

I'm looking for advice on how to optimize this bash script, currently i use the shotgun approach to avoid file io/buffering problems of forks trying to write simultaneously to the same file. i'd like to keep this as a fairly portable bash script rather than writing a C routine.

in a nutshell, there are many conditions in a file that i'm looking to replace strings. any particular file may have some, none or all of the requirements to replace a string.

currently

Code:

Longstring='lots of stuff'
spushd $HOME/somepath

gfind . -depth -name "somefile" -type f -writable -exec gsed -i '1{/^#./! s/.*/'"$Longstring"'/}' {} \;
  gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r 's/ts=4/ts=2/g' {} \;
  gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r 's/sw=4/sw=2/g' {} \;
  gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r 's/tab-width: 4/tab-width: 2/g' {} \;
  gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r 's/mode: tcl/mode: _tcl/g' {} \;
  gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r 's/c-basic-offset: 4/c-basic-offset: 2/g' {} \;
  gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r 's/^\s*(size.*)$/\1/g' {} \;
  gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r 's/^\s*(md.*)$/\1/g' {} \;
  gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r 's/^\s*(rmd.*)$/\1/g' {} \;
  gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r 's/^\s*(sha.*)$/\1/g' {} \;
  gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r 's/^(python.versions.*)$/python.versions 27 36/g' {} \;
  spopd

as you can see, these operations are sequential which can take quite a while.

should i modify the find to do depth first?

can i fork the find and avoid file io problems?

spawn different processes?

thanks

f77hack

View Public Profile for f77hack

Find all posts by f77hack

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

optimizing - to find the number of occurrence

Hi, I need to find the number of occurrence of string in a file, for ex: >cat filename abc abc def ghi ghi ghi ghi abc abc >output would be abc 4 def 1

2. Shell Programming and Scripting

Conditional replacements

Hi, I have a requirement as below Input Jacuzzi,"Jet Rings, Pillows",Accessory,Optional,,9230917,69094,,P556805,69094,FALSE,1,0,, Jacuzzi,"Jet Rings, Pillows, Skirt/Apron",Accessory,Optional,,9230917,69094,,P556805,69094,FALSE,1,0,, Output Jacuzzi,"Jet Rings!@%...

3. Shell Programming and Scripting

Gen random char then sed replacements

Hey guys, I need to first generate some random characters, which I am already doing perfectly as follows: randomize=`cat /dev/urandom | tr -dc "a-z0-9" | fold -w 6 | head -n 1` This is where I am stuck...I need to sed replace some static values with those random characters, but I need each...

4. UNIX for Dummies Questions & Answers

How to use sed to do multiple replacements all at once?

I have a text file where I want to use sed to do multiple replacements all at once (i.e. with a single command) . I want to convert all AA's to 0, all AG's to 1 and all GG's to 2. How do I go about doing that? Thanks!

5. Shell Programming and Scripting

How to get count of replacements done by sed?

Hi , How can i get count of replacements done by sed in a file. I know grep -c is a method. But say if sed had made 10 replacement in a file, can i get number 10 some how?

6. UNIX for Dummies Questions & Answers

Selective Replacements: Using sed or awk to replace letters with numbers in a very specific way

Hello all. I am a beginner UNIX user who is using UNIX to work on a bioinformatics project for my university. I have a bit of a complicated issue in trying to use sed (or awk) to "find and replace" bases (letters) in a genetics data spreadsheet (converted to a text file, can be either...

7. Shell Programming and Scripting

awk delete newline after other replacements

Dear All, could you please help me to remove \n characters after all other replacements have been done as in the code below: { #remove punctuation and starting whitespaces gsub("]"," "); $1=$1; } { #print lines containing 'whatever' if ($1=="whatever") {print} #print...

8. Shell Programming and Scripting

BASH script to read external file to perform text replacements?

Hi all, I have a moderate size (300 lines) BASH Shell script that performs various tasks on different source reports (CSV files). One of the tasks that it performs, is to use SED to replace 'non-conforming' titles with conformant ones. For example "How to format a RAW Report" needs to become...

9. Web Development

Optimizing JS and CSS

Yes. Got few suggestions. - How about minifying resources - mod_expires - Service workers setup https://www.unix.com/attachments/web-programming/7709d1550557731-sneak-preview-new-unix-com-usercp-vuejs-demo-screenshot-png

10. Shell Programming and Scripting

Help building a variable string from a keyword - character replacements!

Hello scripting geniusii! I come to kneel before the alter of your wisdom! I am looking to take a keyword and replace characters within that keyword and add them to a string variable. I would like this to only go through however many characters the word has, which may vary in size. ...

LEARN ABOUT DEBIAN

surfraw-update-path

SURFRAW-UPDATE-PATH(1)					User Contributed Perl Documentation				    SURFRAW-UPDATE-PATH(1)

NAME

       surfraw-update-path - updates PATH in shell config files

SYNOPSIS

       surfraw-update-path [-add] [-remove] [-check] [-sys] [-all] [-help] [-shell=SHELL]

DESCRIPTION

       surfraw-update-path adds the surfraw elvi directory (/usr/lib/surfraw) to your PATH in your shell's config file.

       Currently it supports bash, sh, csh, tcsh, ash, dash, ksh, pdksh, zsh, rc, and es

       Don't forget to login again or source your login files for it to take effect.

OPTIONS

       -check
	   Checks to see if the surfraw config code is present.  This is the default.

       -add
	   Adds the surfraw config code.

       -remove
	   Removes the surfraw config code

       -sys
	   Updates the system-wide shell config instead of the user.  Must be done as root.

       -shell=SHELL
	   Selects the shell to configure. Defaults to the value of the $SHELL environment variable.

	   Currently supported shells are:

	   sh, ash, bash, dash, csh, tcsh, ksh, pdksh, zsh, rc, and es.

       -all
	   Attempts to configure the startup files for all known shells

       -help
	   Gives a usage message

RETURN VALUE

       -check returns 0 if the surfraw code is present in the file, 1 if it is not found, or 2 on error.

       All other options return 0 on success, or 2 on error.

ENVIRONMENT

       SHELL
	   Used to determine which shell to configure, if -shell is not given.

       HOME
	   Used to find users config files.

       ENV Used by posix-compliant shells to specify a startup rc file.

       ZDOTDIR
	   Used to find user config files for zsh. If not set, defaults to HOME.

SEE ALSO

       surfraw(1), sh(1), ash(1), bash(1), dash(1), csh(1), tcsh(1), ksh(1), pdksh(1), zsh(1), rc(1), es(1)

AUTHOR

       Ian Beckwith <ianb@erislabs.net>

perl v5.12.4							    2011-07-12						    SURFRAW-UPDATE-PATH(1)

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

optimizing - to find the number of occurrence

Discussion started by: matrixmadhan

2. Shell Programming and Scripting

Conditional replacements

Discussion started by: kewk

3. Shell Programming and Scripting

Gen random char then sed replacements

Discussion started by: holyearth

4. UNIX for Dummies Questions & Answers

How to use sed to do multiple replacements all at once?

Discussion started by: evelibertine