Sponsored Content
Top Forums Shell Programming and Scripting Optimizing find with many replacements Post 303015926 by f77hack on Monday 16th of April 2018 03:22:15 PM
Old 04-16-2018
Optimizing find with many replacements

Hello,

I'm looking for advice on how to optimize this bash script, currently i use the shotgun approach to avoid file io/buffering problems of forks trying to write simultaneously to the same file. i'd like to keep this as a fairly portable bash script rather than writing a C routine.

in a nutshell, there are many conditions in a file that i'm looking to replace strings. any particular file may have some, none or all of the requirements to replace a string.

currently

Code:
Longstring='lots of stuff'
spushd $HOME/somepath

gfind . -depth -name "somefile" -type f -writable -exec gsed -i '1{/^#./! s/.*/'"$Longstring"'/}' {} \;
  gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r 's/ts=4/ts=2/g' {} \;
  gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r 's/sw=4/sw=2/g' {} \;
  gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r 's/tab-width: 4/tab-width: 2/g' {} \;
  gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r 's/mode: tcl/mode: _tcl/g' {} \;
  gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r 's/c-basic-offset: 4/c-basic-offset: 2/g' {} \;
  gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r 's/^\s*(size.*)$/\1/g' {} \;
  gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r 's/^\s*(md.*)$/\1/g' {} \;
  gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r 's/^\s*(rmd.*)$/\1/g' {} \;
  gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r 's/^\s*(sha.*)$/\1/g' {} \;
  gfind . -depth -name "somefile" -type f -writable -exec gsed -i -r 's/^(python.versions.*)$/python.versions 27 36/g' {} \;
  spopd

as you can see, these operations are sequential which can take quite a while.

should i modify the find to do depth first?

can i fork the find and avoid file io problems?

spawn different processes?

thanks
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

optimizing - to find the number of occurrence

Hi, I need to find the number of occurrence of string in a file, for ex: >cat filename abc abc def ghi ghi ghi ghi abc abc >output would be abc 4 def 1 (10 Replies)
Discussion started by: matrixmadhan
10 Replies

2. Shell Programming and Scripting

Conditional replacements

Hi, I have a requirement as below Input Jacuzzi,"Jet Rings, Pillows",Accessory,Optional,,9230917,69094,,P556805,69094,FALSE,1,0,, Jacuzzi,"Jet Rings, Pillows, Skirt/Apron",Accessory,Optional,,9230917,69094,,P556805,69094,FALSE,1,0,, Output Jacuzzi,"Jet Rings!@%... (6 Replies)
Discussion started by: kewk
6 Replies

3. Shell Programming and Scripting

Gen random char then sed replacements

Hey guys, I need to first generate some random characters, which I am already doing perfectly as follows: randomize=`cat /dev/urandom | tr -dc "a-z0-9" | fold -w 6 | head -n 1` This is where I am stuck...I need to sed replace some static values with those random characters, but I need each... (4 Replies)
Discussion started by: holyearth
4 Replies

4. UNIX for Dummies Questions & Answers

How to use sed to do multiple replacements all at once?

I have a text file where I want to use sed to do multiple replacements all at once (i.e. with a single command) . I want to convert all AA's to 0, all AG's to 1 and all GG's to 2. How do I go about doing that? Thanks! (2 Replies)
Discussion started by: evelibertine
2 Replies

5. Shell Programming and Scripting

How to get count of replacements done by sed?

Hi , How can i get count of replacements done by sed in a file. I know grep -c is a method. But say if sed had made 10 replacement in a file, can i get number 10 some how? (8 Replies)
Discussion started by: abhitanshu
8 Replies

6. UNIX for Dummies Questions & Answers

Selective Replacements: Using sed or awk to replace letters with numbers in a very specific way

Hello all. I am a beginner UNIX user who is using UNIX to work on a bioinformatics project for my university. I have a bit of a complicated issue in trying to use sed (or awk) to "find and replace" bases (letters) in a genetics data spreadsheet (converted to a text file, can be either... (3 Replies)
Discussion started by: Mince
3 Replies

7. Shell Programming and Scripting

awk delete newline after other replacements

Dear All, could you please help me to remove \n characters after all other replacements have been done as in the code below: { #remove punctuation and starting whitespaces gsub("]"," "); $1=$1; } { #print lines containing 'whatever' if ($1=="whatever") {print} #print... (3 Replies)
Discussion started by: shivacoder
3 Replies

8. Shell Programming and Scripting

BASH script to read external file to perform text replacements?

Hi all, I have a moderate size (300 lines) BASH Shell script that performs various tasks on different source reports (CSV files). One of the tasks that it performs, is to use SED to replace 'non-conforming' titles with conformant ones. For example "How to format a RAW Report" needs to become... (3 Replies)
Discussion started by: richardsantink
3 Replies

9. Web Development

Optimizing JS and CSS

Yes. Got few suggestions. - How about minifying resources - mod_expires - Service workers setup https://www.unix.com/attachments/web-programming/7709d1550557731-sneak-preview-new-unix-com-usercp-vuejs-demo-screenshot-png (8 Replies)
Discussion started by: Akshay Hegde
8 Replies

10. Shell Programming and Scripting

Help building a variable string from a keyword - character replacements!

Hello scripting geniusii! I come to kneel before the alter of your wisdom! I am looking to take a keyword and replace characters within that keyword and add them to a string variable. I would like this to only go through however many characters the word has, which may vary in size. ... (10 Replies)
Discussion started by: ghaniba
10 Replies
SYSPROFILE(8)						      System Manager's Manual						     SYSPROFILE(8)

NAME
sysprofile - modular centralized shell configuration DESCRIPTION
sysprofile is a generic approach to configure shell settings in a modular and centralized way mostly aimed at avoiding work for lazy sysad- mins. It has only been tested to work with the bash shell. It basically consists of the small /etc/sysprofile shell script which invokes other small shell scripts having a .bash suffix which are contained in the /etc/sysprofile.d/ directory. The system administrator can drop in any script he wants without any naming convention other than that the scripts need to have a .bash suffix to enable automagic sourcing by /etc/sysprofile. This mechanism is set up by inserting a small shell routine into /etc/profile for login shells and optionally into /etc/bashrc and/or /etc/bash.bashrc for non-login shells from where the actual /etc/sysprofile script is invoked: if [ -f /etc/sysprofile ]; then . /etc/sysprofile fi For using "sysprofile" under X11, one can source it in a similar way from /etc/X11/Xsession or your X display manager's Xsession file to provide the same shell environment as under the console in X11. See the example files in /usr/share/doc/sysprofile/ for illustration. For usage of terminal emulators with a non-login bash shell under X11, take care to enable sysprofile via /etc/bash.bashrc. If not set this way, your terminal emulators won't come up with the environment defined by the scripts in /etc/sysprofile.d/. Users not wanting /etc/sysprofile to be sourced for their environment can easily disable it's automatic mechanism. It can be disabled by simply creating an empty file called $HOME/.nosysprofile in the user's home directory using e.g. the touch(1) command. Any single configuration file in /etc/sysprofile.d/ can be overridden by any user by creating a private $HOME/.sysprofile.d/ directory which may contain a user's own version of any configuration file to be sourced instead of the system default. It's names have just to match exactly the system's default /etc/sysprofile.d/ configuration files. Empty versions of these files contained in the $HOME/.syspro- file.d/ directory automatically disable sourcing of the system wide version. Naturally, users can add and include their own private script inventions to be automagically executed by /etc/sysprofile at login time. OPTIONS
There are no options other than those dictated by shell conventions. Anything is defined within the configuration scripts themselves. SEE ALSO
The README files and configuration examples contained in /etc/sysprofile.d/ and the manual pages bash(1), xdm(1x), xdm.options(5), and wdm(1x). Recommended further reading is everything related with shell programming. If you need a similar mechanism for executing code at logout time check out the related package syslogout(8) which is a very close compan- ion to sysprofile. BUGS
sysprofile in its current form is mainly restricted to bash(1) syntax. In fact it is actually a rather embarrassing quick and dirty hack than anything else - but it works. It serves the practical need to enable a centralized bash configuration until something better becomes available. Your constructive criticism in making this into something better" is very welcome. Before i forget to mention it: we take patches... ;-) AUTHOR
sysprofile was developed by Paul Seelig <pseelig@debian.org> specifically for the Debian GNU/Linux system. Feel free to port it to and use it anywhere else under the conditions of either the GNU public license or the BSD license or both. Better yet, please help to make it into something more worthwhile than it currently is. SYSPROFILE(8)
All times are GMT -4. The time now is 08:02 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy