Sponsored Content
Top Forums Shell Programming and Scripting Optimizing find with many replacements Post 303015931 by f77hack on Monday 16th of April 2018 04:40:33 PM
Old 04-16-2018
Quote:
Originally Posted by Don Cragun
Just to be clear, is it true that you want the output of the sed from the 1st find to be written to standard output (and not be included in the changes made to updated files) while all of the other finds run seds that will make updates to the files and not write anything to standard outputa?

Why not run all of the sed commands in the last 10 invocations of sed in a single invocation of find -execing sed?

And, why not use two -execs in a single invocation of find instead of invoking find eleven times?
Thanks for help.

All of the sed is basically a large 'OR' boolean.

Any particular file, could have any sed condition, 1..."#conditions", so the the sed needs to search for a condition in the file before moving on to the next file.

Basically, the script has expanded over time and now it's getting to the point where I'd like to refactor it.

That is sort of the question, is it more efficient to let find search a massive amount of files and let sed chew on one condition at a time? Which it does now, which is basically unrolling the loops in your suggestion about concatening the sed to two exec commands?

or as you suggest, find pauses its search while let sed grind on one file searching all the conditions at once?

Say average files to search is ~ 100,000 files, average size ~40k/~100k

Thanks for the thoughts.

Last edited by f77hack; 04-16-2018 at 05:47 PM.. Reason: typos
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

optimizing - to find the number of occurrence

Hi, I need to find the number of occurrence of string in a file, for ex: >cat filename abc abc def ghi ghi ghi ghi abc abc >output would be abc 4 def 1 (10 Replies)
Discussion started by: matrixmadhan
10 Replies

2. Shell Programming and Scripting

Conditional replacements

Hi, I have a requirement as below Input Jacuzzi,"Jet Rings, Pillows",Accessory,Optional,,9230917,69094,,P556805,69094,FALSE,1,0,, Jacuzzi,"Jet Rings, Pillows, Skirt/Apron",Accessory,Optional,,9230917,69094,,P556805,69094,FALSE,1,0,, Output Jacuzzi,"Jet Rings!@%... (6 Replies)
Discussion started by: kewk
6 Replies

3. Shell Programming and Scripting

Gen random char then sed replacements

Hey guys, I need to first generate some random characters, which I am already doing perfectly as follows: randomize=`cat /dev/urandom | tr -dc "a-z0-9" | fold -w 6 | head -n 1` This is where I am stuck...I need to sed replace some static values with those random characters, but I need each... (4 Replies)
Discussion started by: holyearth
4 Replies

4. UNIX for Dummies Questions & Answers

How to use sed to do multiple replacements all at once?

I have a text file where I want to use sed to do multiple replacements all at once (i.e. with a single command) . I want to convert all AA's to 0, all AG's to 1 and all GG's to 2. How do I go about doing that? Thanks! (2 Replies)
Discussion started by: evelibertine
2 Replies

5. Shell Programming and Scripting

How to get count of replacements done by sed?

Hi , How can i get count of replacements done by sed in a file. I know grep -c is a method. But say if sed had made 10 replacement in a file, can i get number 10 some how? (8 Replies)
Discussion started by: abhitanshu
8 Replies

6. UNIX for Dummies Questions & Answers

Selective Replacements: Using sed or awk to replace letters with numbers in a very specific way

Hello all. I am a beginner UNIX user who is using UNIX to work on a bioinformatics project for my university. I have a bit of a complicated issue in trying to use sed (or awk) to "find and replace" bases (letters) in a genetics data spreadsheet (converted to a text file, can be either... (3 Replies)
Discussion started by: Mince
3 Replies

7. Shell Programming and Scripting

awk delete newline after other replacements

Dear All, could you please help me to remove \n characters after all other replacements have been done as in the code below: { #remove punctuation and starting whitespaces gsub("]"," "); $1=$1; } { #print lines containing 'whatever' if ($1=="whatever") {print} #print... (3 Replies)
Discussion started by: shivacoder
3 Replies

8. Shell Programming and Scripting

BASH script to read external file to perform text replacements?

Hi all, I have a moderate size (300 lines) BASH Shell script that performs various tasks on different source reports (CSV files). One of the tasks that it performs, is to use SED to replace 'non-conforming' titles with conformant ones. For example "How to format a RAW Report" needs to become... (3 Replies)
Discussion started by: richardsantink
3 Replies

9. Web Development

Optimizing JS and CSS

Yes. Got few suggestions. - How about minifying resources - mod_expires - Service workers setup https://www.unix.com/attachments/web-programming/7709d1550557731-sneak-preview-new-unix-com-usercp-vuejs-demo-screenshot-png (8 Replies)
Discussion started by: Akshay Hegde
8 Replies

10. Shell Programming and Scripting

Help building a variable string from a keyword - character replacements!

Hello scripting geniusii! I come to kneel before the alter of your wisdom! I am looking to take a keyword and replace characters within that keyword and add them to a string variable. I would like this to only go through however many characters the word has, which may vary in size. ... (10 Replies)
Discussion started by: ghaniba
10 Replies
All times are GMT -4. The time now is 10:51 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy