Hi there,
I am using AIX and trying to search and replace a string with another string in multiple files in different directories.
I wanted to search replace in steps so I don't change all of the instance anywhere in the server at once, minimizing impact.
STEP 1:
--------
I first searched all the server for the location of the string in every directory/subdirectory/file, and then send the output to a file Main_fpaths.txt (see below).
I then selected one of the directories that was common for files (/home/FLOC/), and then created another file fpaths.txt as a source to search and replace so I can control the change.
STEP 2:
--------
Next, I want to search and replace the string in all the files in fpaths.txt. Once it is successful, I will take next chunk of directories/subdirectories from Main_fpaths.txt and process them.
Each of the .ksh file in the fpath.txt contains a text 'old string', and I want to replace it with 'new string'.
I wanted to do loop on the fpaths.txt, go to first line, find last character "/" and use the string from column 1 to last column position as a path. and then use this fpath to perform sed like below:
I am new to Unix and may not have it right. Is there a better way to do this? I understand I don't have "-i" switch in AIX so I have to create temp file and then move temp file to original file.
Code:
while read line
do
var1=`echo $line`
end=${var1##*/}
#echo "Last / is in column $((${#var1} - ${#end}))"
lastcol=$((${#var1} - ${#end}))
#echo $lastcol
#fileloc=$var1| cut -c1-$lastcol
filespath=$var1
fileloc=echo $filespath |cut -c1-$lastcol
done < /home/myloc/Main_fpaths.txt
-----------------------------------------------------------------
This is the statement I used to create Main_fpaths.txt file.
Consider me newbie because I came up with this logic and wanted to see if its possible. I searched this forum and found bits and pieces, tried and worked in same directory but having multiple directories recursively like in my question it did not work.
I am sure there is a better way to do the same thing and I am all up for learning.
My only concern is I want to limit the search/replace to directories in my question. Once update 1 is done, then I can change directory to do UPDATE 2 and so on...
The first part of the command that you said produces your list of files:
Code:
find /home/ -TYPE
should produce a diagnostic similar to:
Code:
find: -TYPE: unknown primary or operator
and some of the rest of your code seems to be going out of its way to make your life difficult. There is no reason to process the files in each directory separately.
If I understand what you're trying to do, something like the following seems to be a simpler approach:
Code:
#!/bin/ksh
IAm=${0##*/}
searchDir="/home"
searchRE='search string'
rep='replace string'
tmpf="/tmp/$IAm.$$"
trap 'rm -f "$tmpf"' EXIT
find "$searchDir" -type f -name '*.ksh' -exec grep -l "$searchRE" {} + |
while read -r file
do if sed "s|$searchRE|$rep|g" "$file" > "$tmpf"
then cp "$tmpf" "$file"
fi
done
Note that I used the pipe symbol (|) rather than slash (/) in case your search pattern or replacement strings are manipulating pathnames. You can replace the pipe symbols with any character other than backslash (\) and <newline>. Try to choose a character that you know will never appear in the search and replacement strings.
This User Gave Thanks to Don Cragun For This Post:
Sorry I didn't get a chance to thank you. It worked like a charm, and it ran pretty fast. I am novice to Unix and most of the syntax is unfamiliar with.
Whenever you get a chance, could you please quickly explain the following code, I'd like to understand how it worked so I can try using it next time.
Code:
#!/bin/ksh
IAm=${0##*/} #what is this one used for? #
searchDir="/home"
searchRE='search string'
rep='replace string'
tmpf="/tmp/$IAm.$$"
trap 'rm -f "$tmpf"' EXIT #when does it get used? I thought it would be used within while loop?
find "$searchDir" -type f -name '*.ksh' -exec grep -l "$searchRE" {} + | #what "{}" is used for?
while read -r file
do if sed "s|$searchRE|$rep|g" "$file" > "$tmpf"
then cp "$tmpf" "$file"
fi
done
Moderator's Comments:
Please use code tags next time for your code and data. Thanks
Sorry I didn't get a chance to thank you. It worked like a charm, and it ran pretty fast. I am novice to Unix and most of the syntax is unfamiliar with.
Whenever you get a chance, could you please quickly explain the following code, I'd like to understand how it worked so I can try using it next time.
Code:
#!/bin/ksh
IAm=${0##*/} #what is this one used for? #
searchDir="/home"
searchRE='search string'
rep='replace string'
tmpf="/tmp/$IAm.$$"
trap 'rm -f "$tmpf"' EXIT #when does it get used? I thought it would be used within while loop?
find "$searchDir" -type f -name '*.ksh' -exec grep -l "$searchRE" {} + | #what "{}" is used for?
while read -r file
do if sed "s|$searchRE|$rep|g" "$file" > "$tmpf"
then cp "$tmpf" "$file"
fi
done
Moderator's Comments:
Please use code tags next time for your code and data. Thanks
The IAm (I am) variable is the last component of the pathname used to invoke this script. (If your script is named myScript and it was invoked as /home/me/bin/myScript, IAm will be set to myScript. As shown in red in the listing above, it is used as part of the name of the temporary file it creates. You can find more details about this in a section of the ksh man page (try man ksh) on your system titled something like "Parameter Expansion".
The trap statement in the script removes the temporary file just before the script exits. It will remove the file even if your script is terminated by a signal (any signal except SIGKILL). So even if you kill your script with a CTL-C while sed is running, the temporary file will still be removed. This means that there is less chance that the temp file will be left hanging around (than if we just put in a call to rm as the last line of the script) and it is faster than invoking rm each time we go through the loop if we are processing more than one file. Some systems will have a man page for trap; others will include it in your shell's man page in a section with a heading something like "Special Built-Ins".
The {} in find ... -exec ... {} + is replaced by a list of one or more pathnames of files matched by the search criteria you gave to find. If I had used \; instead of + to terminate the -exec primary, grep would be invoked for each matched file instead of invoking grep to operate on several files at a time. You can see the details on this on the find man page.
I have a environment property file which contains:
Input file:
value1 = url1
value2 = url2
value3 = url3 and so on.
I need to search all *.xml files under directory for value1 and replace it with url1.
Same thing I have to do for all values mentioned in input file. I need script in unix bash... (7 Replies)
Hello,
I have hundreds of files in which I need to change email address. Here is what I am trying to do:
1. All text files are in a directory "a"
2. In the text file, I want to replace email address for preparer. All these lines start with {{PreparerEmail and end with }}. The email... (3 Replies)
I have a list of files all over a file system e.g.
/home/1/foo/bar.x
/www/sites/moose/foo.txtI'm looking for strings in these files and want to replace each occurrence with a replacement string, e.g.
if I find: '#@!^\&@ in any of the files I want to replace it with: 655#@11, etc.
There... (2 Replies)
When I use special characters the command to replace multiple files with a string pattern does nt work.
---------- Post updated at 12:33 PM ---------- Previous update was at 11:38 AM ----------
This works
perl -pi -e 's/100/test/g' *
This does nt work
perl -pi -e 's... (1 Reply)
Hello.
I have five config files in /etc that I want to edit in one click for testing.
I would like to make a script like this :
#!/bin/bash
#
a_file="/etc/file_1"
src_str="src_string_1"
rpl_str="rpl_string_1"
calling_sed_or_awk_or_whatelse $a_file search_for_all $src_str replace_with... (4 Replies)
Hi all
I'm in need of a command which can replace a specified string with another string - across multiple files within multiple sub-directories (I intend to run it from / )
I've used the following to get a list of the files:
find . | xargs grep <string1>
But that's as far as I've got.... (7 Replies)
Hi,
I tried making a shell script which recursively search in a given directory for all files *.txt and then search and replace some text and after that
save each file as $filename.dynamips.txt
I would like to get this done with sed. I tried but couldn't get it to work.
BTW this is not... (1 Reply)
I have a rather complicated search and replace I need to do among several dozen files and over a hundred occurrences. My site is written in PHP and throughout the old code, you will find things like
die("Operation Aborted due to.....");
For my new design skins for the site, I need to get... (2 Replies)
Hi ,
I have a directory structure as dir and subdirectories and files under it and so on.now I need to find the files which contain the search string under every dir and subdir and replace .
my search string is like
searchstring=/a/b
string to be replaced=/a/c/b
please help.
... (7 Replies)
Hello all
I need to search and replace in multiple files that are in directory hierarchy
Im using the :
find . -name "*.dsp" -print | xargs grep -n -o Test.lib" , I like to be able to replace every instance of Test.lib
with empty space .
how can I write one liner that does this ? (3 Replies)
04-10-2014
