Visit Our UNIX and Linux User Community


Search & Replace: Multiple Strings / Multiple Files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Search & Replace: Multiple Strings / Multiple Files
# 1  
Old 10-16-2012
Data Search & Replace: Multiple Strings / Multiple Files

I have a list of files all over a file system e.g.

Code:
/home/1/foo/bar.x
/www/sites/moose/foo.txt

I'm looking for strings in these files and want to replace each occurrence with a replacement string, e.g.

if I find: '#@!^\&@ in any of the files I want to replace it with: 655#@11, etc.

There are around 120 substitutions and around 4,600 files.

One issue: these strings are passwords, I'm hoping not to need to escape them.

So far I've only come up with:

Code:
ggrep -HF -f /home/pass "/home/find-passwords/struc/www/sites/foo.php"

which doesn't do much except list the file and matched string from pass (which contains old passwords),

no escaping is required due to the -F arg to grep.

e.g.

Code:
ggrep -HF -f /home/pass "/home/find-passwords/struc/www/sites/foo.php"

returns:

Code:
/home/find-passwords/struc/www/sites/foo.php:$dbcnx = @mysql_pconnect("blah.com", "foo", "xfgt^5");

where xfgt^5 is the match from my /home/pass file.

I'm struggling with how to replace the old password with the new, and thinking loops with a bunch of ifs might work? Or a lookup table?

I can put the old and new passwords in a file with corresponding columns.

It would be nice to be able to run in a manner that shows what it will do, and then one that does it.

Thanks for any help,

Bill

Last edited by spacegoose; 10-17-2012 at 09:52 AM..
# 2  
Old 10-16-2012
If working with files that are ":" field delimited:

Code:
 
pwds file (pwd:replace pwd):
s/1zg6/2RWAUY,2/lR:@#xyyyy$@#!$@#
JQy74TuvHyqMo:@#$@#!xxxxx$@#
32oVGmESDPBLk:@#$@234234wa#!$@#
VkrDf0FVYbg7.,2/cW:@#$sss@#!$@#
hWTRS2IF9EOQc,2/mW:@#$@aaaa#!$@#
OrfQo.EYJo9q6:@#$35345@#!$@#
2oK927ArLTgnY,B.mW:@#$@23234#!$@#

somefile:
Code:
user:s/1zg6/2RWAUY,2/lR:user:::::
user:JQy74TuvHyqMo:user:::::
user:32oVGmESDPBLk:1162:user:::::
user:VkrDf0FVYbg7.,2/cW:user:::::
user:hWTRS2IF9EOQc,2/mW:user:::::
user:OrfQo.EYJo9q6:user:::::
user:2oK927ArLTgnY,B.mW:user:::::


using awk:
Code:
 
awk -F: '
NR==FNR {pw[$1]=$2; next;}
{ FS=":";
  for (i=1; i<=NF; i++) {
    if (length(pw[$i])>0) $i=pw[$i];
  }
  print;
}
' OFS=: pwds somefile

output:
Code:
 
user:@#xyyyy$@#!$@#:user:::::
user:@#$@#!xxxxx$@#:user:::::
user:@#$@234234wa#!$@#:1162:user:::::
user:@#$sss@#!$@#:user:::::
user:@#$@aaaa#!$@#:user:::::
user:@#$35345@#!$@#:user:::::
user:@#$@23234#!$@#:user:::::

This User Gave Thanks to rdrtx1 For This Post:
# 3  
Old 10-16-2012
Thanks rdtx1,

This looks promising, except the files I need to work on (e.g. somefile) are not delimited, they are free form database configuration files (see what my ggrep returns on an example file).

I can make my own pwds file (with my own delimiter in case : is used in a password) - but unless my somefile files are delimited (and they are not, they are free form), this doesn't appear to work for me.

Regards,
Bill

Previous Thread | Next Thread
Test Your Knowledge in Computers #271
Difficulty: Easy
While working at Bletchley Park, in the first week of June each year Alan Turing would get a bad attack of hay fever, and he would cycle to the office wearing a service gas mask to keep from breathing pollen. In the first week of June each year he would get a bad attack of hay fever, and he would cycle to the office wearing a service gas mask to keep from breathing the pollen.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Search/Replace in multiple files recursively

Hi there, I am using AIX and trying to search and replace a string with another string in multiple files in different directories. I wanted to search replace in steps so I don't change all of the instance anywhere in the server at once, minimizing impact. STEP 1: -------- I first searched... (5 Replies)
Discussion started by: zaino22
5 Replies

2. Shell Programming and Scripting

Search & Replace in Multiple Files by reading a input file

I have a environment property file which contains: Input file: value1 = url1 value2 = url2 value3 = url3 and so on. I need to search all *.xml files under directory for value1 and replace it with url1. Same thing I have to do for all values mentioned in input file. I need script in unix bash... (7 Replies)
Discussion started by: Shamkamde
7 Replies

3. Shell Programming and Scripting

Search and Replace in multiple files

Hello, I have hundreds of files in which I need to change email address. Here is what I am trying to do: 1. All text files are in a directory "a" 2. In the text file, I want to replace email address for preparer. All these lines start with {{PreparerEmail and end with }}. The email... (3 Replies)
Discussion started by: cartrider
3 Replies

4. Shell Programming and Scripting

perl: search replace in multiple files

When I use special characters the command to replace multiple files with a string pattern does nt work. ---------- Post updated at 12:33 PM ---------- Previous update was at 11:38 AM ---------- This works perl -pi -e 's/100/test/g' * This does nt work perl -pi -e 's... (1 Reply)
Discussion started by: w020637
1 Replies

5. Shell Programming and Scripting

Script to find & replace a multiple lines string across multiple php files and subdirectories

Hey guys. I know pratically 0 about Linux, so could anyone please give me instructions on how to accomplish this ? The distro is RedHat 4.1.2 and i need to find and replace a multiple lines string in several php files across subdirectories. So lets say im at root/dir1/dir2/ , when i execute... (12 Replies)
Discussion started by: spfc_dmt
12 Replies

6. Shell Programming and Scripting

awk + gsub to search multiple input values & replace with located string + extra text

Hi all. I have the following command that is successfully searching for any one of the strings on all lines of a file and replacing it with the instructed value. cat inputFile | awk '{gsub(/aaa|bbb|ccc|ddd/,"1234")}1' > outputFile This does in fact replace any occurrence of aaa, bbb,... (2 Replies)
Discussion started by: dazhoop
2 Replies

7. UNIX for Dummies Questions & Answers

how to find and replace strings in multiple files

Hi All, Iam new to unix, I need to find string and replace it in the file name. Like text_123_0.txt,text_123_1.txt,text_123_2.txt. I need to search 123 and replace it with 234 . Is there any unix command to replace them in single command since i have 5 directories. So i need to go each and every... (0 Replies)
Discussion started by: etldeveloper
0 Replies

8. Shell Programming and Scripting

Global search and replace across multiple files

Hi all I'm in need of a command which can replace a specified string with another string - across multiple files within multiple sub-directories (I intend to run it from / ) I've used the following to get a list of the files: find . | xargs grep <string1> But that's as far as I've got.... (7 Replies)
Discussion started by: huskie69
7 Replies

9. Shell Programming and Scripting

Search & Replace in Multiple Files by reading a input file

Hi, I have a folder which contains multiple config.xml files and one input file, Please see the below format. Config Files format looks like :- Code: <application name="SAMPLE-ARCHIVE"> <NVPairs name="Global Variables"> <NameValuePair> ... (0 Replies)
Discussion started by: haiksuresh
0 Replies

10. Shell Programming and Scripting

Find & Replace string in multiple files & folders using perl

find . -type f -name "*.sql" -print|xargs perl -i -pe 's/pattern/replaced/g' this is simple logic to find and replace in multiple files & folders Hope this helps. Thanks Zaheer (0 Replies)
Discussion started by: Zaheer.mic
0 Replies

Featured Tech Videos