Visit Our UNIX and Linux User Community


Search & Replace: Multiple Strings / Multiple Files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Search & Replace: Multiple Strings / Multiple Files
# 1  
Old 10-16-2012
Data Search & Replace: Multiple Strings / Multiple Files

I have a list of files all over a file system e.g.

Code:
/home/1/foo/bar.x
/www/sites/moose/foo.txt

I'm looking for strings in these files and want to replace each occurrence with a replacement string, e.g.

if I find: '#@!^\&@ in any of the files I want to replace it with: 655#@11, etc.

There are around 120 substitutions and around 4,600 files.

One issue: these strings are passwords, I'm hoping not to need to escape them.

So far I've only come up with:

Code:
ggrep -HF -f /home/pass "/home/find-passwords/struc/www/sites/foo.php"

which doesn't do much except list the file and matched string from pass (which contains old passwords),

no escaping is required due to the -F arg to grep.

e.g.

Code:
ggrep -HF -f /home/pass "/home/find-passwords/struc/www/sites/foo.php"

returns:

Code:
/home/find-passwords/struc/www/sites/foo.php:$dbcnx = @mysql_pconnect("blah.com", "foo", "xfgt^5");

where xfgt^5 is the match from my /home/pass file.

I'm struggling with how to replace the old password with the new, and thinking loops with a bunch of ifs might work? Or a lookup table?

I can put the old and new passwords in a file with corresponding columns.

It would be nice to be able to run in a manner that shows what it will do, and then one that does it.

Thanks for any help,

Bill

Last edited by spacegoose; 10-17-2012 at 09:52 AM..
# 2  
Old 10-16-2012
If working with files that are ":" field delimited:

Code:
 
pwds file (pwd:replace pwd):
s/1zg6/2RWAUY,2/lR:@#xyyyy$@#!$@#
JQy74TuvHyqMo:@#$@#!xxxxx$@#
32oVGmESDPBLk:@#$@234234wa#!$@#
VkrDf0FVYbg7.,2/cW:@#$sss@#!$@#
hWTRS2IF9EOQc,2/mW:@#$@aaaa#!$@#
OrfQo.EYJo9q6:@#$35345@#!$@#
2oK927ArLTgnY,B.mW:@#$@23234#!$@#

somefile:
Code:
user:s/1zg6/2RWAUY,2/lR:user:::::
user:JQy74TuvHyqMo:user:::::
user:32oVGmESDPBLk:1162:user:::::
user:VkrDf0FVYbg7.,2/cW:user:::::
user:hWTRS2IF9EOQc,2/mW:user:::::
user:OrfQo.EYJo9q6:user:::::
user:2oK927ArLTgnY,B.mW:user:::::


using awk:
Code:
 
awk -F: '
NR==FNR {pw[$1]=$2; next;}
{ FS=":";
  for (i=1; i<=NF; i++) {
    if (length(pw[$i])>0) $i=pw[$i];
  }
  print;
}
' OFS=: pwds somefile

output:
Code:
 
user:@#xyyyy$@#!$@#:user:::::
user:@#$@#!xxxxx$@#:user:::::
user:@#$@234234wa#!$@#:1162:user:::::
user:@#$sss@#!$@#:user:::::
user:@#$@aaaa#!$@#:user:::::
user:@#$35345@#!$@#:user:::::
user:@#$@23234#!$@#:user:::::

This User Gave Thanks to rdrtx1 For This Post:
# 3  
Old 10-16-2012
Thanks rdtx1,

This looks promising, except the files I need to work on (e.g. somefile) are not delimited, they are free form database configuration files (see what my ggrep returns on an example file).

I can make my own pwds file (with my own delimiter in case : is used in a password) - but unless my somefile files are delimited (and they are not, they are free form), this doesn't appear to work for me.

Regards,
Bill

Previous Thread | Next Thread
Test Your Knowledge in Computers #284
Difficulty: Easy
AIX (Advanced Interactive eXecutive) is a series of proprietary Unix operating systems developed and sold by Digital Equipment Corporation (DEC) for several of its computer platforms.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Search/Replace in multiple files recursively

Hi there, I am using AIX and trying to search and replace a string with another string in multiple files in different directories. I wanted to search replace in steps so I don't change all of the instance anywhere in the server at once, minimizing impact. STEP 1: -------- I first searched... (5 Replies)
Discussion started by: zaino22
5 Replies

2. Shell Programming and Scripting

Search & Replace in Multiple Files by reading a input file

I have a environment property file which contains: Input file: value1 = url1 value2 = url2 value3 = url3 and so on. I need to search all *.xml files under directory for value1 and replace it with url1. Same thing I have to do for all values mentioned in input file. I need script in unix bash... (7 Replies)
Discussion started by: Shamkamde
7 Replies

3. Shell Programming and Scripting

Search and Replace in multiple files

Hello, I have hundreds of files in which I need to change email address. Here is what I am trying to do: 1. All text files are in a directory "a" 2. In the text file, I want to replace email address for preparer. All these lines start with {{PreparerEmail and end with }}. The email... (3 Replies)
Discussion started by: cartrider
3 Replies

4. Shell Programming and Scripting

perl: search replace in multiple files

When I use special characters the command to replace multiple files with a string pattern does nt work. ---------- Post updated at 12:33 PM ---------- Previous update was at 11:38 AM ---------- This works perl -pi -e 's/100/test/g' * This does nt work perl -pi -e 's... (1 Reply)
Discussion started by: w020637
1 Replies

5. Shell Programming and Scripting

Script to find & replace a multiple lines string across multiple php files and subdirectories

Hey guys. I know pratically 0 about Linux, so could anyone please give me instructions on how to accomplish this ? The distro is RedHat 4.1.2 and i need to find and replace a multiple lines string in several php files across subdirectories. So lets say im at root/dir1/dir2/ , when i execute... (12 Replies)
Discussion started by: spfc_dmt
12 Replies

6. Shell Programming and Scripting

awk + gsub to search multiple input values & replace with located string + extra text

Hi all. I have the following command that is successfully searching for any one of the strings on all lines of a file and replacing it with the instructed value. cat inputFile | awk '{gsub(/aaa|bbb|ccc|ddd/,"1234")}1' > outputFile This does in fact replace any occurrence of aaa, bbb,... (2 Replies)
Discussion started by: dazhoop
2 Replies

7. UNIX for Dummies Questions & Answers

how to find and replace strings in multiple files

Hi All, Iam new to unix, I need to find string and replace it in the file name. Like text_123_0.txt,text_123_1.txt,text_123_2.txt. I need to search 123 and replace it with 234 . Is there any unix command to replace them in single command since i have 5 directories. So i need to go each and every... (0 Replies)
Discussion started by: etldeveloper
0 Replies

8. Shell Programming and Scripting

Global search and replace across multiple files

Hi all I'm in need of a command which can replace a specified string with another string - across multiple files within multiple sub-directories (I intend to run it from / ) I've used the following to get a list of the files: find . | xargs grep <string1> But that's as far as I've got.... (7 Replies)
Discussion started by: huskie69
7 Replies

9. Shell Programming and Scripting

Search & Replace in Multiple Files by reading a input file

Hi, I have a folder which contains multiple config.xml files and one input file, Please see the below format. Config Files format looks like :- Code: <application name="SAMPLE-ARCHIVE"> <NVPairs name="Global Variables"> <NameValuePair> ... (0 Replies)
Discussion started by: haiksuresh
0 Replies

10. Shell Programming and Scripting

Find & Replace string in multiple files & folders using perl

find . -type f -name "*.sql" -print|xargs perl -i -pe 's/pattern/replaced/g' this is simple logic to find and replace in multiple files & folders Hope this helps. Thanks Zaheer (0 Replies)
Discussion started by: Zaheer.mic
0 Replies

Featured Tech Videos