Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Recursivley search files and replace the characters

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Recursivley search files and replace the characters
# 1  
Old 11-14-2014
Recursivley search files and replace the characters
Code:
1st_FILE.cnf
Here is my sample test file [TST_FILE].
This will be replaced by [FILE_NAME].

Code:
2nd_FILE.cnf
This is my 2nd test file [2ND_TST_FILE]..This is agian a test file for [REASON] purpose.


Code:
variable.txt
TST_FILE=1st_FILE.txt
FILE_NAME=1st_FILE.txt
2ND_TST_FILE=2nd_FILE.txt
REASON=test


I have 2 sample config file under directory structure /tmp/test where variables are recognize by [].
So the task is, recursilvely go thru parent dir /tmp/test and find out the files with names *.cnf having placeholders for variables.
it's not necessary that all .cnf files got placeholders.

let's say

Code:
/tmp/test/confa/1st_FILE.cnf
/tmp/test/confa/readme.1stfile.txt
/tmp/test/confa/3rd_FILE.cnf 

/tmp/test/confb/2nd_FILE.cnf
/tmp/test/confb/readme.2ndfile.txt

so it will only list /tmp/test/confa/1st_FILE.cnf and /tmp/test/confb/2nd_FILE.cnf where placeholders are there.
not /tmp/test/confa/3rd_FILE.cnf ,
Then it will go each file one by one and replace each place holder to the actual ones based on file /tmp/test/variable.txt

e.g.

after replacement final look should look like
Code:
/tmp/test/confa/1st_FILE.cnf.out

Here is my sample test file 1st_FILE.txt.
This will be replaced by 1st_FILE.txt.

Code:
/tmp/test/confb/2nd_FILE.cnf.out

This is my 2nd test file 2nd_FILE.txt..This is agian a test file for test purpose.

I know, how to find the files recursively that matches [].
but my question is how can I bind variables from variable.txt to the filename that matched.
Code:
find . -type f -exec grep -lIi "\[" {} \; | while read name
> do
> echo $name
> done

--> this will display the files with []

now how can I loop these files with variable.txt and replace the variables with actual values.

any ideas????
Last edited by manas_ranjan; 11-14-2014 at 09:17 AM..
# 2  
Old 11-14-2014
If you have a list of those files, e.g. in a variable, you could try sth like (untested)
Code:
awk 'NR==FNR {R[$1]=$2; next} {for (r in R) gsub ("["r"]", R[r]); print > FILENAME".new"}' FS="=" variable.txt $filelist

# 3  
Old 11-14-2014
Thanks Rudi.
however this is not working

my o/p looks like testing with one config file

Code:
cat 1st_FILE.cnf.new
iHere is my sample test file [1st1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txttest.txt1st1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txttest.txt1st1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txttest.txt1st1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txttest.txt1st1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txttest.txt1st1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txttest.txt1st1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txttest.txt1st1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txttest.txt].
1st1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txttest.txthis will be replaced by [1st1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txttest.txt1st1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txttest.txt1st1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txttest.txt1st1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txttest.txt1st1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txttest.txttesttest1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txt1st2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt2nd_FILE.txt.txttest.txt].

so it takes all the variables and replace it within [] for multiple times.
also not removing [].
Code:
cat ./1st_FILE.cnf
iHere is my sample test file [TST_FILE].
This will be replaced by [FILE_NAME].

Code:
cat config.txt
TST_FILE=1st_FILE.txt
FILE_NAME=1st_FILE.txt
2ND_TST_FILE=2nd_FILE.txt
REASON=test

# 4  
Old 11-14-2014
I didn't think it was THAT picky. Try
Code:
awk 'NR==FNR {R[$1]=$2; next} {for (r in R) gsub ("\["r"\]", R[r]); print > FILENAME".new"}' FS="=" variable.txt $filelist

# 5  
Old 11-14-2014
I'm sorry , I already tried "\[" in AWK, but it throws me below error


Code:
awk 'NR==FNR {R[$1]=$2; next} {for (r in R) gsub ("\["r"\]", R[r]); print > FILENAME".new"}' FS="=" config.txt 1st_FILE.cnf
awk: warning: escape sequence `\[' treated as plain `['
awk: warning: escape sequence `\]' treated as plain `]'

However,it creates the same file with all the variables (although it's showing warning.)
# 6  
Old 11-14-2014
This is my result with above snippet and samples:
Code:
iHere is my sample test file 1st_FILE.txt.
This will be replaced by 1st_FILE.txt.

The problem is awk regards "[" and "]" as special chars to build a regex. You need to somehow persuade it to accept the escaped "[" and "]".

---------- Post updated at 16:45 ---------- Previous update was at 16:43 ----------

OK, got this one:
Code:
awk 'NR==FNR {R[$1]=$2; next} {for (r in R) gsub ("[[]"r"[]]", R[r]); print}' FS="=" file1 file2
iHere is my sample test file 1st_FILE.txt.
This will be replaced by 1st_FILE.txt.

Give it a shot...
This User Gave Thanks to RudiC For This Post:
manas_ranjan
# 7  
Old 11-14-2014
thanks, it's now working fine.
Last edited by manas_ranjan; 11-14-2014 at 12:10 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Search special characters in a file and replace with meaningful text messages like Hello

Search special characters in a file and replace with meaningful text messages like Hello (2 Replies)
Discussion started by: raka_rjit
2 Replies

2. Shell Programming and Scripting

Search and Replace Extended Ascii Characters

We are getting extended Ascii characters in the input file and my requirement is to search and replace them with a space. I am using the following command LANG=C sed -e 's// /g' It is doing a good job, but in some cases it is replacing the extended characters with two spaces. So my input... (12 Replies)
Discussion started by: ysvsr1
12 Replies

3. Shell Programming and Scripting

search pattern and replace x-y characters in nth line after every match

Hi, I am looking for any script which can do the following. have to read a pattern from fileA and copy it to fileB. fileA: ... ... Header ... ... ..p1 ... ... fileB: .... .... Header (3 Replies)
Discussion started by: anilvk
3 Replies

4. Shell Programming and Scripting

Search and replace particular characters in fixed length-file

Masters, I have fixed length input file like FHEAD0000000001XXXX20090901 0000009000Y1000XXX2 THEAD000000000220090901 ITM0000109393813 430143504352N22SP 000000000000RN000000010000EA P0000000000000014390020090901 TTAIL0000000003000000 FTAIL00000000040000000002 Note... (4 Replies)
Discussion started by: bittoo
4 Replies

5. Shell Programming and Scripting

Replace special characters in multiple files - perl

I have 100 files, where i want to search a set of strings and make the replacement by other strings In the first case I want to include a parameter in the name of a file LOG_DCT = $ LOG_DIR/DCT_GERAL_"$DATAINI".log replace to : LOG_DCT = $ LOG_DIR / DCT_GERAL_ $ 1_ "$ DATAINI". log I did... (1 Reply)
Discussion started by: RMSoares
1 Replies

6. Shell Programming and Scripting

Search and Replace between two files

Hi all- I've got 2 files: One is the final results and one is a result set from a query. In the final results files I have placeholder strings in there that need to be replaced by the corresponding strings from the query file. So File#1 (FINAL RESULTS) LINEID CLIENT ID REP ... (1 Reply)
Discussion started by: Cailet
1 Replies

7. Shell Programming and Scripting

search and replace characters in one string

I have lines like: Dog Cat House Mouse Dog Cat House Mouse Dog Cat House Mouse Dog Cat House Mouse I'd like to replace characters only in $3. H -> Z s -> W e -> x Resulting in something like (where $1, $2, and $4 are not changed): Dog Cat ZouWx Mouse Dog Cat ZouWx Mouse... (3 Replies)
Discussion started by: dcfargo
3 Replies

8. Shell Programming and Scripting

Replace Characters for bunch of Files.

Hi, I am new to unix and looking out for some help in reading a file contents and replacing the characters, the requirement is I having a folder and having nearly 300 txt files, all the file contents contains some words we need to iterate all each and every files and need to find and replace it... (1 Reply)
Discussion started by: subrahmaniank
1 Replies

9. Shell Programming and Scripting

Search For Control M characters in files

Hi , I have special character control M in many of my files as below ersNet-Telnet-3.03/Makefile.PL100644 21166 144 612 7113770214 135 77 0ustar jayusers## -*- Perl -*-^M ^M use ExtUtils::MakeMaker qw(WriteMakefile);^M ^M WriteMakefile(NAME => "Net::Telnet",^M ... (4 Replies)
Discussion started by: Mohammed
4 Replies

10. Shell Programming and Scripting

Best way to search files for non-printable characters?

I need to check ftp'd incoming files for characters that are not alphanumeric,<tab>, <cr>, or <lf> characters. Each file would have 10-20,000 line with up to 3,000 characters per line. Should I use awk, sed, or grep and what would the command look like to do such a search? Thanks much to anyone... (2 Replies)
Discussion started by: jvander
2 Replies
Login or Register to Ask a Question