Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Find/replace in file from another file.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Find/replace in file from another file.
# 1  
Old 09-03-2014
Find/replace in file from another file.
Dear Shell Expert,
I have requirement of replacing passwd field in number of files based on username in them.
The matching username and "new" password in available in a separate file. below is the sample for both.

#sample of one of the file content, red highlighted field are username and password.
Code:
<?xml version='1.0' encoding='UTF-8'?>
<jdbc-data-source xmlns="http://xmlns.oracle.com/weblogic/jdbc-data-source" xmlns:sec="http://xmlns.oracle.com/weblogic/security" xmlns:wls="http://xmlns.oracle.com/weblogic/security/wls" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.oracle.com/weblogic/jdbc-data-source http://xmlns.oracle.com/weblogic/jdbc-data-source/1.2/jdbc-data-source.xsd">
  <name>user_1_app_DS</name>
  <jdbc-driver-params>
    <url>jdbc:oracle:thin:@pocof-scan:1520/DEV</url>
    <driver-name>oracle.jdbc.xa.client.OracleXADataSource</driver-name>
    <properties>
      <property>
        <name>user</name>
        <value>user_1_app</value>
      </property>
    </properties>
    <password-encrypted>{AES}y11NPEB1Z1B3Qr+0S5j5e7OUJ2Gmz/P1Y+6auN215sY=</password-encrypted>
  </jdbc-driver-params>
  <jdbc-connection-pool-params>
    <initial-capacity>10</initial-capacity>
    <max-capacity>100</max-capacity>
    <test-table-name>SQL SELECT 1 FROM DUAL</test-table-name>
    <statement-cache-size>100</statement-cache-size>
  </jdbc-connection-pool-params>
  <jdbc-data-source-params>
    <jndi-name>jdbc/user_1_app_DS</jndi-name>
    <global-transactions-protocol>TwoPhaseCommit</global-transactions-protocol>
  </jdbc-data-source-params>
  <jdbc-xa-params>
    <xa-transaction-timeout>180</xa-transaction-timeout>
  </jdbc-xa-params>
</jdbc-data-source>

# sample of username/passwd file.
Code:
user_1_app	{AES}333paUEqje9PJXuoxRreB42dTkejdudHavfDZDhyDUd7AJLkyFfOu0VYxjg47tNz
user_2_db	{AES}333BmY0PIh/GOy4iqFEr4dXLppsT3cpSU0j9ylIG3YY=
john_3_cat	{AES}M/333+JKrlDsnFdHTsO+A5sEKlkR5IWDdG0IrNobggQ=
boby_5_dd	{AES}333NPEB1Z1B3Qr+0S5j5e7OUJ2Gmz/P1Y+6auN215sY=

----username and password is separated by "tab"


can you please help?
Last edited by KDDubai333; 09-04-2014 at 01:26 AM.. Reason: updating passwd file. (Instead of $ separated by tab)
# 2  
Old 09-03-2014
Hello KDDubai333,

Following may help you in same.

Code:
awk -vs1="<password-encrypted>" -vs2="</password-encrypted>" 'NR==FNR{a[$1]=$4;next} {{if($0 ~ /<value>/){v=$0;gsub(/value/,X,v);gsub(/^[[:space:]]+/,X,v);gsub(/[<>/]/,X,v);print $0}} {if($0 ~ s1){$0=s1 a[v] s2}} {if($0 !~ /<value>/) {print $0}}}' FS="$"   paaswd_file input_file

Output will be as follows.

Code:
<?xml version='1.0' encoding='UTF-8'?>
<jdbc-data-source xmlns="Schema Versions for http://xmlns.oracle.com/weblogic/jdbc-data-source" xmlns:sec="Schema Versions for http://xmlns.oracle.com/weblogic/security" xmlns:wls="Schema Versions for http://xmlns.oracle.com/weblogic/security/wls" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="Schema Versions for http://xmlns.oracle.com/weblogic/jdbc-data-source http://xmlns.oracle.com/weblogic/jdb...ata-source.xsd">
  <name>user_1_app_DS</name>
  <jdbc-driver-params>
    <url>jdbc:oracle:thin:@pocof-scan:1520/DEV</url>
    <driver-name>oracle.jdbc.xa.client.OracleXADataSource</driver-name>
    <properties>
      <property>
        <name>user</name>
        <value>user_2_db</value>
      </property>
    </properties>
<password-encrypted>{AES}QKfBmY0PIh/GOy4iqFEr4dXLppsT3cpSU0j9ylIG3YY=</password-encrypted>
  </jdbc-driver-params>
  <jdbc-connection-pool-params>
    <initial-capacity>10</initial-capacity>
    <max-capacity>100</max-capacity>
    <test-table-name>SQL SELECT 1 FROM DUAL</test-table-name>
    <statement-cache-size>100</statement-cache-size>
  </jdbc-connection-pool-params>
  <jdbc-data-source-params>
    <jndi-name>jdbc/user_1_app_DS</jndi-name>
    <global-transactions-protocol>TwoPhaseCommit</global-transactions-protocol>
  </jdbc-data-source-params>
  <jdbc-xa-params>
    <xa-transaction-timeout>180</xa-transaction-timeout>
  </jdbc-xa-params>
</jdbc-data-source>

Where Input file is as follows.

Code:
<?xml version='1.0' encoding='UTF-8'?>
<jdbc-data-source xmlns="Schema Versions for http://xmlns.oracle.com/weblogic/jdbc-data-source" xmlns:sec="Schema Versions for http://xmlns.oracle.com/weblogic/security" xmlns:wls="Schema Versions for http://xmlns.oracle.com/weblogic/security/wls" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="Schema Versions for http://xmlns.oracle.com/weblogic/jdbc-data-source http://xmlns.oracle.com/weblogic/jdb...ata-source.xsd">
  <name>user_1_app_DS</name>
  <jdbc-driver-params>
    <url>jdbc:oracle:thin:@pocof-scan:1520/DEV</url>
    <driver-name>oracle.jdbc.xa.client.OracleXADataSource</driver-name>
    <properties>
      <property>
        <name>user</name>
        <value>user_2_db</value>
      </property>
    </properties>
    <password-encrypted>{AES}y11NPEB1Z1B3Qr+0S5j5e7OUJ2Gmz/P1Y+6auN215sY=</password-encrypted>
  </jdbc-driver-params>
  <jdbc-connection-pool-params>
    <initial-capacity>10</initial-capacity>
    <max-capacity>100</max-capacity>
    <test-table-name>SQL SELECT 1 FROM DUAL</test-table-name>
    <statement-cache-size>100</statement-cache-size>
  </jdbc-connection-pool-params>
  <jdbc-data-source-params>
    <jndi-name>jdbc/user_1_app_DS</jndi-name>
    <global-transactions-protocol>TwoPhaseCommit</global-transactions-protocol>
  </jdbc-data-source-params>
  <jdbc-xa-params>
    <xa-transaction-timeout>180</xa-transaction-timeout>
  </jdbc-xa-params>
</jdbc-data-source>

kindly let me know if you have any firther queries.


Thanks,
R. Singh
Last edited by RavinderSingh13; 09-03-2014 at 08:40 AM.. Reason: Changed code a bit to get better output
# 3  
Old 09-04-2014
Thanks Ravinder for your reply.
I faced below problems with your solution.

1) the awk output is changing when similar other file is supplied.
Code:
#executed on other similar file.

[user@host DS-change]$ awk -vs1="<password-encrypted>" -vs2="</password-encrypted>" 'NR==FNR{a[$1]=$4;next} {{if($0 ~ /<value>/){v=$0;gsub(/value/,X,v);gsub(/^[[:space:]]+/,X,v);gsub(/[<>/]/,X,v);print $0}} {if($0 ~ s1){$0=s1 a[v] s2}} {if($0 !~ /<value>/) {print $0}}}' FS="$"   input.txt file2.xml
<?xml version='1.0' encoding='UTF-8'?>
<jdbc-data-source xmlns="http://xmlns.oracle.com/weblogic/jdbc-data-source" xmlns:sec="http://xmlns.oracle.com/weblogic/security" xmlns:wls="http://xmlns.oracle.com/weblogic/security/wls" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.oracle.com/weblogic/jdbc-data-source http://xmlns.oracle.com/weblogic/jdbc-data-source/1.2/jdbc-data-source.xsd">
  <name>MY_APP_DS</name>
  <jdbc-driver-params>
    <url>jdbc:oracle:thin:@poc-scan:1520:DEV</url>
    <driver-name>oracle.jdbc.xa.client.OracleXADataSource</driver-name>
    <properties>
      <property>
        <name>user</name>
        <value>MY_APP</value>
      </property>
    </properties>
<password-encrypted></password-encrypted>
  </jdbc-driver-params>
  <jdbc-connection-pool-params>
    <test-table-name>SQL SELECT 1 FROM DUAL</test-table-name>
  </jdbc-connection-pool-params>
  <jdbc-data-source-params>
    <jndi-name>jdbc/MY_APP_DS</jndi-name>
    <global-transactions-protocol>TwoPhaseCommit</global-transactions-protocol>
  </jdbc-data-source-params>
</jdbc-data-source>

2) the awk output on above posted sample file also not the expected one. it just shows <password-encrypted></password-encrypted>


3) Output is displayed on console, whereas i want the input file to be updated with latest password. (taken from passwd input file based on user match).


can you please advise.

---------- Post updated at 08:24 AM ---------- Previous update was at 08:13 AM ----------

For another requirement, i am going to modify password file as below.
Code:
user_1_app	{AES}333paUEqje9PJXuoxRreB42dTkejdudHavfDZDhyDUd7AJLkyFfOu0VYxjg47tNz
user_2_db	{AES}333BmY0PIh/GOy4iqFEr4dXLppsT3cpSU0j9ylIG3YY=
john_3_cat	{AES}M/333+JKrlDsnFdHTsO+A5sEKlkR5IWDdG0IrNobggQ=
boby_5_dd	{AES}333NPEB1Z1B3Qr+0S5j5e7OUJ2Gmz/P1Y+6auN215sY=

Last edited by KDDubai333; 09-04-2014 at 01:44 AM.. Reason: data masking :)
# 4  
Old 09-04-2014
Code:
awk 'NR == FNR{split($0, t, /[$][$][$]/); a[t[1]] = t[2]; next} /<value>/ {split($0, t, />|</); v=t[3]} /<password-encrypted>/ {sub(/>.*<\//, ">" a[v] "</")}1' passwd file

for the modified passwd file,
Code:
awk 'NR == FNR{a[$1] = $2; next} /<value>/ {split($0, t, />|</); v=t[3]} /<password-encrypted>/ {sub(/>.*<\//, ">" a[v] "</")}1' passwd file

This User Gave Thanks to SriniShoo For This Post:
KDDubai333
# 5  
Old 09-04-2014
Hi Srinishoo, seems to be partially working...I faced below problem.
<password-encrypted> starting tag changed to closed tag (</password-encrypted>)...and closed tag is missing.
My OS : Linux hostname 2.6.39-200.24.1.el6uek.x86_64

#execution logs
Code:
[user@host DS-change]$ cat File.xml 
<?xml version='1.0' encoding='UTF-8'?>
<jdbc-data-source xmlns="http://xmlns.oracle.com/weblogic/jdbc-data-source" xmlns:sec="http://xmlns.oracle.com/weblogic/security" xmlns:wls="http://xmlns.oracle.com/weblogic/security/wls" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.oracle.com/weblogic/jdbc-data-source http://xmlns.oracle.com/weblogic/jdbc-data-source/1.2/jdbc-data-source.xsd">
  <name>user_1_app_DS</name>
  <jdbc-driver-params>
    <url>jdbc:oracle:thin:@pocof-scan:1520/DEV</url>
    <driver-name>oracle.jdbc.xa.client.OracleXADataSource</driver-name>
    <properties>
      <property>
        <name>user</name>
        <value>user_1_app</value>
      </property>
    </properties>
    <password-encrypted>{AES}y11NPEB1Z1B3Qr+0S5j5e7OUJ2Gmz/P1Y+6auN215sY=</password-encrypted>
  </jdbc-driver-params>
  <jdbc-connection-pool-params>
    <initial-capacity>10</initial-capacity>
    <max-capacity>100</max-capacity>
    <test-table-name>SQL SELECT 1 FROM DUAL</test-table-name>
    <statement-cache-size>100</statement-cache-size>
  </jdbc-connection-pool-params>
  <jdbc-data-source-params>
    <jndi-name>jdbc/user_1_app_DS</jndi-name>
    <global-transactions-protocol>TwoPhaseCommit</global-transactions-protocol>
  </jdbc-data-source-params>
  <jdbc-xa-params>
    <xa-transaction-timeout>180</xa-transaction-timeout>
  </jdbc-xa-params>
</jdbc-data-source>
[user@host DS-change]$ cat passwd.txt
user_1_app	{AES}333paUEqje9PJXuoxRreB42dTkejdudHavfDZDhyDUd7AJLkyFfOu0VYxjg47tNz
user_2_db	{AES}333BmY0PIh/GOy4iqFEr4dXLppsT3cpSU0j9ylIG3YY=
john_3_cat	{AES}M/333+JKrlDsnFdHTsO+A5sEKlkR5IWDdG0IrNobggQ=
boby_5_dd	{AES}333NPEB1Z1B3Qr+0S5j5e7OUJ2Gmz/P1Y+6auN215sY=
[user@host DS-change]$ awk 'NR == FNR{a[$1] = $2; next} /<value>/ {split($0, t, />|</); v=t[3]} /<password-encrypted>/ {sub(/>.*<\//, ">" a[v] "</")}1' passwd.txt File.xml
<?xml version='1.0' encoding='UTF-8'?>
<jdbc-data-source xmlns="http://xmlns.oracle.com/weblogic/jdbc-data-source" xmlns:sec="http://xmlns.oracle.com/weblogic/security" xmlns:wls="http://xmlns.oracle.com/weblogic/security/wls" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.oracle.com/weblogic/jdbc-data-source http://xmlns.oracle.com/weblogic/jdbc-data-source/1.2/jdbc-data-source.xsd">
  <name>user_1_app_DS</name>
  <jdbc-driver-params>
    <url>jdbc:oracle:thin:@pocof-scan:1520/DEV</url>
    <driver-name>oracle.jdbc.xa.client.OracleXADataSource</driver-name>
    <properties>
      <property>
        <name>user</name>
        <value>user_1_app</value>
      </property>
    </properties>
</password-encrypted>ed>{AES}333paUEqje9PJXuoxRreB42dTkejdudHavfDZDhyDUd7AJLkyFfOu0VYxjg47tNz
  </jdbc-driver-params>
  <jdbc-connection-pool-params>
    <initial-capacity>10</initial-capacity>
    <max-capacity>100</max-capacity>
    <test-table-name>SQL SELECT 1 FROM DUAL</test-table-name>
    <statement-cache-size>100</statement-cache-size>
  </jdbc-connection-pool-params>
  <jdbc-data-source-params>
    <jndi-name>jdbc/user_1_app_DS</jndi-name>
    <global-transactions-protocol>TwoPhaseCommit</global-transactions-protocol>
  </jdbc-data-source-params>
  <jdbc-xa-params>
    <xa-transaction-timeout>180</xa-transaction-timeout>
  </jdbc-xa-params>
</jdbc-data-source>
[user@host DS-change]$

# 6  
Old 09-04-2014
Posted by KDDubai333

Quote:
Thanks Ravinder for your reply.
I faced below problems with your solution.

1) the awk output is changing when similar other file is supplied.

Code:
#executed on other similar file.[user@host DS-change]$ awk -vs1="<password-encrypted>" -vs2="</password-encrypted>" 'NR==FNR{a[$1]=$4;next} {{if($0 ~ /<value>/){v=$0;gsub(/value/,X,v);gsub(/^[[:space:]]+/,X,v);gsub(/[<>/]/,X,v);print $0}} {if($0 ~ s1){$0=s1 a[v] s2}} {if($0 !~ /<value>/) {print $0}}}' FS="$" input.txt file2.xml<?xml version='1.0' encoding='UTF-8'?><jdbc-data-source xmlns="http://xmlns.oracle.com/weblogic/jdbc-data-source" xmlns:sec="http://xmlns.oracle.com/weblogic/security" xmlns:wls="http://xmlns.oracle.com/weblogic/security/wls" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.oracle.com/weblogic/jdbc-data-source http://xmlns.oracle.com/weblogic/jdbc-data-source/1.2/jdbc-data-source.xsd"> <name>MY_APP_DS</name> <jdbc-driver-params> <url>jdbcSmilieracle:thin:@poc-scan:1520SmilieEV</url> <driver-name>oracle.jdbc.xa.client.OracleXADataSource</driver-name> <properties> <property> <name>user</name> <value>MY_APP</value> </property> </properties><password-encrypted></password-encrypted> </jdbc-driver-params> <jdbc-connection-pool-params> <test-table-name>SQL SELECT 1 FROM DUAL</test-table-name> </jdbc-connection-pool-params> <jdbc-data-source-params> <jndi-name>jdbc/MY_APP_DS</jndi-name> <global-transactions-protocol>TwoPhaseCommit</global-transactions-protocol> </jdbc-data-source-params></jdbc-data-source>

2) the awk output on above posted sample file also not the expected one. it just shows <password-encrypted></password-encrypted>


3) Output is displayed on console, whereas i want the input file to be updated with latest password. (taken from passwd input file based on user match).


can you please advise.

---------- Post updated at 08:24 AM ---------- Previous update was at 08:13 AM ----------

For another requirement, i am going to modify password file as below.


Code:
user_1_app {AES}333paUEqje9PJXuoxRreB42dTkejdudHavfDZDhyDUd7AJLkyFfOu0VYxjg47tNzuser_2_db {AES}333BmY0PIh/GOy4iqFEr4dXLppsT3cpSU0j9ylIG3YY=john_3_cat {AES}M/333+JKrlDsnFdHTsO+A5sEKlkR5IWDdG0IrNobggQ=boby_5_dd {AES}333NPEB1Z1B3Qr+0S5j5e7OUJ2Gmz/P1Y+6auN215sY=
Hello KDDubai333,

I have taken input file as per your 1st post where user entry MY_APP is not present in .txt file so that is why it is giving no value there. Please try with those values which are present in .txt file it will work. Let me know if you have any queries meanwhile I can try another ways also to complete this.


Thanks,
R. Singh
# 7  
Old 09-04-2014
Try this
Code:
awk 'NR == FNR{a[$1] = $2; next} /<value>/ {split($0, t, />|</); v=t[3]} /<password-encrypted>/ && a[v] {sub(/<password-encrypted>.*<\/password-encrypted>/, ("<password-encrypted>" a[v] "</password-encrypted>"))}1' passwd file

With this code, if the entries are not available in 'passwd' file, it will leave the encrypted password unchanged
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find and replace in a file from another file

Hello, I am writing a phonetic converter for French from written French to IPA. French has the convention of putting an apostrophe and joining 2 words if the first word ends in an e. l'homme d'air s'est Loading all such words in my dictionary just over loads the database. Apart from this... (6 Replies)
Discussion started by: gimley
6 Replies

2. UNIX for Beginners Questions & Answers

Need to find and replace in a file

Hi All, I am having below sample data in a file. I need to find all the line form this file with word ABC and i need to replace the characters at position 120 which is "CO:BOGFDUI"(30chars) in the lines with blank space. I have tried using grep to find the word with ABC (grep ABC filename),... (3 Replies)
Discussion started by: abhi_123
3 Replies

3. Shell Programming and Scripting

Perl script to read string from file#1 and find/replace in file#2

Hello Forum. I have a file called abc.sed with the following commands; s/1/one/g s/2/two/g ... I also have a second file called abc.dat and would like to substitute all occurrences of "1 with one", "2 with two", etc and create a new file called abc_new.dat sed -f abc.sed abc.dat >... (10 Replies)
Discussion started by: pchang
10 Replies

4. Shell Programming and Scripting

Find and Replace in File

Legends, I have a file /tmp/list.txt I want to find "/bin/" and replace it with "/log/" I tried the follwoing but no luck Sandy: /tmp> perl -pi -e 's/\/bin\/\/log\/' /tmp/list.txt >> /tmp/try Substitution pattern not terminated at -e line 1. AND, Sandy: /tmp> perl -pi -e... (2 Replies)
Discussion started by: sdosanjh
2 Replies

5. Shell Programming and Scripting

How to find a certain string in a file and replace it with a value from another file using sed/awk?

Hi Everyone, I am new to this forum and new to sed/awk programming too !! I need to find particular string in file1(text file) and replace it with a value from another text file(file2) the file2 has only one line and the value to be replaced with is in the second column. file 1: (assert (=... (21 Replies)
Discussion started by: paramad
21 Replies

6. Shell Programming and Scripting

find and replace a string in a file without the use of temp file

Hi - I am looking for a replacing a string in a in multiple *.sql files in directory with a new string without using a temporary file Normally I can use sed command as below for W in ls `FILE*.sql` do sed 's/OLD/NEW/g' $W > TEMPFILE.dat mv TEMPFILE.dat $W done But Here in my... (9 Replies)
Discussion started by: raghutapal
9 Replies

7. Shell Programming and Scripting

Find and replace in a gz file

Is there a way to do a find and replace in a .gz file in a single script ? I can always unzip, find and replace and then zip it again but would hate to do this everytime. Thanks ! Vivek (1 Reply)
Discussion started by: vashah
1 Replies

8. Shell Programming and Scripting

Find and replace in a file

Hi everyone, I am new to the world of shell script programming. I have a file named Fnd1.txt which has the contents as below. I need to replace the \t with the tab space. Can any one help me to write a perl scipt for this. USA45V1\tG\t341029 USAV1T1\tG\t450545 USAREJ1\tG\t572645... (5 Replies)
Discussion started by: vinay123
5 Replies

9. Shell Programming and Scripting

find and replace pattren in file

Hi, I have the input file having data as follow: file1.txt 001 aaa_1:abcd 002 bbb_2:abcd I want output as, 001xabcd 002xabcd Here iam trying to replace "{1 space}{alphanumeric string with underscore}{:}" with characrter "x". I tried to achieve this using sed;but Iam not getting this... (5 Replies)
Discussion started by: gopalss
5 Replies

10. UNIX for Dummies Questions & Answers

Find replace within a file?

I build several files by using the cut command to grab select fields(columns) from a really bid csv file. Each file is one column of data. I then put them together using paste command. Here is the code built in tcsh: cut -d , -f 1 some.csv > 1.csv cut -d , -f 10 some.csv > 10.csv paste 1.csv... (2 Replies)
Discussion started by: yankee428
2 Replies
Login or Register to Ask a Question