Replace string and append Indicator to end of record in Linux


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Replace string and append Indicator to end of record in Linux
# 1  
Old 09-12-2014
Hammer & Screwdriver Replace string and append Indicator to end of record in Linux

Hi All,

Could you please help me to achieve below solution.

I have a FILE1.txt as below.
Code:
TEXAS CALIFORNIA TEXAS
DALLAS CALIFORNIA CALIFORNIA DALLAS
DALLAS TEXAS TEXAS DALLAS
NEW YORK NEW YORk

FILE2.txt as below.
Code:
TEXAS,TX
DALLAS,DA

Now I need to compare the string in FILE2.txt (first field) record is present in FILE1.txt but more than once then replace the string with 2nd field from FIEL2.txt and if replacement happens only we need to append indicator "|Y"

The output should be as below.
Code:
TEXAS CALIFORNIA TX|Y
DALLAS CALIFORNIA CALIFORNIA DA|Y
DALLAS TX TEXAS DA|Y
NEW YORK NEW YORk

In the above output for last record only we didn't append 'Y' because there is no match from FILE2.txt

Last edited by rbatte1; 09-12-2014 at 01:32 PM.. Reason: Please only wrap commands and data in CodeTags
# 2  
Old 09-12-2014
Hello ureddy,

I have a few to questions pose in response first:-
  • Is this homework/assignment? There are specific forums for these.
  • What have you tried so far?
  • What output/errors do you get?
  • What OS and version are you using?
  • What are your preferred tools? (C, shell, perl, awk, etc.)
  • What logical process have you considered? (to help steer us to follow what you are trying to achieve)
Most importantly, What have you tried so far?

There are probably many ways to achieve most tasks, so giving us an idea of your style and thoughts will help us guide you to an answer most suitable to you so you can adjust it to suit your needs in future.


We're all here to learn and getting the relevant information will help us all.


Kind regards,
Robin
# 3  
Old 09-12-2014
Hi robin,
This is real time problem and not home work.
I want to achieve this in sed or awk as the data is in millions.
I tried sed but it's replacing correctly but appending the string '|Y' for all the records.
I'm using Linux and as this forum is too good in resolving the issues I posted here also I'm not much proficient in unix so seeking for help.
Sorry if my question is confusing you !

Thanks,
Ureddy
# 4  
Old 09-13-2014
Why is line 3
Code:
DALLAS TX TEXAS DA|Y

and not
Code:
DALLAS TEXAS TX DA|Y

?

---------- Post updated at 10:03 ---------- Previous update was at 09:48 ----------

Try
Code:
awk     'NR==FNR        {T[$1]=$2; next}
                        {for (i in T) if (2 <= gsub (i,"&")) {
                                gsub (i, T[i])
                                sub  (T[i], i)
                                $0=$0 "|Y"
                                gsub ("\|Y\|Y","|Y")
                                }
                        }
         1
        ' FS="," file2 file1
TEXAS CALIFORNIA TX|Y
DALLAS CALIFORNIA CALIFORNIA DA|Y
DALLAS TEXAS TX DA|Y
NEW YORK NEW YORk

# 5  
Old 09-13-2014
Code:
Hi,
 
Thanks for looking into this. I corrected the output and here it is.

TEXAS CALIFORNIA TX|Y
DALLAS CALIFORNIA CALIFORNIA DA|Y
DALLAS TEXAS TX DA|Y
NEW YORK NEW YORk

---------- Post updated at 01:49 PM ---------- Previous update was at 01:37 PM ----------

Code:
Hi,

I just executed the script that you provided but iam getting wired output. see below.

|YT|YE|YX|YA|YS|Y |YC|YA|YL|YI|YF|YO|YR|YN|YI|YA|Y |YT|YX|Y||Y|Y                
|YD|YA|YL|YL|YA|YS|Y |YC|YA|YL|YI|YF|YO|YR|YN|YI|YA|Y |YC|YA|YL|YI|YF|YO|YR|YN|Y
I|YA|Y |YD|YA|Y||Y|Y                                                            
|YD|YA|YL|YL|YA|YS|Y |YT|YE|YX|YA|YS|Y |YT|YE|YX|YA|YS|Y |YD|YA|Y||Y|Y          
NEW YORK NEW YORk

# 6  
Old 09-13-2014
Strange... try
Code:
awk     'NR==FNR        {T[$1]=$2; next}
                        {for (i in T) if (2 <= gsub (i,"&")) {
                                gsub (i, T[i])
                                sub  (T[i], i)
                                sub  ("[^Y]$", "&|Y")
                                }
                        }
         1
        ' FS="," file2 file1

This User Gave Thanks to RudiC For This Post:
# 7  
Old 09-13-2014
Hi RudiC,

working perfectly :-) . Thank you so much for this help.

Regards,
Ureddy
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with awk script to append seq num at end of record

Hi Unix forum. I have the following requirement to add a sequence value to each record in a file but only if it meets certain conditions. Field value in pos. 1 and 2 must be '0B' or 'OA' else leave as is. Sequence value must be preserved for each OB and OA pair. Data Before: 123 456... (5 Replies)
Discussion started by: pchang
5 Replies

2. Shell Programming and Scripting

Replace a string for every record after the 1st record

I have data coming in the below format for each record <?xml version="1.0" encoding="UTF-8" standalone="no"?><test_sox xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><testdetials>....</test_sox> <?xml version="1.0" encoding="UTF-8" standalone="no"?><test_sox... (8 Replies)
Discussion started by: dsravanam
8 Replies

3. Shell Programming and Scripting

Append this string to end of each line

Platform: Solaris 10 I have a file like below $ cat languages.txt Spanish Norwegian English Persian German Portugese Chinese Korean Hindi Malayalam Bengali Italian Greek Arabic I want to append the string " is a great language" at end of each line in this file. (3 Replies)
Discussion started by: omega3
3 Replies

4. Shell Programming and Scripting

Search for a string, then append character to end of that line only

I have 2 files that I am working with $ cat file1 server1 server3 server5 server6 server8 $ cat file2 server1;Solaris; server2; SLES; server3;Linux; server4; Solaris; server5;SLES; server6;SLES; server7;Solaris; server8;Linux; (1 Reply)
Discussion started by: snoman1
1 Replies

5. Shell Programming and Scripting

Append the end of each line in a file with a given string

Hi friends, I have a file containing many lines as follows. M:\mmarimut_v6.4.0_pit_01\java\build.xml@@\main\v6.4.0_pit_a M:\mmarimut_v6.4.0_pit_01\ADBasicView.java@@\main\v6.4.0_pit_a I would like to append the string "\0" at the end of each line in the file. The output should look... (10 Replies)
Discussion started by: nmattam
10 Replies

6. Shell Programming and Scripting

Append a string at the end of every line in a file

Hi Friends, I have a file with many lines as shown below. /START SAMPLE LINE/ M:\mmarimut_v6.4.0_pit_01\java\build.xml@@\main\v6.4.0_pit_a M:\mmarimut_v6.4.0_pit_01\port\Post.java@@\main\v6.4.0_pit_a M:\mmarimut_v6.4.0_pit_01\switchview\View.java@@\main\v6.4.0_pit_a /END SAMPLE LINE/ I... (1 Reply)
Discussion started by: nmattam
1 Replies

7. Shell Programming and Scripting

How do I append a ^M to the end of each 129 character string

Hello all, I have a stumper of a problem. I am trying to append a ^M or "newline" to the end of each 129 character string in a huge file in unix. Each string starts with A00. I am trying to get the file to go from... A00vswjdv1 Test Junk Junk A00vswjdv2 Test Junk Junk ... (6 Replies)
Discussion started by: Captain
6 Replies

8. Shell Programming and Scripting

How can I append a string at the end of a line in a file

Hi, guys. I have one question: I have a file called "group", the contents of it is below: ******************************** ... test:x:203: sales:x:204: repair:x:205: research:x:206:brownj ... *********** Now I want to add string ",sherrys" at the end of "research:x:206:brownj", so... (5 Replies)
Discussion started by: daikeyang
5 Replies

9. Shell Programming and Scripting

append a record at the end of a file

Hi all, i have to append a record at the end of the file(a file which is already with some records).how do i do?please help me? is there any way of doing this with "SED" command.i am not sure.plz help me on this. would appreciate your ideas!!!! bye rao. (3 Replies)
Discussion started by: raoscb
3 Replies

10. UNIX for Dummies Questions & Answers

HOWTO Append String to End of Line

I have a comma delimited text file and need to appened ",000000" to the end of every line. For example: Before: "D700000","2006" ,"5000","Open Year" ,"Conversion" ,"Wk64","Productive Payroll $" ,1103.45 After: "D700000","2006" ,"5000","Open Year" ,"Conversion" ,"Wk64","Productive Payroll... (3 Replies)
Discussion started by: bggibson
3 Replies
Login or Register to Ask a Question