Replacing content from a list


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Replacing content from a list
# 1  
Old 12-15-2010
Replacing content from a list

Okay, so I am not quite sure the best way of going about this.

I have a whole stack of files that need items replaced with a code corresponding to it. In particular, I am looking to replace the names of countries with the two letter iso country code. The files themselves are in csv format with the country names in the first column. I also have the list of countries with their codes in another csv file. I am playing around with the un databases, so really i would be editing millions of entries, by hand is not an option.

many thanks.
john
# 2  
Old 12-15-2010
if country_code.csv is
Code:
 
country_name1,country_code1
country_name2,country_code2
.
.

and inputFile.csv is
Code:
 
country_name1,gfgdfg,dfgfdgfdg
country_name2,trtert,rrter,trtrt
.
.

Then
Code:
 
while read line
do
    name=$(echo $line | cut -d, -f1
    code=$(echo $line | cut -d, -f2
    sed -i "s/$name/$code/" inputFile.csv
done < country_code.csv

# 3  
Old 12-15-2010
Above sed script need read inputFile.csv many times, depend on the lines of country_code.csv

below awk command will only read one time.

Code:
awk -F, 'NR==FNR{a[$1]=$2;next} $1 in a {$1=a[$1]}1' OFS=, country_code.csv inputFile.csv

And the sed can be updated as:

Code:
IFS=,
while read name code
do  
   sed -i "s/$name/$code/" inputFile.csv
done < country_code.csv


Last edited by rdcwayx; 12-15-2010 at 08:21 PM..
# 4  
Old 12-16-2010
thank you very much- I'll give them a go on monday


/john
# 5  
Old 12-17-2010
Here is a Perl solution:
Code:
#!/usr/bin/perl

use strict;
use warnings;
use Text::CSV;

my $lookup = 'isocodes';
my $csv = Text::CSV->new();
my %hash_table = ();

open (CSV, "<", $lookup) or die $!;
while (<CSV>) {
   if ($csv->parse($_)) {
       my @columns = $csv->fields();
       # print "@columns\n";
       $hash_table{$columns[0]} = $columns[1];
   } else {
       my $err = $csv->error_input;
       print "Failed to parse line: $err";
   }
}
close CSV;

# print "$_ = $hash_table{$_}\n" foreach keys %hash_table;

my $file = 'example';
$csv = Text::CSV->new();

open (CSV, "<", $file) or die $!;
while (<CSV>) {
   if ($csv->parse($_)) {
       my @columns = $csv->fields();
       $columns[0] = $hash_table{$columns[0]};
       { local $" = ','; print "@columns\n"; }
   } else {
       my $err = $csv->error_input;
       print "Failed to parse line: $err";
   }
}
close CSV;

As an example if "isocodes" contains
Code:
Ireland,IE
United States,US
Philippines,PH

and "example" contains
Code:
Ireland,column2,column3
Philippines,column2,column3
United States,column2,column3

then the output is
Code:
IE,column2,column3
PH,column2,column3
US,column2,column3

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Replacing stopwords based on a list

Dear all, I have Files with lines of text in them, I want to replace the stopwords in them with ",". I have create a file which contain the stopwords... I have been trying for last 3 hours but no success I have managed to replace one using "sed" and delete the line containing them using... (3 Replies)
Discussion started by: A-V
3 Replies

2. Shell Programming and Scripting

Replacing partial content of a file.

Hi All, Please help me in the below issue.I had a file called env.prop.In that i need to change service.url.If i select chocie 2. it should come as https://dev2.ecif.info53.com:30102/soap/default 30102 port is not fixed .It varies when the file is updated.So i want to keep orginal port in... (3 Replies)
Discussion started by: bhas85
3 Replies

3. Shell Programming and Scripting

Pattern Matching & replacing of content in file1 with file2

I have file 1 & file 2 with content mentioned below. I want to get the output as shown in file3. Requirement: check the content of column 1 & column 2, if value of column 1 in file1 matches with first column of file2 then remaining columns(2&3) of file2 should get replaced, also if value of... (4 Replies)
Discussion started by: siramitsharma
4 Replies

4. Solaris

Replacing a string in a long list of files

I have a script that needs to read a file with a long list of /path/filenames - replace the name of the server in each file - and write the file to the same path with a date extension. This is the script that I have so far #!/bin/ksh umask 022 LIST=`scripts.list` for i in $LIST do ... (2 Replies)
Discussion started by: bjdamon
2 Replies

5. Shell Programming and Scripting

how to get list of files changed by sed while replacing

Hi We have a shell script to find a string in a large set of files and replace it using the sed command.in the same time we also want to store the list of files it checked. *** The existing script is as below #!/bin/bash #1 _r1="inst_group=ems7770" _r2="inst_group=dba" # Escape path... (4 Replies)
Discussion started by: sabkan
4 Replies

6. Shell Programming and Scripting

How to list the content of a directory

Hi I want to write a script that reads a directory name, checks if the directory exists and lists the content of that directory. That's what I have at the moment. function listDirectory { echo "Give in a directory name" read name #Here I want to check if the... (4 Replies)
Discussion started by: hss
4 Replies

7. Shell Programming and Scripting

Query for replacing a string and keeping the non-replaced content

Hi experts, As i am a novice unix player...so need help for the below query...banged my head from quite a while...:confused: i have a set of html files, in which i need to search for string "Page"(case sensitive) and then replace the same with some numeric code ,say, "XXX1234". Here in... (2 Replies)
Discussion started by: rahulfhp
2 Replies

8. Shell Programming and Scripting

Replacing from a list file using awk

I have a set of user name password pairs in one file and I have a set of files which have user names and a random string as the password. How can I read the user name and password from the first file and replace the random password in the second file. my list file looks like this user1, pass1... (2 Replies)
Discussion started by: thaKid
2 Replies

9. Shell Programming and Scripting

replacing text in file1 with list from file2

I am trying to automate a process of searching through a set of files and replace all occurrences of a formatted text with the next item in the list of a second file. Basically i need to replace all instances of T????CLK???? with an IP address from a list in a second file. the second file is one IP... (9 Replies)
Discussion started by: dovetail
9 Replies

10. Shell Programming and Scripting

Replacing a paragraph between pattern , with the content 4m another file

hi, i wanted to put the output of file f1 into the pattern space of file f2 f1: wjwjwjwjwjwjwj //these line go in file f2 jwjwjwjwjwjjwjw wjwjwjwjjwjwjwj f2: Pattern_start __________ //these are the line to be replaced __________ Pattern_end i m... (4 Replies)
Discussion started by: go4desperado
4 Replies
Login or Register to Ask a Question