Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

csv file to array, match field1, replace in flat file field1 to field2

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting csv file to array, match field1, replace in flat file field1 to field2
# 1  
Old 02-01-2009
Hammer & Screwdriver csv file to array, match field1, replace in flat file field1 to field2
Hello, i am bit stuck with making script for automatic procedure.

Case:

Two files. One is flat file, other is csv file.
csv file has two column/fields with comma delimited data.

Here is what i need (explained way)

CSV file:

field1 | field2
"hello","byebye"
"hello2","byebye2"
"hello3","byebye3"

Flat file has matching data:

"hello"
"hello2"
"hello3"

If found:
"hello"
"hello2"
"hello3"
..in flat file, then it must be replaced from csv file with FIELD2 according
to search match.

But i have to search "hello" from csv file field1 data with flat file together and if found match in
flat file, it must be replaced from csv file with field2 data.

I know i could split csv file to two files, one containing field1 data and
second file field2 data but that would be not so dynamic way.
Though then i could easily use loop with cat + sed

Idea is to make nagios mass updating hosts naming changes in config.
So im wondering could i do it with awk or sed more universal way?

Any examples would be great help.

Thanks in advance..

Frankie
Last edited by frankie_konin; 02-01-2009 at 06:21 PM..
# 2  
Old 02-01-2009
Code:
$ cat csv
field1 | field2
"hello","byebye"
"other","data"
"hello2","byebye2"
"hello3","byebye3"
"other","data"

Code:
$ cat flat
"hello"
"hello2"
"hello3"

Code:
 awk -F, 'NR==FNR { a[$0]; next } $1 in a { print $2 }' flat csv


Output:

Code:
"byebye"
"byebye2"
"byebye3"

# 3  
Old 02-01-2009
Code:
grep -f flat csv | cut -d, -f2

# 4  
Old 02-02-2009
Quote:
Originally Posted by cfajohnson
Code:
grep -f flat csv | cut -d, -f2

Hello, thanks for reply but this only prints csv field2 data.

I need to search from csv file first field1 and same time from
flat file and if both csv and flat file have a match, then replace
founded match in flat file with csv field2 data.

here is what csv file contains:

Code:
cat test.csv

"hello","byebye"
"other","data"
"hello2","byebye2"
"hello3","byebye3"
"other","data"

Now flat file called nagios.cfg contains data:

Code:
define host{
        use                             generic-host
        host_name                    hello
        alias                             hello.hello
        address                         127.0.0.1
        contact_groups               mymail
        parents                         hello3
}

So if i found match "hello" in test.csv and nagios.cfg
then in nagios.cfg it needs to be replaced dynamic way
to "byebye" and "hello3" to "byebye3" from test.csv field2/colum

Need that way.. kinda hard to explain evenSmilie
# 5  
Old 02-02-2009
something to start with.
nawk -f fran.awk test.csv nagios.cfg
Code:
BEGIN {
  FScsv=","
  FScfg=FS

  qq=sprintf("%c", 034)
}
FNR==NR && FNR==1 {FS=FScsv;$1=$1}
FNR!=NR && FNR==1 {FS=FScfg; $1=$1}
FNR==NR {gsub(qq, "", $1);gsub(qq, "", $2); csv[$1]=$2; next}

!/{/ { if ($2 in csv) $2=csv[$2] }
1

# 6  
Old 02-02-2009
Quote:
Originally Posted by vgersh99
something to start with.
nawk -f fran.awk test.csv nagios.cfg
Code:
BEGIN {
  FScsv=","
  FScfg=FS

  qq=sprintf("%c", 034)
}
FNR==NR && FNR==1 {FS=FScsv;$1=$1}
FNR!=NR && FNR==1 {FS=FScfg; $1=$1}
FNR==NR {gsub(qq, "", $1);gsub(qq, "", $2); csv[$1]=$2; next}

!/{/ { if ($2 in csv) $2=csv[$2] }
1

Seems we are getting there:

did run your code.
output:
Code:
 nawk -f fran.awk test.csv nagios.cfg
define host{
        use                          generic-host
        host_name                    hello
        alias                        hello.hello
        address                      127.0.0.1
        contact_groups               mymail
parents byebye3
}

So parents which was hello3 got changed to right one from csv
field2. But other in host_name hello and in alias hello.hello didn't

Can you help me more with it, why didn't change?

thanks very much for any help..

frankie
# 7  
Old 02-02-2009
Quote:
Originally Posted by frankie_konin
Seems we are getting there:

did run your code.
output:
Code:
 nawk -f fran.awk test.csv nagios.cfg
define host{
        use                          generic-host
        host_name                    hello
        alias                        hello.hello
        address                      127.0.0.1
        contact_groups               mymail
parents byebye3
}

So parents which was hello3 got changed to right one from csv
field2. But other in host_name hello and in alias hello.hello didn't

Can you help me more with it, why didn't change?

thanks very much for any help..

frankie
this is what I got back given your sample test.csv file from above:
Code:
nawk -f fran.awk fran.csv fran.cfg
define host{
        use                             generic-host
host_name byebye
        alias                             hello.hello
        address                         127.0.0.1
        contact_groups               mymail
parents byebye3
}

The 'alias hello.hello' didn't change because you had no matching entry in 'test.csv' file for it.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Converting csv file to flat file

Hi All, I have a csv file which is comma seperated. I need to convert to flat file with preferred column length country,id Australia,1234 Africa,12399999 Expected output country id Australia 1234 Africa 12399999 the flat file should predefined length on respective... (8 Replies)
Discussion started by: rohit_shinez
8 Replies

2. Shell Programming and Scripting

awk to update file based on partial match in field1 and exact match in field2

I am trying to create a cronjob that will run on startup that will look at a list.txt file to see if there is a later version of a database using database.txt as the source. The matching lines are written to output. $1 in database.txt will be in list.txt as a partial match. $2 of database.txt... (2 Replies)
Discussion started by: cmccabe
2 Replies

3. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

4. Shell Programming and Scripting

Replace value in csv file with match from another file

Good morning everyone and hello unix.com! I'm trying to solve a problem and was unable to find a solution after a forum research. So hopefully you can help me get this solved. I have these two csv files: file1.csv ID,REFERENCE,STATUS 1,2,0 2,4,1 3,1,0 file2.csv... (2 Replies)
Discussion started by: RKos
2 Replies

5. Shell Programming and Scripting

Match list of strings in File A and compare with File B, C and write to a output file in CSV format

Hi Friends, I'm a great fan of this forum... it has helped me tone my skills in shell scripting. I have a challenge here, which I'm sure you guys would help me in achieving... File A has a list of job ids and I need to compare this with the File B (*.log) and File C (extend *.log) and copy... (6 Replies)
Discussion started by: asnandhakumar
6 Replies

6. Shell Programming and Scripting

reading a csv file and creating a flat file

hi i have written a script for reading a csv file and creating a flat file, suggest if this script can be optimized #---------------- FILENAME="$1" SCRIPT=$(basename $0) #-----------------------------------------// function usage { echo "\nUSAGE: $THIS_SCRIPT file_to_process\n"... (3 Replies)
Discussion started by: mprakasheee
3 Replies

7. Programming

Insert value from field1 to field2 with additional info

Need help - hopefully I explain it correctly I have a table in mysql database with multiple fields. 2 of the fields are called id and id_link id id_link 93 http://test_server/testpage.cgi?93 95 96 97 98 I need to find the correct sql to update all of the id_link... (2 Replies)
Discussion started by: frustrated1
2 Replies

8. Shell Programming and Scripting

Flat file to csv conversion

Hi Guy's can someone help me in converting the following I have a flat text file which has several thousand lines which I need to convert to a csv it's got a consistent format but basically want every time it hit's txt to create a new line with the subsequent lines comma delimited for example ... (6 Replies)
Discussion started by: p1_ben
6 Replies

9. Shell Programming and Scripting

Awk to convert a flat file to CSV file

Hi , I have a file with contents as below: Contract Cancellation Report UARCNCL LOS CODE DATE REAS TYPE AMOUNT AMOUNT LETTER BY ========= ======= ==== ==== ==== ========= ==== ==== 8174739 7641509 1S NONE CRCD 30-JUN-2008 NPAR N .00 .00 CCAN 8678696 8091709 1S NONE DDEB 30-JUN-2008... (14 Replies)
Discussion started by: rkumudha
14 Replies

10. Shell Programming and Scripting

Remove 5th character from Field1 & Print

Hi , I need to remove the 5th character of column1 and print the rest. Can anybody give some advice? Input: 0001c xx 0001r gg jj 0002y vv 0002p kk 0003q gg ll 0003v tt 0003t gg pp kk Output: 0001 xx 0001 gg jj (9 Replies)
Discussion started by: Raynon
9 Replies
Login or Register to Ask a Question