Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

csv file to array, match field1, replace in flat file field1 to field2

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting csv file to array, match field1, replace in flat file field1 to field2
# 15  
Old 02-03-2009
given fran.csv:
Code:
"hello","byebye"
"other","data"
"hello2","byebye2"
"hello3","byebye3"
"other","data"

and services.txt:
Code:
define service{
use generic-service
service_description TCP_80
host_name hello,hello2,hello3,nochangematch
        contact_groups
        check_command
}

nawk -v sep=',' -f fran.awk fran.csv services.txt
produces:
Code:
define service{
use generic-service
service_description TCP_80
host_name byebye,byebye2,byebye3,nochangematch
        contact_groups
        check_command
}

given nagios.cfg
Code:
define host{
use generic-host
host_name hello
alias hello.hello
address 127.0.0.1
contact_groups mymail
parents hello3
}

running:
nawk -v sep='.' -f fran.awk fran.csv nagios.cfg
produces:
Code:
define host{
use generic-host
host_name byebye
alias byebye.byebye
address 127.0.0.1
contact_groups mymail
parents byebye3
}

Check all your files against my samples - it all works for me.
# 16  
Old 02-03-2009
Very weird. Checked, proof:

cat fran4.awk
Code:
BEGIN {
  FScsv=","
  FScfg=FS

  if (sep=="") sep="."

  qq=sprintf("%c", 034)
}
FNR==NR && FNR==1 {FS=FScsv;$1=$1}
FNR!=NR && FNR==1 {FS=FScfg; $1=$1}
FNR==NR {gsub(qq, "", $1);gsub(qq, "", $2); csv[$1]=$2; next}

!/{/ {
    n=split($2, _a,sep)
    for(i=1; i<=n; i++)
      if (_a[i] in csv) _a[i]=csv[_a[i]]
    for(i=1; i<=n; i++)
      $2=(i==1) ? _a[i] : $2 sep _a[i]
}
1

cat test.csv

Code:
"hello","byebye"
"other","data"
"hello2","byebye2"
"hello3","byebye3"
"other","data"

cat services.txt

Code:
define service{
        use                             generic-service
        service_description             TCP_80
        host_name                       hello,hello2,hello3,nochangematch
        contact_groups                  
        check_command                
}

Running code:

nawk -v sep=',' -f fran4.awk test.csv services.txt

Code:
define service{
use generic-service
service_description TCP_80
host_name hello,byebye2,byebye3,nochangematch
        contact_groups                  
        check_command                
}

Didn't change first hello

cat nagios.cfg.txt

Code:
define host{
        use                          generic-host
        host_name                    hello
        alias                        hello.hello
        address                      127.0.0.1
        contact_groups               mymail
        parents                      hello3
}

Running code:

nawk -v sep='.' -f fran4.awk test.csv nagios.cfg.txt

Code:
define host{
use generic-host
host_name hello
alias hello.hello
address 127.0.0.1
contact_groups mymail
parents byebye3
}

Changed only parents field.

Is fran4.awk latest code, matches yours?

Just in case did for all files dos2unix as well and:

file -i fran4.awk test.csv nagios.cfg.txt services.txt

Code:
fran4.awk:      text/plain charset=us-ascii
test.csv:       text/plain charset=us-ascii
nagios.cfg.txt: text/plain charset=us-ascii
services.txt:   text/plain charset=us-ascii

# 17  
Old 02-03-2009
strange, your awk code is the same as mine.
hmmm... - it must be something in the data files.
couxld you 'cat -vet nagios.cfg.txt' and 'cat -vet test.csv' and post the results here within the code tags, pls.
# 18  
Old 02-03-2009
cat -vet nagios.cfg.txt

Code:
define host{$
        use                          generic-host$
        host_name                    hello$
        alias                        hello.hello$
        address                      127.0.0.1$
        contact_groups               mymail$
        parents                      hello3$
}$

cat -vet test.csv

Code:
"hello","byebye"$
"other","data"$
"hello2","byebye2"$
"hello3","byebye3"$
"other","data"$

cat -vet services.txt

Code:
 
define service{$
        use                             generic-service$
        service_description             TCP_80$
        host_name                       hello,hello2,hello3,nochangematch$
        contact_groups                  $
        check_command                $
}$

cat -vet fran4.awk

Code:
 
BEGIN {$
  FScsv=","$
  FScfg=FS$
$
  if (sep=="") sep="."$
$
  qq=sprintf("%c", 034)$
}$
FNR==NR && FNR==1 {FS=FScsv;$1=$1}$
FNR!=NR && FNR==1 {FS=FScfg; $1=$1}$
FNR==NR {gsub(qq, "", $1);gsub(qq, "", $2); csv[$1]=$2; next}$
$
!/{/ {$
    n=split($2, _a,sep)$
    for(i=1; i<=n; i++)$
      if (_a[i] in csv) _a[i]=csv[_a[i]]$
    for(i=1; i<=n; i++)$
      $2=(i==1) ? _a[i] : $2 sep _a[i]$
}$
1$

# 19  
Old 02-03-2009
hmm...... this is bizarre - works just fine here. Looks like you're having issues with the lookup of "hello" in the test.csv file.
could change nagios.cfg.txt like so:
Code:
define host{
        use                          generic-host
        host_name                    other
        alias                        other.other
        address                      127.0.0.1
        contact_groups               mymail
        parents                      hello3
}

and:
nawk -v sep='.' -f fran4.awk test.csv nagios.cfg.txt
# 20  
Old 02-03-2009
Yep, if changing to "other" then works Smilie

cat nagios.cfg.txt
Code:
define host{
        use                          generic-host
        host_name                    other
        alias                        other.other
        address                      127.0.0.1
        contact_groups               mymail
        parents                      hello3
}

nawk -v sep='.' -f fran4.awk test.csv nagios.cfg.txt

Code:
define host{
use generic-host
host_name data
alias data.data
address 127.0.0.1
contact_groups mymail
parents byebye3
}

Works, like a champ

Then again with this concept:

cat nagios.cfg.txt

Code:
define host{
        use                          generic-host
        host_name                    hello
        alias                        other.hello2
        address                      127.0.0.1
        contact_groups               mymail
        parents                      hello3
}

nawk -v sep='.' -f fran4.awk test.csv nagios.cfg.txt

Code:
define host{
use generic-host
host_name hello
alias data.byebye2
address 127.0.0.1
contact_groups mymail
parents byebye3
}

You see, hello in host_name field didn't change.
all other fields okey.

This is like my awk hates "hello". It want's it to be "hell' Smilie
Just joking.. but seriously.. Very weird case.
# 21  
Old 02-03-2009
heh Smilie
try this test.csv:
Code:
"other","data"
"hello","byebye"
"hello2","byebye2"
"hello3","byebye3"
"other","data"
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Converting csv file to flat file

Hi All, I have a csv file which is comma seperated. I need to convert to flat file with preferred column length country,id Australia,1234 Africa,12399999 Expected output country id Australia 1234 Africa 12399999 the flat file should predefined length on respective... (8 Replies)
Discussion started by: rohit_shinez
8 Replies

2. Shell Programming and Scripting

awk to update file based on partial match in field1 and exact match in field2

I am trying to create a cronjob that will run on startup that will look at a list.txt file to see if there is a later version of a database using database.txt as the source. The matching lines are written to output. $1 in database.txt will be in list.txt as a partial match. $2 of database.txt... (2 Replies)
Discussion started by: cmccabe
2 Replies

3. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

4. Shell Programming and Scripting

Replace value in csv file with match from another file

Good morning everyone and hello unix.com! I'm trying to solve a problem and was unable to find a solution after a forum research. So hopefully you can help me get this solved. I have these two csv files: file1.csv ID,REFERENCE,STATUS 1,2,0 2,4,1 3,1,0 file2.csv... (2 Replies)
Discussion started by: RKos
2 Replies

5. Shell Programming and Scripting

Match list of strings in File A and compare with File B, C and write to a output file in CSV format

Hi Friends, I'm a great fan of this forum... it has helped me tone my skills in shell scripting. I have a challenge here, which I'm sure you guys would help me in achieving... File A has a list of job ids and I need to compare this with the File B (*.log) and File C (extend *.log) and copy... (6 Replies)
Discussion started by: asnandhakumar
6 Replies

6. Shell Programming and Scripting

reading a csv file and creating a flat file

hi i have written a script for reading a csv file and creating a flat file, suggest if this script can be optimized #---------------- FILENAME="$1" SCRIPT=$(basename $0) #-----------------------------------------// function usage { echo "\nUSAGE: $THIS_SCRIPT file_to_process\n"... (3 Replies)
Discussion started by: mprakasheee
3 Replies

7. Programming

Insert value from field1 to field2 with additional info

Need help - hopefully I explain it correctly I have a table in mysql database with multiple fields. 2 of the fields are called id and id_link id id_link 93 http://test_server/testpage.cgi?93 95 96 97 98 I need to find the correct sql to update all of the id_link... (2 Replies)
Discussion started by: frustrated1
2 Replies

8. Shell Programming and Scripting

Flat file to csv conversion

Hi Guy's can someone help me in converting the following I have a flat text file which has several thousand lines which I need to convert to a csv it's got a consistent format but basically want every time it hit's txt to create a new line with the subsequent lines comma delimited for example ... (6 Replies)
Discussion started by: p1_ben
6 Replies

9. Shell Programming and Scripting

Awk to convert a flat file to CSV file

Hi , I have a file with contents as below: Contract Cancellation Report UARCNCL LOS CODE DATE REAS TYPE AMOUNT AMOUNT LETTER BY ========= ======= ==== ==== ==== ========= ==== ==== 8174739 7641509 1S NONE CRCD 30-JUN-2008 NPAR N .00 .00 CCAN 8678696 8091709 1S NONE DDEB 30-JUN-2008... (14 Replies)
Discussion started by: rkumudha
14 Replies

10. Shell Programming and Scripting

Remove 5th character from Field1 & Print

Hi , I need to remove the 5th character of column1 and print the rest. Can anybody give some advice? Input: 0001c xx 0001r gg jj 0002y vv 0002p kk 0003q gg ll 0003v tt 0003t gg pp kk Output: 0001 xx 0001 gg jj (9 Replies)
Discussion started by: Raynon
9 Replies
Login or Register to Ask a Question