06-09-2008
awk updating one file with another, comparing, updating
Hello,
I read and search through this wonderful forum and tried different approaches but it seems I lack some knowledge and neurones ^^
Here is what I'm trying to achieve :
file1:
test filea 3495;
test fileb 4578;
test filec 7689;
test filey 9978;
test filez 12300;
file2:
test filea 3495;
test filed 4578;
test filec 7689;
test filex 8978;
results:
test filea 3495;
test filed 4578;
test filec 7689;
test filex 8978;
test filey 9978;
test filez 12300;
comparison in based on last field (field $3), new content from file2 (here content with "key" 8978 is new) should be added to final output and content that is different in file2 (test filed 4578; here) should replace file1 one.
here is where I am now:
awk 'NF { key=$NF;keys[key]++ } NR == FNR { key1[key] = $NF ORS;rec1[key] = $0 ORS;next } { key2[key] = $NF ORS;rec2[key] = $0 ORS;next } END { for (k in keys) { if (key1[k] == key2[k]) { print rec2[k] } else { print rec1[k] } } }' $file1 $file2 > $file1.updated
for readability:
awk '
NF
{
key=$NF;
keys[key]++
}
NR == FNR
{
key1[key] = $NF ORS;
rec1[key] = $0 ORS;
next
}
{
key2[key] = $NF ORS;
rec2[key] = $0 ORS;
next
}
END
{
for (k in keys)
{
if (key1[k] == key2[k])
{
print rec2[k]
}
else
{
print rec1[k]
}
}
}'
$file1 $file2 > $file1.updated
but.. this doesn't work well :/
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
I have a log file which is continuously added to, called log.file. I'd like to
monitor this file, and when certain lines are found, update some totals in
another file. I've played around with tail -f, grep, and awk, but can't seem
to hit the right note, so to speak.
The lines I'm... (0 Replies)
Discussion started by: nortonloaf
0 Replies
2. UNIX for Dummies Questions & Answers
Hi,
I have two text files containing records in following format:
file1 format is:
name1 age1 nickname1 path1
name2 age2 nickname2 path2
file 1 example is:
abcd 13 abcd.13 /home/temp/abcd.13
efgh 15 efgh.15 /home/temp/new/efgh.15 (4 Replies)
Discussion started by: alrinno
4 Replies
3. UNIX for Dummies Questions & Answers
Can someone help me again, I think with awk? I have a file with 4 columns (pipe-delimited):
I just want to convert the last field so that e1 is now 'message 1', e2 is 'message 2', e0 is 'message 3', etc. I don't want to change any other columns because the e0-e10 code may appear as part of a... (4 Replies)
Discussion started by: giannicello
4 Replies
4. Shell Programming and Scripting
I have an extremely large csv file that I need to search the second field, and upon matches update the last field...
I can pull the line with awk.. but apparently you cant use awk to directly update the file? So im curious if I can use sed to do this... The good news is the field I want to... (5 Replies)
Discussion started by: trey85stang
5 Replies
5. Shell Programming and Scripting
I have a file ("modtest") in which I want to update the last date/time block in the lines beginning with a period. Here is a sample:
.ROMULT 10150908EDT 10270908EDT 10010908EDT RANGE
RAWV2 1.00
.ROMULT 10150908EDT 10270908EDT 10010908EDT FGROUP
CHOWANRV 1.00
.RRIMULT 10150908EDT... (10 Replies)
Discussion started by: chrismcg24
10 Replies
6. Shell Programming and Scripting
file1: (unique files)
1 /pub/atomicbk/catalog/catalog.gif 693
2 /pub/atomicbk/catalog/home.gif 813
3 /pub/atomicbk/catalog/logo2.gif 12871
4 /pub/atomicbk/catalog/sleazbk.html 18338
file2: (duplicate filenames allowed)
28/Aug/1995:00:00:38 1 /pub/atomicbk/catalog/home.gif 813... (2 Replies)
Discussion started by: jontjioe
2 Replies
7. Shell Programming and Scripting
I have another question I am stuck at :wall:
I have a text file with two columns, like so...
2 0.0627279
3 0.0794451
4 0.108705
5 0.137739
6 0.190394
7 0.217407
8 0.241764
9 0.344458
10 0.460762
I'd like to go through the file line by line until the value in the second column... (3 Replies)
Discussion started by: origamisven
3 Replies
8. Shell Programming and Scripting
I can not seem to figure out how to update the attached match.txt column 2 using update.txt. However, only the text before the period in updat.txt appears in match.txt.
For example, in update.txt NM_001613.2 matches NM_001613 in match.txt, so is it possible to update the record in match.txt to... (8 Replies)
Discussion started by: cmccabe
8 Replies
9. Shell Programming and Scripting
Hi,
I have a file(testfile.txt) that contains list of variables as shown below. T
$$FirstName=James
$$LastName=Fox
$$Dateofbirth=1980-02-04
……and so on there are 50 different variables.
I am writing a script(script1.sh) that will update the above three variable one by one with the values... (6 Replies)
Discussion started by: Saanvi1
6 Replies
10. UNIX for Beginners Questions & Answers
hi,
i have an csv(which is a month's log file containing userid and log in date ) file which has to be appended to another file say master.log.I need to compare the next month's log data to master.log file .In case, there is new log date for userid it has to get updated in master.log file or i... (2 Replies)
Discussion started by: preema
2 Replies
LEARN ABOUT DEBIAN
opendkim-testkey
opendkim-testkey(8) System Manager's Manual opendkim-testkey(8)
NAME
opendkim-testkey - DKIM filter installation test
SYNOPSIS
opendkim-testkey [-d domain] [-s selector] [-k keypath] [-v] [-x configfile]
DESCRIPTION
opendkim-testkey verifies the setup of signing and verifying (private and public) keys for use with opendkim(8).
The test program will read a domain name and selector from the command line, configuration file or a key table, then query and parse the
resulting DKIM key(s), reporting any errors found.
If a key path is also provided, the test program will read the private key named and attempt to confirm that the private key specified by
keypath (or in the key table) and the public DKIM key retrieved match.
OPTIONS
-d domain
Names the domain in which signing is to be done. More specifically, names the domain in which the public key matching the provided
private key will be found. This parameter must be provided either explicitly, or in the configuration file, or via a KeyTable (see
opendkim.conf(5) for details).
-k keypath
Specifies the path to the private key file which should be used for this test. This parameter is optional
-s selector
Names the selector within the specified domain whose public key should be retrieved and tested, comparing it to the private key if
provided. This parameter must be provided either explicitly, or in the configuration file, or via a KeyTable (see opendkim.conf(5)
for details).
-v Increases the amount of output (verbosity) of the program. May be specified multiple times for increased output.
-x conffile
Names a configuration file to be parsed. See the opendkim.conf(5) man page for details. The only values used are Domain, Selector,
KeyFile, KeyTable, TrustAnchorFile and UnboundConfigFile. The default is /etc/opendkim.conf.
NOTES
The test program will also complain if a private key file is readable by anyone other than the user executing the program.
VERSION
This man page covers the version of opendkim-testkey that shipped with version 2.6.8 of OpenDKIM.
COPYRIGHT
Copyright (c) 2007, 2008, Sendmail, Inc. and its suppliers. All rights reserved.
Copyright (c) 2009-2012, The OpenDKIM Project. All rights reserved.
SEE ALSO
opendkim(8)
RFC6376 - DomainKeys Identified Mail
The OpenDKIM Project opendkim-testkey(8)