Sponsored Content
Top Forums Shell Programming and Scripting awk updating one file with another, comparing, updating Post 302203573 by mecano on Monday 9th of June 2008 07:41:39 AM
Old 06-09-2008
awk updating one file with another, comparing, updating

Hello,
I read and search through this wonderful forum and tried different approaches but it seems I lack some knowledge and neurones ^^

Here is what I'm trying to achieve :

file1:
test filea 3495;
test fileb 4578;
test filec 7689;
test filey 9978;
test filez 12300;

file2:
test filea 3495;
test filed 4578;
test filec 7689;
test filex 8978;

results:
test filea 3495;
test filed 4578;
test filec 7689;
test filex 8978;
test filey 9978;
test filez 12300;

comparison in based on last field (field $3), new content from file2 (here content with "key" 8978 is new) should be added to final output and content that is different in file2 (test filed 4578; here) should replace file1 one.

here is where I am now:

awk 'NF { key=$NF;keys[key]++ } NR == FNR { key1[key] = $NF ORS;rec1[key] = $0 ORS;next } { key2[key] = $NF ORS;rec2[key] = $0 ORS;next } END { for (k in keys) { if (key1[k] == key2[k]) { print rec2[k] } else { print rec1[k] } } }' $file1 $file2 > $file1.updated

for readability:

awk '
NF
{
key=$NF;
keys[key]++
}
NR == FNR
{
key1[key] = $NF ORS;
rec1[key] = $0 ORS;
next
}
{
key2[key] = $NF ORS;
rec2[key] = $0 ORS;
next
}
END
{
for (k in keys)
{
if (key1[k] == key2[k])
{
print rec2[k]
}
else
{
print rec1[k]
}
}
}'
$file1 $file2 > $file1.updated

but.. this doesn't work well :/
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Constantly updating log files (tail -f? grep? awk?)

I have a log file which is continuously added to, called log.file. I'd like to monitor this file, and when certain lines are found, update some totals in another file. I've played around with tail -f, grep, and awk, but can't seem to hit the right note, so to speak. The lines I'm... (0 Replies)
Discussion started by: nortonloaf
0 Replies

2. UNIX for Dummies Questions & Answers

Korn shell awk use for updating two files

Hi, I have two text files containing records in following format: file1 format is: name1 age1 nickname1 path1 name2 age2 nickname2 path2 file 1 example is: abcd 13 abcd.13 /home/temp/abcd.13 efgh 15 efgh.15 /home/temp/new/efgh.15 (4 Replies)
Discussion started by: alrinno
4 Replies

3. UNIX for Dummies Questions & Answers

Updating specific fields with awk using conditions

Can someone help me again, I think with awk? I have a file with 4 columns (pipe-delimited): I just want to convert the last field so that e1 is now 'message 1', e2 is 'message 2', e0 is 'message 3', etc. I don't want to change any other columns because the e0-e10 code may appear as part of a... (4 Replies)
Discussion started by: giannicello
4 Replies

4. Shell Programming and Scripting

Updating a line in a large csv file, with sed/awk?

I have an extremely large csv file that I need to search the second field, and upon matches update the last field... I can pull the line with awk.. but apparently you cant use awk to directly update the file? So im curious if I can use sed to do this... The good news is the field I want to... (5 Replies)
Discussion started by: trey85stang
5 Replies

5. Shell Programming and Scripting

AWK and sub/gsub: updating a date/time block

I have a file ("modtest") in which I want to update the last date/time block in the lines beginning with a period. Here is a sample: .ROMULT 10150908EDT 10270908EDT 10010908EDT RANGE RAWV2 1.00 .ROMULT 10150908EDT 10270908EDT 10010908EDT FGROUP CHOWANRV 1.00 .RRIMULT 10150908EDT... (10 Replies)
Discussion started by: chrismcg24
10 Replies

6. Shell Programming and Scripting

Comparing 2 files with awk and updating 2nd file

file1: (unique files) 1 /pub/atomicbk/catalog/catalog.gif 693 2 /pub/atomicbk/catalog/home.gif 813 3 /pub/atomicbk/catalog/logo2.gif 12871 4 /pub/atomicbk/catalog/sleazbk.html 18338 file2: (duplicate filenames allowed) 28/Aug/1995:00:00:38 1 /pub/atomicbk/catalog/home.gif 813... (2 Replies)
Discussion started by: jontjioe
2 Replies

7. Shell Programming and Scripting

awk - updating variable in if statement

I have another question I am stuck at :wall: I have a text file with two columns, like so... 2 0.0627279 3 0.0794451 4 0.108705 5 0.137739 6 0.190394 7 0.217407 8 0.241764 9 0.344458 10 0.460762 I'd like to go through the file line by line until the value in the second column... (3 Replies)
Discussion started by: origamisven
3 Replies

8. Shell Programming and Scripting

Help updating a file

I can not seem to figure out how to update the attached match.txt column 2 using update.txt. However, only the text before the period in updat.txt appears in match.txt. For example, in update.txt NM_001613.2 matches NM_001613 in match.txt, so is it possible to update the record in match.txt to... (8 Replies)
Discussion started by: cmccabe
8 Replies

9. Shell Programming and Scripting

Updating variables using sed or awk

Hi, I have a file(testfile.txt) that contains list of variables as shown below. T $$FirstName=James $$LastName=Fox $$Dateofbirth=1980-02-04 ……and so on there are 50 different variables. I am writing a script(script1.sh) that will update the above three variable one by one with the values... (6 Replies)
Discussion started by: Saanvi1
6 Replies

10. UNIX for Beginners Questions & Answers

Updating in file

hi, i have an csv(which is a month's log file containing userid and log in date ) file which has to be appended to another file say master.log.I need to compare the next month's log data to master.log file .In case, there is new log date for userid it has to get updated in master.log file or i... (2 Replies)
Discussion started by: preema
2 Replies
TEST(1) 						    BSD General Commands Manual 						   TEST(1)

NAME
test, [ -- condition evaluation utility SYNOPSIS
test expression [ expression ] DESCRIPTION
The test utility evaluates expression and, if it evaluates to true, returns a zero (true) exit status; otherwise it returns 1 (false). If expression is not given, test also returns 1 (false). All operators and flags are separate arguments to the test utility. The following primaries are used to construct expression: -b file True if file exists and is a block special file. -c file True if file exists and is a character special file. -d file True if file exists and is a directory. -e file True if file exists (regardless of type). -f file True if file exists and is a regular file. -g file True if file exists and its set group ID flag is set. -h file True if file exists and is a symbolic link. -k file True if file exists and its sticky bit is set. -n string True if the length of string is nonzero. -p file True if file exists and is a named pipe (FIFO). -r file True if file exists and is readable. -s file True if file exists and has a size greater than zero. -t file_descriptor True if the file whose file descriptor number is file_descriptor is open and is associated with a terminal. -u file True if file exists and its set user ID flag is set. -w file True if file exists and is writable. True indicates only that the write flag is on. The file is not writable on a read-only file system even if this test indicates true. -x file True if file exists and is executable. True indicates only that the execute flag is on. If file is a directory, true indi- cates that file can be searched. -z string True if the length of string is zero. -L file True if file exists and is a symbolic link. This operator is retained for compatibility with previous versions of this pro- gram. Do not rely on its existence; use -h instead. -O file True if file exists and its owner matches the effective user id of this process. -G file True if file exists and its group matches the effective group id of this process. -S file True if file exists and is a socket. file1 -nt file2 True if file1 exists and is newer than file2. file1 -ot file2 True if file1 exists and is older than file2. file1 -ef file2 True if file1 and file2 exist and refer to the same file. string True if string is not the null string. s1 = s2 True if the strings s1 and s2 are identical. s1 != s2 True if the strings s1 and s2 are not identical. s1 < s2 True if string s1 comes before s2 based on the ASCII value of their characters. s1 > s2 True if string s1 comes after s2 based on the ASCII value of their characters. n1 -eq n2 True if the integers n1 and n2 are algebraically equal. n1 -ne n2 True if the integers n1 and n2 are not algebraically equal. n1 -gt n2 True if the integer n1 is algebraically greater than the integer n2. n1 -ge n2 True if the integer n1 is algebraically greater than or equal to the integer n2. n1 -lt n2 True if the integer n1 is algebraically less than the integer n2. n1 -le n2 True if the integer n1 is algebraically less than or equal to the integer n2. These primaries can be combined with the following operators: ! expression True if expression is false. expression1 -a expression2 True if both expression1 and expression2 are true. expression1 -o expression2 True if either expression1 or expression2 are true. ( expression ) True if expression is true. The -a operator has higher precedence than the -o operator. Note that all file tests with the exception of -h and -L follow symbolic links and thus evaluate the test for the file pointed at. GRAMMAR AMBIGUITY
The test grammar is inherently ambiguous. In order to assure a degree of consistency, the cases described in IEEE Std 1003.2 (``POSIX.2'') section 4.62.4, are evaluated consistently according to the rules specified in the standards document. All other cases are subject to the ambiguity in the command semantics. EXIT STATUS
The test utility exits with one of the following values: 0 expression evaluated to true. 1 expression evaluated to false or was missing. >1 An error occurred. STANDARDS
The test utility implements a superset of the IEEE Std 1003.2 (``POSIX.2'') specification. BSD
November 10, 2009 BSD
All times are GMT -4. The time now is 08:18 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy