awk updating one file with another, comparing, updating


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk updating one file with another, comparing, updating
# 8  
Old 06-10-2008
Some suggestions,
IMHO:

- quote your variables to avoid problems with "pathological" characters: embedded spaces or other special characters.
This:

Code:
if [ -f $filename ] ...

Should become:


Code:
if [ -f "$filename" ] ...

- use printf instead of echo for portability
- reduce the awk code when possible:

this:

Code:
awk '{ printf substr($NF, 1, length($NF)-1);$NF = "";printf " %s\n",$0 }' $filename | sort -n | awk '{ printf "%s%s;\n",$0,$1 }' | awk '{$1="";sub(/^ +/, "");printf "%s\n",$0}' > $filename.sorted

could be written as:

Code:
awk '{ print | "sort -nk" NF }' "$filename" > "$filename.sorted"

and this:

Code:
awk '{ printf substr($NF, 1, length($NF)-1);$NF = "";printf "\n" }' $filename | sort -n | awk '{ if ($1 == prev) { printf "%d\n",$0;num++ };prev=$1 } END { printf "\n%d duplicates were found...\n",num }'

could become (the output is not sorted):

Code:
awk 'END { 
  for (k in _) 
    if (_[k] > 1)
      printf "%d\t-->\t%d\n", k, _[k]
      print (c ? c : 0) " duplicates were found..." 
    }
{ sub(/;$/,""); c = _[$NF]++ ? ++c : c }
' "$filename"

etc.

- you're writing the same code again and again:

Code:
if [ -f $filename ]
  then
    ...
else
  echo -e "\ncan't find file $filename\n"

You may use a function:

Code:
exists () { 
  [ -f "$1" ] || { 
    printf " ... " >&2
    return 1
    }
}

# 9  
Old 09-12-2008
Hi am very new to awk & unix, my requirement is very similar to this..

I want to compare by first column,

file1:
0000-00058|Green
0000-00059|Green
0000-00060|Green
0402-01055|Green
0402-01058|Green
0402-01059|Green
0402-01061|Green
0402-01065|Green



file2:
0000-00057|Red
0000-00058|Blue
0000-00059|Red
0000-00060|Blue


My result should be

0000-00058|Blue
0000-00059|Red
0000-00060|Blue
0402-01055|Green
0402-01058|Green
0402-01059|Green
0402-01061|Green
0402-01065|Green

0000-00057|Red

awaiting for your reply...
Thank u.
# 10  
Old 09-13-2008
Not tested on Solaris (you may try nawk or /usr/xpg4/bin/awk):

Code:
awk -F\| 'END { for (k in _) print k, _[k] }
NR == FNR { _[$1] = $2; next }
$1 in _ { $2 = _[$1]; delete _[$1] } -3
' OFS=\| file2 file1

# 11  
Old 09-14-2008
perl:

Code:
open(FH1,"<a");
while(<FH1>){
	@arr=split(" ",$_);
	$hash{$arr[2]}=$_;
}
close(FH1);
open(FH2,"<b");
while(<FH2>){
	@arr=split(" ",$_);
	$hash{$arr[2]}=$_;
}
close(FH1);
for $key (sort {$a<=>$b} keys %hash){
	print $hash{$key};
}

# 12  
Old 09-14-2008
With GNU Awk (sorted output):

Code:
WHINY_USERS=1 awk -F\| 'END {
  for(k in _) print k, _[k]
  }
{ _[$1] = $2 }' OFS=\| file1 file2


Or:

Code:
perl -F'\|' -ane'
  $x{$F[0]}=$F[1];
  END{print map{$_,"|",$x{$_}}sort keys %x}
  ' file1 file2


Last edited by radoulov; 09-14-2008 at 07:59 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Updating in file

hi, i have an csv(which is a month's log file containing userid and log in date ) file which has to be appended to another file say master.log.I need to compare the next month's log data to master.log file .In case, there is new log date for userid it has to get updated in master.log file or i... (2 Replies)
Discussion started by: preema
2 Replies

2. Shell Programming and Scripting

Updating variables using sed or awk

Hi, I have a file(testfile.txt) that contains list of variables as shown below. T $$FirstName=James $$LastName=Fox $$Dateofbirth=1980-02-04 ……and so on there are 50 different variables. I am writing a script(script1.sh) that will update the above three variable one by one with the values... (6 Replies)
Discussion started by: Saanvi1
6 Replies

3. Shell Programming and Scripting

Help updating a file

I can not seem to figure out how to update the attached match.txt column 2 using update.txt. However, only the text before the period in updat.txt appears in match.txt. For example, in update.txt NM_001613.2 matches NM_001613 in match.txt, so is it possible to update the record in match.txt to... (8 Replies)
Discussion started by: cmccabe
8 Replies

4. Shell Programming and Scripting

awk - updating variable in if statement

I have another question I am stuck at :wall: I have a text file with two columns, like so... 2 0.0627279 3 0.0794451 4 0.108705 5 0.137739 6 0.190394 7 0.217407 8 0.241764 9 0.344458 10 0.460762 I'd like to go through the file line by line until the value in the second column... (3 Replies)
Discussion started by: origamisven
3 Replies

5. Shell Programming and Scripting

Comparing 2 files with awk and updating 2nd file

file1: (unique files) 1 /pub/atomicbk/catalog/catalog.gif 693 2 /pub/atomicbk/catalog/home.gif 813 3 /pub/atomicbk/catalog/logo2.gif 12871 4 /pub/atomicbk/catalog/sleazbk.html 18338 file2: (duplicate filenames allowed) 28/Aug/1995:00:00:38 1 /pub/atomicbk/catalog/home.gif 813... (2 Replies)
Discussion started by: jontjioe
2 Replies

6. Shell Programming and Scripting

AWK and sub/gsub: updating a date/time block

I have a file ("modtest") in which I want to update the last date/time block in the lines beginning with a period. Here is a sample: .ROMULT 10150908EDT 10270908EDT 10010908EDT RANGE RAWV2 1.00 .ROMULT 10150908EDT 10270908EDT 10010908EDT FGROUP CHOWANRV 1.00 .RRIMULT 10150908EDT... (10 Replies)
Discussion started by: chrismcg24
10 Replies

7. Shell Programming and Scripting

Updating a line in a large csv file, with sed/awk?

I have an extremely large csv file that I need to search the second field, and upon matches update the last field... I can pull the line with awk.. but apparently you cant use awk to directly update the file? So im curious if I can use sed to do this... The good news is the field I want to... (5 Replies)
Discussion started by: trey85stang
5 Replies

8. UNIX for Dummies Questions & Answers

Updating specific fields with awk using conditions

Can someone help me again, I think with awk? I have a file with 4 columns (pipe-delimited): I just want to convert the last field so that e1 is now 'message 1', e2 is 'message 2', e0 is 'message 3', etc. I don't want to change any other columns because the e0-e10 code may appear as part of a... (4 Replies)
Discussion started by: giannicello
4 Replies

9. UNIX for Dummies Questions & Answers

Korn shell awk use for updating two files

Hi, I have two text files containing records in following format: file1 format is: name1 age1 nickname1 path1 name2 age2 nickname2 path2 file 1 example is: abcd 13 abcd.13 /home/temp/abcd.13 efgh 15 efgh.15 /home/temp/new/efgh.15 (4 Replies)
Discussion started by: alrinno
4 Replies

10. UNIX for Dummies Questions & Answers

Constantly updating log files (tail -f? grep? awk?)

I have a log file which is continuously added to, called log.file. I'd like to monitor this file, and when certain lines are found, update some totals in another file. I've played around with tail -f, grep, and awk, but can't seem to hit the right note, so to speak. The lines I'm... (0 Replies)
Discussion started by: nortonloaf
0 Replies
Login or Register to Ask a Question