awk "intelligently" paste two files together.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk "intelligently" paste two files together.
# 1  
Old 08-28-2012
awk "intelligently" paste two files together.

Hi everyone,

I am trying to join two files together based on a field present in both files but not necessarily in the same order.

File 1:
Code:
123 user1
245 user2
559 user3

File 2:
Code:
123 user1
246 user4
544 user2

Also, sometimes a user may not even be in both files. What I want awk to do is look at field 2 and, when it finds the user in both files, add up the numbers in column one and print the username and the total. So desired output

Code:
246 user1
789 user2

I have seen a couple of similar examples here on the unix forums. However, nothing is exactly what I am trying to do and I just don't understand awk well enough to adapt for this purpose. Any help is greatly appreciated. Thanks so much.


Moderator's Comments:
Mod Comment Please use code tags next time for your code and data.

Last edited by zaxxon; 08-29-2012 at 11:50 AM.. Reason: code tags
# 2  
Old 08-28-2012
If it doesn't have to be awk, then this would probably do the job:
Code:
#!/bin/sh
while read file1line
do
  value=`echo "$file1line" | cut -d ' ' -f 1`
  user=`echo "$file1line" | cut -d ' ' -f 2`
  # Depending on your os, you might need egrep here instead of grep -e
  file2value=`grep -e " $user\$" $2 | tail -1 | cut -d ' ' -f 1`
  if [ -n "$file2value" ]
  then
    # We've found one
    value=`expr $value + $file2value`
    echo "$value $user"
  fi
done < $1

(Untested)

Usage:
Code:
  ./yourscript file1 file2 > outputfile

This will have the following potential issues:
If a user appears twice in the first file, it will be output twice as though it were two different users (use another grep over the first file to locate additional identical users to fix this if it matters).
If a user appears twice in the second file, only the last detected occurance of the user will ever be used (use a while loop on the second file to fix this if it matters).
This User Gave Thanks to Smiling Dragon For This Post:
# 3  
Old 08-28-2012
Code:
awk '{sum[$2]+=$1}END{for(i in sum) print sum[i],i}' file1 file2

This User Gave Thanks to complex.invoke For This Post:
# 4  
Old 08-29-2012
Thanks so much for the replies. Both work well. I thought that making tweaks would be a simple matter, but I am having a bit of trouble making a small change. I would also like to be able to return the difference between the numbers in the two files as well as the sum. I am having trouble altering the awk one liner solution for this purpose.

Smiling Dragon, your solution works great. Also, I was able to tailor it to what I need. Man, sometimes with AWK I just can't tell what's going on.

Last edited by collards; 08-29-2012 at 12:29 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash script - Print an ascii file using specific font "Latin Modern Mono 12" "regular" "9"

Hello. System : opensuse leap 42.3 I have a bash script that build a text file. I would like the last command doing : print_cmd -o page-left=43 -o page-right=22 -o page-top=28 -o page-bottom=43 -o font=LatinModernMono12:regular:9 some_file.txt where : print_cmd ::= some printing... (1 Reply)
Discussion started by: jcdole
1 Replies

2. Shell Programming and Scripting

Delete all log files older than 10 day and whose first string of the first line is "MSH" or "<?xml"

Dear Ladies & Gents, I have a requirement to delete all the log files in /var/log/test directory that are older than 10 days and their first line begin with "MSH" or "<?xml" or "FHS". I've put together the following BASH script, but it's erroring out: for filename in $(find /var/log/test... (2 Replies)
Discussion started by: Hiroshi
2 Replies

3. Web Development

How would I mod_rewrite "/~a1Pha" and "/=a1Pha" to "/paste/a1Pha.htm"? (internally & externally)

Basically I want to shorten URLs on my html pasting site (pasteht.ml), by using "/~a1Pha" instead of "/paste/a1Pha". The ID is 5 numbers and letters, both cases. For example: /~idnum serves /paste/idnum.htm /=idnum serves /paste/idnum.htm /paste/idnum redirects to /~idnum (to update any old... (0 Replies)
Discussion started by: phillips1012
0 Replies

4. Shell Programming and Scripting

how to use "cut" or "awk" or "sed" to remove a string

logs: "/home/abc/public_html/index.php" "/home/abc/public_html/index.php" "/home/xyz/public_html/index.php" "/home/xyz/public_html/index.php" "/home/xyz/public_html/index.php" how to use "cut" or "awk" or "sed" to get the following result: abc abc xyz xyz xyz (8 Replies)
Discussion started by: timmywong
8 Replies

5. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

6. Shell Programming and Scripting

"Join" or "Merge" more than 2 files into single output based on common key (column)

Hi All, I have working (Perl) code to combine 2 input files into a single output file using the join function that works to a point, but has the following limitations: 1. I am restrained to 2 input files only. 2. Only the "matched" fields are written out to the "matched" output file and... (1 Reply)
Discussion started by: Katabatic
1 Replies

7. Shell Programming and Scripting

Delete files older than "x" if directory size is greater than "y"

I wrote a script to delete files which are older than "x" days, if the size of the directory is greater than "y" #!/bin/bash du -hs $1 while read SIZE ENTRY do if ; then find $1 -mtime +$2 -exec rm -f {} \; echo "Files older than $2 days deleted" else echo "free Space available"... (4 Replies)
Discussion started by: JamesCarter
4 Replies

8. Shell Programming and Scripting

cat $como_file | awk /^~/'{print $1","$2","$3","$4}' | sed -e 's/~//g'

hi All, cat file_name | awk /^~/'{print $1","$2","$3","$4}' | sed -e 's/~//g' Can this be done by using sed or awk alone (4 Replies)
Discussion started by: harshakusam
4 Replies

9. Shell Programming and Scripting

help for saving vertical datas to horizontal with "awk" or "cut"

hi, i have a file having datas like that ./a.txt 12344 12345 12346 12347 ..... ..... ... i want to save this datas to another file like that ./b.txt 12344 12345 12346 12347 ... ... ... i think awk can make this but how? :) waiting for ur help. (3 Replies)
Discussion started by: mercury
3 Replies

10. UNIX for Dummies Questions & Answers

Explain the line "mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'`"

Hi Friends, Can any of you explain me about the below line of code? mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'` Im not able to understand, what exactly it is doing :confused: Any help would be useful for me. Lokesha (4 Replies)
Discussion started by: Lokesha
4 Replies
Login or Register to Ask a Question