Simple two file compare with twist


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Simple two file compare with twist
# 8  
Old 05-17-2012
Yes, that is correct. Awk uses so-called associative arrays..
This User Gave Thanks to Scrutinizer For This Post:
# 9  
Old 05-18-2012
understood, thanks.

speaking of making the code for efficient, the current method will become expensive soon as file 2 will grow to thousands of lines whereas file1 will only be around 20-30 lines at most. so currently, the code will iterate through each line of file2 and compare against file1 (which is loaded in the array).

it would be more economical to switch this around i.e load file2 into the array and lookup file1 so if file1 only has 5 lines, it will stop at that and hence be quicker. is my logic correct?
# 10  
Old 07-25-2012
Quote:
Originally Posted by Scrutinizer
Try:
Code:
awk -F, 'FNR==NR {arr[$1]=$2; next} $3 in arr {print $2,$4,$5,arr[$3]}' OFS=, file1 file2

found my first little "bug" with this, more like "shortcoming" as i didnt plan for it.
if file one has field1 appearing more than once (on a separate line), i would need that in the output with its corresponding field2. eg:

cat file1
Code:
foo,cmd1
bar,cmd2
foo,cmd3

cat file2
Code:
Hello,World,foo
Alice,Bob,bar
Egg,Spam,ham

desired output:
Code:
World,fi,fom,cmd1
Bob,bie,doll,cmd2
World,fi,fom,cmd3


currently this is not the case as the script looks in file2 and if field3 is found in array A, prints the output. this will only ever match once as field3 will only be in file2 once. hope that makes sense.

i think we need to do a reverse look up perhaps? i.e use field1,file1 and lookup against file2?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

File Listing, with a Twist?

Greetings! I have a quick question which must be deferred to those with greater skill than myself :) In this situation, I wish to create a list of all the files on an entire partition in descending order sorted by date. I tried numerous switches for ls, and found this line to be the closest... (4 Replies)
Discussion started by: LinQ
4 Replies

2. Shell Programming and Scripting

Simple two file compare with twist

I have file1 and file2 I lookup field3 from file2 in field1 of file1 and if there is a match, output field 2,3,5 from file2. I now want to add field2 of file1 in the output. I suspect what I have to do is read the entire line of file1 into a 2 dim array? pls help. (1 Reply)
Discussion started by: tmonk1
1 Replies

3. Shell Programming and Scripting

Simple awk command to compare two files and print first difference

Hello, I have two text files, each with a single column, file 1: 124152970 123899868 123476854 54258288 123117283 file 2: 124152970 123899868 54258288 123117283 122108330 (5 Replies)
Discussion started by: LMHmedchem
5 Replies

4. Shell Programming and Scripting

Multiple File renaming with a twist

Hi I can do simple file renaming but this task is slightly more troublesome Ive got a guy that gives me multiple .pdf filles in a directory named something like 3412345.pdf 4565465.pdf 8534534.pdf And he also gives me a html file which is tabled with which shows the filenames above... (2 Replies)
Discussion started by: messiah1
2 Replies

5. Shell Programming and Scripting

Section Removal With sed; and With a Twist . . .

Hello folks! Raised a bump on my head trying to figure this one out ;) I have an xml file which needs to be edited, removing an entire property section in the work. Here's what the target section layout looks like: <property name="something"> {any number of lines go here} </property>... (7 Replies)
Discussion started by: LinQ
7 Replies

6. Shell Programming and Scripting

Incrementing with a twist - please help

I'm currently trying to write a ksh or csh script that would change the name of a file found in directories and attach to the name an incrementing three digit number. I know how to write a script that will go: 000, 001, 002, 003, etc The twist is I need more increments then allowed by a 3... (11 Replies)
Discussion started by: Rust
11 Replies

7. UNIX for Dummies Questions & Answers

file count with a twist

Hello Everyone, I am using the korn shell. I was hoping to find a set of commands to count files in a directory. I am using: ls /home/name/abc* | wc -l This command works fine when a file matches abc* (returns only the file count) , however when no file(s) are found I get... (2 Replies)
Discussion started by: robert4732
2 Replies

8. UNIX for Advanced & Expert Users

building a kernel (with a twist)

Hey all, I am working on a static analysis tool and I wan't to see if it can find bugs in the linux kernel, it uses LLVM framework to analyse the instructions. Long story short I need to build the kernel with a custom compiler. The compiler will create byte code files where binaries usually... (2 Replies)
Discussion started by: zigga15
2 Replies

9. Shell Programming and Scripting

Compare 2 files yet again but with a twist

Ok so I have a file which contains 2 columns/fields and I have another file with 2 columns. The files look like: file1: 1 33 5 345 18 2 45 1 78 31 file2: 1 c1d2t0 2 c1d3t0 3 c1d4t0 4 c1d4t0 5 c2d1t0 6 c2d1t0 7 c2d1t0 8 c2d1t0 9 c2d1t0 10 c2d1t0 (11 Replies)
Discussion started by: Autumn Tree
11 Replies

10. UNIX for Dummies Questions & Answers

how do I log into this machine - with a twist...

I know this topic has been covered in one form or another, but it hasn't been covered to handle my problem. I was given a Sparc4 running Solaris 2.5.1 The root password is unknown. This machine has no cdrom drive and it has no floppy drive. I tried booting into the single user mode, but... (1 Reply)
Discussion started by: xyyz
1 Replies
Login or Register to Ask a Question