Shell Programming and Scripting

I can't see a simple solution without sorting the data. This approach has only one sort. It works.

Code:

cat sourcefile1 sourcefile2 | sort | uniq -u > file3

Btw and off topic. I'm sure that the uuoc advocates can find some complicated way of concatonating the files.

Certainly not a UUOC advocate, but isn't that the same as:

Code:

sort -u sourcefile1 sourcefile2 > file3

Nope. sort -u prints one instance of duped lines. uniq -u does not print those lines (which in this case are unwanted).

I missed the -u uniq option in methyl's post. Oops, sorry

(at least the cat isn't neccessary )

Wonderful, thanks! It looks like both of these work

Code:

grep -vxf file1 file2 > file3
grep -vxf file2 file1 >> file3

using a single grep wouldn't see changes to the other file. The second version looks more elegant:

Code:

cat file1 file2 | sort | uniq -u > file3

though I'm not sure which method would use less cpu/mem? I'm sorting a few million records with this so resource usage may be an issue. Actually I have to repeat this process for many text files in a directory, so I may want to automate piping the ls of the dir and to this process, so at the end I get a dump of only unique records from all the files.