Unix/Linux Go Back    

Extract common data out of multiple files

UNIX for Dummies Questions & Answers

Kindly Note - This is a Single User Post by Forum Member Scrutinizer Regarding:
Extract common data out of multiple files.
Please Follow The Primary Link Above to View the Full Discussion.

Old Unix and Linux 12-24-2012   -   Original Discussion by macmath
Scrutinizer's Unix or Linux Image
Scrutinizer Scrutinizer is offline Forum Staff  
Join Date: Nov 2008
Last Activity: 22 February 2018, 4:42 AM EST
Location: Amsterdam
Posts: 11,734
Thanks: 536
Thanked 3,406 Times in 3,001 Posts
If there is maximum of 1 entry per file:

awk '++A[$0]>=ARGC-1' file*

This would then be a bit more robust:

awk '{$1=$1} ++A[$0]>=ARGC-1' file*

But 1000 files is probably going to be too many for the command line length.

Otherwise try:

  set -- file*
  for f
    cat "$f"
  done | awk '{$1=$1} ++A[$0]>=c' c=$# 

Last edited by Scrutinizer; 12-24-2012 at 08:14 AM..
The Following User Says Thank You to Scrutinizer For This Useful Post:
dovah (11-15-2015)