Unix/Linux Go Back    



Extract common data out of multiple files

UNIX for Dummies Questions & Answers




Kindly Note - This is a Single User Post by Forum Member Scrutinizer Regarding:
Extract common data out of multiple files.
Please Follow The Primary Link Above to View the Full Discussion.

   
Old Unix and Linux 12-24-2012   -   Original Discussion by macmath
Scrutinizer's Unix or Linux Image
Scrutinizer Scrutinizer is offline Forum Staff  
Moderator
 
Join Date: Nov 2008
Last Activity: 22 November 2017, 8:17 PM EST
Location: Amsterdam
Posts: 11,625
Thanks: 518
Thanked 3,381 Times in 2,980 Posts
If there is maximum of 1 entry per file:

Code:
awk '++A[$0]>=ARGC-1' file*

This would then be a bit more robust:

Code:
awk '{$1=$1} ++A[$0]>=ARGC-1' file*

But 1000 files is probably going to be too many for the command line length.

Otherwise try:

Code:
( 
  set -- file*
  for f
  do
    cat "$f"
  done | awk '{$1=$1} ++A[$0]>=c' c=$# 
)


Last edited by Scrutinizer; 12-24-2012 at 08:14 AM..
The Following User Says Thank You to Scrutinizer For This Useful Post:
dovah (11-15-2015)