find common data Post: 302606404

Sponsored Content

Top Forums Shell Programming and Scripting find common data Post 302606404 by drl on Sunday 11th of March 2012 08:55:13 AM

03-11-2012

Registered User

Hi.

If the data files are as posted, then a form of grep or fgrep can be used:

Code:

#!/usr/bin/env bash

# @(#) s1	Demonstrate finding common lines, special case, grep.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
edges() { local _f _n _l;: ${1?"edges: need file"}; _f=$1;_l=$(wc -l $_f);
  head -${_n:=3} $_f ; pe "--- ( $_l: lines total )" ; tail -$_n $_f ; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C grep

MAIN=${1-data1}
shift
PATS=${1-data2}

pl " Input data files $MAIN $PATS:"
edges $MAIN
pe
edges $PATS

pl " Results:"
grep -f $PATS $MAIN

exit 0

producing:

Code:

% ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0.8 (lenny) 
bash GNU bash 3.2.39
grep GNU grep 2.5.3

-----
 Input data files data1 data2:
a,1
b,2
c,343
--- ( 6 data1: lines total )
d,343
e,4343
f,4544

a,
d,
e,
--- ( 3 data2: lines total )
a,
d,
e,

-----
 Results:
a,1
d,343
e,4343

Best wishes ... cheers, drl

drl

View Public Profile for drl

Find all posts by drl

9 More Discussions You Might Find Interesting

1. Ubuntu

How to compare two columns and fetch the common data with additional column

Dear All, I am new to this forum and please ignore my little knowledge :p I have two types of data (a subset is given below) data version 1: 439798 2 1 451209 1 2 508696 2 1 555760 2 1 582757 1 2 582889 1 2 691827...

2. Shell Programming and Scripting

Find common entries in 2 list and write data before it

Hi all, I have 2 files: second file I want if entries in one file will match in other file. It shuld wite approve before it so output shuld be

3. Shell Programming and Scripting

Help with duplicate common data content

Input file: #data_131 0 >content..._* 1 >content..._at_+/97.20% #data_137 0 >content..._* 1 >content..._at_+/97.20% 2 >seq..._* 3 >content..._at_+/97.20% 4 >content..._at_+/97.20% #data_141 0 >content..._* #data_150 0 >content..._* 1 >content..._at_+/97.20% 2 >seq..._* 3...

4. Shell Programming and Scripting

Find common entries

Hi all I have to compare two files and find common entries First file is like this XVY CVY ZYN MNA In second file I have to search these entries in even number of columns 5 XVY 7 hdfj 8 CVY 9 if there is common entries then out put shuld be 5 XVY(approved) 7 hdfj 8...

5. UNIX for Dummies Questions & Answers

Extract common data out of multiple files

I am trying to extract common list of Organisms from different files For example I took 3 files and showed expected result. In real I have more than 1000 files. I am aware about the useful use of awk and grep but unaware in depth so need guidance regarding it. I want to use awk/ grep/ cut/...

6. Shell Programming and Scripting

Find Common Values Across Two Files

7. Shell Programming and Scripting

Help on writing data from 2 different files to one based on a common factor

Hello all, I have 2 text files. For example: File1.txt contains data A B C D ****NEXT**** X Y Z ****NEXT**** L M N and File2.txt contains data

8. Shell Programming and Scripting

Find the common values

Hi, I have two files with the below values. file1 305231921 1.0 ben/Ben_Determination_Appeals 1348791394 2.0 ben/Ben_Determination_Appeals] 1305231921 1.0 ben/Cancel_Refund_Payment_JLRS 1348791394 2.0 ben/Cancel_Refund_Payment_JLRS 1305231921 ...

9. Shell Programming and Scripting

Find common words

Hi, I have 10 files which needs to be print common words from those all files. Is there any command to find out.

LEARN ABOUT DEBIAN

dpkg-awk

DPKG-AWK(1)						      General Commands Manual						       DPKG-AWK(1)

NAME

       dpkg-awk - Utility to read a dpkg style db file

SYNOPSIS

       dpkg-awk [(-f|--file) filename] [(-d|--debug) ##] [(-s|--sort) list] [(-rs|--rec_sep) ??] '<fieldname>:<regex>' ... -- <out_fieldname> ..

DESCRIPTION

       dpkg-awk  Parses  a  dpkg status file (or other similarly formatted file) and outputs the resulting records.  It can use regex on the field
       values to limit the returned records, it can also be told which fields to output, and it can sort the matched fields.

OPTIONS

       -f filename
       --file filename
	      The file to parse.  The default is /var/lib/dpkg/status.

       -d [#]
       --debug [#]
	      Each time this is specified, it increased the debug level.

       -s field(s)
       --sort field(s)
	      A space or comma separated list of fields to sort on.

       -n field(s)
       --numeric field(s)
	      A space or comma separated list of fields that should be interpreted as numeric in value.

       -rs ??
       --rec_sep ??
	      Output this string at the end of each output paragraph.

       -h
       --help Display some help.

       fieldname
	      The fields from the file, that are matched with the regex given.	The fieldnames are case insensitive.

       out_fieldname
	      The fields from the file, that are output for each record.  If the first field listed begins with ^, then the list  of  fields  that
	      follows will NOT be output.

BUGS

       Be  warned  that  the author has only a shallow understanding of the dpkg packaging system, so there are probably tons of bugs in this pro-
       gram.

       This program comes with no warranties.  If running this program causes fire and brimstone to rain down upon the earth, you will be on  your
       own.

       This program accesses the dpkg database directly in places, querying for data that cannot be gotten via dpkg.

AUTHOR

       Adam Heath <doogie@debian.org>

DEBIAN
								 Debian Utilities						       DPKG-AWK(1)