comparing 2 text files to get unique values??

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Merge 4 bim files by keeping only the overlapping variants (unique rs values )

Dear community, I am facing a problem and I kindly ask your help: I have 4 different data sets consisted from 3 different types of array. On each file, column 1 is chromosome position, column 2 is SNP id etc... Lets say I have the following (bim) datasets: x2014: 1 rs3094315...

2. Shell Programming and Scripting

How to identify varying unique fields values from a text file in UNIX?

Hi, I have a huge unsorted text file. We wanted to identify the unique field values in a line and consider those fields as a primary key for a table in upstream system. Basically, the process or script should fetch the values from each line that are unique compared to the rest of the lines in...

3. Shell Programming and Scripting

How to merge two files with unique values matching.?

I have one script as below: #!/bin/ksh Outputfile1="/home/OutputFile1.xls" Outputfile2="/home/OutputFile2.xls" InputFile1="/home/InputFile1.sql" InputFile2="/home/InputFile2.sql" echo "Select hobby, class, subject, sports, rollNumber from Student_Table" >> InputFile1 echo "Select rollNumber...

4. Shell Programming and Scripting

Count Unique values from multiple lists of files

Looking for a little help here. I have 1000's of text files within a multiple folders. YYYY/ /MM /1000's Files Eg. 2014/01/1000 files 2014/02/1237 files 2014/03/1400 files There are folders for each year and each month, and within each monthly folder there are...

5. Shell Programming and Scripting

Comparing the values of two files

Hi Am trying to compare the values of two files.. One is a big file that has many values and the other is a small file.. The big file has all values present in small file.. # cat SmallFile 4456602 22347881 7471282 15859891 8257690 21954701 7078068 18219229 2883826 6094959 100000 ...

6. UNIX for Dummies Questions & Answers

Comparing two test files and printing out the values that do not match

Hi, I have two text files with matching first columns. Some of the values in the second column do not match. I want to write a script to print out the rows (only the first column) where the values in the second column do not match. Example: Input 1 A 1 B 2 C 3 D 4 Input 2 A 2 B 2...

7. UNIX for Dummies Questions & Answers

Comparing two text files by a column and printing values that do not match

I have two text files where the first three columns are exactly the same. I want to compare the fourth column of the text files and if the values are different, print that row into a new output file. How do I go about doing that? File 1: 100 rs3794811 0.01 0.3434 100 rs8066551 0.01...

8. UNIX for Dummies Questions & Answers

Comparing two files and dividing the values

Hi all, I am new to unix and I am trying hard to get this requirement, but no luck. I am trying to compare two cloumns in two files and if it matches, the last column in file1 must be divided by file2 and the output must be written in a new file. To elaborate the 2nd column in file1 (EUR) must be...

9. Shell Programming and Scripting

Comparing 2 files and return the unique lines in first file

Hi, I have 2 files file1 ******** 01-05-09|java.xls| 02-05-08|c.txt| 08-01-09|perl.txt| 01-01-09|oracle.txt| ******** file2 ******** 01-02-09|windows.xls| 02-05-08|c.txt| 01-05-09|java.xls| 08-02-09|perl.txt| 01-01-09|oracle.txt| ********

10. UNIX for Dummies Questions & Answers

Need to find only unique values for a given tag across the files

Need to find only unique values for a given tag across the files: For eg: Test1: <Tag1>aaa</Tag1> <Tag2>bbb</Tag2> <Tag3>ccc</Tag3> Test2: <Tag1>aaa</Tag1> <Tag2>ddd</Tag2> <Tag3>eee</Tag3> Test3: <Tag1>aaa</Tag1> <Tag2>ddd</Tag2> <Tag3>eee</Tag3> Test4:

LEARN ABOUT SUNOS

comm

comm(1) 							   User Commands							   comm(1)

NAME

       comm - select or reject lines common to two files

SYNOPSIS

       comm [-123] file1 file2

DESCRIPTION

       The comm utility reads file1 and file2, which must be ordered in the current collating sequence, and produces three text columns as output:
       lines only in file1; lines only in file2; and lines in both files.

       If the input files were ordered according to the collating sequence of the current locale, the lines  written  will  be	in  the  collating
       sequence of the original lines. If not, the results are unspecified.

OPTIONS

       The following options are supported:

       -1	Suppresses the output column of lines unique to file1.

       -2	Suppresses the output column of lines unique to file2.

       -3	Suppresses the output column of lines duplicated in file1 and file2.

OPERANDS

       The following operands are supported:

       file1	A path name of the first file to be compared. If file1 is -, the standard input is used.

       file2	A path name of the second file to be compared. If file2 is -, the standard input is used.

USAGE

       See largefile(5) for the description of the behavior of comm when encountering files greater than or equal to 2 Gbyte ( 2**31 bytes).

EXAMPLES

       Example 1: Printing a list of utilities specified by files

       If file1, file2, and file3 each contain a sorted list of utilities, the command

       example% comm -23 file1 file2  | comm -23 - file3

       prints a list of utilities in file1 not specified by either of the other files. The entry:

       example% comm -12 file1 file2 | comm -12 - file3

       prints a list of utilities specified by all three files. And the entry:

       example% comm -12  file2 file3 | comm -23 -file1

       prints a list of utilities specified by both file2 and file3, but not specified in file1.

ENVIRONMENT VARIABLES

       See  environ(5)	for  descriptions  of  the  following  environment  variables that affect the execution of comm: LANG, LC_ALL, LC_COLLATE,
       LC_CTYPE, LC_MESSAGES, and NLSPATH.

EXIT STATUS

       The following exit values are returned:

       0	All input files were successfully output as specified.

       >0	An error occurred.

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |Availability		     |SUNWesu			   |
       +-----------------------------+-----------------------------+
       |CSI			     |enabled			   |
       +-----------------------------+-----------------------------+
       |Interface Stability	     |Standard			   |
       +-----------------------------+-----------------------------+

SEE ALSO

       cmp(1), diff(1), sort(1), uniq(1), attributes(5), environ(5), largefile(5), standards(5)

SunOS 5.10							    3 Mar 2004								   comm(1)

Shell Programming and Scripting