Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

comparing 2 text files to get unique values??

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting comparing 2 text files to get unique values??
# 8  
Old 12-16-2008
Code:
nawk -v OFS='\t' '$1==1 {$1=$1;print}'

" -v OFS='\t' " - set the OutputFieldSeparator to '\t' (tab)
" $1 == 1" - if the value of field 1 ($1) is 1 (the number of occurrences in the combined file), then do {...}

" {$1=$1; print} " - force the reevaluation of the current record/line - forcing the '\t' delimited fields

" | cut -f2- " - given a default field delimiter of '\t', 'cut' everything starting at field 2.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Merge 4 bim files by keeping only the overlapping variants (unique rs values )

Dear community, I am facing a problem and I kindly ask your help: I have 4 different data sets consisted from 3 different types of array. On each file, column 1 is chromosome position, column 2 is SNP id etc... Lets say I have the following (bim) datasets: x2014: 1 rs3094315... (4 Replies)
Discussion started by: fondan
4 Replies

2. Shell Programming and Scripting

How to identify varying unique fields values from a text file in UNIX?

Hi, I have a huge unsorted text file. We wanted to identify the unique field values in a line and consider those fields as a primary key for a table in upstream system. Basically, the process or script should fetch the values from each line that are unique compared to the rest of the lines in... (13 Replies)
Discussion started by: manikandan23
13 Replies

3. Shell Programming and Scripting

How to merge two files with unique values matching.?

I have one script as below: #!/bin/ksh Outputfile1="/home/OutputFile1.xls" Outputfile2="/home/OutputFile2.xls" InputFile1="/home/InputFile1.sql" InputFile2="/home/InputFile2.sql" echo "Select hobby, class, subject, sports, rollNumber from Student_Table" >> InputFile1 echo "Select rollNumber... (3 Replies)
Discussion started by: Sharma331
3 Replies

4. Shell Programming and Scripting

Count Unique values from multiple lists of files

Looking for a little help here. I have 1000's of text files within a multiple folders. YYYY/ /MM /1000's Files Eg. 2014/01/1000 files 2014/02/1237 files 2014/03/1400 files There are folders for each year and each month, and within each monthly folder there are... (4 Replies)
Discussion started by: whegra
4 Replies

5. Shell Programming and Scripting

Comparing the values of two files

Hi Am trying to compare the values of two files.. One is a big file that has many values and the other is a small file.. The big file has all values present in small file.. # cat SmallFile 4456602 22347881 7471282 15859891 8257690 21954701 7078068 18219229 2883826 6094959 100000 ... (3 Replies)
Discussion started by: Priya Amaresh
3 Replies

6. UNIX for Dummies Questions & Answers

Comparing two test files and printing out the values that do not match

Hi, I have two text files with matching first columns. Some of the values in the second column do not match. I want to write a script to print out the rows (only the first column) where the values in the second column do not match. Example: Input 1 A 1 B 2 C 3 D 4 Input 2 A 2 B 2... (6 Replies)
Discussion started by: evelibertine
6 Replies

7. UNIX for Dummies Questions & Answers

Comparing two text files by a column and printing values that do not match

I have two text files where the first three columns are exactly the same. I want to compare the fourth column of the text files and if the values are different, print that row into a new output file. How do I go about doing that? File 1: 100 rs3794811 0.01 0.3434 100 rs8066551 0.01... (8 Replies)
Discussion started by: evelibertine
8 Replies

8. UNIX for Dummies Questions & Answers

Comparing two files and dividing the values

Hi all, I am new to unix and I am trying hard to get this requirement, but no luck. I am trying to compare two cloumns in two files and if it matches, the last column in file1 must be divided by file2 and the output must be written in a new file. To elaborate the 2nd column in file1 (EUR) must be... (6 Replies)
Discussion started by: smadderla
6 Replies

9. Shell Programming and Scripting

Comparing 2 files and return the unique lines in first file

Hi, I have 2 files file1 ******** 01-05-09|java.xls| 02-05-08|c.txt| 08-01-09|perl.txt| 01-01-09|oracle.txt| ******** file2 ******** 01-02-09|windows.xls| 02-05-08|c.txt| 01-05-09|java.xls| 08-02-09|perl.txt| 01-01-09|oracle.txt| ******** (8 Replies)
Discussion started by: shekhar_v4
8 Replies

10. UNIX for Dummies Questions & Answers

Need to find only unique values for a given tag across the files

Need to find only unique values for a given tag across the files: For eg: Test1: <Tag1>aaa</Tag1> <Tag2>bbb</Tag2> <Tag3>ccc</Tag3> Test2: <Tag1>aaa</Tag1> <Tag2>ddd</Tag2> <Tag3>eee</Tag3> Test3: <Tag1>aaa</Tag1> <Tag2>ddd</Tag2> <Tag3>eee</Tag3> Test4: (8 Replies)
Discussion started by: sudheshnaiyer
8 Replies
Login or Register to Ask a Question

LEARN ABOUT DEBIAN

h5diff

h5diff(1)						      General Commands Manual							 h5diff(1)

NAME

       h5diff - Compares two HDF5 files and reports the differences.

SYNOPSIS

       h5diff file1 file2 [OPTIONS] [object1 [object2 ] ]

DESCRIPTION

       h5diff is a command line tool that compares two HDF5 files, file1 and file2, and reports the differences between them.

       Optionally,  h5diff  will compare two objects within these files. If only one object, object1, is specified, h5diff will compare object1 in
       file1 with object1 in file2. In two objects, object1 and object2, are specified, h5diff will compare  object1  in  file1  with  object2	in
       file2. These objects must be HDF5 datasets.

       object1 and object2 must be expressed as absolute paths from the respective file's root group.

       Additional information, with several sample cases, can be found in the document H5diff Examples.

OPTIONS

       file1 file2
	      The HDF5 files to be compared.

       -h     Print all differences.

       -r     Print  only  the	names  of  objects that differ; do not print the differences. These objects may be HDF5 datasets, groups, or named
	      datatypes.

       -n count
	      Print difference up to count differences, then stop. count must be a positive integer.

       -d delta
	      Print only differences that are greater than the limit delta. delta must be a positive number. The comparison criterion  is  whether
	      the  absolute  value of the difference of two corresponding values is greater than delta (e.g., |a-b| > delta, where a is a value in
	      file1 and b is a value in file2).

       -p relative
	      Print only differences that are greater than a relative error. relative must be a  positive  number.  The  comparison  criterion	is
	      whether the absolute value of the difference 1 and the ratio of two corresponding values is greater than relative (e.g., |1-(b/a)| >
	      relative where a is a value in file1 and b is a value in file2).

       object1 object2
	      Specific object(s) within the files to be compared.

EXAMPLES

       The following h5diff call compares the object /a/b in file1 with the object /a/c in file2:
	   h5diff file1 file2 /a/b /a/c

       This h5diff call compares the object /a/b in file1 with the same object in file2:
	   h5diff file1 file2 /a/b

       And this h5diff call compares all objects in both files:
	   h5diff file1 file2

SEE ALSO

       h5dump(1), h5ls(1), h5repart(1), h5import(1), gif2h5(1), h52gif(1), h5perf(1)

																	 h5diff(1)