Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Find common numbers from two very large files using awk or the like Post 302799617 by hanson44 on Friday 26th of April 2013 08:05:48 PM
Old 04-26-2013
Quote:
There is no point in specifying a numeric sort
You're right. In this case, the numeric sort has no effect for good or ill, but is superfluous, should not be used:
Code:
$ cat scottie.sh
sed "s/.*/1 &/" file1 > file1.lbl
sed "s/.*/2 &/" file2 > file2.lbl
cat file1.lbl file2.lbl | sort -k 2 > all.lbl
uniq -d -f 1 all.lbl | cut -f 2 -d " "
rm file1.lbl file2.lbl all.lbl

Code:
$ ./scottie.sh
123456000017214
123456000017255
123456000018300
123456000100123
123456000100253
223456000001212
223456000013212

-------------------------------
Or, as suggested by alister:
Code:
$ comm -1 -2 file1 file2
123456000017214
123456000017255
123456000018300
123456000100123
123456000100253
223456000001212
223456000013212

But the OP said there was some problem with this. Smilie
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Get un common numbers from two files

Hi, I have two files: abc : 50040 123123 31703 cde: 104 97 50040 123123 31703 36609 50534 (3 Replies)
Discussion started by: jingi1234
3 Replies

2. Shell Programming and Scripting

To find all common lines from 'n' no. of files

Hi, I have one situation. I have some 6-7 no. of files in one directory & I have to extract all the lines which exist in all these files. means I need to extract all common lines from all these files & put them in a separate file. Please help. I know it could be done with the help of... (11 Replies)
Discussion started by: The Observer
11 Replies

3. UNIX for Dummies Questions & Answers

Grep alternative to handle large numbers of files

I am looking for a file with 'MCR0000000716214' in it. I tried the following command: grep MCR0000000716214 * The problem is that the folder I am searching in has over 87000 files and I am getting the following: bash: /bin/grep: Arg list too long Is there any command I can use that can... (6 Replies)
Discussion started by: runnerpaul
6 Replies

4. Shell Programming and Scripting

Drop common lines at head/tail of a large set of files

Hi! I have a large set of pairs of text files (each pair in their own subdirectory) and each pair shares head/tail (a couple of first and last lines) but differs in the middle part. I need to delete the heads/tails and keep only the middle portions in which they differ. The lengths of heads/tails... (1 Reply)
Discussion started by: dobryden
1 Replies

5. UNIX for Advanced & Expert Users

Find common Strings in two large files

Hi , I have a text file in the format DB2: DB2: WB: WB: WB: WB: and a second text file of the format Time=00:00:00.473 Time=00:00:00.436 Time=00:00:00.016 Time=00:00:00.027 Time=00:00:00.471 Time=00:00:00.436 the last string in both the text files is of the... (4 Replies)
Discussion started by: kanthrajgowda
4 Replies

6. Shell Programming and Scripting

finding common numbers (contents) across 2 or 3 files

I have 3 files which are tab delimited and have numbers in it. file 1 1 2 3 4 5 6 7 File 2 3 5 7 8 File 3 1 (4 Replies)
Discussion started by: Lucky Ali
4 Replies

7. Shell Programming and Scripting

Find common numbers and print yes or no

Hi I have 2 files with following data First file, sp|Q676U5|A16L1_HUMAN, Autophagy-related protein 16-1 OS=Homo sapiens GN=ATG16L1 PE=1 SV=2, Maximum coiled-coil residue probability: 0.657 in position 163. Maximum dimeric residue probability: 0.288 in position 163. ... (1 Reply)
Discussion started by: manigrover
1 Replies

8. Shell Programming and Scripting

Find Common Values Across Two Files

Hi All, I have two files like below: File1 MYFILE_28012012_1112.txt|4 MYFILE_28012012_1113.txt|51 MYFILE_28012012_1114.txt|57 MYFILE_28012012_1115.txt|57 MYFILE_28012012_1116.txt|57 MYFILE_28012012_1117.txt|57 File2 MYFILE_28012012_1110.txt|57 MYFILE_28012012_1111.txt|57... (2 Replies)
Discussion started by: angshuman
2 Replies

9. Shell Programming and Scripting

Find common files between two directories

I have two directories Dir 1 /home/sid/release1 Dir 2 /home/sid/release2 I want to find the common files between the two directories Dir 1 files /home/sid/release1>ls -lrt total 16 -rw-r--r-- 1 sid cool 0 Jun 19 12:53 File123 -rw-r--r-- 1 sid cool 0 Jun 19 12:53... (5 Replies)
Discussion started by: sidnow
5 Replies
h5diff(1)						      General Commands Manual							 h5diff(1)

NAME
h5diff - Compares two HDF5 files and reports the differences. SYNOPSIS
h5diff file1 file2 [OPTIONS] [object1 [object2 ] ] DESCRIPTION
h5diff is a command line tool that compares two HDF5 files, file1 and file2, and reports the differences between them. Optionally, h5diff will compare two objects within these files. If only one object, object1, is specified, h5diff will compare object1 in file1 with object1 in file2. In two objects, object1 and object2, are specified, h5diff will compare object1 in file1 with object2 in file2. These objects must be HDF5 datasets. object1 and object2 must be expressed as absolute paths from the respective file's root group. Additional information, with several sample cases, can be found in the document H5diff Examples. OPTIONS
file1 file2 The HDF5 files to be compared. -h Print all differences. -r Print only the names of objects that differ; do not print the differences. These objects may be HDF5 datasets, groups, or named datatypes. -n count Print difference up to count differences, then stop. count must be a positive integer. -d delta Print only differences that are greater than the limit delta. delta must be a positive number. The comparison criterion is whether the absolute value of the difference of two corresponding values is greater than delta (e.g., |a-b| > delta, where a is a value in file1 and b is a value in file2). -p relative Print only differences that are greater than a relative error. relative must be a positive number. The comparison criterion is whether the absolute value of the difference 1 and the ratio of two corresponding values is greater than relative (e.g., |1-(b/a)| > relative where a is a value in file1 and b is a value in file2). object1 object2 Specific object(s) within the files to be compared. EXAMPLES
The following h5diff call compares the object /a/b in file1 with the object /a/c in file2: h5diff file1 file2 /a/b /a/c This h5diff call compares the object /a/b in file1 with the same object in file2: h5diff file1 file2 /a/b And this h5diff call compares all objects in both files: h5diff file1 file2 SEE ALSO
h5dump(1), h5ls(1), h5repart(1), h5import(1), gif2h5(1), h52gif(1), h5perf(1) h5diff(1)
All times are GMT -4. The time now is 09:35 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy