02-03-2012
Compare multiple files and print unique lines
Hi friends,
I have multiple files. For now, let's say I have two of the following style
cat 1.txt
Quote:
A B C 1 2 3 4 5 6 7 8
D E F 9 8 7 6 5 4 3 2
G H I 0 1 2 3 4 5 6 7
cat 2.txt
Quote:
A B C -1 -2 3 4 5 6 -7 80
D E F 90 88 76 54 32 1 0 1
M N O 99 65 34 22 13 9 4 3
output.txt
Quote:
G H I 0 1 2 3 4 5 6 7 1.txt
M N O 99 65 34 22 13 9 4 3 2.txt
Please note that my files are not sorted and in the output file I need another extra column that says the file from which it is coming. I have more than 100 files to do this.
All helps are highly appreciated.
Thanks
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
hey guys, I have two files both with two columns, I have already created an
awk code to ignore certain lines (e.g lines that start with 963) as they wou
ld begin with a certain string, however, the rest I have added together and
calculated the average.
At the moment the code also displays... (3 Replies)
Discussion started by: chlfc
3 Replies
2. Shell Programming and Scripting
I have two files like this:
#FILE 1
ABCD 4322 26485
JMTJ 5311 97248
XMPJ 4321 58978
#FILE 2
ABCD 4321 26485
JMTJ 5311 97248
XMPJ 4321 68978
What to do: Compare the two files and find those lines that doesn't match. And have a new file like this:
#FILE 3
"from file 1"
ABCD 4322 26485... (11 Replies)
Discussion started by: kingpeejay
11 Replies
3. Shell Programming and Scripting
Hi,
i have an input text file like this:
Student 1
maths
science
=
Student 2
maths
science
=
Student 3
maths
science
i would like to print each student information into separate files, each student id is separated by "=". (1 Reply)
Discussion started by: saint2006
1 Replies
4. Shell Programming and Scripting
Hi.
I have a tab separated file that has a couple nearly identical lines. When doing:
sort file | uniq > file.new
It passes through the nearly identical lines because, well, they still are unique.
a)
I want to look only at field x for uniqueness and if the content in field x is the... (1 Reply)
Discussion started by: rocket_dog
1 Replies
5. Shell Programming and Scripting
hi
my problem is little complicated one. i have 2 files which appear like this
file 1
abbsss:aa:22:34:as akl abc 1234
mkilll:as:ss:23:qs asc abc 0987
mlopii:cd:wq:24:as asd abc 7866
file2
lkoaa:as:24:32:sa alk abc 3245
lkmo:as:34:43:qs qsa abc 0987
kloia:ds:45:56:sa acq abc 7805
i... (5 Replies)
Discussion started by: anurupa777
5 Replies
6. Shell Programming and Scripting
Good morning all,
I have a problem that is one step beyond a standard awk compare.
I would like to compare three files which have several thousand records against a fourth file. All of them have a value in each row that is identical, and one value in each of those rows which may be duplicated... (1 Reply)
Discussion started by: nashton
1 Replies
7. UNIX for Dummies Questions & Answers
I would like to print unique lines without sort or unique. Unfortunately the server I am working on does not have sort or unique. I have not been able to contact the administrator of the server to ask him to add it for several weeks. (7 Replies)
Discussion started by: cokedude
7 Replies
8. Shell Programming and Scripting
Hi,
I have multiple files that each contain one column of strings:
File1:
123abc
456def
789ghi
File2:
123abc
456def
891jkl
File3:
234mno
123abc
456def
In total I have 25 of these type of file. (5 Replies)
Discussion started by: owwow14
5 Replies
9. Shell Programming and Scripting
Hello,
I would like to ask for help with csh script.
An example of an input in .txt file is below, the number of lines varies from file to file and I have 2 or 3 columns with values. I would like to read all the values (probably one by one) and set them to independent unique variables that... (7 Replies)
Discussion started by: FMMOLA
7 Replies
10. UNIX for Beginners Questions & Answers
I have a directory of files, I can show the number of lines in each file and order them from lowest to highest with:
wc -l *|sort
15263 Image.txt
16401 reference.txt
40459 richtexteditor.txt
How can I also print the number of unique lines in each file?
15263 1401 Image.txt
16401... (15 Replies)
Discussion started by: spacegoose
15 Replies
cat(1) General Commands Manual cat(1)
Name
cat - concatenate and print data
Syntax
cat [ -b ] [ -e ] [ -n ] [ -s ] [ -t ] [ -u ] [ -v ] file...
Description
The command reads each file in sequence and displays it on the standard output. Therefore, to display the file on the standard output you
type:
cat file
To concatenate two files and place the result on the third you type:
cat file1 file2 > file3
To concatenate two files and append them to a third you type:
cat file1 file2 >> file3
If no input file is given, or if a minus sign (-) is encountered as an argument, reads from the standard input file. Output is buffered in
1024-byte blocks unless the standard output is a terminal, in which case it is line buffered. The utility supports the processing of 8-bit
characters.
Options
-b Ignores blank lines and precedes each output line with its line number.
-e Displays a dollar sign ($) at the end of each output line.
-n Precedes all output lines (including blank lines) with line numbers.
-s Squeezes adjacent blank lines from output and single spaces output.
-t Displays non-printing characters (including tabs) in output. In addition to those representations used with the -v option, all tab
characters are displayed as ^I.
-u Unbuffers output.
-v Displays non-printing characters (excluding tabs and newline) as the ^x. If the character is in the range octal 0177 to octal 0241,
it is displayed as M-x. The delete character (octal 0177) displays as ^?. For example, is displayed as ^X.
See Also
cp(1), ex(1), more(1), pr(1), tail(1)
cat(1)