12-13-2013
@zozoo :
You should sort before using uniq, in current example it's fine, but in some cases there will be chances of printing duplicates.
This User Gave Thanks to Akshay Hegde For This Post:
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi,
i'm new here (and to scripting too).
I was hoping for some help in comparing two files.
i have a file called 'file1' with a list of names in the following format:
adam jones
paul higgins
kelly lowe
i also have another file which may contain some of the names but with a lot of... (4 Replies)
Discussion started by: ibking
4 Replies
2. UNIX for Dummies Questions & Answers
I have a file that looks like this
452 025_E3
8 025_E3
82 025_F5
135 025_F5
5 025_F5
23 025_G2
38 025_G2
71 025_G2
9 026_A12
81 026_A12
10 026_A12
some of the elements in column2 are repeated.
I want an output file that will extract the... (1 Reply)
Discussion started by: FelipeAd
1 Replies
3. Shell Programming and Scripting
Hi,
I have a file like this
ACC 2 2 21 aaa
AC 443 3 22 aaa
GCT 76 1 33 xxx
TCG 34 2 33 aaa
ACGT 33 1 22 ggg
TTC 99 3 44 wee
CCA 33 2 33 ggg
AAC 1 3 55 ddd
TTG 10 1 22 ddd
TTGC 98 3 22 ddd
GCT 23 1 21 sds
GTC 23 4 32 sds
ACGT 32 2 33 vvv
CGT 11 2 33 eee
CCC 87 2 44... (1 Reply)
Discussion started by: polsum
1 Replies
4. Shell Programming and Scripting
I have the following records from multiple files.
415 A G
415 A G
415 A T
415 A .
415 A .
421 G A
421 G A,C
421 G A
421 G A
421 G A,C
421 G .
427 A C
427 A ... (3 Replies)
Discussion started by: empyrean
3 Replies
5. UNIX for Dummies Questions & Answers
Hi all,
I know how to merge two tables and to remove the duplicated lines based on a field (Column 2) . My next challenge is to be able to identify in a new column those common elements between table A & B, those elements in table A not present in table B and vice versa. A simple count would be... (6 Replies)
Discussion started by: lsantome
6 Replies
6. Shell Programming and Scripting
hi all,
i want to get the most comon column w.r.t another
this is my file
Tom|london
Tom|london
Tom|Paris
Adam|Madrid
Adam|NY
the Output to get me :
Tom|london
Adamn|Madrid
ive tried (10 Replies)
Discussion started by: teefa
10 Replies
7. Shell Programming and Scripting
Hi all
I have a great challenge that I am not able to resolve.
Briefly, I have a file like this:
ID_1 chr1 100 -
ID_2 chr2 300 +
and another file like this:
name_1 chr1 150 no -
name_2 chr1 250 yes -
name_3 chr2 350 yes +
name_4 chr2 280 yes +
Well, for each entry in file1 I would... (2 Replies)
Discussion started by: giuliangiuseppe
2 Replies
8. Shell Programming and Scripting
Hi,
I have a table to be imported for R as matrix or data.frame but I first need to edit it because I've got several lines with the same identifier (1st column), so I want to sum the each column (2nd -nth) of each identifier (1st column)
The input is for example, after sorted:
K00001 1 1 4 3... (8 Replies)
Discussion started by: sargotrons
8 Replies
9. UNIX for Beginners Questions & Answers
Hi,
Sure it's an easy one, but it drives me insane.
input ("|" separated):
1|A,B,C,A
2|A,D,D
3|A,B,B
I would like to count the occurence of each capital letters in $2 across the entire file, knowing that duplicates in each record count as 1.
I am trying to get this output... (5 Replies)
Discussion started by: beca123456
5 Replies
10. UNIX for Beginners Questions & Answers
Input file is as follows:
1 | 6
2 | 7
3 | 8
4 | 9
5 | 10
Output reuired (sum of the first column $1*$2)
1 | 6 | 90
2 | 7 | 105
3 | 8 | 120
4 |9 | 135
5 |10 | 150
Please enclose sample input, sample output, and code... (5 Replies)
Discussion started by: Sagar Singh
5 Replies
UNIQ(1) User Commands UNIQ(1)
NAME
uniq - report or omit repeated lines
SYNOPSIS
uniq [OPTION]... [INPUT [OUTPUT]]
DESCRIPTION
Filter adjacent matching lines from INPUT (or standard input), writing to OUTPUT (or standard output).
With no options, matching lines are merged to the first occurrence.
Mandatory arguments to long options are mandatory for short options too.
-c, --count
prefix lines by the number of occurrences
-d, --repeated
only print duplicate lines, one for each group
-D, --all-repeated[=METHOD]
print all duplicate lines groups can be delimited with an empty line METHOD={none(default),prepend,separate}
-f, --skip-fields=N
avoid comparing the first N fields
--group[=METHOD]
show all items, separating groups with an empty line METHOD={separate(default),prepend,append,both}
-i, --ignore-case
ignore differences in case when comparing
-s, --skip-chars=N
avoid comparing the first N characters
-u, --unique
only print unique lines
-z, --zero-terminated
end lines with 0 byte, not newline
-w, --check-chars=N
compare no more than N characters in lines
--help display this help and exit
--version
output version information and exit
A field is a run of blanks (usually spaces and/or TABs), then non-blank characters. Fields are skipped before chars.
Note: 'uniq' does not detect repeated lines unless they are adjacent. You may want to sort the input first, or use 'sort -u' without
'uniq'. Also, comparisons honor the rules specified by 'LC_COLLATE'.
GNU coreutils online help: <http://www.gnu.org/software/coreutils/> Report uniq translation bugs to <http://translationproject.org/team/>
AUTHOR
Written by Richard M. Stallman and David MacKenzie.
COPYRIGHT
Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
SEE ALSO
comm(1), join(1), sort(1)
The full documentation for uniq is maintained as a Texinfo manual. If the info and uniq programs are properly installed at your site, the
command
info coreutils 'uniq invocation'
should give you access to the complete manual.
GNU coreutils 8.22 June 2014 UNIQ(1)