01-15-2008
Uniq using only the first field
Hi all,
I have a file that contains a list of codes (shown below).
I want to 'uniq' the file using only the first field. Anyone know an easy way of doing it?
Cheers,
Dave
##### Input File #####
1xr1 1xws 1yxt 1yxu 1yxv 1yxx 2o3p 2o63 2o64 2o65
1xr1 1xws 1yxt 1yxv 1yxx 2o3p 2o63 2o64 2o65
1xr1 1yxt 1yxv 1yxx 2o3p 2o63 2o64 2o65
1y8y 2c68 2c69 2c6i 2c6k 2c6m 2c6o
1y8y 2c69 2c6i 2c6k
1zlt 2br1 2brb 2brn 2bro 2c3k 2cgu 2cgv 2cgx
1zlt 2br1 2brb 2brn 2bro 2cgu 2cgv 2cgx
1zlt 2br1 2brb 2brn 2bro 2cgv 2cgx
1zlt 2br1 2brb 2brn 2bro 2cgx
1zlt 2br1 2brb 2bro
##### Desired Output File #######
1xr1 1xws 1yxt 1yxu 1yxv 1yxx 2o3p 2o63 2o64 2o65
1y8y 2c68 2c69 2c6i 2c6k 2c6m 2c6o
1zlt 2br1 2brb 2brn 2bro 2c3k 2cgu 2cgv 2cgx
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi ;
I have a question regarding the uniq command in unix
How do I uniq 3rd field in a file ?
original file :
zoom coord 39 18652 39 18652
zoom coord 39 18653 39 18653
zoom coord 39 18818 39 18818
zoom coord 39 18840 39 18840
zoom coord 41 15096 41 15096
zoom... (1 Reply)
Discussion started by: babycakes
1 Replies
2. Shell Programming and Scripting
How can I use uniq on a certain field or what else could I use? If I want to use uniq on the second field and the output would remove one of the lines with a 5.
bob 5 hand
jane 3 leg
jon 4 head
chris 5 lungs (1 Reply)
Discussion started by: Bandit390
1 Replies
3. UNIX for Dummies Questions & Answers
Dear all,
It's not entirely clear to me from manpage the difference between them.
Why we still need "-u" flag?
- monkfan (3 Replies)
Discussion started by: monkfan
3 Replies
4. UNIX for Dummies Questions & Answers
Hello all, new to unix and have just found the forum.
I think I will be here quite often, and hope that in time i will be able to provide soem help, role on not being a newbie anymore :)
I have a question which iI am hoping someone could help me with.
If i have a file with lines in in thus... (8 Replies)
Discussion started by: grom
8 Replies
5. Shell Programming and Scripting
Hi New to unix.
I want to display only the unrepeated lines from a file using first field.
Ex:
1234 uname1 status1
1235 uname2 status2
1234 uname3 status3
1236 uname5 status5
I used
sort filename | uniq -u
output:
1234 uname1 status1
1235 uname2 status2
1234 uname3 status3
1236... (10 Replies)
Discussion started by: venummca
10 Replies
6. Shell Programming and Scripting
I have a flatfile A.txt
2012/12/04 14:06:07 |trees|Boards 2, 3|denver|mekong|mekong12
2012/12/04 17:07:22 |trees|Boards 2, 3|denver|mekong|mekong12
2012/12/04 17:13:27 |trees|Boards 2, 3|denver|mekong|mekong12
2012/12/04 14:07:39 |rain|Boards 1|tampa|merced|merced11
How do i sort and get... (3 Replies)
Discussion started by: sabercats
3 Replies
7. Shell Programming and Scripting
Hi All,
I am searching for a script which will produce an output file with the uniq first field with the second field having highest value among all the duplicates..
The output file will produce only the uniqs which are duplicate 3 times..
Input file
X 9
B 5
A 1
Z 9
T 4
C 9
A 4... (13 Replies)
Discussion started by: ailnilanjan
13 Replies
8. Linux
I have a .CSV file (file.csv) whose data are all enclosed in double quotes. Sample format of the file is as below:
column1,column2,column3,column4,column5,column6, column7, Column8, Column9, Column10
"12","B000QRIGJ4","4432","string with quotes, and with a comma, and colon: in... (3 Replies)
Discussion started by: dhruuv369
3 Replies
9. Shell Programming and Scripting
Hello, I want to filter column based on string value. All substring matches are filtered out and only unique master strings are picked up.
infile:
1 abcd
2 abc
3 abcd
4 cdef
5 efgh
6 efgh
7 efx
8 fgh
Outfile:
1 abcd
4 cdef
5 efgh
7 efxI have tried
awk '!a++; match(a, $2)>0'... (32 Replies)
Discussion started by: yifangt
32 Replies
10. Shell Programming and Scripting
Hi All,
I have a file like this(having 2 column).
Column 1: like a,b,c....
Column 2: having numbers.
I want to segregate those numbers based on column 1.
Example:
file.
a 5
b 9
b 620
a 710
b 230
a 330
b 1910 (4 Replies)
Discussion started by: Raza Ali
4 Replies
UNIQ(1) FSF UNIQ(1)
NAME
uniq - remove duplicate lines from a sorted file
SYNOPSIS
uniq [OPTION]... [INPUT [OUTPUT]]
DESCRIPTION
Discard all but one of successive identical lines from INPUT (or standard input), writing to OUTPUT (or standard output).
Mandatory arguments to long options are mandatory for short options too.
-c, --count
prefix lines by the number of occurrences
-d, --repeated
only print duplicate lines
-D, --all-repeated[=delimit-method] print all duplicate lines
delimit-method={none(default),prepend,separate} Delimiting is done with blank lines.
-f, --skip-fields=N
avoid comparing the first N fields
-i, --ignore-case
ignore differences in case when comparing
-s, --skip-chars=N
avoid comparing the first N characters
-u, --unique
only print unique lines
-w, --check-chars=N
compare no more than N characters in lines
--help display this help and exit
--version
output version information and exit
A field is a run of whitespace, then non-whitespace characters. Fields are skipped before chars.
AUTHOR
Written by Richard Stallman and David MacKenzie.
REPORTING BUGS
Report bugs to <bug-coreutils@gnu.org>.
COPYRIGHT
Copyright (C) 2002 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICU-
LAR PURPOSE.
SEE ALSO
The full documentation for uniq is maintained as a Texinfo manual. If the info and uniq programs are properly installed at your site, the
command
info uniq
should give you access to the complete manual.
uniq (coreutils) 4.5.3 February 2003 UNIQ(1)