Could this be more efficient?
.... or if multiple input files to awk would confuse things, try:-
An alternate (which may be horribly slow, I don't know) could be:-
.... although this will fail for excessive number of input files because the command grows too long. I suppose you could also wrap it in a find like this:-
It will be one of those that you have to try variations to see which one works best for your data.
Dear All,
I want to get help for below case.
I have a file like this.
saman 1
gihan 2
saman 4
ravi 1
ravi 2
so i want to get the result,
saman 5
gihan 2
ravi 3 like this.
Pls help me. (17 Replies)
Hi, Iam new to unix. I have one input file .
Input file :
ID1~Name1~Place1
ID2~Name2~Place2
ID3~Name3~Place3
I need output such that only first column should change to fixed width column of 15 characters of length.
Output File:
ID1<<12 spaces>>Name1~Place1
ID2<<12... (5 Replies)
I have a data set that has 4 columns, I want to know if I can delete duplicate lines while ignoring one of the columns, for example
10 chr1 ASF 30
15 chr1 ASF 20
5 chr1 ASF 30
6 chr2 EBC 15
4 chr2 EBC 30
...
I want to know if I can delete duplicate lines while ignoring column 1, so the... (5 Replies)
Hi ,
Can You Please let Know How use unix uniq command on a single column for deleting records from file
with Below Structure.Pipe Delimter File .
Source
Name | Account_Id
A | 101
B... (2 Replies)
I met a challenge to filter ~70 millions of sequence rows and I want using awk with conditions:
1) longest string of each pattern in column 2, ignore any sub-string, as the index;
2) all the unique patterns after 1);
3) print the whole row;
input:
1 ABCDEFGHI longest_sequence1
2 ABCDEFGH... (12 Replies)
Hello
How can I get a number of occurrence count for this file;
ERR315389.1000156 CTTGAAGAAGAATTGAAAACTGTGACGAACAACTTGAAGTCACTGGAGGCTCAGGCTGAGAAGTACTCGCAGAAGGAAGACAGATATGAGGAAGAG
ERR315389.1000281 ... (3 Replies)
I want to bring values in the second column into single line for uniq value in the first column.
My input
jvm01, Web 2.0 Feature Pack Library
jvm01, IBM WebSphere JAX-RS
jvm01, Custom01 Shared Library
jvm02, Web 2.0 Feature Pack Library
jvm02, IBM WebSphere JAX-RS
jvm03, Web 2.0 Feature... (10 Replies)
Hi All,
I am trying to output uniq values per column. see file below. can you please assist? Thank you in advance.
cat names
joe allen ibm
joe smith ibm
joe allen google
joe smith google
rachel allen google
desired output is:
joe allen google
rachel smith ibm (5 Replies)
Hi All,
I have a file like this(having 2 column).
Column 1: like a,b,c....
Column 2: having numbers.
I want to segregate those numbers based on column 1.
Example:
file.
a 5
b 9
b 620
a 710
b 230
a 330
b 1910 (4 Replies)
Discussion started by: Raza Ali
4 Replies
LEARN ABOUT OPENDARWIN
uniq
UNIQ(1) BSD General Commands Manual UNIQ(1)NAME
uniq -- report or filter out repeated lines in a file
SYNOPSIS
uniq [-c | -d | -u] [-i] [-f num] [-s chars] [input_file [output_file]]
DESCRIPTION
The uniq utility reads the specified input_file comparing adjacent lines, and writes a copy of each unique input line to the output_file. If
input_file is a single dash ('-') or absent, the standard input is read. If output_file is absent, standard output is used for output. The
second and succeeding copies of identical adjacent input lines are not written. Repeated lines in the input will not be detected if they are
not adjacent, so it may be necessary to sort the files first.
The following options are available:
-c Precede each output line with the count of the number of times the line occurred in the input, followed by a single space.
-d Only output lines that are repeated in the input.
-f num Ignore the first num fields in each input line when doing comparisons. A field is a string of non-blank characters separated from
adjacent fields by blanks. Field numbers are one based, i.e. the first field is field one.
-s chars
Ignore the first chars characters in each input line when doing comparisons. If specified in conjunction with the -f option, the
first chars characters after the first num fields will be ignored. Character numbers are one based, i.e. the first character is
character one.
-u Only output lines that are not repeated in the input.
-i Case insensitive comparison of lines.
DIAGNOSTICS
The uniq utility exits 0 on success, and >0 if an error occurs.
COMPATIBILITY
The historic +number and -number options have been deprecated but are still supported in this implementation.
SEE ALSO sort(1)STANDARDS
The uniq utility is expected to be IEEE Std 1003.2 (``POSIX.2'') compatible.
HISTORY
A uniq command appeared in Version 3 AT&T UNIX.
BSD June 6, 1993 BSD