05-12-2009
filter the uniq record problem
Anyone can help for filter the uniq record for below example? Thank you very much
Input file
20090503011111|test|abc
20090503011112|tet1|abc|def
20090503011112|test1|bcd|def
20090503011131|abc|abc
20090503011131|bbc|bcd
20090503011152|bcd|abc
20090503011151|abc|abc
20090503011151|abc|bcd
20090503011162|bcd|abc
Expect outupt
20090503011111|test|abc
20090503011152|bcd|abc
20090503011162|bcd|abc
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi All,
I have a text file with the folowing content.
BANGALORE|1417|2010-02-04 08:41:04.174|dob|xxx
BANGALORE|1416|2010-02-04 08:23:19.566|dob|yyy
BANGALORE|1415|2010-02-04 08:20:14.497|dob|aaa
BANGALORE|1414|2010-02-04 08:19:40.065|dob|vvv
BANGALORE|1413|2010-02-04... (4 Replies)
Discussion started by: Karpak
4 Replies
2. Shell Programming and Scripting
Hi,
I want to filter record from a file if the records in the second column matches the data in another file. I tried the below awk command but it filters the records in the filter file. I want the opposite, to include only the records in the filter file.
I tried this:
awk -F'|'... (8 Replies)
Discussion started by: gpaulose
8 Replies
3. Shell Programming and Scripting
Reposting since I didnt not get any reply.
I have a problem while filtering records from a file. Can somebody help please?
For eg: Consider the below files
Record file:
0003@00000000000190@20100401@201004012010040120100401@003@... (1 Reply)
Discussion started by: gpaulose
1 Replies
4. Shell Programming and Scripting
Hi folks,
Below is the content of a file 'tmp.dat', and I want to keep the uniq record (key by first column). However, the uniq record should be the last record.
302293022|2|744124889|744124889
302293022|3|744124889|744124889
302293022|4|744124889|744124889
302293022|5|744124889|744124889... (4 Replies)
Discussion started by: ChicagoBlues
4 Replies
5. Shell Programming and Scripting
Hello;
I have the output data set from:
egrep -i 'warning| error| fail' /var/adm/syslog/syslog.log
Jan 31 12:02:18 fidsrv vmunix: LVM: WARNING: VG 128 0x001000: LV 5: Some I/O requests to this LV are waiting
Jan 31 12:02:23 fidsrv vmunix: Asynchronous write failed on LUN (dev=0x100000f)... (3 Replies)
Discussion started by: delphys
3 Replies
6. Shell Programming and Scripting
I have file1
3049
3138
4672
22631
45324
112382
121240
125470
130289
186128
193996
194002
202776
228002
253221
273523
284601
284605
641858 (8 Replies)
Discussion started by: biomed
8 Replies
7. Shell Programming and Scripting
Hi i want to fetch 100k record from a file which is looking like as below.
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
... (17 Replies)
Discussion started by: lathigara
17 Replies
8. Shell Programming and Scripting
Dear friend,
I have a file 2 files with column wise
FILE_A
------------------------------
x,1,@
y,3,$
x,5,%
FILE_B
--------------------
x,1,@
i like to delete the all lines in FILE_A ,if first column available in FILE_B.
output (in FILE_A)
y,3,$
x,5,% (10 Replies)
Discussion started by: Jewel
10 Replies
9. Shell Programming and Scripting
Hello, I want to filter column based on string value. All substring matches are filtered out and only unique master strings are picked up.
infile:
1 abcd
2 abc
3 abcd
4 cdef
5 efgh
6 efgh
7 efx
8 fgh
Outfile:
1 abcd
4 cdef
5 efgh
7 efxI have tried
awk '!a++; match(a, $2)>0'... (32 Replies)
Discussion started by: yifangt
32 Replies
10. Shell Programming and Scripting
Hi Experts,
I have csv file with 30, 40 columns
Pasting just 2 column for problem description.
Need to print error if below combination is not present in file
check for column-1 (DocumentNumber) and filter columns where value in DocumentNumber field is same.
For all such rows, the field... (7 Replies)
Discussion started by: as7951
7 Replies
UNIQ(1) General Commands Manual UNIQ(1)
NAME
uniq - report repeated lines in a file
SYNOPSIS
uniq [ -udc [ +n ] [ -n ] ] [ input [ output ] ]
DESCRIPTION
Uniq reads the input file comparing adjacent lines. In the normal case, the second and succeeding copies of repeated lines are removed;
the remainder is written on the output file. Note that repeated lines must be adjacent in order to be found; see sort(1). If the -u flag
is used, just the lines that are not repeated in the original file are output. The -d option specifies that one copy of just the repeated
lines is to be written. The normal mode output is the union of the -u and -d mode outputs.
The -c option supersedes -u and -d and generates an output report in default style but with each line preceded by a count of the number of
times it occurred.
The n arguments specify skipping an initial portion of each line in the comparison:
-n The first n fields together with any blanks before each are ignored. A field is defined as a string of non-space, non-tab charac-
ters separated by tabs and spaces from its neighbors.
+n The first n characters are ignored. Fields are skipped before characters.
SEE ALSO
sort(1), comm(1)
UNIQ(1)