05-13-2009
Quote:
Originally Posted by
System Shock
... so, if a string of numbers appears in 2 lines, both lines get deleted?
Or are you saying that all of the numbers in red are duplicates of each other?
yes, both lines get deleted.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi All,
I have a text file with the folowing content.
BANGALORE|1417|2010-02-04 08:41:04.174|dob|xxx
BANGALORE|1416|2010-02-04 08:23:19.566|dob|yyy
BANGALORE|1415|2010-02-04 08:20:14.497|dob|aaa
BANGALORE|1414|2010-02-04 08:19:40.065|dob|vvv
BANGALORE|1413|2010-02-04... (4 Replies)
Discussion started by: Karpak
4 Replies
2. Shell Programming and Scripting
Hi,
I want to filter record from a file if the records in the second column matches the data in another file. I tried the below awk command but it filters the records in the filter file. I want the opposite, to include only the records in the filter file.
I tried this:
awk -F'|'... (8 Replies)
Discussion started by: gpaulose
8 Replies
3. Shell Programming and Scripting
Reposting since I didnt not get any reply.
I have a problem while filtering records from a file. Can somebody help please?
For eg: Consider the below files
Record file:
0003@00000000000190@20100401@201004012010040120100401@003@... (1 Reply)
Discussion started by: gpaulose
1 Replies
4. Shell Programming and Scripting
Hi folks,
Below is the content of a file 'tmp.dat', and I want to keep the uniq record (key by first column). However, the uniq record should be the last record.
302293022|2|744124889|744124889
302293022|3|744124889|744124889
302293022|4|744124889|744124889
302293022|5|744124889|744124889... (4 Replies)
Discussion started by: ChicagoBlues
4 Replies
5. Shell Programming and Scripting
Hello;
I have the output data set from:
egrep -i 'warning| error| fail' /var/adm/syslog/syslog.log
Jan 31 12:02:18 fidsrv vmunix: LVM: WARNING: VG 128 0x001000: LV 5: Some I/O requests to this LV are waiting
Jan 31 12:02:23 fidsrv vmunix: Asynchronous write failed on LUN (dev=0x100000f)... (3 Replies)
Discussion started by: delphys
3 Replies
6. Shell Programming and Scripting
I have file1
3049
3138
4672
22631
45324
112382
121240
125470
130289
186128
193996
194002
202776
228002
253221
273523
284601
284605
641858 (8 Replies)
Discussion started by: biomed
8 Replies
7. Shell Programming and Scripting
Hi i want to fetch 100k record from a file which is looking like as below.
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
... (17 Replies)
Discussion started by: lathigara
17 Replies
8. Shell Programming and Scripting
Dear friend,
I have a file 2 files with column wise
FILE_A
------------------------------
x,1,@
y,3,$
x,5,%
FILE_B
--------------------
x,1,@
i like to delete the all lines in FILE_A ,if first column available in FILE_B.
output (in FILE_A)
y,3,$
x,5,% (10 Replies)
Discussion started by: Jewel
10 Replies
9. Shell Programming and Scripting
Hello, I want to filter column based on string value. All substring matches are filtered out and only unique master strings are picked up.
infile:
1 abcd
2 abc
3 abcd
4 cdef
5 efgh
6 efgh
7 efx
8 fgh
Outfile:
1 abcd
4 cdef
5 efgh
7 efxI have tried
awk '!a++; match(a, $2)>0'... (32 Replies)
Discussion started by: yifangt
32 Replies
10. Shell Programming and Scripting
Hi Experts,
I have csv file with 30, 40 columns
Pasting just 2 column for problem description.
Need to print error if below combination is not present in file
check for column-1 (DocumentNumber) and filter columns where value in DocumentNumber field is same.
For all such rows, the field... (7 Replies)
Discussion started by: as7951
7 Replies
uniq(1) General Commands Manual uniq(1)
Name
uniq - report repeated lines in a file
Syntax
uniq [-udc[+n][-n]] [input[output]]
Description
The command reads the input file comparing adjacent lines. In the normal case, the second and succeeding copies of repeated lines are
removed; the remainder is written on the output file. Note that repeated lines must be adjacent in order to be found. For further infor-
mation, see
Options
The n arguments specify skipping an initial portion of each line in the comparison:
-n Skips specified number of fields. A field is defined as a string of non-space, non-tab characters separated by tabs and spaces from its
neighbors.
+n Skips specified number of characters in addition to fields. Fields are skipped before characters.
-c Displays number of repetitions, if any, for each line.
-d Displays only lines that were repeated.
-u Displays only unique (nonrepeated) lines.
If the -u flag is used, just the lines that are not repeated in the original file are output. The -d option specifies that one copy of
just the repeated lines is to be written. The normal mode output is the union of the -u and -d mode outputs.
The -c option supersedes -u and -d and generates an output report in default style but with each line preceded by a count of the number of
times it occurred.
See Also
comm(1), sort(1)
uniq(1)