|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
Script to remove duplicates
Post 302417203 by antointoronto on Thursday 29th of April 2010 12:15:50 AM
04-29-2010
|
|
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hello Experts,
I have two files named old and new. Below are my example files. I need to compare and print the records that only exist in my new file. I tried the below awk script, this script works perfectly well if the records have exact match, the issue I have is my old file has got extra... (4 Replies)
Discussion started by: forumthreads
4 Replies
2. Shell Programming and Scripting
Hi,
I am writing a shell script that needs to remove duplicate lines within a file by category.
example:
section a
a
c
b
a
section b
a
b
a
c
I need to remove the duplicates within th category with out removing the duplicates from the 2 different sections (one of the a's in section... (1 Reply)
Discussion started by: RichElks
1 Replies
3. Shell Programming and Scripting
Hi,
I need to remove duplicates from a file. The file will be like this
0003 10101 20100120 abcdefghi
0003 10101 20100121 abcdefghi
0003 10101 20100122 abcdefghi
0003 10102 20100120 abcdefghi
0003 10103 20100120 abcdefghi
0003 10103 20100121 abcdefghi
Here if the first colum and... (6 Replies)
Discussion started by: gpaulose
6 Replies
4. Shell Programming and Scripting
Hi,
I'm using the below command to sort and remove duplicates in a file. But, i need to make this applied to the same file instead of directing it to another.
Thanks (6 Replies)
Discussion started by: dvah
6 Replies
5. Shell Programming and Scripting
I need to use a bash script to remove duplicate files from a download list, but I cannot use uniq because the urls are different.
I need to go from this:
http://***/fae78fe/file1.wmv
http://***/39du7si/file1.wmv
http://***/d8el2hd/file2.wmv
http://***/h893js3/file2.wmv
to this:
... (2 Replies)
Discussion started by: locoroco
2 Replies
6. UNIX for Dummies Questions & Answers
Hello experts!
I'd like a way to remove duplicates per line. Strings are enclosed in brackets, and I would prefer to maintain the order of the file:
example input
(56)(63)
(56)(70)(56)(70)(24)
(25)(78)
(12)(33)(12)
(10)
(10)
desired output
(56)(63)
(56)(70)(24)
(25)(78)... (5 Replies)
Discussion started by: torchij
5 Replies
7. Shell Programming and Scripting
I have a file with the following format:
fields seperated by "|"
title1|something class|long...content1|keys
title2|somhing class|log...content1|kes
title1|sothing class|lon...content1|kes
title3|shing cls|log...content1|ks
I want to remove all duplicates with the same "title field"(the... (3 Replies)
Discussion started by: dtdt
3 Replies
8. Shell Programming and Scripting
hello ,
I have a requirement with input in below format
abc 123 xyz
bcd 365 kii
abc 987 876
cdf 987 uii
abc 456 yuu
bcd 654 rrr
Expecting Output
abc 456 yuu
bcd 654 rrr
cdf 987 uii (1 Reply)
Discussion started by: Tomlight
1 Replies
9. Shell Programming and Scripting
Hi I have a below file structure.
200,1245,E1,1,E1,,7611068,KWH,30, ,,,,,,,,
200,1245,E1,1,E1,,7611070,KWH,30, ,,,,,,,,
300,20140223,0.001,0.001,0.001,0.001,0.001
300,20140224,0.001,0.001,0.001,0.001,0.001
300,20140225,0.001,0.001,0.001,0.001,0.001
300,20140226,0.001,0.001,0.001,0.001,0.001... (1 Reply)
Discussion started by: tejashavele
1 Replies
10. Shell Programming and Scripting
values=(1 2 3 5 4 2 3 1 6 8 3 5 )
#i need the output like this by removing the duplicates
1
2
3
5
4
6
8
#i dont need sorting in my program
#plz explain me as simple using for loop
#os-ubuntu ,shell=bash (5 Replies)
Discussion started by: Meeran Rizvi
5 Replies
LEARN ABOUT OSX
rwbytype.d
rwbytype.d(1m) USER COMMANDS rwbytype.d(1m) NAME
rwbytype.d - read/write bytes by vnode type. Uses DTrace. SYNOPSIS
rwbytype.d DESCRIPTION
This program identifies the vnode type of read/write activity - whether that is for regular files, sockets, character special devices, etc. This is measuring at the application level, so file activity may well be cached by the system. Since this uses DTrace, only users with root privileges can run this command. EXAMPLES
This samples until Ctrl-C is hit. # rwbytype.d FIELDS
PID process ID CMD process name VNODE vnode type (describes I/O type) DIR direction, Read or Write BYTES total bytes DOCUMENTATION
See the DTraceToolkit for further documentation under the Docs directory. The DTraceToolkit docs may include full worked examples with ver- bose descriptions explaining the output. EXIT
rwbytype.d will sample until Ctrl-C is hit. AUTHOR
Brendan Gregg [Sydney, Australia] SEE ALSO
rwbypid.d(1M), rwbbypid.d(1M), dtrace(1M) version 0.70 Jan 11, 2006 rwbytype.d(1m)