Sponsored Content
Full Discussion: Script to remove duplicates
Top Forums Shell Programming and Scripting Script to remove duplicates Post 302417092 by anbu23 on Wednesday 28th of April 2010 01:00:30 PM
Old 04-28-2010
Code:
$ cat file
abcd.23
abcd.24
abcd.25
qwer.25
qwer.26
qwer.98
$ sort -u -t"." -k1,1 file
abcd.23
qwer.25

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove duplicates

Hello Experts, I have two files named old and new. Below are my example files. I need to compare and print the records that only exist in my new file. I tried the below awk script, this script works perfectly well if the records have exact match, the issue I have is my old file has got extra... (4 Replies)
Discussion started by: forumthreads
4 Replies

2. Shell Programming and Scripting

Shell script to remove duplicates lines in a file

Hi, I am writing a shell script that needs to remove duplicate lines within a file by category. example: section a a c b a section b a b a c I need to remove the duplicates within th category with out removing the duplicates from the 2 different sections (one of the a's in section... (1 Reply)
Discussion started by: RichElks
1 Replies

3. Shell Programming and Scripting

Remove duplicates from a file

Hi, I need to remove duplicates from a file. The file will be like this 0003 10101 20100120 abcdefghi 0003 10101 20100121 abcdefghi 0003 10101 20100122 abcdefghi 0003 10102 20100120 abcdefghi 0003 10103 20100120 abcdefghi 0003 10103 20100121 abcdefghi Here if the first colum and... (6 Replies)
Discussion started by: gpaulose
6 Replies

4. Shell Programming and Scripting

remove duplicates and sort

Hi, I'm using the below command to sort and remove duplicates in a file. But, i need to make this applied to the same file instead of directing it to another. Thanks (6 Replies)
Discussion started by: dvah
6 Replies

5. Shell Programming and Scripting

bash - remove duplicates

I need to use a bash script to remove duplicate files from a download list, but I cannot use uniq because the urls are different. I need to go from this: http://***/fae78fe/file1.wmv http://***/39du7si/file1.wmv http://***/d8el2hd/file2.wmv http://***/h893js3/file2.wmv to this: ... (2 Replies)
Discussion started by: locoroco
2 Replies

6. UNIX for Dummies Questions & Answers

script to remove duplicates per line

Hello experts! I'd like a way to remove duplicates per line. Strings are enclosed in brackets, and I would prefer to maintain the order of the file: example input (56)(63) (56)(70)(56)(70)(24) (25)(78) (12)(33)(12) (10) (10) desired output (56)(63) (56)(70)(24) (25)(78)... (5 Replies)
Discussion started by: torchij
5 Replies

7. Shell Programming and Scripting

Remove duplicates

I have a file with the following format: fields seperated by "|" title1|something class|long...content1|keys title2|somhing class|log...content1|kes title1|sothing class|lon...content1|kes title3|shing cls|log...content1|ks I want to remove all duplicates with the same "title field"(the... (3 Replies)
Discussion started by: dtdt
3 Replies

8. Shell Programming and Scripting

Remove top 3 duplicates

hello , I have a requirement with input in below format abc 123 xyz bcd 365 kii abc 987 876 cdf 987 uii abc 456 yuu bcd 654 rrr Expecting Output abc 456 yuu bcd 654 rrr cdf 987 uii (1 Reply)
Discussion started by: Tomlight
1 Replies

9. Shell Programming and Scripting

Remove duplicates

Hi I have a below file structure. 200,1245,E1,1,E1,,7611068,KWH,30, ,,,,,,,, 200,1245,E1,1,E1,,7611070,KWH,30, ,,,,,,,, 300,20140223,0.001,0.001,0.001,0.001,0.001 300,20140224,0.001,0.001,0.001,0.001,0.001 300,20140225,0.001,0.001,0.001,0.001,0.001 300,20140226,0.001,0.001,0.001,0.001,0.001... (1 Reply)
Discussion started by: tejashavele
1 Replies

10. Shell Programming and Scripting

How to remove duplicates using for loop?

values=(1 2 3 5 4 2 3 1 6 8 3 5 ) #i need the output like this by removing the duplicates 1 2 3 5 4 6 8 #i dont need sorting in my program #plz explain me as simple using for loop #os-ubuntu ,shell=bash (5 Replies)
Discussion started by: Meeran Rizvi
5 Replies
QDeepCopy(3qt)															    QDeepCopy(3qt)

NAME
QDeepCopy - Template class which ensures that SYNOPSIS
All the functions in this class are reentrant when Qt is built with thread support.</p> #include <qdeepcopy.h> Public Members QDeepCopy () QDeepCopy ( const T & t ) QDeepCopy<T> & operator= ( const T & t ) operator T () DESCRIPTION
The QDeepCopy class is a template class which ensures that implicitly shared and explicitly shared classes reference unique data. Normally, shared copies reference the same data to optimize memory use and for maximum speed. In the example below, s1, s2, s3, s4 and s5 share data. // all 5 strings share the same data QString s1 = "abcd"; QString s2 = s1; QString s3 = s2; QString s4 = s3; QString s5 = s2; QDeepCopy can be used several ways to ensure that an object references unique, unshared data. In the example below, s1, s2 and s5 share data, while neither s3 nor s4 share data. // s1, s2 and s5 share the same data, neither s3 nor s4 are shared QString s1 = "abcd"; QString s2 = s1; QDeepCopy<QString> s3 = s2; // s3 is a deep copy of s2 QString s4 = s3; // s4 is a deep copy of s3 QString s5 = s2; In the example below, s1, s2 and s5 share data, and s3 and s4 share data. // s1, s2 and s5 share the same data, s3 and s4 share the same data QString s1 = "abcd"; QString s2 = s1; QString s3 = QDeepCopy<QString>( s2 ); // s3 is a deep copy of s2 QString s4 = s3; // s4 is a shallow copy of s3 QString s5 = s2; QDeepCopy can also provide safety in multithreaded applications that use shared classes. In the example below, the variable global_string is used safely since the data contained in global_string is always a deep copy. This ensures that all threads get a unique copy of the data, and that any assignments to global_string will result in a deep copy. QDeepCopy<QString> global_string; // global string data QMutex global_mutex; // mutex to protext global_string ... void setGlobalString( const QString &str ) { global_mutex.lock(); global_string = str; // global_string is a deep copy of str global_mutex.unlock(); } ... void MyThread::run() { global_mutex.lock(); QString str = global_string; // str is a deep copy of global_string global_mutex.unlock(); // process the string data ... // update global_string setGlobalString( str ); } Warning: It is the application developer's responsibility to protect the object shared across multiple threads. The examples above use QString, which is an implicitly shared class. The behavior of QDeepCopy is the same when using explicitly shared classes like QByteArray. Currently, QDeepCopy works with the following classes: QMemArray (including subclasses like QByteArray and QCString) QMap QString QValueList (including subclasses like QStringList and QValueStack) QValueVector See also Thread Support in Qt, Implicitly and Explicitly Shared Classes, and Non-GUI Classes. MEMBER FUNCTION DOCUMENTATION
QDeepCopy::QDeepCopy () Constructs an empty instance of type T. QDeepCopy::QDeepCopy ( const T & t ) Constructs a deep copy of t. QDeepCopy::operator T () Returns a deep copy of the encapsulated data. QDeepCopy<;T> & QDeepCopy::operator= ( const T & t ) Assigns a deep copy of t. SEE ALSO
http://doc.trolltech.com/qdeepcopy.html http://www.trolltech.com/faq/tech.html COPYRIGHT
Copyright 1992-2001 Trolltech AS, http://www.trolltech.com. See the license file included in the distribution for a complete license statement. AUTHOR
Generated automatically from the source code. BUGS
If you find a bug in Qt, please report it as described in http://doc.trolltech.com/bughowto.html. Good bug reports help us to help you. Thank you. The definitive Qt documentation is provided in HTML format; it is located at $QTDIR/doc/html and can be read using Qt Assistant or with a web browser. This man page is provided as a convenience for those users who prefer man pages, although this format is not officially supported by Trolltech. If you find errors in this manual page, please report them to qt-bugs@trolltech.com. Please include the name of the manual page (qdeepcopy.3qt) and the Qt version (3.1.1). Trolltech AS 9 December 2002 QDeepCopy(3qt)
All times are GMT -4. The time now is 03:12 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy