Sponsored Content
Top Forums UNIX for Dummies Questions & Answers How to redirect duplicate lines from a file???? Post 302115490 by marwan on Wednesday 25th of April 2007 05:53:55 AM
Old 04-25-2007
I think it can work by this way also:
sort file1 |uniq -d > file2
this will extract the duplicated lines and redirects them to file2

you can also specify the number of duplication of every line at the begining of the line:
sort file1 |uniq -c > file2
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Duplicate lines in the file

Hi, I have a file with duplicate lines in it. I want to keep only the duplicate lines and delete the non duplicates. Can some one please help me? Regards Narayana Gupta (3 Replies)
Discussion started by: guptan
3 Replies

2. UNIX for Dummies Questions & Answers

Remove Duplicate lines from File

I have a log file "logreport" that contains several lines as seen below: 04:20:00 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping 06:38:08 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping 07:11:05 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but... (18 Replies)
Discussion started by: Nysif Steve
18 Replies

3. Shell Programming and Scripting

removing the duplicate lines in a file

Hi, I need to concatenate three files in to one destination file.In this if some duplicate data occurs it should be deleted. eg: file1: ----- data1 value1 data2 value2 data3 value3 file2: ----- data1 value1 data4 value4 data5 value5 file3: ----- data1 value1 data4 value4 (3 Replies)
Discussion started by: Sharmila_P
3 Replies

4. AIX

Grep multiple lines and redirect to file

I have setof files with data and with same fields multiple times in each of the files. for example: file 1 name = mary kate last name = kate address = 123 street = abc name = mary mark last name = mark address = 456 street = bcd file 2 name = mary kate last name = kate... (2 Replies)
Discussion started by: relearner
2 Replies

5. Shell Programming and Scripting

Duplicate lines in a file

Hi All, I am trying to remove the duplicate entries in a file and print them just once. For example, if my input file has: 00:44,37,67,56,15,12 00:44,34,67,56,15,12 00:44,58,67,56,15,12 00:44,35,67,56,15,12 00:59,37,67,56,15,12 00:59,34,67,56,15,12 00:59,35,67,56,15,12... (7 Replies)
Discussion started by: faiz1985
7 Replies

6. Shell Programming and Scripting

Redirect lines to another file

Hi, I have a file with a size of 10 mb and I need to redirect some specific lines to a new files. For eg. Executed Restore for 11227.EDCS.551.01.201110 from /tmp/bk/restore/CR81500/content/S24U15VA2.2010-10-29.16:49.EDT/ArchiveFile_11227.EDCS.551.01.201110.zip Operation output: <U+FEFF>Oct... (4 Replies)
Discussion started by: gsiva
4 Replies

7. UNIX for Advanced & Expert Users

In a huge file, Delete duplicate lines leaving unique lines

Hi All, I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space. I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies

8. Shell Programming and Scripting

bash keep only duplicate lines in file

hello all in my bash script I have a file and I only want to keep the lines that appear twice in the file.Is there a way to do this? thanks in advance! (4 Replies)
Discussion started by: vlm
4 Replies

9. UNIX for Dummies Questions & Answers

Duplicate lines in a file

I have a file with following data A B C I would like to print like this n times(For eg:5 times) A B C A B C A B C A B C A (7 Replies)
Discussion started by: nsuresh316
7 Replies

10. Shell Programming and Scripting

Remove duplicate lines from a file

Hi, I have a csv file which contains some millions of lines in it. The first line(Header) repeats at every 50000th line. I want to remove all the duplicate headers from the second occurance(should not remove the first line). I don't want to use any pattern from the Header as I have some... (7 Replies)
Discussion started by: sudhakar T
7 Replies
lp-set-dup(1)						      General Commands Manual						     lp-set-dup(1)

NAME
lp-set-dup - mark one or more bugs as duplicate of another bug SYNOPSIS
lp-set-dup [-f] <main bug> <duplicate bug> [<duplicate bug> ...] lp-set-dup --help DESCRIPTION
lp-set-dup allow to easily mark one or more bug as duplicate of another bug. It checks for permission to operate on a given bug first, then perform required tasks on Launchpad. OPTIONS
Listed below are the command line options for lp-set-dup: -h, --help Display a help message and exit. -f Skip confirmation prompt. -l INSTANCE, --lpinstance=INSTANCE Use the specified instance of Launchpad (e.g. "staging"), instead of the default of "production". --no-conf Do not read any configuration files, or configuration from environment variables. ENVIRONMENT
All of the CONFIGURATION VARIABLES below are also supported as environment variables. Variables in the environment take precedence to those in configuration files. CONFIGURATION VARIABLES
The following variables can be set in the environment or in ubuntu-dev-tools(5) configuration files. In each case, the script-specific variable takes precedence over the package-wide variable. LP_SET_DUP_LPINSTANCE, UBUNTUTOOLS_LPINSTANCE The default value for --lpinstance. SEE ALSO
ubuntu-dev-tools(5) AUTHORS
lp-set-dup was written by Loic Minier <lool@dooz.org>, and this manual page was written by Luca Falavigna <dktrkranz@debian.org>. Both are released under the terms of the GNU General Public License, version 2. lptools March 6 2010 lp-set-dup(1)
All times are GMT -4. The time now is 01:59 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy