Top Forums Shell Programming and Scripting bash keep only duplicate lines in file Post 302653565 by vlm on Saturday 9th of June 2012 01:23:49 PM
bash keep only duplicate lines in file

hello all

in my bash script I have a file and I only want to keep the lines that appear twice in the file.Is there a way to do this?
thanks in advance!
Test Your Knowledge in Computers #49
Difficulty: Easy
A USB hard drive is considered to be a primary storage device for personal computers.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove Duplicate Lines in File

I am doing KSH script to remove duplicate lines in a file. Let say the file has format below. FileA 1253-6856 3101-4011 1827-1356 1822-1157 1822-1157 1000-1410 1000-1410 1822-1231 1822-1231 3101-4011 1822-1157 1822-1231 and I want to simply it with no duplicate line as file... (5 Replies)
Discussion started by: Teh Tiack Ein
5 Replies

2. UNIX for Advanced & Expert Users

Duplicate lines in the file

Hi, I have a file with duplicate lines in it. I want to keep only the duplicate lines and delete the non duplicates. Can some one please help me? Regards Narayana Gupta (3 Replies)
Discussion started by: guptan
3 Replies

3. UNIX for Dummies Questions & Answers

removing duplicate lines from a file

Hi, I am trying to remove duplicate lines from a file. For example the contents of example.txt is: this is a test 2342 this is a test 34343 this is a test 43434 and i want to remove the "this is a test" lines only and end up with the numbers in the file, that is, end up with: 2342... (4 Replies)
Discussion started by: ocelot
4 Replies

4. UNIX for Dummies Questions & Answers

How to redirect duplicate lines from a file????

Hi, I am having a file which contains many duplicate lines. I wanted to redirect these duplicate lines into another file. Suppose I have a file called file_dup.txt which contains some line as file_dup.txt A100-R1 ACCOUNTING-CONTROL ACTONA-ACTASTOR ADMIN-AUTH-STATS ACTONA-ACTASTOR... (3 Replies)
Discussion started by: zing_foru
3 Replies

5. UNIX for Dummies Questions & Answers

Remove Duplicate lines from File

I have a log file "logreport" that contains several lines as seen below: 04:20:00 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping 06:38:08 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping 07:11:05 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but... (18 Replies)
Discussion started by: Nysif Steve
18 Replies

6. Shell Programming and Scripting

removing the duplicate lines in a file

Hi, I need to concatenate three files in to one destination file.In this if some duplicate data occurs it should be deleted. eg: file1: ----- data1 value1 data2 value2 data3 value3 file2: ----- data1 value1 data4 value4 data5 value5 file3: ----- data1 value1 data4 value4 (3 Replies)
Discussion started by: Sharmila_P
3 Replies

7. Shell Programming and Scripting

Duplicate lines in a file

Hi All, I am trying to remove the duplicate entries in a file and print them just once. For example, if my input file has: 00:44,37,67,56,15,12 00:44,34,67,56,15,12 00:44,58,67,56,15,12 00:44,35,67,56,15,12 00:59,37,67,56,15,12 00:59,34,67,56,15,12 00:59,35,67,56,15,12... (7 Replies)
Discussion started by: faiz1985
7 Replies

8. UNIX for Advanced & Expert Users

In a huge file, Delete duplicate lines leaving unique lines

Hi All, I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space. I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies

9. UNIX for Dummies Questions & Answers

Duplicate lines in a file

I have a file with following data A B C I would like to print like this n times(For eg:5 times) A B C A B C A B C A B C A (7 Replies)
Discussion started by: nsuresh316
7 Replies

10. Shell Programming and Scripting

Remove duplicate lines from a file

Hi, I have a csv file which contains some millions of lines in it. The first line(Header) repeats at every 50000th line. I want to remove all the duplicate headers from the second occurance(should not remove the first line). I don't want to use any pattern from the Header as I have some... (7 Replies)
Discussion started by: sudhakar T
7 Replies
LIBBASH(7)							  libbash Manual							LIBBASH(7)

libbash -- A bash shared libraries package. DESCRIPTION
libbash is a package that enables bash dynamic-like shared libraries. Actually its a tool for managing bash scripts whose functions you may want to load and use in scripts of your own. It contains a 'dynamic loader' for the shared libraries ( ldbash(1)), a configuration tool (ldbashconfig(8)), and some libraries. Using ldbash(1) you are able to load loadable bash libraries, such as getopts(1) and hashstash(1). A bash shared library that can be loaded using ldbash(1) must answer 4 requirments: 1. It must be installed in $LIBBASH_PREFIX/lib/bash (default is /usr/lib/bash). 2. It must contain a line that begins with '#EXPORT='. That line will contain (after the '=') a list of functions that the library exports. I.e. all the function that will be usable after loading that library will be listed in that line. 3. It must contain a line that begins with '#REQUIRE='. That line will contain (after the '=') a list of bash libraries that are required for our library. I.e. every bash library that is in use in our bash library must be listed there. 4. The library must be listed (For more information, see ldbashconfig(8)). Basic guidelines for writing library of your own: 1. Be aware, that your library will be actually sourced. So, basically, it should contain (i.e define) only functions. 2. Try to declare all variables intended for internal use as local. 3. Global variables and functions that are intended for internal use (i.e are not defined in '#EXPORT=') should begin with: __<library_name>_ For example, internal function myfoosort of hashstash library should be named as __hashstash_myfoosort This helps to avoid conflicts in global name space when using libraries that come from different vendors. 4. See html manual for full version of this guide. AUTHORS
Hai Zaar <> Gil Ran <> SEE ALSO
ldbash(1), ldbashconfig(8), getopts(1), hashstash(1) colors(1) messages(1) urlcoding(1) locks(1) Linux Epoch Linux

Featured Tech Videos

All times are GMT -4. The time now is 04:40 AM.
Unix & Linux Forums Content Copyright 1993-2020. All Rights Reserved.
Privacy Policy