Sponsored Content
Full Discussion: Duplicate lines in a file
Top Forums UNIX for Dummies Questions & Answers Duplicate lines in a file Post 302661431 by alister on Monday 25th of June 2012 09:08:17 AM
Old 06-25-2012
Quote:
Originally Posted by nsuresh316
My shell is csh
That's the type of information that should always be included in the original post, especially since most people will provide posix (bourne like) sh solutions.

Regards,
Alister
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove Duplicate Lines in File

I am doing KSH script to remove duplicate lines in a file. Let say the file has format below. FileA 1253-6856 3101-4011 1827-1356 1822-1157 1822-1157 1000-1410 1000-1410 1822-1231 1822-1231 3101-4011 1822-1157 1822-1231 and I want to simply it with no duplicate line as file... (5 Replies)
Discussion started by: Teh Tiack Ein
5 Replies

2. UNIX for Advanced & Expert Users

Duplicate lines in the file

Hi, I have a file with duplicate lines in it. I want to keep only the duplicate lines and delete the non duplicates. Can some one please help me? Regards Narayana Gupta (3 Replies)
Discussion started by: guptan
3 Replies

3. UNIX for Dummies Questions & Answers

How to redirect duplicate lines from a file????

Hi, I am having a file which contains many duplicate lines. I wanted to redirect these duplicate lines into another file. Suppose I have a file called file_dup.txt which contains some line as file_dup.txt A100-R1 ACCOUNTING-CONTROL ACTONA-ACTASTOR ADMIN-AUTH-STATS ACTONA-ACTASTOR... (3 Replies)
Discussion started by: zing_foru
3 Replies

4. UNIX for Dummies Questions & Answers

Remove Duplicate lines from File

I have a log file "logreport" that contains several lines as seen below: 04:20:00 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping 06:38:08 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping 07:11:05 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but... (18 Replies)
Discussion started by: Nysif Steve
18 Replies

5. Shell Programming and Scripting

Duplicate lines in a file

Hi All, I am trying to remove the duplicate entries in a file and print them just once. For example, if my input file has: 00:44,37,67,56,15,12 00:44,34,67,56,15,12 00:44,58,67,56,15,12 00:44,35,67,56,15,12 00:59,37,67,56,15,12 00:59,34,67,56,15,12 00:59,35,67,56,15,12... (7 Replies)
Discussion started by: faiz1985
7 Replies

6. UNIX for Advanced & Expert Users

In a huge file, Delete duplicate lines leaving unique lines

Hi All, I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space. I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies

7. Shell Programming and Scripting

How do I remove the duplicate lines in this file?

Hey guys, need some help to fix this script. I am trying to remove all the duplicate lines in this file. I wrote the following script, but does not work. What is the problem? The output file should only contain five lines: Later! (5 Replies)
Discussion started by: Ernst
5 Replies

8. UNIX for Advanced & Expert Users

Inserting duplicate lines in a file

Hi, I copied the contents of a binary file into a .text file using hd (hexdump) command. The data in binary file is such that I get in many places like following 00000250 00 00 00 00 3f 2d 91 68 3f 69 fb e7 00 00 00 00 |....?-.h?i......| 00000260 00 00 00 00 00 00 00 00 00 00 00 00 00... (2 Replies)
Discussion started by: KidD312
2 Replies

9. Shell Programming and Scripting

bash keep only duplicate lines in file

hello all in my bash script I have a file and I only want to keep the lines that appear twice in the file.Is there a way to do this? thanks in advance! (4 Replies)
Discussion started by: vlm
4 Replies

10. Shell Programming and Scripting

Remove duplicate lines from a file

Hi, I have a csv file which contains some millions of lines in it. The first line(Header) repeats at every 50000th line. I want to remove all the duplicate headers from the second occurance(should not remove the first line). I don't want to use any pattern from the Header as I have some... (7 Replies)
Discussion started by: sudhakar T
7 Replies
USE(1)								   User Commands							    USE(1)

NAME
use - Frontend to the Usepackage Environment Manager SYNOPSIS
csh and derivatives: source /usr/share/usepackage/use.csh bourne shell and derivatives: source /usr/share/usepackage/use.bsh korn shell: . /usr/share/usepackage/use.ksh use [-vs] [-f file] package ... use -l DESCRIPTION
Usepackage is an environment management program. It is based on the principle of packages - collections of executables that share a common set of necessary environment variables, such as PATH, MANPATH or LD_LIBRARY_PATH. For each given package, use sources the appropriate environment information into the current shell. The environment information is speci- fied in a configuration file, see usepackage(1). OPTIONS -v Output verbose information to the standard error stream. -s Silence warnings for un-matched packages. This is useful in a shell rc script when a package is known not to be available on all architectures that the shell is used on. -f file Specify an alternate initial configuration file. -l List available packages and groups. FILES
/usr/share/usepackage/usepackage.conf The default configuration file. /usr/share/usepackage/use.csh Shell setup for csh and derivatives. /usr/share/usepackage/use.bsh Shell setup for bourne shell and derivatives. /usr/share/usepackage/use.ksh Shell setup for ksh. /usr/bin/usepackage The underlying Usepackage executable. ENVIRONMENT
Other than the reading and re-definition of environment variables for package setup, use also uses the following environment variables for user configuration: PACKAGES_PATH Colon-separated path list giving the directories to search for configuration files. Shell-style tilde (~) user-directory escapes are expanded. HOME If present in the environment, this is used to provide the expansion for the tilde (~) user-directory. SHELL If present in the environment, the last path component of this is used for shell matching (see SYNTAX) and detecting the style of environment output that should be used, see usepackage(1). COPYRIGHT
Usepackage Environment Manager Copyright (C) 1995-2005 Jonathan Hogg This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MER- CHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA SEE ALSO
usepackage(1), csh(1), sh(1), ksh(1) Usepackage $Date: 2005/12/11 16:42:09 $ USE(1)
All times are GMT -4. The time now is 07:36 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy