Sponsored Content
Full Discussion: remove duplicating lines
Top Forums UNIX for Dummies Questions & Answers remove duplicating lines Post 302355260 by chipcmc on Tuesday 22nd of September 2009 07:42:02 AM
Old 09-22-2009
can you copy/paste the content of your file?
look in man page for command sort in your system:
Code:
man sort

the command in my enviorement work's,
Code:
$cat /etc/issue; echo $SHELL
Red Hat Enterprise Linux AS release 3 (Taroon Update 6)
Kernel \r on an \m
/bin/bash

 

10 More Discussions You Might Find Interesting

1. Post Here to Contact Site Administrators and Moderators

Sorry for duplicating posts

SORRY FOR DUPLICATING POSTS, COULD U JUST REMOVE THE FIRST ONE. Thank u. (1 Reply)
Discussion started by: vrguha
1 Replies

2. Red Hat

Duplicating ethernet speed

Hi guys, Suppose you have a server with two ethernet cards (1GB each) and each cards are connecting to two different switches cisco 3750. My question is: How can I setup my server's network interfaces to increase the throughput up to 2GB? is it possible? If not, do you know another way to up... (3 Replies)
Discussion started by: iga3725
3 Replies

3. Shell Programming and Scripting

remove blank lines and merge lines in shell

Hi, I'm not a expert in shell programming, so i've come here to take help from u gurus. I'm trying to tailor a csv file that i got to make it work for the LOAD FROM command. I've a datatable csv of the below format - --in file format xx,xx,xx ,xx , , , , ,,xx, xxxx,, ,, xxx,... (11 Replies)
Discussion started by: dvah
11 Replies

4. Shell Programming and Scripting

mv duplicating directories

Hi Folks, I've put together a script for sorting my backup files into sub folders to be run from a cron job. Each file is named username.tar.gz and the file /etc/trueuserowners contains all users and their owner in the format "user: owner". The script works fine identifying users and their owners... (10 Replies)
Discussion started by: beddo
10 Replies

5. UNIX for Dummies Questions & Answers

duplicating a line

I have a text file which is the results of running a tests hundreds of times. For simplicity let's say that each test consists of 5 lines of text numbered 1-5 e.g. 1 aaa aaa aaa 2 bbb bbb bbb 3 ccc ccc ccc 4 ddd ddd ddd 5 eee eee eee 1 aaa aaa aaa 2 bbb bbb bbb 3 ccc... (4 Replies)
Discussion started by: millsy5
4 Replies

6. Shell Programming and Scripting

Duplicating and changing sh file

Hello, I have a file named file_1.sh that I want to duplicate into file_2.sh, file_3.sh,..., etc. I also need to change the text within each file so that it would fit the file name. For example, in file_1.sh there is a command to save some output as 'output_1.txt', and also there is an input... (3 Replies)
Discussion started by: haguyw
3 Replies

7. UNIX for Dummies Questions & Answers

Want to remove all lines but not latest 50 lines from a file

Hi, I have a huge file which has Lacs of lines. File system got full. I want your guys help to suggest me a solution so that I can remove all lines from that file but not last 50,000 lines. I want solution which can remove lines from existing file so that I can have some space left with. (28 Replies)
Discussion started by: prashant2507198
28 Replies

8. Shell Programming and Scripting

Two files, remove lines from second based on lines in first

I have two files, a keepout.txt and a database.csv. They're unsorted, but could be sorted. keepout: user1 buser3 anuser19 notheruser27 database: user1,2343,"information about",field,blah,34 user2,4231,"mo info",etc,stuff,43 notheruser27,4344,"hiya",thing,more thing,423... (4 Replies)
Discussion started by: esoffron
4 Replies

9. Shell Programming and Scripting

Remove lines that are subsets of other lines in File

Hello everyone, Although it seems easy, I've been stuck with this problem for a moment now and I can't figure out a way to get it done. My problem is the following: I have a file where each line is a sequence of IP addresses, example : 10.0.0.1 10.0.0.2 10.0.0.5 10.0.0.1 10.0.0.2... (5 Replies)
Discussion started by: MisterJellyBean
5 Replies

10. Shell Programming and Scripting

awk to remove lines that do not start with digit and combine line or lines

I have been searching and trying to come up with an awk that will perform the following on a converted text file (original is a pdf). 1. Since the first two lines are (begin with) text they are removed 2. if $1 is a number then all text is merged (combined) into one line until the next... (3 Replies)
Discussion started by: cmccabe
3 Replies
UNIQ(1) 						    BSD General Commands Manual 						   UNIQ(1)

NAME
uniq -- report or filter out repeated lines in a file SYNOPSIS
uniq [-cdu] [-f fields] [-s chars] [input_file [output_file]] DESCRIPTION
The uniq utility reads the standard input comparing adjacent lines, and writes a copy of each unique input line to the standard output. The second and succeeding copies of identical adjacent input lines are not written. Repeated lines in the input will not be detected if they are not adjacent, so it may be necessary to sort the files first. The following options are available: -c Precede each output line with the count of the number of times the line occurred in the input, followed by a single space. -d Don't output lines that are not repeated in the input. -f fields Ignore the first fields in each input line when doing comparisons. A field is a string of non-blank characters separated from adja- cent fields by blanks. Field numbers are one based, i.e. the first field is field one. -s chars Ignore the first chars characters in each input line when doing comparisons. If specified in conjunction with the -f option, the first chars characters after the first fields fields will be ignored. Character numbers are one based, i.e. the first character is character one. -u Don't output lines that are repeated in the input. If additional arguments are specified on the command line, the first such argument is used as the name of an input file, the second is used as the name of an output file. The uniq utility exits 0 on success, and >0 if an error occurs. COMPATIBILITY
The historic +number and -number options have been deprecated but are still supported in this implementation. SEE ALSO
sort(1) STANDARDS
The uniq utility is expected to be IEEE Std 1003.2 (``POSIX.2'') compatible. BSD
January 6, 2007 BSD
All times are GMT -4. The time now is 05:57 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy