Sponsored Content
Full Discussion: grep unique occurrences
Top Forums UNIX for Dummies Questions & Answers grep unique occurrences Post 302117940 by Shell_Life on Thursday 17th of May 2007 09:18:45 AM
Old 05-17-2007
Quote:
Also taking it a bit further is there a way to clean up a file so it only has unique occurrences? Say I don't know the patterns to search for, I just want to remove all duplicates. Is this possible?
Code:
sort input_file | uniq

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Getting unique list of numbers using grep

Hi, I am going to fetch a list of numbers that starts with "0032" from a file with a format like the given below: " 0032459999 0032458888 0032457777 0032451111 0032452222 0032453333 0032459999 0032458888 0032457777 0032451111 0032452222 0032453333 " I want to get a unique... (6 Replies)
Discussion started by: tinku
6 Replies

2. Shell Programming and Scripting

grep for unique value

If i want to grep for a value in a file but display only unique value then which option can i use. ex: Values in the file IP <1.2.3.4> value <2> IP <1.2.3.4> value <2> IP <1.2.3.4> value <3> IP <1.2.3.5> value <1> i should get only the unique value (3 Replies)
Discussion started by: vls1210
3 Replies

3. UNIX for Dummies Questions & Answers

Grep Unique

Hello, I have a file with a list of car makes and specific information for each make. An example is: @Audi:Warranty @Audi:Pricing @Audi:Colors @Acura:Warranty @Acura:Pricing @Acura:Colors and so on through a bunch of makes. I need to make a list in a selection box of just one name of... (4 Replies)
Discussion started by: macbb1117
4 Replies

4. Homework & Coursework Questions

Du without directory and Grep for occurrences of a word

Assistance on work Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: Files stored in ... (1 Reply)
Discussion started by: alindner
1 Replies

5. Shell Programming and Scripting

grep with date & unique output

alert.log has the entries with ORA-XXXX, .... Mon Sep 24 15:08:09 2012 WARNING: inbound connection timed out (ORA-3136) Mon Sep 24 15:08:09 2012 WARNING: inbound connection timed out (ORA-3136) Mon Sep 24 15:08:09 2012 WARNING: inbound connection timed out (ORA-3136) Mon Sep 24 15:15:01... (4 Replies)
Discussion started by: Daniel Gate
4 Replies

6. Shell Programming and Scripting

How to grep on unique id which has request and response on different lines?

Hi I want to find out those unique uids from the log file which have request and response. The log file format is as follows. This log has other irrelevant lines too but each uid should have request and reponse, I need those uids only 2013-04-03 10:51:01,808 INFO <?xml version="1.0"... (4 Replies)
Discussion started by: random_thoughts
4 Replies

7. Shell Programming and Scripting

GREP between last occurrences of two strings

Hi I have the server.log file as: Server Stopped ABC DEF GHI JKL Server Started MNO PQR STU Server Stopped VWX YZ ABC Server Started Server Stopped 123 456 789 (9 Replies)
Discussion started by: ankur328
9 Replies

8. UNIX for Dummies Questions & Answers

Grep unique 1st column

Hello, I'm trying to used awk but am new to this. I have a file like this: Bob is a good boy Bob is a strange person Bob is a good dancer Jane can party Jane is a good girl Jane is batty I'd like to get this: Bob is a good boy is a strange person is a good dancer Jane... (4 Replies)
Discussion started by: Billyjo
4 Replies

9. UNIX for Beginners Questions & Answers

Grep or awk a unique and specific word across many fields

Hi there, I have data with similar structure as this: CHR START-SNP END-SNP REF ALT PATIENT1 PATIENT2 PATIENT3 PATIENT4 chr1 69511 69511 A G homo hetero homo hetero chr2 69513 69513 T C . hetero homo hetero chr3 69814 69814 G C . . homo homo chr4 69815 69815 C A hetero . . hetero is... (10 Replies)
Discussion started by: daashti
10 Replies

10. Shell Programming and Scripting

How to strictly grep (with before and after args) for last N number of occurrences?

Here is my grep string to print only the last occurrence, with before and after lines. Note that the tail Argument is sum of B and A args + 1, so that it prints the data of only the last 1 match. Now I need to print last 2 such matches. I thought of doubling the tail arg like 5+5+1 (For -- line),... (2 Replies)
Discussion started by: samjna
2 Replies
uniq(1) 						      General Commands Manual							   uniq(1)

NAME
uniq - report repeated lines in a file SYNOPSIS
fields] chars] [input_file [output_file]] DESCRIPTION
reads the input text file input_file, comparing adjacent lines, and copies the result to output_file. If input_file is not specified, the standard input and standard output are used. If input_file is specified, but output_file is not, results are printed to standard output. input_file and output_file must not be the same file. Line-Comparison Options recognizes the following options when comparing adjacent lines: Print those lines that are repeated in the original file. Print copy only of each repeated line in the input file. Generate an output report in default style except that each line is preceded by a count of the number of times it occurred. If this option is specified, the and options are ignored if either or both are also present. If none of the options or are present, prints the results of the union of the and options, producing a copy of the original input file with the second and succeeding copies of any repeated lines removed. (Note that repeated lines must be adjacent in order to be found -- see sort(1)). Field-Skip Options Two options are provided for skipping an initial portion of each line when making comparisons: Ignore the first fields fields, together with any blanks before each. fields is a positive decimal integer. A field is defined as a string of non-space, non-tab characters separated by tabs and/or spaces from its neighbors. Ignore the first chars characters. chars is a positive decimal integer. Each line in the input is assumed to be terminated with a new line character for purposes of comparison. Fields are skipped before characters. EXTERNAL INFLUENCES
Environment Variables must be equal to the value it had when the input files were sorted. determines the interpretation of text within files as single- and/or multi-byte characters, and defines a space character when the or option is used. determines the language in which messages are displayed. If or is not specified in the environment or is set to the empty string, the value of is used as a default for each unspecified or empty variable. If is not specified or is set to the empty string, a default of "C" (see lang(5)) is used instead of If any internationalization variable contains an invalid setting, behaves as if all internationalization variables are set to "C". See environ(5). International Code Set Support Single- and multi-byte character code sets are supported. RETURN VALUE
Exit values are: 0 Successful completion. >0 Error condition occurred. AUTHOR
was developed by OSF and HP. SEE ALSO
comm(1), sort(1). STANDARDS CONFORMANCE
uniq(1)
All times are GMT -4. The time now is 02:38 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy