Sponsored Content
Full Discussion: How to count unique strings
Top Forums Shell Programming and Scripting How to count unique strings Post 302339429 by KevinADC on Thursday 30th of July 2009 12:40:33 PM
Old 07-30-2009
Assuming by unique you mean 100% unique, and things like case or white space are included in the uniqueness:

Code:
use strict;
use warnings;
my %unique;
open my $FH, '/path/to/your/file' or die "$!";
while(<$FH>) {
    chomp;#<-- remove this line if the record seperator should be incuded
    $unique{$_};
}
close $FH;
foreach my $line (keys %unique) {
     print "$line = $unique{$line}\n";
}

Next time some effort on your part to first try and solve your programming requirements would be nice to see.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Finding Unique strings which match pattern

I need to grep for a pattern in a file. Files are huge and have several repeated occurances of the strings which match pattern. I just need the strings which contain the pattern in the output. For eg. The contents of my file are as follows. The pattern I want to match by is ABCD ... (5 Replies)
Discussion started by: tektips
5 Replies

2. Shell Programming and Scripting

count identical strings print last row and count

I have a sorted file like: Apple 3 Apple 5 Apple 8 Banana 2 Banana 3 Grape 31 Orange 7 Orange 13 I'd like to search $1 and if $1 is not the same as $1 in the previous row print that row and print the number of times $1 was found. so the output would look like: Apple 8 3 Banana... (2 Replies)
Discussion started by: dcfargo
2 Replies

3. Shell Programming and Scripting

Unique count from flat file

Hello Guys I have a flat file with '|~|' delimited When I use to record count using below command awk -FS"+" ' {print $colno}' filename | wc -l the count is fine But when I am trying to find the unique number of record the o/p is always 1 awk -FS"+" ' {print $colno}'... (11 Replies)
Discussion started by: Pratik4891
11 Replies

4. Shell Programming and Scripting

calculating unique strings values

Hi, Im looking for a script which will calculate the unique strings column 2 & 3 values in a log as mentioned in example eg:- bag 12 12 bag 18 15 bags 15 13 bags 15 14 blazer 24 24 blazer 33 32 boots 19 15 Result should be:- bag 30 27 bags 30 27... (9 Replies)
Discussion started by: Paulwintech
9 Replies

5. Shell Programming and Scripting

Command to show unique strings in two files

how to display the unique strings in two files using shell script or commands. I tried diff and cmp but it shows the entire line, i need only the mismatched strings. File1: sat,sun,mon,tue rose,lilly,lotus white,red,blue,green,pink File2: sat,sun,mon,tue rose,sunflower,lotus... (4 Replies)
Discussion started by: Arun_Linux
4 Replies

6. Shell Programming and Scripting

awk to count using each unique value

Im looking for an awk script that will take the unique values in column 5, then print and count the unique values in column 6. CA001011500 11111 11111 -9999 201301 AAA CA001012040 11111 11111 -9999 201301 AAA CA001012573 11111 11111 -9999 201301 BBB CA001012710 11111 11111 -9999 201301... (4 Replies)
Discussion started by: ncwxpanther
4 Replies

7. Shell Programming and Scripting

Count of unique lines in field 4

When I use the below awk to count the unique lines in $4 for the input it seems to work. The answer is 3 because $4 is only unique 3 times in all the entries. However, when I use the same on actual data I get 56,536 and I know the answer should be 56,548. My question is there a better way to... (8 Replies)
Discussion started by: cmccabe
8 Replies

8. Shell Programming and Scripting

Count occurrence of column one unique value having unique second column value

Hello Team, I need your help on the following: My input file a.txt is as below: 3330690|373846|108471 3330690|373846|108471 0640829|459725|100001 0640829|459725|100001 3330690|373847|108471 Here row 1 and row 2 of column 1 are identical but corresponding column 2 value are... (4 Replies)
Discussion started by: angshuman
4 Replies

9. UNIX for Beginners Questions & Answers

Count unique words

Dear all, I would like to know how to list and count unique words in thousands number of text files. Please help me out thanks in advance (9 Replies)
Discussion started by: imranrasheedamu
9 Replies

10. UNIX for Beginners Questions & Answers

Count unique column

Hello, I am trying to count unique rows in my file based on 4 columns (2-5) and to output its frequency in a sixth column. My file is tab delimited My input file looks like this: Colum1 Colum2 Colum3 Colum4 Coulmn5 1.1 100 100 a b 1.1 100 100 a c 1.2 200 205 a d 1.3 300 301 a y 1.3 300... (6 Replies)
Discussion started by: nans
6 Replies
SLAPO-UNIQUE(5) 						File Formats Manual						   SLAPO-UNIQUE(5)

NAME
slapo-unique - Attribute Uniqueness overlay to slapd SYNOPSIS
/etc/openldap/slapd.conf DESCRIPTION
The Attribute Uniqueness overlay can be used with a backend database such as slapd-bdb(5) to enforce the uniqueness of some or all attributes within a scope. This subtree defaults to all objects within the subtree of the database for which the Uniqueness overlay is con- figured. Uniqueness is enforced by searching the subtree to ensure that the values of all attributes presented with an add, modify or modrdn opera- tion are unique within the scope. For example, if uniqueness were enforced for the uid attribute, the subtree would be searched for any other records which also have a uid attribute containing the same value. If any are found, the request is rejected. The search is performed using the rootdn of the database, to avoid issues with ACLs preventing the overlay from seeing all of the relevant data. As such, the database must have a rootdn configured. CONFIGURATION
These slapd.conf options apply to the Attribute Uniqueness overlay. They should appear after the overlay directive. unique_uri <[strict ][ignore ]URI[URI...]...> Configure the base, attributes, scope, and filter for uniqueness checking. Multiple URIs may be specified within a domain, allowing complex selections of objects. Multiple unique_uri statements or olcUniqueURI attributes will create independent domains, each with their own independent lists of URIs and ignore/strict settings. The LDAP URI syntax is a subset of RFC-4516, and takes the form: ldap:///[base dn]?[attributes...]?scope[?filter] The base dn defaults to that of the back-end database. Specified base dns must be within the subtree of the back-end database. If no attributes are specified, the URI applies to all non-operational attributes. The scope component is effectively mandatory, because LDAP URIs default to base scope, which is not valid for uniqueness, because groups of one object are always unique. Scopes of sub (for subtree) and one for one-level are valid. The filter component causes the domain to apply uniqueness constraints only to matching objects. e.g. ldap:///?cn?sub?(sn=e*) would require unique cn attributes for all objects in the subtree of the back-end database whose sn starts with an e. It is possible to assert uniqueness upon all non-operational attributes except those listed by prepending the keyword ignore If not configured, all non-operational (e.g., system) attributes must be unique. Note that the attributes list of an ignore URI should gen- erally contain the objectClass, dc, ou and o attributes, as these will generally not be unique, nor are they operational attributes. It is possible to set strict checking for the uniqueness domain by prepending the keyword strict. By default, uniqueness is not enforced for null values. Enabling strict mode extends the concept of uniqueness to include null values, such that only one attribute within a subtree will be allowed to have a null value. Strictness applies to all URIs within a uniqueness domain, but some domains may be strict while others are not. It is not possible to set both URIs and legacy slapo-unique configuration parameters simultaneously. In general, the legacy configuration options control pieces of a single unfiltered subtree domain. unique_base <basedn> This legacy configuration parameter should be converted to the base dn component of the above unique_uri style of parameter. unique_ignore <attribute...> This legacy configuration parameter should be converted to a unique_uri parameter with ignore keyword as described above. unique_attributes <attribute...> This legacy configuration parameter should be converted to a unique_uri parameter, as described above. unique_strict This legacy configuration parameter should be converted to a strict keyword prepended to a unique_uri parameter, as described above. CAVEATS
unique_uri cannot be used with the old-style of configuration, and vice versa. unique_uri can implement everything the older system can do, however. Typical attributes for the ignore ldap:///... URIs are intentionally not hardcoded into the overlay to allow for maximum flexibility in meeting site-specific requirements. FILES
/etc/openldap/slapd.conf default slapd configuration file SEE ALSO
slapd.conf(5), slapd-config(5). OpenLDAP 2.4.21 2009/12/20 SLAPO-UNIQUE(5)
All times are GMT -4. The time now is 04:03 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy