Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Finding unique reocrds at a particular field

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Finding unique reocrds at a particular field
# 8  
Old 12-11-2007
Quote:
Originally Posted by Franklin52
Try this:

Code:
awk '
BEGIN {FS="|"}
{cnt[$4]++;line[$4]=$0}
END {
  for(i in cnt) {
    if(cnt[i]==1) {
      print line[i]
    }
  }
}' file

> output_file

Regards
Simply redirect the output to a new file using the shell's redirection mechanism
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Count of unique lines in field 4

When I use the below awk to count the unique lines in $4 for the input it seems to work. The answer is 3 because $4 is only unique 3 times in all the entries. However, when I use the same on actual data I get 56,536 and I know the answer should be 56,548. My question is there a better way to... (8 Replies)
Discussion started by: cmccabe
8 Replies

2. Shell Programming and Scripting

awk to print unique text in field

I am trying to use awk to print the unique entries in $2 So in the example below there are 3 lines but 2 of the lines match in $2 so only one is used in the output. File.txt chr17:29667512-29667673 NF1:exon.1;NF1:exon.2;NF1:exon.38;NF1:exon.4;NF1:exon.46;NF1:exon.47 703.807... (5 Replies)
Discussion started by: cmccabe
5 Replies

3. Shell Programming and Scripting

Finding a text in files & replacing it with unique strings

Hallo Everyone. I have to admit I'm shell scripting illiterate . I need to find certain strings in several text files and replace each of the string by unique & corresponding text. I prepared a csv file with 3 columns: <filename>;<old_pattern>;<new_pattern> ... (5 Replies)
Discussion started by: gordom
5 Replies

4. Shell Programming and Scripting

Finding unique values in a hash (Perl)

Hi, I have a hash with unique keys associated with some data. my %FINALcontigs = ( 'mira_rep_c765:119reads**', 'ctctactggaagactgac', 'mira_rep_c7454:54reads**', 'atggatactgcgctgttgctaactactgga', 'mira_rep_c6803:12reads**', 'atcgactggatgcagggttgtggtttcta', ... (2 Replies)
Discussion started by: jdilts
2 Replies

5. Shell Programming and Scripting

Unique Field

I have this input file tilenet_test:clar_r5_performance:server_2:4.80762:0%:APM00083103999-009E,APM00083103999-009F tilenet_int:clar_r5_performance:server_2:4.80762:0%:APM00083103999-00C4... (3 Replies)
Discussion started by: greycells
3 Replies

6. Shell Programming and Scripting

Finding the number of unique words in a file

find the number of unique words in a file using sort com- mand. (7 Replies)
Discussion started by: abhikamune
7 Replies

7. Shell Programming and Scripting

Finding unique entries without sorting

Hi Guys, I have two files that I am using: File1 is as follows: wwe khfgv jfo jhgfd hoaha hao lkahe This is like a master file which has entries in the order which I want. (4 Replies)
Discussion started by: npatwardhan
4 Replies

8. Shell Programming and Scripting

Need help with finding unique string in log file

Shell script help Here is 3 sample lines from a log file <date> INFO <java.com.blah> abcd:ID= user login <date> DEBUG <java.com.blah> <nlah bla> abcd:ID=123 user login <date> INFO <java.com.blah> abcd:ID=3243 user login I want to find unique "ID" from this log... (3 Replies)
Discussion started by: gubbu
3 Replies

9. Shell Programming and Scripting

Perl sort unique by one field only

Hi all, I've searched the forum and I can find some code to sort uniquely in perl but not by a single field. I have a file with data such as the following: 1,test,34 1,test2,65 2,test,35, 1,test3,34 2,test,34 What i want to do is sort it uniqely by the first field only so I'd end... (2 Replies)
Discussion started by: Donkey25
2 Replies

10. UNIX for Dummies Questions & Answers

Finding Unique strings which match pattern

I need to grep for a pattern in a file. Files are huge and have several repeated occurances of the strings which match pattern. I just need the strings which contain the pattern in the output. For eg. The contents of my file are as follows. The pattern I want to match by is ABCD ... (5 Replies)
Discussion started by: tektips
5 Replies
Login or Register to Ask a Question

LEARN ABOUT HPUX

ttys

ttys(4) 						     Kernel Interfaces Manual							   ttys(4)

NAME

       ttys - terminal control database file for trusted systems

SYNOPSIS

DESCRIPTION

       The system supports a single terminal control database containing entries for each local terminal that can log into the system. Authentica-
       tion programs use information contained in the terminal control database to determine if login from the terminal is  permitted.	Additional
       fields are maintained for informational purposes.

       The  format  of	the terminal control database file is identical to other system authentication database files. For more information on the
       file format, see authcap(4).  The file consists of keyword field identifiers and values for those fields. The keyword identifiers supported
       and their use include:

       This field defines the terminal device name for the entry. The terminal device
			 is  expected  to  be  contained  in  the  directory, therefore this prefix should not be specified. If the terminal entry
			 describes the device, the field should contain

       This field records the user id of the last user to successfully login using
			 the terminal device.

       This		 field records the last successful login time to the terminal device.

       This		 field records the last unsuccessful login time to the terminal device.

       This field records the number of consecutive unsuccessful login attempts to
			 the terminal device.

       This field specifies the maximum number of consecutive unsuccessful login
			 attempts permitted using the terminal before the terminal is locked. Once the terminal is locked, it must be unlocked	by
			 an authorized administrator.

       This field specifies the login timeout value (sec).

       This field specifies the delay between login tries (sec).

       This flag field indicates whether the terminal device has been administratively
			 locked or not. This field is manipulated by authorized administrators only.

EXAMPLES

       The following is an example of a terminal control database entry:

       This  entry  is	for the system console device, The most recent successful login session was for the user The entry records the system time
       for the current successful login and the time of the most recent unsuccessful login attempt.

WARNINGS

       Remote terminals (ptys) should not be added to the or databases.  Device name formats treated as ptys by login are:

	      where	  x is a letter, and y is a hex number
	      where	  x is a letter, and y is a hex number

	      HP-UX 11i Version 3 is the last release to support
	      trusted systems functionality.

AUTHOR

       was developed by HP.

FILES

       Terminal control database file

SEE ALSO

       login(1), getprtcent(3), devassign(4), authcap(4), default(4).

								  TO BE OBSOLETED							   ttys(4)