Extracting duplicates from a desired field Post: 302474181

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing duplicates in a sorted file by field.

I have data like this: It's sorted by the 2nd field (TID). envoy,90000000000000634600010001,04/11/2008,23:19:27,RB00266,0015,DETAIL,ERROR, envoy,90000000000000634600010001,04/12/2008,04:23:45,RB00266,0015,DETAIL,ERROR,...

2. Shell Programming and Scripting

Awk to find duplicates in 2nd field

I want to find duplicates in file on 2nd field i wrote this code: nawk '{a++} END{for i in a {if (a>1) print}}' temp Could not find whats wrong with this. Appreciate help

3. Shell Programming and Scripting

extracting desired value from cmd o/p

Hi I m using vxassist cmd to get avaialbe space in DG o/p is like this, wat I need is only numeric value of the o/p , I have a solution is anyone can provide better x1# vxassist -g appdg1 maxsize Maximum volume size: 90406912 (44144Mb) I need only 44144 value command I m using rite...

4. Shell Programming and Scripting

need Shell script for Sort BASED ON FIRST FIELD and PRINT THE WHOLE FILE WITHOUT DUPLICATES

Can some one provide me a shell script. I have file with many columns and many rows. need to sort the first column and then remove the duplicates records if exists.. finally print the full data with first coulm as unique. Sort BASED ON FIRST FIELD and remove the duplicates if exists...

5. Shell Programming and Scripting

Remove the partial duplicates by checking the length of a field

Hi Folks - I'm quite new to awk and didn't come across such issues before. The problem statement is that, I've a file with duplicate records in 3rd and 4th fields. The sample is as below: aaaaaa|a12|45|56 abbbbaaa|a12|45|56 bbaabb|b1|51|45 bbbbbabbb|b2|51|45 aaabbbaaaa|a11|45|56 ...

6. Shell Programming and Scripting

CSV with commas in field values, remove duplicates, cut columns

Hi Description of input file I have: ------------------------- 1) CSV with double quotes for string fields. 2) Some string fields have Comma as part of field value. 3) Have Duplicate lines 4) Have 200 columns/fields 5) File size is more than 10GB Description of output file I need:...

7. UNIX for Dummies Questions & Answers

remove duplicates based on a field and criteria

Hi, I have a file with fields like below: A;XYZ;102345;222 B;XYZ;123243;333 C;ABC;234234;444 D;MNO;103345;222 E;DEF;124243;333 desired output: C;ABC;234234;444 D;MNO;103345;222 E;DEF;124243;333 ie, if the 4rth field is a duplicate.. i need only those records where...

8. Shell Programming and Scripting

Remove duplicates based on a field's value

Hi All, I have a text file with three columns. I would like a simple script that removes lines in which column 1 has duplicate entries, but use the largest value in column 3 to decide which one to keep. For example: Input file: 12345a rerere.rerere len=23 11111c fsdfdf.dfsdfdsf len=33 ...

9. Shell Programming and Scripting

Deleting only 2nd and third duplicates in field 2

10. Shell Programming and Scripting

Trying to remove duplicates based on field and row

I am trying to see if I can use awk to remove duplicates from a file. This is the file: -==> Listvol <== deleting /vol/eng_rmd_0941 deleting /vol/eng_rmd_0943 deleting /vol/eng_rmd_0943 deleting /vol/eng_rmd_1006 deleting /vol/eng_rmd_1012 rearrange /vol/eng_rmd_0943 ...

LEARN ABOUT OSX

grpck

grpck(1B)					     SunOS/BSD Compatibility Package Commands						 grpck(1B)

NAME

       grpck - check group database entries

SYNOPSIS

       /etc/grpck [filename]

DESCRIPTION

       The grpck utility checks that a file in group(4) does not contain any errors; it checks the /etc/group file by default.

FILES

       /etc/group

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |Availability		     |SUNWcsu			   |
       +-----------------------------+-----------------------------+

SEE ALSO

       groups(1), group(4), passwd(4), attributes(5)

DIAGNOSTICS

       Too many/few fields

	   An entry in the group file does not have the proper number of fields.

       No group name

	   The group name field of an entry is empty.

       Bad character(s) in group name

	   The group name in an entry contains characters other than lower-case letters and digits.

       Invalid GID

	   The group ID field in an entry is not numeric or is greater than 65535.

       Null login name

	   A login name in the list of login names in an entry is null.

       Logname not found in password file

	   A login name in the list of login names in an entry is not in the password file.

       Line too long

	   A line (including the newline character) in the group file exceeds the maximum length of 512 characters.

       Duplicate logname entry

	   A login name appears more than once in the list of login names for a group file entry.

       Out of memory

	   The program cannot allocate memory in order to continue.

       Maximum groups exceeded for logname

	   A login name's group membership exceeds the maximum, NGROUPS_MAX.

SunOS 5.10							    10 Jan 2000 							 grpck(1B)