Sponsored Content
Top Forums Shell Programming and Scripting find out duplicate records in file? Post 302296801 by tiger2000 on Wednesday 11th of March 2009 10:48:45 PM
Old 03-11-2009
find out duplicate records in file?

Dear All,

I have one file which looks like :

account1:passwd1
account2:passwd2
account3:passwd3
account1:passwd4
account5:passwd5
account6:passwd6

you can see there're two records for account1. and is there any shell command which can find out : account1 is the duplicate record in file?

Thanks.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Delete Duplicate records from a tilde delimited file

Hi All, I want to delete duplicate records from a tilde delimited file. Criteria is considering the first 2 fields, the combination of which has to be unique, below is a sample of records in the input file 1620000010338~2446694087~0~20061130220000~A00BCC1CT... (5 Replies)
Discussion started by: irshadm
5 Replies

2. Shell Programming and Scripting

Remove all instances of duplicate records from the file

Hi experts, I am new to scripting. I have a requirement as below. File1: A|123|NAME1 A|123|NAME2 B|123|NAME3 File2: C|123|NAME4 C|123|NAME5 D|123|NAME6 1) I have 2 merge both the files. 2) need to do a sort ( key fields are first and second field) 3) remove all the instances... (3 Replies)
Discussion started by: vukkusila
3 Replies

3. UNIX for Advanced & Expert Users

Duplicate records from oracle to text file.

Hi, I want to fetch duplicate records from an external table to a text file. Pls suggest me. Thanks (1 Reply)
Discussion started by: shilendrajadon
1 Replies

4. Shell Programming and Scripting

How to find Duplicate Records in a text file

Hi all pls help me by providing soln for my problem I'm having a text file which contains duplicate records . Example: abc 1000 3452 2463 2343 2176 7654 3452 8765 5643 3452 abc 1000 3452 2463 2343 2176 7654 3452 8765 5643 3452 tas 3420 3562 ... (1 Reply)
Discussion started by: G.Aavudai
1 Replies

5. Shell Programming and Scripting

find duplicate records... again

Hi all: Let's suppose I have a file like this (but with many more records). XX ME 342 8688 2006 7 6 3c 60.029 -38.568 2901 0001 74 4 7603 8 969.8 958.4 3.6320 34.8630 985.5 973.9 3.6130 34.8600 998.7 986.9 3.6070 34.8610 1003.6 991.7 ... (4 Replies)
Discussion started by: rleal
4 Replies

6. Shell Programming and Scripting

compare fields in a file with duplicate records

Hi: I've been searching the net but didnt find a clue. I have a file in which, for some records, some fields coincide. I want to compare one (or more) of the dissimilar fields and retain the one record that fulfills a certain condition. For example, on this file: 99 TR 1991 5 06 ... (1 Reply)
Discussion started by: rleal
1 Replies

7. Shell Programming and Scripting

Find Duplicate records in first Column in File

Hi, Need to find a duplicate records on the first column, ANU4501710430989 0000000W20389390 ANU4501710430989 0000000W67065483 ANU4501130050520 0000000W80838713 ANU4501210170685 0000000W69246611... (3 Replies)
Discussion started by: Murugesh
3 Replies

8. UNIX for Dummies Questions & Answers

CSV file:Find duplicates, save original and duplicate records in a new file

Hi Unix gurus, Maybe it is too much to ask for but please take a moment and help me out. A very humble request to you gurus. I'm new to Unix and I have started learning Unix. I have this project which is way to advanced for me. File format: CSV file File has four columns with no header... (8 Replies)
Discussion started by: arvindosu
8 Replies

9. Shell Programming and Scripting

Remove somewhat Duplicate records from a flat file

I have a flat file that contains records similar to the following two lines; 1984/11/08 7 700000 123456789 2 1984/11/08 1941/05/19 7 700000 123456789 2 The 123456789 2 represents an account number, this is how I identify the duplicate record. The ### signs represent... (4 Replies)
Discussion started by: jolney
4 Replies

10. Shell Programming and Scripting

Deleting duplicate records from file 1 if records from file 2 match

I have 2 files "File 1" is delimited by ";" and "File 2" is delimited by "|". File 1 below (3 record shown): Doc1;03/01/2012;New York;6 Main Street;Mr. Smith 1;Mr. Jones Doc2;03/01/2012;Syracuse;876 Broadway;John Davis;Barbara Lull Doc3;03/01/2012;Buffalo;779 Old Windy Road;Charles... (2 Replies)
Discussion started by: vestport
2 Replies
acctdisk(8)						      System Manager's Manual						       acctdisk(8)

NAME
acctdisk, acctdusg - Perform disk-usage accounting SYNOPSIS
acctdisk acctdusg [-u file] [-p file] FLAGS
Searches file as the alternate file for login names and numbers, instead of searching /etc/passwd. Writes records of file names for which it does not charge into file. DESCRIPTION
Normally the acctdisk and acctdusg commands are called from the dodisk shell procedure to do disk-usage accounting. The dodisk shell proce- dure is invoked when the cron daemon executes commands in the /usr/spool/cron/crontabs/[filename] file. In the usual case, the output of the diskusg command is the redirected input to the acctdisk command. When a more thorough, but slower, version of disk accounting is needed, specify the -o flag with the dodisk command. This is not normally done in the /usr/spool/cron/crontabs/[filename] file. When the -o flag is used, the acctdusg command replaces the diskusg command. acctdisk Normally, the acctdisk command reads a temporary output file produced by the diskusg or the acctdusg command from standard input, converts each record into a total disk-accounting record, and writes it to standard output. These records are merged with other accounting records with the acctmerg command to produce a daily accounting report. acctdusg The acctdusg command is called when the -o flag is used with the dodisk command. This produces a more thorough, but slower, version of disk-accounting records. Otherwise, the dodisk shell procedure invokes the diskusg command. The acctdusg command reads a list of files from standard input (usually piped from a find / -print command), computes the number of disk blocks (including indirect blocks) allocated to each file divided by the number of hard links then writes an individual record for each user to standard output. To find the user who is charged for the file, compare each file pathname with the login directories of the users. The user who has the longest pathname component match is charged for the file. Therefore, the relevant information for charging users is not ownership of a file but the directory where it is stored. The acctdusg command searches the /etc/passwd file, or the alternate password file specified with the -p flag, for login names, numbers and login directories. Each output record has the following format: uid login #blocks EXAMPLES
To start normal disk-accounting procedures, add a line similar to the following to the /usr/spool/cron/crontabs/[filename] file: 0 2 * * 4 /usr/sbin/acct/dodisk The foregoing example is a typical, periodically invoked command that the cron daemon reads and executes. The period is expressed by a 6-field entry having the format: mm hh daymon monyr wkday cmd. For any field requiring digits, numbers are integers. These six fields have the following significance: Purpose Time variable that has the value 0 through 59 expressing minutes past the hour. Time variable for the hour of the day in 24-hour clock notation. Time variable for the day of the month. Time variable for month of the year. Time variable for the day or days of the week, where 0 is Sunday and inclusive days are separated with a hyphen (-). Command the cron daemon must execute. Whenever you write any of the time variables described in the foregoing table, an unspecified value must be noted with an * (aster- isk) to define an empty field. In the foregoing example, the dodisk shell procedure runs at 02:00 hours (2) every Thursday (4). The dodisk shell procedure calls the acctdusg command to redirect its input to a temporary file and then calls the acctdisk shell procedure to redirect disk-usage records from the temporary file as input to a /var/adm/acct/nite/[filename] file as output. The file stored in the /var/adm/acct/nite subdirectory is a permanent binary record of disk usage for the specified period. To initiate a slower, more thorough disk-accounting procedure, add a line similar to the following to the /usr/spool/cron/crontabs/[filename] file. 0 2 * * 0-4 /usr/sbin/acct/dodisk -o The dodisk shell procedure calls the acctdusg command and the acctdisk command to write disk-usage records to the /var/adm/acct/nite/[filename] file just as in the previous example. The dodisk procedure runs at 2 a.m. every day (0-6) including Sunday. FILES
Specifies the command path. Specifies the command path. User database file. The active login/logout database file. Accounting header files that define formats for writing accounting files. RELATED INFORMATION
Commands: acct(8), acctmerg(8), cron(8), diskusg(8), dodisk(8), runacct(8) Functions: acct(2) delim off acctdisk(8)
All times are GMT -4. The time now is 03:49 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy