Sponsored Content
Top Forums Shell Programming and Scripting Filtering out duplicates with the highest version number Post 302723263 by Chubler_XL on Monday 29th of October 2012 07:16:52 PM
Old 10-29-2012
Another awk solution:

Code:
awk -F_ '$2>0+V[$1]{V[$1]=$2;F[$1]=$0} END{for(k in V) print F[k]}' OFS=_ infile

This should work for quite large files, however the output will be unsorted, you didn't specify if the file order was important.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

find the highest number in the file

Hi, I have a file a.txt and it has values in it Eg :- I need to read through the file and find the number that is the greatest in them all. Can any one assit me on this. Thanks (30 Replies)
Discussion started by: systemali
30 Replies

2. Shell Programming and Scripting

Extract the highest number out

Hi Gurus, I've using HPUX B.11.23 U ia64 with shell = sh. I've been having some problem get the highest number of this script. Actually I wanted to get the highest number from this listing (TEST123 data and based on this highest number, there will be email being sent out. For example,... (6 Replies)
Discussion started by: superHonda123
6 Replies

3. Shell Programming and Scripting

Find highest number - working but need help!

Hello all, I am new to this and need some help or maybe steer me to the right direction! I wrote a script to get the highest number and prints it on the screen, the script basically asks the user to input numbers, and then prints the highest number! very simple it works like this $sh max.sh... (8 Replies)
Discussion started by: unknownsolo
8 Replies

4. Shell Programming and Scripting

Only print the entries with the highest number?

Just want to say this is great resources for all thing Unix!! cat tmp.txt A 3 C 19 A 2 B 5 A 1 A 0 C 13 B 9 C 1 Desired output: A 3 B 9 C 19 The following work but I am wondering if there is a better way to do it: (4 Replies)
Discussion started by: chirish
4 Replies

5. UNIX for Dummies Questions & Answers

Filtering the duplicates

Hello, I want to filter all the duplicates of a record to one place. Sample input and output will give you better idea. I am new to unix. Can some one help me on this? Input: 7488 7389 chr1.fa chr1.fa 3546 9887 chr5.fa chr9.fa 7387 7898 chrX.fa chr3.fa 7488 7389 chr1.fa chr1.fa... (2 Replies)
Discussion started by: koneru_18
2 Replies

6. Shell Programming and Scripting

Need to print duplicate row along with highest version of original

There are some duplicate field on description column .I want to print duplicate row along with highest version of number and corresponding description column. file1.txt number Description === ============ 34567 nl21a00is-centerdb001:ncdbareq:Error in loading init 34577 ... (7 Replies)
Discussion started by: vijay_rajni
7 Replies

7. Shell Programming and Scripting

PERL "filtering the log file removing the duplicates

Hi folks, I have a log file in the below format and trying to get the output of the unique ones based on mnemonic IN PERL. Could any one please let me know with the code and the logic ? Severity Mnemonic Log Message 7 CLI_SCHEDULER Logfile for scheduled CLI... (3 Replies)
Discussion started by: scriptscript
3 Replies

8. Shell Programming and Scripting

Filtering duplicates based on lookup table and rules

please help solving the following. I have access to redhat linux cluster having 32gigs of ram. I have duplicate ids for variable names, in the file 1,2 are duplicates;3,4 and 5 are duplicates;6 and 7 are duplicates. My objective is to use only the first occurrence of these duplicates. Lookup... (4 Replies)
Discussion started by: ritakadm
4 Replies

9. Shell Programming and Scripting

Sort from highest to lowest number

Hi Guys, I am looking for a way to sort the output below from the "Inuse" count from Highest to Lowest. Is it possible? Thanks in advance. user1 0.12 0.06 0 0.12 User Inuse Pin Pgsp Virtual Unit:... (4 Replies)
Discussion started by: jaapar
4 Replies

10. Shell Programming and Scripting

Need to show highest version line from the list

Hi All, Need help here, can you tell me the syntax to line grep the highest file version? 0 04-05-2016 08:00 lib/SBSSchemaProject.jar/schemas/ 0 04-05-2016 08:00 lib/SBSSchemaProject.jar/schemas/airprice/ 0 04-05-2016 08:00 ... (2 Replies)
Discussion started by: 100rin
2 Replies
makedbm(8)						      System Manager's Manual							makedbm(8)

NAME
makedbm - Makes a Network Information Service (NIS) dbm file SYNOPSIS
/var/yp/makedbm [-i yp_input_file] [-s yp_secure_name] [-a method] [-o yp_output_name] [-d yp_domain_name] [-m yp_master_name] infile out- file /var/yp/makedbm [-u dbmfilename] OPTIONS
Specifies that NIS maps are to be stored in one of the following formats: btree -- Recommended when creating and maintaining very large maps. dbm/ndbm -- For backward compatibility. This is the default. hash -- A potentially quicker method for managing small maps. Cre- ates a special entry with the key yp_input_file. Creates a special entry with the key yp_secure_file. This causes the makedbm command to write a secure map. The key value ypserver looks for YP_SECURE. Creates a special entry with the key yp_output_name. Creates a special entry with the key yp_domain_name. Creates a special entry with the key yp_master_name. If no master host name is specified, yp_mas- ter_name will be set to the local host name. Undoes a dbm file. That is, prints out a dbm file one entry per line, with a single space separating keys from values. DESCRIPTION
The makedbm command takes the file specified by the argument infile and converts it to a single file or a pair of files in dbm(3), btree(3), or hash(3) format. The dbm(3) files are stored as outfile.pag and outfile.dir. The btree(3) files are stored as outfile.btree. Each line of the input file is converted to a single dbm record. All characters up to the first tab or space form the key, and the rest of the line is defined as the key's associated data. If a line ends with a backslash (), the data for that record is continued onto the next line. It is left for the Network Information Service (NIS) clients to interpret the number sign (#); makedbm does not treat it as a comment character. The infile parameter can be a hyphen (-), in which case makedbm reads the standard input. The makedbm command is meant to be used in generating database files for NIS. The makedbm command generates a special entry with the key yp_last_modified, which is the date of infile. RESTRICTIONS
You must use the same database format for each map in a domain. In addition, a server serving multiple NIS domains must use the same data- base format for all domains. Although a Tru64 UNIX NIS server that takes advantage of btree files will be able to store very large maps, NIS slave servers that lack this feature might have a much smaller limit on the number of map entries they can handle. It may not be possible to distribute very large maps from a Tru64 UNIX NIS master server to a slave server that lacks support for very large maps. NIS clients are not affected by these enhancements. EXAMPLES
The following example shows how a combination of commands can be used to make the NIS dbm files passwd.byname.pag and passwd.byname.dir from the /etc/passwd file. The percent sign (%) signifies the system prompt. % awk 'BEGIN { FS = ":"; OFS = " "; } { print $1, $0 }' /etc/passwd > ptmp % makedbm ptmp passwd.byname % rm ptmp The awk command creates the file ptmp which is in a form usable by makedbm. The makedbm command uses the ptmp file to create the database files. The rm command removes the ptmp file. The following is an example of the makedb command used with the btree format database routine to store NIS maps. makedbm -a b ... SEE ALSO
Commands: yppasswd(1), ypmake(8) Functions: btree(3), dbm(3), dbopen(3), hash(3), ndbm(3) makedbm(8)
All times are GMT -4. The time now is 06:53 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy