Sponsored Content
Full Discussion: Matching by key fields
Top Forums Shell Programming and Scripting Matching by key fields Post 302282664 by ChicagoBlues on Sunday 1st of February 2009 09:36:33 AM
Old 02-01-2009
Thanks danmero. This works very well for a small dataset. I wonder how well it will perform with 800 million records extract and a 300K records key.

Thanks,

- CB

Last edited by ChicagoBlues; 02-01-2009 at 10:55 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Fill the Key fields : Please help us

Hi .... we are having the below file .Column 1, Column 2 ,column 3 are key fields... In the below ...for 2 nd , 3 rd row the repeated key column is missing .... i want the new file to be populated with all missing key columns. ... (11 Replies)
Discussion started by: charandevu
11 Replies

2. Shell Programming and Scripting

matching 2 exact fields

Dear experts, I have a file1 that looks like 60127930928 2091 60129382039 2092 60126382937 2091 60128937928 2061 60127329389 2062 60123748730 2061 60128730293 2061 and file 2 that looks like 60127930928 2091 60129382039 2092 60126382937 2093 60128937928 2061 60127329389... (2 Replies)
Discussion started by: aismann
2 Replies

3. Shell Programming and Scripting

Perl function to sort a file based on key fields

Hi, I am new to PERL.I want to sort all the lines in a file based on 1,2 and 4th filelds. Can U suggest me a command/function in perl for this operation.. (5 Replies)
Discussion started by: karthikd214
5 Replies

4. Shell Programming and Scripting

awk should output if one input file doesnt have matching key

nawk -F, 'FNR==NR{a= $3 ;next} $2 in a{print $1, 'Person',$2, a}' OFS=, filea fileb Input filea Input fileb output i am getting : (2 Replies)
Discussion started by: pinnacle
2 Replies

5. Linux

matching two fields

Hi I am having 2 fields and if f1=f2 i wanna print that line eg 1 2 1 3 1 9 2 2 3 5 9 9 In the abov eg. the highlighted lines shud be printed 2 2 9 9 Thanking u (3 Replies)
Discussion started by: binnybio
3 Replies

6. Shell Programming and Scripting

Compare Fields from two text files using key columns

Hi All, I have two files to compare. Each has 10 columns with first 4 columns being key index together. The rest of the columns have monetary values. Using Perl, I want to read one file into hash; check for the key value availability in file 2; then compare the values in the rest of 6... (2 Replies)
Discussion started by: Sangtha
2 Replies

7. Shell Programming and Scripting

Rsa public private key matching

Hi All, I have a requirement where i need to check if an rsa public key corresponds to a private key and hence return success or failure. Currently i am using the command diff <( ssh-keygen -y -e -f "$PRIVKEY" ) <( ssh-keygen -y -e -f "$PUBLICKEY" ) and its solving my purpose. This is in... (1 Reply)
Discussion started by: mritusmoi
1 Replies

8. UNIX for Dummies Questions & Answers

awk - Print lines if only matching key is found

I am looking to move matching lines (01 - 07) from File1 and 77 tab the matching string from File2, to File3.txt. I am almost done but - Currently, script is not printing lines to File3.txt in order. Thanks a lot. Any help is appreciated. Script I am using: awk 'FNR == NR && ! /^]*$/ {... (9 Replies)
Discussion started by: High-T
9 Replies

9. UNIX for Dummies Questions & Answers

File updation on matching key

I have input file like Input.dat with below content RRD 0Z91YUn000000Lk 9000100001 103020151117 STMT151117155527001 0000 2 000000 000004 RRD 0Z91YUn00000ysj 9000100001 103020151117 STMT151117155527001 0000 3 000000 000003 RRD 0Z91YUn00001vGh 9000100002... (12 Replies)
Discussion started by: PRAMOD 96
12 Replies

10. UNIX for Beginners Questions & Answers

Matching 2 files based on key

Hi all I have two files I need to match record from first file and second file on column 1,8 and and output only match records on file1 File1: 020059801803180116130926800002090000800231000245204003160000000002000461OUNCE000000350000100152500BM01007W0000 ... (5 Replies)
Discussion started by: arunkumar_mca
5 Replies
cdbmake(1)						      General Commands Manual							cdbmake(1)

NAME
cdbmake - create a constant database SYNOPSIS
cdbmake cdb cdb.tmp DESCRIPTION
cdbmake reads a series of encoded records from its standard input and writes a constant database to cdb. cdbmake ensures that cdb is updated atomically, so programs reading cdb never have to wait for cdbmake to finish. It does this by first writing the database to cdb.tmp and then moving cdb.tmp on top of cdb. If cdb.tmp already exists, it is destroyed. The directories con- taining cdb.tmp and cdb must be writable to cdbmake; they must also be on the same filesystem. cdbmake always makes sure that cdb.tmp is safely written to disk before it replaces cdb. If the input is in a bad format or if cdbmake has any trouble writing cdb.tmp to disk, cdbmake complains and leaves cdb alone. RECORD FORMAT Records are indexed by keys. A key is a string. cdb is structured so that another program, starting from a key, can quickly find the rel- evant record. cdbmake allows several records with the same key, although most readers take only the first record, and cdbmake slows down somewhat if there are many records with the same key. cdbmake and cdbdump(1) preserve the order of records. A record is encoded for cdbmake as +klen,dlen:key->data followed by a newline. Here klen is the number of bytes in key and dlen is the number of bytes in data. The end of data is indicated by an extra newline. For example: +3,5:one->Hello +3,7:two->Goodbye key and data may contain any characters, including colons, dashes, newlines, and nulls. Keys and data do not have to fit into memory. A database cannot exceed 4 gigabytes. cdb is portable across machines. SEE ALSO
cdbdump(1), cdbget(1), cdbstats(1) cdbmake(1)
All times are GMT -4. The time now is 08:49 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy