02-01-2009
Thanks danmero. This works very well for a small dataset. I wonder how well it will perform with 800 million records extract and a 300K records key.
Thanks,
- CB
Last edited by ChicagoBlues; 02-01-2009 at 10:55 AM..
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi ....
we are having the below file .Column 1, Column 2 ,column 3 are key fields...
In the below ...for 2 nd , 3 rd row the repeated key column is missing ....
i want the new file to be populated with all missing key columns.
... (11 Replies)
Discussion started by: charandevu
11 Replies
2. Shell Programming and Scripting
Dear experts,
I have a file1 that looks like
60127930928 2091
60129382039 2092
60126382937 2091
60128937928 2061
60127329389 2062
60123748730 2061
60128730293 2061
and file 2 that looks like
60127930928 2091
60129382039 2092
60126382937 2093
60128937928 2061
60127329389... (2 Replies)
Discussion started by: aismann
2 Replies
3. Shell Programming and Scripting
Hi,
I am new to PERL.I want to sort all the lines in a file based on 1,2 and 4th filelds.
Can U suggest me a command/function in perl for this operation.. (5 Replies)
Discussion started by: karthikd214
5 Replies
4. Shell Programming and Scripting
nawk -F, 'FNR==NR{a= $3 ;next} $2 in a{print $1, 'Person',$2, a}' OFS=, filea fileb
Input filea
Input fileb
output i am getting : (2 Replies)
Discussion started by: pinnacle
2 Replies
5. Linux
Hi
I am having 2 fields and if f1=f2 i wanna print that line
eg
1 2
1 3
1 9
2 2
3 5
9 9
In the abov eg. the highlighted lines shud be printed
2 2
9 9
Thanking u (3 Replies)
Discussion started by: binnybio
3 Replies
6. Shell Programming and Scripting
Hi All,
I have two files to compare. Each has 10 columns with first 4 columns being key index together. The rest of the columns have monetary values.
Using Perl, I want to read one file into hash; check for the key value availability in file 2; then compare the values in the rest of 6... (2 Replies)
Discussion started by: Sangtha
2 Replies
7. Shell Programming and Scripting
Hi All,
I have a requirement where i need to check if an rsa public key corresponds to a private key and hence return success or failure. Currently i am using the command
diff <( ssh-keygen -y -e -f "$PRIVKEY" ) <( ssh-keygen -y -e -f "$PUBLICKEY" )
and its solving my purpose. This is in... (1 Reply)
Discussion started by: mritusmoi
1 Replies
8. UNIX for Dummies Questions & Answers
I am looking to move matching lines (01 - 07) from File1 and 77 tab the matching string from File2, to File3.txt. I am almost done but
- Currently, script is not printing lines to File3.txt in order.
Thanks a lot.
Any help is appreciated.
Script I am using:
awk 'FNR == NR && ! /^]*$/ {... (9 Replies)
Discussion started by: High-T
9 Replies
9. UNIX for Dummies Questions & Answers
I have input file like Input.dat with below content
RRD 0Z91YUn000000Lk 9000100001 103020151117 STMT151117155527001 0000 2 000000 000004
RRD 0Z91YUn00000ysj 9000100001 103020151117 STMT151117155527001 0000 3 000000 000003
RRD 0Z91YUn00001vGh 9000100002... (12 Replies)
Discussion started by: PRAMOD 96
12 Replies
10. UNIX for Beginners Questions & Answers
Hi all
I have two files I need to match record from first file and second file on column 1,8 and and output only match records on file1
File1:
020059801803180116130926800002090000800231000245204003160000000002000461OUNCE000000350000100152500BM01007W0000 ... (5 Replies)
Discussion started by: arunkumar_mca
5 Replies
LEARN ABOUT DEBIAN
cdbmake
cdbmake(1) General Commands Manual cdbmake(1)
NAME
cdbmake - create a constant database
SYNOPSIS
cdbmake cdb cdb.tmp
DESCRIPTION
cdbmake reads a series of encoded records from its standard input and writes a constant database to cdb.
cdbmake ensures that cdb is updated atomically, so programs reading cdb never have to wait for cdbmake to finish. It does this by first
writing the database to cdb.tmp and then moving cdb.tmp on top of cdb. If cdb.tmp already exists, it is destroyed. The directories con-
taining cdb.tmp and cdb must be writable to cdbmake; they must also be on the same filesystem.
cdbmake always makes sure that cdb.tmp is safely written to disk before it replaces cdb. If the input is in a bad format or if cdbmake has
any trouble writing cdb.tmp to disk, cdbmake complains and leaves cdb alone.
RECORD FORMAT
Records are indexed by keys. A key is a string. cdb is structured so that another program, starting from a key, can quickly find the rel-
evant record. cdbmake allows several records with the same key, although most readers take only the first record, and cdbmake slows down
somewhat if there are many records with the same key.
cdbmake and cdbdump(1) preserve the order of records.
A record is encoded for cdbmake as +klen,dlen:key->data followed by a newline. Here klen is the number of bytes in key and dlen is the
number of bytes in data. The end of data is indicated by an extra newline. For example:
+3,5:one->Hello
+3,7:two->Goodbye
key and data may contain any characters, including colons, dashes, newlines, and nulls.
Keys and data do not have to fit into memory. A database cannot exceed 4 gigabytes.
cdb is portable across machines.
SEE ALSO
cdbdump(1), cdbget(1), cdbstats(1)
cdbmake(1)