06-25-2010
Sorry, others may disagree but I don't think that shell script is a non-starter for this project because of the need to cross-check with serial reads through a large flat file in high volume. The number of serial reads will be the multiple of the number of records in each file - say one million times one million!. The database "seek" approach avoids a serial pass through a large flat file for every record providing that the database engine is reasonably efficient and that a "seek" does not retrieve the record - all we want to know is whether record exists.
If the order or the output is unimportant then sorting the files prior to processing in unix script is a simple option.
Overall this is an application design problem.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
hey guys, I have two files both with two columns, I have already created an
awk code to ignore certain lines (e.g lines that start with 963) as they wou
ld begin with a certain string, however, the rest I have added together and
calculated the average.
At the moment the code also displays... (3 Replies)
Discussion started by: chlfc
3 Replies
2. Shell Programming and Scripting
Hi guys,
I need some help to come out with a solution . I have seven such files but I am showing only three for convenience.
filea
a5 20
a8 16
fileb
a3 42
a7 14
filec
a5 23
a3 07
The output file shoud contain the data in table form showing first field of... (7 Replies)
Discussion started by: smriti_shridhar
7 Replies
3. Shell Programming and Scripting
You have two files to compare by searching keyword from one file into another file
File A
23 >pp_ANSWER
24 >aa hello
25 >jau head wear
66 >jss oops
872 >aqq olps ploww oww sss
722 >GG_KILLER
..... large files
File B
Beta done
KILLER
John Mayor
calix meyers
... (5 Replies)
Discussion started by: cdfd123
5 Replies
4. UNIX for Advanced & Expert Users
All,
PLease can you help me with a shell script which can compare two xml files and print the difference to a output file.
I have attached one such file for you reference.
<Group>
<Member ID=":Year_Quad:41501" childCount="4" fullPath="PEPSICO Year-Quad-Wk : FOLDER.52 Weeks Ending Dec... (2 Replies)
Discussion started by: kanthrajgowda
2 Replies
5. Shell Programming and Scripting
Hello. I have two files. FILE1 was extracted from FILE2 and modified thanks to help from this post. Now I need to replace the extracted, modified lines into the original file (FILE2) to produce the FILE3.
FILE1
1466 55.27433 14.72050 -2.52E+03 3.00E-01 1.05E+04 2.57E+04
1467 55.27433... (1 Reply)
Discussion started by: jm4smtddd
1 Replies
6. Shell Programming and Scripting
I have 2 files:
email_1.out
1 abc@yahoo.com
2 abc_1@yahoo.com
3 abc_2@yahoo.com
data_1.out
<tr> 1 MAIL # 1 TO src_1 </tr>
<tr><td class="hcol">col_id</td> <td class="hcol">test_dt</td> <td class="hcol">user_type</td> <td class="hcol">ct</td></tr>
<tr><td... (1 Reply)
Discussion started by: sol_nov
1 Replies
7. Shell Programming and Scripting
compare to flat files using awk .but in 4th field contains non ordered substring. how to do that.
file1.txt
john|0.0|4|**:25;JP:50;UY:25
file2.txt
andy|0.0|4|JP:50;**:25;UY:25 (4 Replies)
Discussion started by: veeruasu
4 Replies
8. Shell Programming and Scripting
Hi,
I have multiple files that each contain one column of strings:
File1:
123abc
456def
789ghi
File2:
123abc
456def
891jkl
File3:
234mno
123abc
456def
In total I have 25 of these type of file. (5 Replies)
Discussion started by: owwow14
5 Replies
9. Shell Programming and Scripting
Hi,
I want to compare two columns from file1 with another two column of file2 and print matched and unmatched column like this
File1
1 rs1 abc
3 rs4 xyz
1 rs3 stu
File2
1 kkk rs1 AA 10
1 aaa rs2 DD 20
1 ccc ... (2 Replies)
Discussion started by: justinjj
2 Replies
10. UNIX for Beginners Questions & Answers
Hi All,
i am trying to compare two files in Centos 6.
F1: /tmp/d21
NAME="xvda" TYPE="disk" SIZE="40G" OWNER="root" GROUP="disk" MODE="brw-rw----" MOUNTPOINT=""
NAME="xvda1" TYPE="part" SIZE="500M" OWNER="root" GROUP="disk" MODE="brw-rw----" MOUNTPOINT="/boot"
NAME="xvda2" TYPE="part"... (2 Replies)
Discussion started by: balu1234
2 Replies
LEARN ABOUT DEBIAN
ldns-read-zone
ldns-read-zone(1) General Commands Manual ldns-read-zone(1)
NAME
ldns-read-zone - read a zonefile and print it
SYNOPSIS
ldns-read-zone ZONEFILE
DESCRIPTION
ldns-read-zone reads a DNS zone file and prints it. The output has 1 resource record per line, and no pretty-printing makeup.
OPTIONS
-c Canonicalize all resource records in the zone before printing
-d Only print DNSSEC data from the zone. This option skips every record that is not of type NSEC, NSEC3, RRSIG or DNSKEY. DS records
are not printed.
-h Show usage and exit
-n Do not print the SOA record
-s Strip DNSSEC data from the zone. This option skips every record that is of type NSEC, NSEC3, RRSIG or DNSKEY. DS records are still
printed.
-S [[+|0]number | YYYYMMDDxx | unixtime ]
Set serial number to the given number, or when preceded by a sign, offset the exisiting number with it. When giving the literal
strings YYYYMMDDxx or unixtime, the serial number is tried to be reset in datecounter or in unixtime format respectively. Though is
the updated serial number is smaller than the original one, the original one is simply increased by one.
When updating a serial number, records of type NSEC, NSEC3, RRSIG and DNSKEY will be skipped when printing the zone.
-v Show the version and exit
-z Sort the zone before printing (this implies -c)
AUTHOR
Written by the ldns team as an example for ldns usage.
REPORTING BUGS
Report bugs to <ldns-team@nlnetlabs.nl>.
COPYRIGHT
Copyright (C) 2005 NLnet Labs. This is free software. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PUR-
POSE.
30 May 2005 ldns-read-zone(1)