Sponsored Content
Top Forums Shell Programming and Scripting awk to compare flat files and print output to another file Post 302432692 by methyl on Friday 25th of June 2010 09:58:19 PM
Old 06-25-2010
Sorry, others may disagree but I don't think that shell script is a non-starter for this project because of the need to cross-check with serial reads through a large flat file in high volume. The number of serial reads will be the multiple of the number of records in each file - say one million times one million!. The database "seek" approach avoids a serial pass through a large flat file for every record providing that the database engine is reasonably efficient and that a "seek" does not retrieve the record - all we want to know is whether record exists.
If the order or the output is unimportant then sorting the files prior to processing in unix script is a simple option.
Overall this is an application design problem.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to compare lines of two files and print output on screen

hey guys, I have two files both with two columns, I have already created an awk code to ignore certain lines (e.g lines that start with 963) as they wou ld begin with a certain string, however, the rest I have added together and calculated the average. At the moment the code also displays... (3 Replies)
Discussion started by: chlfc
3 Replies

2. Shell Programming and Scripting

compare columns from seven files and print the output

Hi guys, I need some help to come out with a solution . I have seven such files but I am showing only three for convenience. filea a5 20 a8 16 fileb a3 42 a7 14 filec a5 23 a3 07 The output file shoud contain the data in table form showing first field of... (7 Replies)
Discussion started by: smriti_shridhar
7 Replies

3. Shell Programming and Scripting

compare two files and search keyword and print output

You have two files to compare by searching keyword from one file into another file File A 23 >pp_ANSWER 24 >aa hello 25 >jau head wear 66 >jss oops 872 >aqq olps ploww oww sss 722 >GG_KILLER ..... large files File B Beta done KILLER John Mayor calix meyers ... (5 Replies)
Discussion started by: cdfd123
5 Replies

4. UNIX for Advanced & Expert Users

Shell Script to compare xml files and print output to a file

All, PLease can you help me with a shell script which can compare two xml files and print the difference to a output file. I have attached one such file for you reference. <Group> <Member ID=":Year_Quad:41501" childCount="4" fullPath="PEPSICO Year-Quad-Wk : FOLDER.52 Weeks Ending Dec... (2 Replies)
Discussion started by: kanthrajgowda
2 Replies

5. Shell Programming and Scripting

awk compare specific columns from 2 files, print new file

Hello. I have two files. FILE1 was extracted from FILE2 and modified thanks to help from this post. Now I need to replace the extracted, modified lines into the original file (FILE2) to produce the FILE3. FILE1 1466 55.27433 14.72050 -2.52E+03 3.00E-01 1.05E+04 2.57E+04 1467 55.27433... (1 Reply)
Discussion started by: jm4smtddd
1 Replies

6. Shell Programming and Scripting

Compare two files and print using awk

I have 2 files: email_1.out 1 abc@yahoo.com 2 abc_1@yahoo.com 3 abc_2@yahoo.com data_1.out <tr> 1 MAIL # 1 TO src_1 </tr> <tr><td class="hcol">col_id</td> <td class="hcol">test_dt</td> <td class="hcol">user_type</td> <td class="hcol">ct</td></tr> <tr><td... (1 Reply)
Discussion started by: sol_nov
1 Replies

7. Shell Programming and Scripting

Compare to flat files using awk

compare to flat files using awk .but in 4th field contains non ordered substring. how to do that. file1.txt john|0.0|4|**:25;JP:50;UY:25 file2.txt andy|0.0|4|JP:50;**:25;UY:25 (4 Replies)
Discussion started by: veeruasu
4 Replies

8. Shell Programming and Scripting

Compare columns of multiple files and print those unique string from File1 in an output file.

Hi, I have multiple files that each contain one column of strings: File1: 123abc 456def 789ghi File2: 123abc 456def 891jkl File3: 234mno 123abc 456def In total I have 25 of these type of file. (5 Replies)
Discussion started by: owwow14
5 Replies

9. Shell Programming and Scripting

[Solved] awk compare two different columns of two files and print all from both file

Hi, I want to compare two columns from file1 with another two column of file2 and print matched and unmatched column like this File1 1 rs1 abc 3 rs4 xyz 1 rs3 stu File2 1 kkk rs1 AA 10 1 aaa rs2 DD 20 1 ccc ... (2 Replies)
Discussion started by: justinjj
2 Replies

10. UNIX for Beginners Questions & Answers

Compare two files and print output

Hi All, i am trying to compare two files in Centos 6. F1: /tmp/d21 NAME="xvda" TYPE="disk" SIZE="40G" OWNER="root" GROUP="disk" MODE="brw-rw----" MOUNTPOINT="" NAME="xvda1" TYPE="part" SIZE="500M" OWNER="root" GROUP="disk" MODE="brw-rw----" MOUNTPOINT="/boot" NAME="xvda2" TYPE="part"... (2 Replies)
Discussion started by: balu1234
2 Replies
ldns-read-zone(1)					      General Commands Manual						 ldns-read-zone(1)

NAME
ldns-read-zone - read a zonefile and print it SYNOPSIS
ldns-read-zone ZONEFILE DESCRIPTION
ldns-read-zone reads a DNS zone file and prints it. The output has 1 resource record per line, and no pretty-printing makeup. OPTIONS
-c Canonicalize all resource records in the zone before printing -d Only print DNSSEC data from the zone. This option skips every record that is not of type NSEC, NSEC3, RRSIG or DNSKEY. DS records are not printed. -h Show usage and exit -n Do not print the SOA record -s Strip DNSSEC data from the zone. This option skips every record that is of type NSEC, NSEC3, RRSIG or DNSKEY. DS records are still printed. -S [[+|0]number | YYYYMMDDxx | unixtime ] Set serial number to the given number, or when preceded by a sign, offset the exisiting number with it. When giving the literal strings YYYYMMDDxx or unixtime, the serial number is tried to be reset in datecounter or in unixtime format respectively. Though is the updated serial number is smaller than the original one, the original one is simply increased by one. When updating a serial number, records of type NSEC, NSEC3, RRSIG and DNSKEY will be skipped when printing the zone. -v Show the version and exit -z Sort the zone before printing (this implies -c) AUTHOR
Written by the ldns team as an example for ldns usage. REPORTING BUGS
Report bugs to <ldns-team@nlnetlabs.nl>. COPYRIGHT
Copyright (C) 2005 NLnet Labs. This is free software. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PUR- POSE. 30 May 2005 ldns-read-zone(1)
All times are GMT -4. The time now is 05:44 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy