Sponsored Content
Top Forums Shell Programming and Scripting Perl: Need help comparing huge files Post 302671279 by mrn6430 on Friday 13th of July 2012 10:09:33 AM
Old 07-13-2012
Quote:
Originally Posted by birei
Hi mrn6430,

Value 1233 isn't found in inputfile2, and similar issue for 1244. Did you forget it or did I miss anything?


Yes. I updated my reply to include it. Besides the point, need a way to deal with such huge files. That is the mean issue. Thanks
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

comparing Huge Files - Performance is very bad

Hi All, Can you please help me in resolving the following problem? My requirement is like this: 1) I have two files YESTERDAY_FILE and TODAY_FILE. Each one is having nearly two million data. 2) I need to check each record of TODAY_FILE in YESTERDAY_FILE. If exists we can skip that by... (5 Replies)
Discussion started by: madhukalyan
5 Replies

2. Shell Programming and Scripting

Comparing two huge files

Hi, I have two files file A and File B. File A is a error file and File B is source file. In the error file. First line is the actual error and second line gives the information about the record (client ID) that throws error. I need to compare the first field (which doesnt start with '//') of... (11 Replies)
Discussion started by: kmkbuddy_1983
11 Replies

3. Shell Programming and Scripting

Perl script error to split huge data one by one.

Below is my perl script: #!/usr/bin/perl open(FILE,"$ARGV") or die "$!"; @DATA = <FILE>; close FILE; $join = join("",@DATA); @array = split( ">",$join); for($i=0;$i<=scalar(@array);$i++){ system ("/home/bin/./program_name_count_length MULTI_sequence_DATA_FILE -d... (5 Replies)
Discussion started by: patrick87
5 Replies

4. Shell Programming and Scripting

Compare 2 folders to find several missing files among huge amounts of files.

Hi, all: I've got two folders, say, "folder1" and "folder2". Under each, there are thousands of files. It's quite obvious that there are some files missing in each. I just would like to find them. I believe this can be done by "diff" command. However, if I change the above question a... (1 Reply)
Discussion started by: jiapei100
1 Replies

5. Shell Programming and Scripting

Problem running Perl Script with huge data files

Hello Everyone, I have a perl script that reads two types of data files (txt and XML). These data files are huge and large in number. I am using something like this : foreach my $t (@text) { open TEXT, $t or die "Cannot open $t for reading: $!\n"; while(my $line=<TEXT>){ ... (4 Replies)
Discussion started by: ad23
4 Replies

6. Shell Programming and Scripting

Comparing two huge files on field basis.

Hi all, I have two large files and i want a field by field comparison for each record in it. All fields are tab seperated. file1: Email SELVAKUMAR RAMACHANDRAN Email SHILPA SAHU Web NIYATI SONI Web NIYATI SONI Email VIINII DOSHI Web RAJNISH KUMAR Web ... (4 Replies)
Discussion started by: Suman Singh
4 Replies

7. Shell Programming and Scripting

Comparing 2 huge text files

I have this 2 files: k5login sanwar@systems.nyfix.com jjamnik@systems.nyfix.com nisha@SYSTEMS.NYFIX.COM rdpena@SYSTEMS.NYFIX.COM service/backups-ora@SYSTEMS.NYFIX.COM ivanr@SYSTEMS.NYFIX.COM nasapova@SYSTEMS.NYFIX.COM tpulay@SYSTEMS.NYFIX.COM rsueno@SYSTEMS.NYFIX.COM... (11 Replies)
Discussion started by: linuxgeek
11 Replies

8. Shell Programming and Scripting

Perl: Comparing to two files and displaying the differences

Hi, I'm new to perl and i have to write a perl script that will compare to log/txt files and display the differences. Unfortunately I'm not allowed to use any complied binaries or applications like diff or comm. So far i've across a code like this: use strict; use warnings; my $list1;... (2 Replies)
Discussion started by: dont_be_hasty
2 Replies

9. Shell Programming and Scripting

Removing Dupes from huge file- awk/perl/uniq

Hi, I have the following command in place nawk -F, '!a++' file > file.uniq It has been working perfectly as per requirements, by removing duplicates by taking into consideration only first 3 fields. Recently it has started giving below error: bash-3.2$ nawk -F, '!a++'... (17 Replies)
Discussion started by: makn
17 Replies

10. Shell Programming and Scripting

Need help in comparing two files using shell or Perl

I have these two file that I am trying to compare using shell arrays. I need to find out the changed or the missing enteries from File2. For example. The line "f nsd1" in file2 is different from file1 and the line "g nsd6" is missing from file2. I dont want to use "for loop" because my files... (2 Replies)
Discussion started by: sags007_99
2 Replies
math::fuzzy(n)							 Tcl Math Library						    math::fuzzy(n)

__________________________________________________________________________________________________________________________________________________

NAME
math::fuzzy - Fuzzy comparison of floating-point numbers SYNOPSIS
package require Tcl ?8.3? package require math::fuzzy ?0.2? ::math::fuzzy::teq value1 value2 ::math::fuzzy::tne value1 value2 ::math::fuzzy::tge value1 value2 ::math::fuzzy::tle value1 value2 ::math::fuzzy::tlt value1 value2 ::math::fuzzy::tgt value1 value2 ::math::fuzzy::tfloor value ::math::fuzzy::tceil value ::math::fuzzy::tround value ::math::fuzzy::troundn value ndigits _________________________________________________________________ DESCRIPTION
The package Fuzzy is meant to solve common problems with floating-point numbers in a systematic way: o Comparing two numbers that are "supposed" to be identical, like 1.0 and 2.1/(1.2+0.9) is not guaranteed to give the intuitive result. o Rounding a number that is halfway two integer numbers can cause strange errors, like int(100.0*2.8) != 28 but 27 The Fuzzy package is meant to help sorting out this type of problems by defining "fuzzy" comparison procedures for floating-point numbers. It does so by allowing for a small margin that is determined automatically - the margin is three times the "epsilon" value, that is three times the smallest number eps such that 1.0 and 1.0+$eps canbe distinguished. In Tcl, which uses double precision floating-point numbers, this is typically 1.1e-16. PROCEDURES
Effectively the package provides the following procedures: ::math::fuzzy::teq value1 value2 Compares two floating-point numbers and returns 1 if their values fall within a small range. Otherwise it returns 0. ::math::fuzzy::tne value1 value2 Returns the negation, that is, if the difference is larger than the margin, it returns 1. ::math::fuzzy::tge value1 value2 Compares two floating-point numbers and returns 1 if their values either fall within a small range or if the first number is larger than the second. Otherwise it returns 0. ::math::fuzzy::tle value1 value2 Returns 1 if the two numbers are equal according to [teq] or if the first is smaller than the second. ::math::fuzzy::tlt value1 value2 Returns the opposite of [tge]. ::math::fuzzy::tgt value1 value2 Returns the opposite of [tle]. ::math::fuzzy::tfloor value Returns the integer number that is lower or equal to the given floating-point number, within a well-defined tolerance. ::math::fuzzy::tceil value Returns the integer number that is greater or equal to the given floating-point number, within a well-defined tolerance. ::math::fuzzy::tround value Rounds the floating-point number off. ::math::fuzzy::troundn value ndigits Rounds the floating-point number off to the specified number of decimals (Pro memorie). Usage: if { [teq $x $y] } { puts "x == y" } if { [tne $x $y] } { puts "x != y" } if { [tge $x $y] } { puts "x >= y" } if { [tgt $x $y] } { puts "x > y" } if { [tlt $x $y] } { puts "x < y" } if { [tle $x $y] } { puts "x <= y" } set fx [tfloor $x] set fc [tceil $x] set rounded [tround $x] set roundn [troundn $x $nodigits] TEST CASES
The problems that can occur with floating-point numbers are illustrated by the test cases in the file "fuzzy.test": o Several test case use the ordinary comparisons, and they fail invariably to produce understandable results o One test case uses [expr] without braces ({ and }). It too fails. The conclusion from this is that any expression should be surrounded by braces, because otherwise very awkward things can happen if you need accuracy. Furthermore, accuracy and understandable results are enhanced by using these "tolerant" or fuzzy comparisons. Note that besides the Tcl-only package, there is also a C-based version. REFERENCES
Original implementation in Fortran by dr. H.D. Knoble (Penn State University). P. E. Hagerty, "More on Fuzzy Floor and Ceiling," APL QUOTE QUAD 8(4):20-24, June 1978. Note that TFLOOR=FL5 took five years of refereed evolution (publication). L. M. Breed, "Definitions for Fuzzy Floor and Ceiling", APL QUOTE QUAD 8(3):16-23, March 1978. D. Knuth, Art of Computer Programming, Vol. 1, Problem 1.2.4-5. BUGS, IDEAS, FEEDBACK This document, and the package it describes, will undoubtedly contain bugs and other problems. Please report such in the category math :: fuzzy of the Tcllib SF Trackers [http://sourceforge.net/tracker/?group_id=12883]. Please also report any ideas for enhancements you may have for either package and/or documentation. KEYWORDS
floating-point, math, rounding math 0.2 math::fuzzy(n)
All times are GMT -4. The time now is 06:24 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy