Sponsored Content
Top Forums UNIX for Advanced & Expert Users Solution for the Massive Comparison Operation Post 302429219 by raghav288 on Sunday 13th of June 2010 01:58:08 PM
Old 06-13-2010
Another Idea for the same solution..

Hi

Thanks for the solution.. We had come up with a solution for comparing the huge data..

Since we are comparing huge data of flat file records, the follwing can be done

A hash function may be used like you mentioned below for each rows on the flat files, making the comparison easier.

But Is there a utility hash function in unix same as that of orahash in oracle that wud encrypt each new row uniquely within a few set of characters or numbers.

Then we cud use only those hashed codes to compare with the old hash codes of the prev day file and which wud make processing faster too...
 

5 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Looking for AWK Solution for column comparison in a single file

- I am looking for different kind of awk solution which I don't think is mentioned before in these forums. Number of rows in the file are fixed Their are two columns in file1.txt 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10 I am looking for 3... (1 Reply)
Discussion started by: softwarekids23
1 Replies

2. Shell Programming and Scripting

Column operation : cosne and sine operation

I have a txt file with several columns and i want to peform an operation on two columns and output it to a new txt file . file.txt 900.00000 1 1 1 500.00000 500.00000 100000.000 4 4 1.45257346E-07 899.10834 ... (4 Replies)
Discussion started by: shashi792
4 Replies

3. Homework & Coursework Questions

having massive trouble with 5 questions about egrep!

Hi all! I need help to do a few things with a .txt file using egrep. 1. I need to list all sequences where the vowel letters 'a, e, i, o, u' occur in that order, possibly separated by characters other than a, e, i, o, u; consisting of one or more complete words, possibly including punctuation. ... (1 Reply)
Discussion started by: dindiqotu
1 Replies

4. Shell Programming and Scripting

Massive Copy With Base Directory

I have a script that I am using to copy around 40-70k files to a NFS NAS. I have posted my code below in hopes that someone can help me figure out a faster way of achieving this. At the end of the script i need to have all the files in the list, copied over to the nas with source directory... (8 Replies)
Discussion started by: nitrobass24
8 Replies

5. Shell Programming and Scripting

Massive ftp

friends good morning FTP works perfect but I have a doubt if I want to transport 10 files, I imagine that I should not open 10 connections as I can transfer more than 1 file? ftp -n <<!EOF open caburga user ephfact ephfact cd /users/efactura/docONE/entrada bin mput EPH`date... (16 Replies)
Discussion started by: tricampeon81
16 Replies
hash(1) 						      General Commands Manual							   hash(1)

NAME
hash - Remembers or reports utility locations SYNOPSIS
hash [utility] hash -r STANDARDS
Interfaces documented on this reference page conform to industry standards as follows: hash: XCU5.0 Refer to the standards(5) reference page for more information about industry standards and associated tags. OPTIONS
Forgets all previously remembered utility locations. OPERANDS
The name of a utility to be searched for and added to the list of remembered locations. If utility contains one or more slashes, the results are unspecified. DESCRIPTION
The hash utility affects the way the current shell environment remembers the locations of utilities found. Depending on the arguments spec- ified, it adds utility locations to its list of remembered locations or it purges the contents of the list. When no arguments are specified, hash reports on the contents of the list. This list consists of those utilities named in previous hash invocations that have been invoked, and those invoked and found through the normal command search process. This list includes the path name of each utility in the list of remembered locations for the current shell environment. NOTES
The use of hash with utility names is unnecessary for most applications, but may provide a performance improvement. The effects of hash -r can also be achieved by resetting the value of PATH. RESTRICTIONS
If hash is called in a separate utility execution environment, such as one of the following it will not affect the command search process of the caller's environment. nohup hash -r find . -type f | xargs hash Utilities provided as built-ins to the shell are not reported by hash. EXIT STATUS
The following exit values are returned: Successful completion. An error occurred. ENVIRONMENT VARIABLES
The following environment variables affect the execution of hash: Provides a default value for the internationalization variables that are unset or null. If LANG is unset or null, the corresponding value from the default locale is used. If any of the internationalization vari- ables contains an invalid setting, the utility behaves as if none of the variables had been defined. If set to a non-empty string value, overrides the values of all the other internationalization variables. Determines the locale for the interpretation of sequences of bytes of text data as characters (for example, single-byte as opposed to multibyte characters in arguments). Determines the locale that should be used to affect the format and contents of diagnostic messages written to standard error. Determines the location of message catalogues for the processing of LC_MESSAGES. Determines the location of name. SEE ALSO
Commands: command(1), type(1) Standards: standards(5) hash(1)
All times are GMT -4. The time now is 10:43 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy