The UNIX and Linux Forums  
Hello and Welcome from United States to the UNIX and Linux Forums! Thank You for Visiting and Joining Our Global Community.

Go Back   The UNIX and Linux Forums > Special Forums > High Performance Computing
Huge Files to be Joined on Ux instead of ORACLE Huge Files to be Joined on Ux instead of ORACLE
.
google unix.com

Forums Register Forum RulesLinksAlbums FAQ Members List Calendar Search Today's Posts Mark Forums Read


High Performance Computing Message Passing Interface (MPI) programming and tuning, MPI library installation and management, parallel administration tools, cluster monitoring, cluster optimization, and more HPC topics.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Huge files manipulation Klashxx UNIX for Advanced & Expert Users 8 12-01-2008 06:15 PM
Difference between two huge files pyaranoid UNIX for Dummies Questions & Answers 13 09-16-2008 10:11 AM
Comparing two huge files kmkbuddy_1983 Shell Programming and Scripting 11 09-03-2008 07:44 AM
Compare 2 huge files wrt to a key using awk Ranjani Shell Programming and Scripting 13 07-15-2008 04:58 AM
splitting huge xml into multiple files uttamhoode Shell Programming and Scripting 5 06-17-2008 07:56 AM

Reply
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 06-02-2009
magedfawzy magedfawzy is offline
Registered User
  
 

Join Date: Mar 2008
Posts: 8
Thumbs up Huge Files to be Joined on Ux instead of ORACLE
we have one file (11 Million) line that is being matched with (10 Billion) line.

the proof of concept we are trying , is to join them on Unix :

All files are delimited and they have composite keys..

could unix be faster than Oracle in This regards..

Please advice
  #2 (permalink)  
Old 06-02-2009
figaro figaro is offline
Registered User
  
 

Join Date: Jan 2007
Posts: 267
If performance is your concern (rather than data integrity), then a lot more factors are at play than just the platform you are using. For instance the structure of you data might be a factor. For large datasets, databases are almost always better, but if you have a test machine I am keen to find out which of the two methods you prefer and what your observations are.
Reply

Bookmarks

« MPI, recovering node | Newbie question about using a Cluster: using memory »
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Hybrid Mode Hybrid Mode
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On



All times are GMT -4. The time now is 01:16 AM.

The UNIX and Linux Forums - Learn UNIX and UNIX Commands RSS Feeds - Contact Us - The UNIX and Linux Forums - Learn UNIX and UNIX Commands - Archive - Top

Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0