06-02-2009
Huge Files to be Joined on Ux instead of ORACLE
we have one file (11 Million) line that is being matched with (10 Billion) line.
the proof of concept we are trying , is to join them on Unix :
All files are delimited and they have composite keys..
could unix be faster than Oracle in This regards..
Please advice
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi All,
I'm using the following script to produce a result:
#!/bin/sh
awk ' $0 ~ /\+\+\+\+\+\+\+\+\+\+\+\+\+\+\+\+Interface\+\+\+\+\+\+\+\+\+\+\+\+\+\+\+\+\+\+\+\+\+/ {
match_str="YES"; line_cnt=0; next; }
{
if((line_cnt < 5) && ( match_str=="YES"))
{
print $0;
line_cnt += 1;
}
else... (0 Replies)
Discussion started by: ntgobinath
0 Replies
2. Shell Programming and Scripting
Hi,
I have two files file A and File B. File A is a error file and File B is source file. In the error file. First line is the actual error and second line gives the information about the record (client ID) that throws error. I need to compare the first field (which doesnt start with '//') of... (11 Replies)
Discussion started by: kmkbuddy_1983
11 Replies
3. UNIX for Dummies Questions & Answers
Hi,
As per my requirement, I need to take difference between two big files(around 6.5 GB) and get the difference to a output file without any line numbers or '<' or '>' in front of each new line.
As DIFF command wont work for big files, i tried to use BDIFF instead.
I am getting incorrect... (13 Replies)
Discussion started by: pyaranoid
13 Replies
4. UNIX for Advanced & Expert Users
Hi , i need a fast way to delete duplicates entrys from very huge files ( >2 Gbs ) , these files are in plain text.
I tried all the usual methods ( awk / sort /uniq / sed /grep .. ) but it always ended with the same result (memory core dump)
In using HP-UX large servers.
Any advice will... (8 Replies)
Discussion started by: Klashxx
8 Replies
5. Shell Programming and Scripting
I have a file with 20 million records. I need to read each record and process it.
Which will be faster? Perl, Shell or awk?
and what is the best method to read huge files line by line? (3 Replies)
Discussion started by: tene
3 Replies
6. Shell Programming and Scripting
Hi, all:
I've got two folders, say, "folder1" and "folder2".
Under each, there are thousands of files.
It's quite obvious that there are some files missing in each. I just would like to find them. I believe this can be done by "diff" command.
However, if I change the above question a... (1 Reply)
Discussion started by: jiapei100
1 Replies
7. Shell Programming and Scripting
I have this 2 files:
k5login
sanwar@systems.nyfix.com
jjamnik@systems.nyfix.com
nisha@SYSTEMS.NYFIX.COM
rdpena@SYSTEMS.NYFIX.COM
service/backups-ora@SYSTEMS.NYFIX.COM
ivanr@SYSTEMS.NYFIX.COM
nasapova@SYSTEMS.NYFIX.COM
tpulay@SYSTEMS.NYFIX.COM
rsueno@SYSTEMS.NYFIX.COM... (11 Replies)
Discussion started by: linuxgeek
11 Replies
8. UNIX for Dummies Questions & Answers
i have something like this
abc 123 3234 1234 * qqoiki * abc 4533 34 1234 * lloiki *
i want to make it two lines i,e.,abc 123 3234 1234 * qqoiki *
abc 4533 34 1234 * lloiki * how to do that ? (13 Replies)
Discussion started by: anurupa777
13 Replies
9. Shell Programming and Scripting
Hi all,
I need help on getting difference between 2 .csv files.
I have 2 large . csv files which has equal number of columns. I nned to compare them and get output in new file which will have difference olny.
E.g.
File1.csv
Name, Date, age,number
Sakshi, 16-12-2011, 22, 56
Akash,... (10 Replies)
Discussion started by: Dimple
10 Replies
10. Shell Programming and Scripting
Hi Friends !!
I am facing a hash total issue while performing over a set of files of huge volume:
Command used:
tail -n +2 <File_Name> |nawk -F"|" -v '%.2f' qq='"' '{gsub(qq,"");sa+=($156<0)?-$156:$156}END{print sa}' OFMT='%.5f'
Pipe delimited file and 156 column is for hash totalling.... (14 Replies)
Discussion started by: Ravichander
14 Replies
LEARN ABOUT MOJAVE
dbix::class::sqlmaker::oraclejoins
DBIx::Class::SQLMaker::OracleJoins(3) User Contributed Perl Documentation DBIx::Class::SQLMaker::OracleJoins(3)
NAME
DBIx::Class::SQLMaker::OracleJoins - Pre-ANSI Joins-via-Where-Clause Syntax
PURPOSE
This module is used with Oracle < 9.0 due to lack of support for standard ANSI join syntax.
SYNOPSIS
Not intended for use directly; used as the sql_maker_class for schemas and components.
DESCRIPTION
Implements pre-ANSI joins specified in the where clause. Instead of:
SELECT x FROM y JOIN z ON y.id = z.id
It will write:
SELECT x FROM y, z WHERE y.id = z.id
It should properly support left joins, and right joins. Full outer joins are not possible due to the fact that Oracle requires the entire
query be written to union the results of a left and right join, and by the time this module is called to create the where query and table
definition part of the sql query, it's already too late.
METHODS
select
Overrides DBIx::Class::SQLMaker's select() method, which calls _oracle_joins() to modify the column and table list before calling
next::method().
BUGS
Does not support full outer joins (however neither really does DBIC itself)
SEE ALSO
DBIx::Class::Storage::DBI::Oracle::WhereJoins - Storage class using this
DBIx::Class::SQLMaker - Parent module
DBIx::Class - Duh
AUTHOR
Justin Wheeler "<jwheeler@datademons.com>"
CONTRIBUTORS
David Jack Olrik "<djo@cpan.org>"
LICENSE
This module is licensed under the same terms as Perl itself.
perl v5.18.2 2014-01-22 DBIx::Class::SQLMaker::OracleJoins(3)