08-10-2010
Quote:
Originally Posted by
kurumi
Ok, so the OP has 2000x 600000 columns. that is over 2GB in file size. do you slurp more than 2GB into memory to do such stuff? What about other processes?
Hi kurumi:
What about other processes? Perhaps there are none besides the basic os services. Perhaps there are thousands. Perhaps there is a dedicated machine for this problem. Perhaps not. Perhaps the hardware available to solve this problem is an old pentium with only a few megabytes of free ram. Perhaps it's a 64-bit monster with many gigabytes of free ram. I do not know. You do not know.
What we do know is that your solution is horribly slow and inefficient even on comparitively small datasets, but it may be the best and only possible approach if there is insufficient ram. My solution is much faster, but horribly ram hungry. This type of trade-off (ram vs i/o-time) is common in algorithmic design. Choice is good and the original poster can choose which best suits his situation, if indeed either of these solutions is suitable.
If I had that 64-bit monster at my disposal, I know which approach I'd choose.
Regards,
Alister
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi I have an input file and I want to transpose it but I need to take care that if any field is missing for a record it should be popoulated with space for that field - using a shell script
INFILE
----------
emp=1
sal=2
loc=abc
emp=2
sal=21
sal=22
loc=xyz
emp=5
loc=abc
OUTFILE... (10 Replies)
Discussion started by: 46019
10 Replies
2. Shell Programming and Scripting
I'm aware there are a lot of resources dedicated to the question of transposing rows and columns, but I'm a total newbie at this and the task appears to be beyond me.
I have 40 text files with content that looks like this:
Dokument 1 von 146
Orange County Register (California)
June 26, 2010... (2 Replies)
Discussion started by: spindoctor
2 Replies
3. Shell Programming and Scripting
I have a data
A 1
B 2
C 3
D 4
E 5
i would like to change the data
A B C D E
1 2 3 4 5
Pls suggest how we can do it in UNIX.
Start using code tags, thanks. Also start reading your PM's you get from Mods as well read the Forum Rules. That might not do any harm. (24 Replies)
Discussion started by: aravindj80
24 Replies
4. Shell Programming and Scripting
Any tips on how I can awk the input data to display the desired output per below? Thanking you in advance.
input test data:
2
2010-02-16 10:00:00
111111111111 bytes
99999999999 bytes
90%
4
2010-02-16 12:00:00
333333333333 bytes
77777777777 bytes
88%
5
2010-02-16 11:00:00... (4 Replies)
Discussion started by: ux4me
4 Replies
5. Shell Programming and Scripting
Hello. very new to shell scripting and would like to know if anyone could help me.
I have data thats being pulled into a txt file and currently have to manually transpose the data which is taking a long time to do.
here is what the data looks like.
Server1 -- Date -- Other -- value... (7 Replies)
Discussion started by: Mikes88
7 Replies
6. Shell Programming and Scripting
Hi Friends,
Hope all is well.
I have an input file like this
a gene1 10
b gene1 2
c gene2 20
c gene3 10
d gene4 5
e gene5 6
Steps to reach output.
1. Print unique values of column1 as column of the matrix, which will be
a
b
c (5 Replies)
Discussion started by: jacobs.smith
5 Replies
7. Shell Programming and Scripting
input:
a1
a2
a3
a4
a5
a6
a7
b1
b2
b3
..
b7
..
z1
..
z7 (12 Replies)
Discussion started by: ux4me
12 Replies
8. Shell Programming and Scripting
Input:
IN,A,1
IN,B,3
IN,B,2
IN,C,7
BR,A,1
BR,A,5
BR,C,9
AR,C,9
Output:
CNTRY,A,B,C
IN,1,5,7
BR,6,0,9
AR,0,0,9 (7 Replies)
Discussion started by: unme
7 Replies
9. Shell Programming and Scripting
Hello,
I have a bilingual database with the following structure
a,b,c=d,e,f
The right half is in a Left to right script and the second is in a Right to left script as the examples below show
What I need is to separate out the database such that the first word on the left hand matches the first... (4 Replies)
Discussion started by: gimley
4 Replies
10. UNIX for Beginners Questions & Answers
Hello,
I have the following data and I want to use awk to transpose each value to a certain column , so in case the value is not available the column should be empty.
Example:
Box Name: BoxA
Weight: 1
Length :2
Depth :3
Color: red
Box Name: BoxB
Weight: 3
Length :4
Color: Yellow... (5 Replies)
Discussion started by: rahman.ahmed
5 Replies
LEARN ABOUT DEBIAN
strsyl
strsyl.f(3) LAPACK strsyl.f(3)
NAME
strsyl.f -
SYNOPSIS
Functions/Subroutines
subroutine strsyl (TRANA, TRANB, ISGN, M, N, A, LDA, B, LDB, C, LDC, SCALE, INFO)
STRSYL
Function/Subroutine Documentation
subroutine strsyl (characterTRANA, characterTRANB, integerISGN, integerM, integerN, real, dimension( lda, * )A, integerLDA, real, dimension(
ldb, * )B, integerLDB, real, dimension( ldc, * )C, integerLDC, realSCALE, integerINFO)
STRSYL
Purpose:
STRSYL solves the real Sylvester matrix equation:
op(A)*X + X*op(B) = scale*C or
op(A)*X - X*op(B) = scale*C,
where op(A) = A or A**T, and A and B are both upper quasi-
triangular. A is M-by-M and B is N-by-N; the right hand side C and
the solution X are M-by-N; and scale is an output scale factor, set
<= 1 to avoid overflow in X.
A and B must be in Schur canonical form (as returned by SHSEQR), that
is, block upper triangular with 1-by-1 and 2-by-2 diagonal blocks;
each 2-by-2 diagonal block has its diagonal elements equal and its
off-diagonal elements of opposite sign.
Parameters:
TRANA
TRANA is CHARACTER*1
Specifies the option op(A):
= 'N': op(A) = A (No transpose)
= 'T': op(A) = A**T (Transpose)
= 'C': op(A) = A**H (Conjugate transpose = Transpose)
TRANB
TRANB is CHARACTER*1
Specifies the option op(B):
= 'N': op(B) = B (No transpose)
= 'T': op(B) = B**T (Transpose)
= 'C': op(B) = B**H (Conjugate transpose = Transpose)
ISGN
ISGN is INTEGER
Specifies the sign in the equation:
= +1: solve op(A)*X + X*op(B) = scale*C
= -1: solve op(A)*X - X*op(B) = scale*C
M
M is INTEGER
The order of the matrix A, and the number of rows in the
matrices X and C. M >= 0.
N
N is INTEGER
The order of the matrix B, and the number of columns in the
matrices X and C. N >= 0.
A
A is REAL array, dimension (LDA,M)
The upper quasi-triangular matrix A, in Schur canonical form.
LDA
LDA is INTEGER
The leading dimension of the array A. LDA >= max(1,M).
B
B is REAL array, dimension (LDB,N)
The upper quasi-triangular matrix B, in Schur canonical form.
LDB
LDB is INTEGER
The leading dimension of the array B. LDB >= max(1,N).
C
C is REAL array, dimension (LDC,N)
On entry, the M-by-N right hand side matrix C.
On exit, C is overwritten by the solution matrix X.
LDC
LDC is INTEGER
The leading dimension of the array C. LDC >= max(1,M)
SCALE
SCALE is REAL
The scale factor, scale, set <= 1 to avoid overflow in X.
INFO
INFO is INTEGER
= 0: successful exit
< 0: if INFO = -i, the i-th argument had an illegal value
= 1: A and B have common or very close eigenvalues; perturbed
values were used to solve the equation (but the matrices
A and B are unchanged).
Author:
Univ. of Tennessee
Univ. of California Berkeley
Univ. of Colorado Denver
NAG Ltd.
Date:
November 2011
Definition at line 164 of file strsyl.f.
Author
Generated automatically by Doxygen for LAPACK from the source code.
Version 3.4.1 Sun May 26 2013 strsyl.f(3)