07-13-2011
millions of records ?
it is any table data dump from the db ? ( if yes, then you can sort it in the db itself, as you may have some index -which is faster)
sorting the millions of records takes time in unix commands
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I have a large csv file that looks like this:
14 ,M0081,+000000001,200302,+00000100500,
14 ,M0081,+000000004,200301,+00000100500,
14 ,M0081,+000000005,200305,+00000100500,
14 ,M0081,+000000000,200205,+00000100500,
14 ,M0081,+000000008,200204,+00000100500,
... (2 Replies)
Discussion started by: edog
2 Replies
2. Shell Programming and Scripting
HI ALL,
i have a problem when i do a sort sum with many fields.
Is there a limit for fields?
Do you know a solution?
thanks in advance.
the shell is:
# SORT1
SORT1_rcode=777
if ; then
echo "USE $DARSEQ/OTPU.FTPEPREC RECORD F,1000 " > $DARPARSRT/TPEKL508.SORT1_$$.srt
... (6 Replies)
Discussion started by: tullo
6 Replies
3. Shell Programming and Scripting
Hi I have following fixed width file and I have to sort on 2 fields
ABC 111222333002555 77788
ABC 111222333004555 77788
ABC 111222333001555 77788
ABC 111222333003555 77788
ABC 111222333005555 77788
one is from field1 to field 3 "ABC" and another is on 14 to 16 "002" (based on first... (1 Reply)
Discussion started by: mgirinath
1 Replies
4. Shell Programming and Scripting
Hi experts,
I am trying sort command with my data but still not getting the expected results.
For example, I have 5 fields data here
c,18:12:45,c,c,c
d,12:34:34,d,d,d
a,13:50:10,a,a,a
b,13:50:50,b,b,b
a,13:50:50,a,a,a
b,14:10:01,b,b,b
c,10:12:45,c,c,c
I want to get
... (3 Replies)
Discussion started by: lalelle
3 Replies
5. UNIX for Dummies Questions & Answers
Hi I would like to sort a csv file. It has 50 fields and approx 1400000 lines. I want to sort by three columns as follows. Say sort on coulmn 5, if entries are equal sort on column 3 if this is also equal sort on column 6.
It seems that this is possible for 5 then 3 i.e.
sort -t "," -k 5,5 -k... (1 Reply)
Discussion started by: turquoise_man
1 Replies
6. UNIX for Dummies Questions & Answers
Hey,
I have a file i want to sort. It contains these kind of lines:
FirstName LastName buyID buyType
Eg:
John Doe 22 Car
Jane Simpson 4 Headset
John Doe 11 Telephone
Now if I use the sort command on it
cat purchases.txt | sort -k1,1 -k2,2
it would also sort the third and 4th field:... (4 Replies)
Discussion started by: ce3c
4 Replies
7. Shell Programming and Scripting
I have a file with contents below
123,502
123,506
123,702
234,101
235,104
456,104
456,100
i want to sort such that i get a unique value in column A, and for those with multiple value in A, i want the lowest value in B.
output should be
123,502
234,101
235,104
456,100 (3 Replies)
Discussion started by: dealerso
3 Replies
8. Shell Programming and Scripting
I have another file with three columns A,B,C as below
123,1,502
123,2,506
123,3,702
234,4,101
235,5,104
456,6,104
456,7,100
i want to sort such that i get a unique value in column A, and for those with multiple value in A, i want the lowest value in C.
output should be
Code:... (3 Replies)
Discussion started by: dealerso
3 Replies
9. Shell Programming and Scripting
Please advice in this.
Input file
100,vvvt
201,unb
100,sos
301,abc
99,gang
desired output
99,gang
100,vvvt
100,sos
201,unb
301,abc
Means if first fields are same ( here 100) the do not sort. (4 Replies)
Discussion started by: theshashi
4 Replies
10. Shell Programming and Scripting
Hi
I have a file as below
<field1> <field2> <field3> ... <field_num1> <field_num2>
Trying to sort based on difference of <field_num1> and <field_num2> in desceding order and print all fields.
I tried this and it doesn't sort on the difference field .. Appreciate your help.
cat... (9 Replies)
Discussion started by: newstart
9 Replies
LEARN ABOUT REDHAT
cluster
CLUSTER(7) SQL Commands CLUSTER(7)
NAME
CLUSTER - cluster a table according to an index
SYNOPSIS
CLUSTER indexname ON tablename
INPUTS
indexname
The name of an index.
table The name (possibly schema-qualified) of a table.
OUTPUTS
CLUSTER
The clustering was done successfully.
DESCRIPTION
CLUSTER instructs PostgreSQL to cluster the table specified by table based on the index specified by indexname. The index must already have
been defined on tablename.
When a table is clustered, it is physically reordered based on the index information. Clustering is a one-time operation: when the table is
subsequently updated, the changes are not clustered. That is, no attempt is made to store new or updated tuples according to their index
order. If one wishes, one can periodically re-cluster by issuing the command again.
NOTES
In cases where you are accessing single rows randomly within a table, the actual order of the data in the heap table is unimportant. How-
ever, if you tend to access some data more than others, and there is an index that groups them together, you will benefit from using CLUS-
TER.
Another place where CLUSTER is helpful is in cases where you use an index to pull out several rows from a table. If you are requesting a
range of indexed values from a table, or a single indexed value that has multiple rows that match, CLUSTER will help because once the index
identifies the heap page for the first row that matches, all other rows that match are probably already on the same heap page, saving disk
accesses and speeding up the query.
During the cluster operation, a temporary copy of the table is created that contains the table data in the index order. Temporary copies of
each index on the table are created as well. Therefore, you need free space on disk at least equal to the sum of the table size and the
index sizes.
CLUSTER preserves GRANT, inheritance, index, foreign key, and other ancillary information about the table.
Because the optimizer records statistics about the ordering of tables, it is advisable to run ANALYZE on the newly clustered table. Other-
wise, the optimizer may make poor choices of query plans.
There is another way to cluster data. The CLUSTER command reorders the original table using the ordering of the index you specify. This can
be slow on large tables because the rows are fetched from the heap in index order, and if the heap table is unordered, the entries are on
random pages, so there is one disk page retrieved for every row moved. (PostgreSQL has a cache, but the majority of a big table will not
fit in the cache.) The other way to cluster a table is to use
SELECT columnlist INTO TABLE newtable
FROM table ORDER BY columnlist
which uses the PostgreSQL sorting code in the ORDER BY clause to create the desired order; this is usually much faster than an index scan
for unordered data. You then drop the old table, use ALTER TABLE...RENAME to rename newtable to the old name, and recreate the table's
indexes. However, this approach does not preserve OIDs, constraints, foreign key relationships, granted privileges, and other ancillary
properties of the table --- all such items must be manually recreated.
USAGE
Cluster the employees relation on the basis of its ID attribute:
CLUSTER emp_ind ON emp;
COMPATIBILITY
SQL92
There is no CLUSTER statement in SQL92.
SQL - Language Statements 2002-11-22 CLUSTER(7)