07-23-2002
Hmmmmmm....
Maybe you could create a single UberTable with a union of all the fields in the database that are intelligently populated for similiar properties and do away with an XML based data-dictionary (I could see that getting pretty nitty gritty quicky). The downside would be that the table would have a bunch of columns that may not be fully utilized and could be rather large in size but the upside would be that the columns would have actual names without an extra lookup during the query process and be easier for humans to follow.
The real question is how disparate the meta-data for each category really is. It wouldn't surprise me if the categories had a lot of similiar data... "Catalog_Number" is the same as "CatNo" is the same as "Cat_No" (et cetera, et cetera). If you could find the intersection of the common fields and then just append the balance to the table you could reduce total number of columns and be easier to query and index the database.
I am trying to remember the proper mathematical terminology, but my Discrete Math is a tad bit rusty at the moment.
The parser would have to have some logic to split the fields into the proper columns. Instead of one hash, one hash per category/manufacturer/grouping that contains the From/To when populating the UberTable. A lot of work would have to be put into the From/To relationship during conversion but the amount of work saved on the database end my justify the cost on the front side of the conversion.
Hopefully I am not just running on about things that have already been discussed by your own development team.
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
i am a unix-super-beginner (swaddled and weaned on windows) and am trying to import a database from a unix directory into winnt. can someone help me or am i a hopeless case? (4 Replies)
Discussion started by: sadiecutie
4 Replies
2. UNIX for Advanced & Expert Users
what is a flat file in unix?
i have to import a unix flat files from windows based programme.
my question is not to export from unix but only to import from windows only.
how to build that flat files?
how to create export to windows
how to import from windows (3 Replies)
Discussion started by: tunirayavarapu
3 Replies
3. UNIX for Advanced & Expert Users
I want to collect the database tables(Ex: Emp,Sal,Bonus...etc)
in a file & give this file at the command prompt, when prompted for the tables list at command prompt. How can i do that ?
Ex:- Import jason/jason1 tables=emp,sal,bonus log=j1.log
i want to change this into
Ex:- Import... (3 Replies)
Discussion started by: dreams5617
3 Replies
4. Shell Programming and Scripting
Hi,
I am trying to import 22 .dmp files but facing the problem with the last table file it never ends the import command, only the table is created but the rows of the table don't get imported.
This is the problem with only ine table rest 21 tables are being imported properly.
Thanks in... (2 Replies)
Discussion started by: anushilrai
2 Replies
5. Shell Programming and Scripting
can i make a dump to happen from my script in a databse.and download that dump.
I want to schedule a script i cron so that i will make a dump and download that inot my server. (1 Reply)
Discussion started by: arunkumar_mca
1 Replies
6. UNIX for Dummies Questions & Answers
Hi...
I have dump in unix machine...How can I this import dump to Oracle database?
Many thanks in advance. (2 Replies)
Discussion started by: agarwal
2 Replies
7. UNIX for Dummies Questions & Answers
I really need help in this :(
I have a file and would like to calculate the cosine similarity of the values in it...
For now I do use R which has an easy function for doing so
test <- as.matrix(read.csv(file="file.csv", sep=",", header=FALSE))
result<- cosine(t(test))
I am using unix of... (3 Replies)
Discussion started by: A-V
3 Replies
8. Boot Loaders
1)It is a great experience to achieve what I got today. I had downloaded GParted iso file (135 MB size). Now to make a bootable usb of it, I used unetbootin and to surprise I succeded in entering Gparted application and modified my partitions.
Earlier I was in a notion that it is used only for... (0 Replies)
Discussion started by: ravisingh
0 Replies
9. AIX
Hi All,
I am facing one problem related to importing Oracle Dump file.
We have two different version of AIX boxes with oracle (version 10.2.0.4.0) installed.
On one AIX box (version 6.1) we are creating oracle dump file (*.dmp) using oracle exp utility and importing it onto another AIX box... (1 Reply)
Discussion started by: gunjan_thakur
1 Replies
10. UNIX for Advanced & Expert Users
Hi,
I want your help to see if there is a way wherein we can load a set of files which are lying in unix server into my database using sql loader.
Also the person who will be running sql loader does not have access to unix system.
So is there a way i can provide any sql loader script wherein... (1 Reply)
Discussion started by: Vivekit82
1 Replies
LEARN ABOUT LINUX
analyze
ANALYZE(7) SQL Commands ANALYZE(7)
NAME
ANALYZE - collect statistics about a database
SYNOPSIS
ANALYZE [ VERBOSE ] [ table [ ( column [, ...] ) ] ]
DESCRIPTION
ANALYZE collects statistics about the contents of tables in the database, and stores the results in the pg_statistic system catalog. Subse-
quently, the query planner uses these statistics to help determine the most efficient execution plans for queries.
With no parameter, ANALYZE examines every table in the current database. With a parameter, ANALYZE examines only that table. It is further
possible to give a list of column names, in which case only the statistics for those columns are collected.
PARAMETERS
VERBOSE
Enables display of progress messages.
table The name (possibly schema-qualified) of a specific table to analyze. Defaults to all tables in the current database.
column The name of a specific column to analyze. Defaults to all columns.
OUTPUTS
When VERBOSE is specified, ANALYZE emits progress messages to indicate which table is currently being processed. Various statistics about
the tables are printed as well.
NOTES
In the default PostgreSQL configuration, in the documentation takes care of automatic analyzing of tables when they are first loaded with
data, and as they change throughout regular operation. When autovacuum is disabled, it is a good idea to run ANALYZE periodically, or just
after making major changes in the contents of a table. Accurate statistics will help the planner to choose the most appropriate query plan,
and thereby improve the speed of query processing. A common strategy is to run VACUUM [vacuum(7)] and ANALYZE once a day during a low-usage
time of day.
ANALYZE requires only a read lock on the target table, so it can run in parallel with other activity on the table.
The statistics collected by ANALYZE usually include a list of some of the most common values in each column and a histogram showing the
approximate data distribution in each column. One or both of these can be omitted if ANALYZE deems them uninteresting (for example, in a
unique-key column, there are no common values) or if the column data type does not support the appropriate operators. There is more infor-
mation about the statistics in in the documentation.
For large tables, ANALYZE takes a random sample of the table contents, rather than examining every row. This allows even very large tables
to be analyzed in a small amount of time. Note, however, that the statistics are only approximate, and will change slightly each time ANA-
LYZE is run, even if the actual table contents did not change. This might result in small changes in the planner's estimated costs shown by
EXPLAIN [explain(7)]. In rare situations, this non-determinism will cause the planner's choices of query plans to change after ANALYZE is
run. To avoid this, raise the amount of statistics collected by ANALYZE, as described below.
The extent of analysis can be controlled by adjusting the default_statistics_target configuration variable, or on a column-by-column basis
by setting the per-column statistics target with ALTER TABLE ... ALTER COLUMN ... SET STATISTICS (see ALTER TABLE [alter_table(7)]). The
target value sets the maximum number of entries in the most-common-value list and the maximum number of bins in the histogram. The default
target value is 100, but this can be adjusted up or down to trade off accuracy of planner estimates against the time taken for ANALYZE and
the amount of space occupied in pg_statistic. In particular, setting the statistics target to zero disables collection of statistics for
that column. It might be useful to do that for columns that are never used as part of the WHERE, GROUP BY, or ORDER BY clauses of queries,
since the planner will have no use for statistics on such columns.
The largest statistics target among the columns being analyzed determines the number of table rows sampled to prepare the statistics.
Increasing the target causes a proportional increase in the time and space needed to do ANALYZE.
COMPATIBILITY
There is no ANALYZE statement in the SQL standard.
SEE ALSO
VACUUM [vacuum(7)], vacuumdb [vacuumdb(1)], in the documentation, in the documentation
SQL - Language Statements 2010-05-14 ANALYZE(7)