Tim Bass
Mon, 27 Aug 2007 16:58:45 +0000
Charles Young kindly calls out my blog post,
Bending Rules for CEP,*and the discussion on rules and Bayesian analytics in his post,
Rules Engines and Bayes’ Theorem.
FWIW, implementing a very simple Bayes network with a rules engine,*as in Charles interesting example,*does little to counter the argument that rules engines are not an efficient implementation for larger Bayes nets and more complex data sets.* It is not convincing to implement somewhat trivial solutions to make a point that does not hold up to the state-of-the-art in Bayesian networks for complex problems.** The complexity and inefficiency (with rule-based systems) comes with larger data sets; a long standing problem with rules.
If rule-based systems were efficient for large data sets, complex spam filters would use rule-based systems, but they don’t.*** Most modern fraud detection algorithms are implemented with Bayesian algorithms (not rules).* Few, if any, credible large companies uses rules for these classes of problems anymore - they all use specially formulated Bayesian algorithms.
In fact, in the late 90s at
Langley Air Force Base*we soon discovered the same problem dealing with massive distributed email bomb attacks on the Internet (
one example reference, also see
Popular Science article,
WAR.COM, by Frank Vizard).* After documenting our rule-based approach, subsequent researchers and implementations all commented that a rules-based approach is
primitive (paraphrased) compared to modern Bayesian techniques.
These comments are not designed, BTW, to disparage rules or* rule-based systems.* Rules are great; but they*are not very efficient in large, complex problems.
I can provide more historical and current references on this topic if anyone is interested.
Source...
