05-05-2011
4,673,
588
Join Date: Oct 2010
Last Activity: 1 February 2016, 3:35 PM EST
Location: Southern NJ, USA (Nord)
Posts: 4,673
Thanks Given: 8
Thanked 588 Times in 561 Posts
You want a data structure that expands, updates, deletes in a other-user-invisible way, like leaf-to-root modification. Users coming down the old tree are not bothered by new trees you build to replace, or new subtrees.
A lot of code and though goes into dealing with kill-words, words and phrases that happen so often you never want to index them. You can discover them as they hit a threshold, or just trim them as needed for space.
JAVA using persistent objects may work well for this. You might want to make your own persistent objects out of map'd flat files.