11-03-2010
10m documents = 10,000,000 files?
500 keywords of ~10 characters?
Well, grep -E would be a bit challenged. If you have the resources, you could for every file get every word as a short line: word file# line#, sort them all eliminating duplicates, merge with a sorted keyword list using join, and now you have an index.
The intermediate list is a very big sort, but the join efficiently trims the list. Perhaps it would help to remove obvious nuisance words like: 'a', 'the', 'and'. The join command needs a flat file, as it likes to seek back to where it started, necessary when doing a cartesian product, but not when one list is unique. I have a streaming join m1join.c that can do this merge on a pipe from the sort.
7 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi,
I am wanting to enable logging of all ftp sessions on my Solaris 8 host. I want to at least log all ftp logins and if possible any commands that the user executes.
I have tried various settings in syslog.conf then rereading syslogd but logging still does not happen.
I have... (1 Reply)
Discussion started by: blp001
1 Replies
2. OS X (Apple)
Dear all,
I am an inexperienced man with Macitosh and green in Apple OS X . I had tried very hard to use Unix, in fact the Terminal, with its Email function.
I read some books and came to know that it has Mail, mailx or mail functions that we can use for simple mail. I have try every... (3 Replies)
Discussion started by: Larry LAM
3 Replies
3. Programming
I have searched the internet (including these forums) and perhaps I'm not using the right wording.
What I'm looking for is a function (preferably C) that analyzes the similitude of two numerical or near-numerical values, and returns either a true/false (match/nomatch) or a return code that... (4 Replies)
Discussion started by: marcus121
4 Replies
4. UNIX for Dummies Questions & Answers
Hello
I have a SunBlade 1000 workstation and I cannot login via CDE. All I get is a console login prompt. I then have to login via root and I just get the command line interface.
I have being doing some research on the UNIX forum and the problem may lie with the content in etc/hosts file.... (7 Replies)
Discussion started by: tjwops
7 Replies
5. Hardware
Hi there
Not sure if I'm posting this in the right section...but here goes.
I'm using an HP Compaq nc8430 laptop. Graphics card according to specs is an ATI Mobility Radeon X1600.
It's the first time I installed Linux for use on my personal laptop and I'm having trouble using it with a... (0 Replies)
Discussion started by: notreallyhere
0 Replies
6. Shell Programming and Scripting
Hi All
I have a whole block of pages that have come in from various sources, unfortunately the pages in many instances have blocks of corrupted text. What I'm trying to do is write a sed line that will just delete non alphanumeric characters if they're in a block of say three or four... (5 Replies)
Discussion started by: Bashingaway
5 Replies
7. Shell Programming and Scripting
Hi Forum:
I have struggle with it and decide to use my eye ball to accomplish this.
Basically I am looking for sequence of date inside a file.
If one of the sequence repeat 2-3 time or skip once; it's still consider a match.
input text file:
Sep 6 A
Sep 6 A
Sep 10 A
Sep 7 B
Sep 8... (7 Replies)
Discussion started by: chirish
7 Replies
disco(1) General Commands Manual disco(1)
NAME
disco - Mono's Web Service Discovery Tool
SYNOPSIS
disco [options] url
DESCRIPTION
disco is a tool for discovering web services and for retireving the documents that describe those services.
url is the location of a DISCO document, which includes a list of WSDL documents, XML schemas and references to other DISCO documents.
disco downloads the DISCO document and all referenced documents (unless the -nosave option is specified), and saves them to disk.
OPTIONS
The following options are available:
-nologo
Supress the startup logo.
-nosave
Do not save the discovered documents to disk. The default is to save the documents.
-o:directory , -out:directory
The directory where to save the discovered documents. By default, documents are saved in the current directory.
-u:username , -user:username
The user name to use when connecting to the server.
-p:password , -password:password
The password to use when connecting to the server.
-d:domain , -domain:domain
The domain to use when connecting to the server.
-proxy:url
The url of the proxy server to use for http requests.
-proxyusername:username
The user name to use when connecting to the proxy.
-proxypassword:password
The password to use when connecting to the proxy.
-proxydomain:domain
The domain to use when connecting to the proxy.
AUTHORS
Lluis Sanchez Gual (lluis@ximian.com)
LICENSE
disco is released under the terms of the GNU GPL.
SEE ALSO
wsdl(1), soapsuds(1), mono(1), mcs(1)
disco(1)