Sponsored Content
Full Discussion: Search Engine
Top Forums Web Development Search Engine Post 302486102 by pludi on Friday 7th of January 2011 03:22:33 AM
Old 01-07-2011
I'd say you need (at least) 3 components:
  1. A crawler that downloads pages, and follows links on those pages.
  2. An indexer that builds a list of words used on each page (maybe in relation to other words nearby), and saves that to a database.
  3. A front-end to query the database.
For the crawler you can use just about any language since the main limitation is the network speed. For the indexer I'd recommend either C/C++ (for speed) or a language geared towards natural language processing (like Perl). For the front-end you can again choose whatever language you're comfortable with.
 

5 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Search Engine

How do you write a search engline to search offline scripts? (3 Replies)
Discussion started by: hawaiifiver
3 Replies

2. Programming

Search Engine in C

Hello everybody, I need help with this, I need to design a CGI search engine in C but i have no idea on what or how to do it. Do i have to open all the html files one by one and search for the given strings? i think this process will be slow, and will take too much of the server processing... (11 Replies)
Discussion started by: semash!
11 Replies

3. Shell Programming and Scripting

About search engine in unix

Hello, How to create a search engine in unix using commands?...Atleast guidelines to craete this search engine...Thank you in advance. (10 Replies)
Discussion started by: Sindhu R
10 Replies

4. Homework & Coursework Questions

About search engine in unix

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: How to create a search engine in unix using commands?...Atleast guidelines to craete this search engine...Thank... (1 Reply)
Discussion started by: Sindhu R
1 Replies

5. What is on Your Mind?

YouTube: Search Engine Optimization | How To Fix Soft 404 Errors and A.I. Tales from Google Search

Getting a bit more comfortable making quick YT videos in 4K, here is: Search Engine Optimization | How To Fix Soft 404 Errors and A.I. Tales from Google Search Console https://youtu.be/I6b9T2qcqFo (0 Replies)
Discussion started by: Neo
0 Replies
URI::URL(3pm)						User Contributed Perl Documentation					     URI::URL(3pm)

NAME
URI::URL - Uniform Resource Locators SYNOPSIS
$u1 = URI::URL->new($str, $base); $u2 = $u1->abs; DESCRIPTION
This module is provided for backwards compatibility with modules that depend on the interface provided by the "URI::URL" class that used to be distributed with the libwww-perl library. The following differences exist compared to the "URI" class interface: o The URI::URL module exports the url() function as an alternate constructor interface. o The constructor takes an optional $base argument. The "URI::URL" class is a subclass of "URI::WithBase". o The URI::URL->newlocal class method is the same as URI::file->new_abs. o URI::URL::strict(1) o $url->print_on method o $url->crack method o $url->full_path: same as ($uri->abs_path || "/") o $url->netloc: same as $uri->authority o $url->epath, $url->equery: same as $uri->path, $uri->query o $url->path and $url->query pass unescaped strings. o $url->path_components: same as $uri->path_segments (if you don't consider path segment parameters) o $url->params and $url->eparams methods o $url->base method. See URI::WithBase. o $url->abs and $url->rel have an optional $base argument. See URI::WithBase. o $url->frag: same as $uri->fragment o $url->keywords: same as $uri->query_keywords o $url->localpath and friends map to $uri->file. o $url->address and $url->encoded822addr: same as $uri->to for mailto URI o $url->groupart method for news URI o $url->article: same as $uri->message SEE ALSO
URI, URI::WithBase COPYRIGHT
Copyright 1998-2000 Gisle Aas. perl v5.14.2 2012-02-11 URI::URL(3pm)
All times are GMT -4. The time now is 10:41 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy