Login or Register to Ask a Question and Join Our Community


Web Development

sitemap.xml from crawler or CMS?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Web Development sitemap.xml from crawler or CMS?
# 1  
Old 01-20-2009
sitemap.xml from crawler or CMS?
Hello,
I have question. Is it better to use site Crawler or scripts from CMS/blog to do that?
# 2  
Old 01-29-2009
I would say the scripts. With the crawler you might get stuff you don't actually want.
# 3  
Old 01-29-2009
Quote:
Originally Posted by gstoychev
Hello,
I have question. Is it better to use site Crawler or scripts from CMS/blog to do that?
Sitemaps should be build from the CMS scripts (not from crawling). You can assign priority to the links as well as expiration / update frequency.
Login or Register to Ask a Question

Previous Thread | Next Thread

2 More Discussions You Might Find Interesting

1. Homework & Coursework Questions

crawler in bash

hello i'm doing a assignment in bash building a crawler.. i downloaded a url into a file and i need to check if the url is an html format and also to save in a different file all the html's links.. please help me thx (1 Reply)
Discussion started by: yanivlug
1 Replies

2. Shell Programming and Scripting

Website crawler

Hi, I want to build a crawler that seeks for a keyword on certain websites. This is what the website looks like: website.com/xxxxAA11xxxx I want that the crawler automatically changes the letters alphanumerically and if a certain keyword is found, the website got to be logged. But... (12 Replies)
Discussion started by: yaylol
12 Replies
Login or Register to Ask a Question