01-27-2015
Bumping up posts or double posting is not permitted in these forums.
Please read the
rules, which you agreed to when you registered, if you have not already done so.
You may receive an infraction for this. If so, don't worry, just try to follow the rules more carefully. The infraction will expire in the near future
Thank You.
The UNIX and Linux Forums.
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
Hello,
I've been tasked with migrating users from one linux server to another new linux server. This is how I was thinking of doing it:
1.) Open up an NFS connection between the two servers, cp -Rp /home/ to the new server and then append the old /etc/group & /etc/passwd to the existing... (2 Replies)
Discussion started by: vancouver_joe
2 Replies
2. IP Networking
I have an RS6000 server running AIX and on occasion all users are logged out of the server "connection closed by foreign host" is the error message. Normally a user can press enter and get a Login prompt, but they get the message "connection refused" and then the users can wait a minute or so and... (2 Replies)
Discussion started by: Docboyeee
2 Replies
3. IP Networking
We have two NIC cards in our IBM RS/6000 F50 running AIX 4.3.3
We are trying to make sure we have moved all users to log in through the new NIC.
10.22.x.y (old)
10.22.x.z (new)
How can I tell which users are still using the old address for logging in so I can update their work station to... (5 Replies)
Discussion started by: cburtgo
5 Replies
4. Shell Programming and Scripting
in unix what is the syntax to find out how many users are currently logged in (4 Replies)
Discussion started by: trichyselva
4 Replies
5. Post Here to Contact Site Administrators and Moderators
how to find out users who logged out within 5 minutes (1 Reply)
Discussion started by: roshni
1 Replies
6. Shell Programming and Scripting
I have searched the forums but have not mangaed to quite find what im looking for. I have used to /etc/passwd command to present me a list of all users the who command to present all users currently logged on, but what i want to know is what command can i use to display users that are registered... (12 Replies)
Discussion started by: warlock129
12 Replies
7. UNIX for Dummies Questions & Answers
How do I find this out? I have a feeling its a simple command such as who, but I just don't know what it is. I've had a search on here but either I can't put it into the right search criteria or there isn't a topic on it.
Thanks.
EDIT: Delete this thread, as I posted it I noticed the... (0 Replies)
Discussion started by: chris_rabz
0 Replies
8. Red Hat
I have 2 systems. (1) RHEL5 and (2) winXP pro
from xpPRO putty i ssh into rhel5 : user root
from xpPRO i ftp into rhel5 : user abc123
when i run #uptime it only shows 1 user
when i do #ps -u abc123 : it shows vsftpd deamon PID
is there a command that can be used to show all currently... (4 Replies)
Discussion started by: dplinux
4 Replies
9. Shell Programming and Scripting
how to find out total number of users logged in a server from uptime . i mean to say i need the total output of unix command . who gives the out put at a particular time . I need at all time from which machine who has connected , (3 Replies)
Discussion started by: amiya.te@gmail
3 Replies
10. Linux
Scenario:
Log into a linux server interface as root.
Inititiate an SSH session with the server using Putty and a valid user account (e.g. fakeuser).
Log into TTY2 of the linux server interface using another valid user account (e.g. faketester).
Issue:
With these three login sessions,... (8 Replies)
Discussion started by: walterthered
8 Replies
LEARN ABOUT REDHAT
www::robotrules
WWW::RobotRules(3) User Contributed Perl Documentation WWW::RobotRules(3)
NAME
WWW::RobotsRules - Parse robots.txt files
SYNOPSIS
require WWW::RobotRules;
my $robotsrules = new WWW::RobotRules 'MOMspider/1.0';
use LWP::Simple qw(get);
$url = "http://some.place/robots.txt";
my $robots_txt = get $url;
$robotsrules->parse($url, $robots_txt);
$url = "http://some.other.place/robots.txt";
my $robots_txt = get $url;
$robotsrules->parse($url, $robots_txt);
# Now we are able to check if a URL is valid for those servers that
# we have obtained and parsed "robots.txt" files for.
if($robotsrules->allowed($url)) {
$c = get $url;
...
}
DESCRIPTION
This module parses a /robots.txt file as specified in "A Standard for Robot Exclusion", described in
<http://info.webcrawler.com/mak/projects/robots/norobots.html> Webmasters can use the /robots.txt file to disallow conforming robots access
to parts of their web site.
The parsed file is kept in the WWW::RobotRules object, and this object provides methods to check if access to a given URL is prohibited.
The same WWW::RobotRules object can parse multiple /robots.txt files.
The following methods are provided:
$rules = WWW::RobotRules->new($robot_name)
This is the constructor for WWW::RobotRules objects. The first argument given to new() is the name of the robot.
$rules->parse($robot_txt_url, $content, $fresh_until)
The parse() method takes as arguments the URL that was used to retrieve the /robots.txt file, and the contents of the file.
$rules->allowed($uri)
Returns TRUE if this robot is allowed to retrieve this URL.
$rules->agent([$name])
Get/set the agent name. NOTE: Changing the agent name will clear the robots.txt rules and expire times out of the cache.
ROBOTS.TXT
The format and semantics of the "/robots.txt" file are as follows (this is an edited abstract of
<http://info.webcrawler.com/mak/projects/robots/norobots.html>):
The file consists of one or more records separated by one or more blank lines. Each record contains lines of the form
<field-name>: <value>
The field name is case insensitive. Text after the '#' character on a line is ignored during parsing. This is used for comments. The
following <field-names> can be used:
User-Agent
The value of this field is the name of the robot the record is describing access policy for. If more than one User-Agent field is
present the record describes an identical access policy for more than one robot. At least one field needs to be present per record. If
the value is '*', the record describes the default access policy for any robot that has not not matched any of the other records.
Disallow
The value of this field specifies a partial URL that is not to be visited. This can be a full path, or a partial path; any URL that
starts with this value will not be retrieved
ROBOTS.TXT EXAMPLES
The following example "/robots.txt" file specifies that no robots should visit any URL starting with "/cyberworld/map/" or "/tmp/":
User-agent: *
Disallow: /cyberworld/map/ # This is an infinite virtual URL space
Disallow: /tmp/ # these will soon disappear
This example "/robots.txt" file specifies that no robots should visit any URL starting with "/cyberworld/map/", except the robot called
"cybermapper":
User-agent: *
Disallow: /cyberworld/map/ # This is an infinite virtual URL space
# Cybermapper knows where to go.
User-agent: cybermapper
Disallow:
This example indicates that no robots should visit this site further:
# go away
User-agent: *
Disallow: /
SEE ALSO
LWP::RobotUA, WWW::RobotRules::AnyDBM_File
libwww-perl-5.65 2001-04-20 WWW::RobotRules(3)