04-25-2019
Bumping up posts or double posting is not permitted in these forums.
Please read the
rules, which you agreed to when you registered, if you have not already done so.
You may receive an infraction for this. If so, don't worry, just try to follow the rules more carefully. The infraction will expire in the near future
Thank You.
The UNIX and Linux Forums.
This User Gave Thanks to Corona688 For This Post:
9 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
This is what i have in a file sortdata
IGOT C9.1.20.2DBSU10 - -
CVGRETesting C9.1.20.2DBSU11 - -
BalInqMail C9.1.20.2DBSU11 - -
SecScore C9.1.20.2DBSU8 - -
IGOT C9.1.20.2DBSU8 - -
IGOT C9.1.20.2DBSU9 - -
SecScore C9.1.20.2DBSU9 - -
When i Sort i am getting
BalInqMail... (4 Replies)
Discussion started by: pulluru
4 Replies
2. UNIX for Advanced & Expert Users
Hi,
I am trying to attach tape drive to sun V890 running Solaris 9 on it.
I have installed HBA(qlogic) in slot 1 of 0-8 slots and booted the system. I do not see HBAin prtdiag output. The tape drive is not attached to HBA. The tape drive I am going to attach is Sony AIT3.
1.How can I make... (3 Replies)
Discussion started by: sriny
3 Replies
3. Shell Programming and Scripting
I have the following file. Its an output from a du command with certain conditions attached to it. I used du -ah as I need the 1st column to look human readable. sort -nr is not giving me the output I need, nor is sort -dr. Please help out.
cat testout
121K ./OMautomation/pvd
14M ... (6 Replies)
Discussion started by: vivek.bharadwaj
6 Replies
4. Shell Programming and Scripting
Let's say that I have a database that I call part ID. This database has the following grouping:
Dart1=4
Dart2=8
Dart3=12
Fork1=68
Fork2=72
Fork3=64
Bike1=28
Bike2=24
Bike3=20
Car1=44
Car2=40
Car3=36
I want to write a program that would read this database and tell me when the... (19 Replies)
Discussion started by: Ernst
19 Replies
5. UNIX for Advanced & Expert Users
hi everyone, I am kind of new to this forum. I need help in sorting this data out accordingly, I am actually doing a traceroute application and wants my AS path displayed in front of my address like this;
192.168.1.1 AS28513 AS65534 AS5089 AS5089 .... till the last AS number and if possible... (1 Reply)
Discussion started by: sam127
1 Replies
6. UNIX for Dummies Questions & Answers
Hi!
As a part of a larger script I'm using the comand du -k to get the sizes of various directories, the output is printed like this:
1324 ./images/backup/2012
1333 ./images/backup
1400 ./images
22 ./video/backup
44 ./video
1450 .
I'm trying to print it the other way around with... (2 Replies)
Discussion started by: Tralaraloro
2 Replies
7. Shell Programming and Scripting
Hi Friends,
Newbie to shell scripting. Currently i have used the below to sort data based on filenames and datestamp
$ printf '%s\n' *.dat* | sort -t. -k3,4
filename_1.dat.20120430.Z
filename_2.dat.20120430.Z
filename_3.dat.20120430.Z
filename_1.dat.20120501.Z
filename_2.dat.20120501.Z... (1 Reply)
Discussion started by: robertbrown624
1 Replies
8. Shell Programming and Scripting
I am trying to write a shell script which takes an input file as an arguement in the terminal e.g. bash shellscriptname.sh input.txt. I would like for the file to be read line by line each time checking if the .txt file contains certain words or letters(validating the syntax). If the line being... (1 Reply)
Discussion started by: Gurdza32
1 Replies
9. UNIX for Beginners Questions & Answers
Hello Everyone, I need help here . I need to sort a file for one of my requirement , The file has to be sorted using a key with 4 columns. Sorting is working fine on those 4 columns but when the key is matching for many rows the other columns are also getting sorted which is not required .
... (11 Replies)
Discussion started by: richa_240889
11 Replies
LEARN ABOUT REDHAT
www::robotrules
WWW::RobotRules(3) User Contributed Perl Documentation WWW::RobotRules(3)
NAME
WWW::RobotsRules - Parse robots.txt files
SYNOPSIS
require WWW::RobotRules;
my $robotsrules = new WWW::RobotRules 'MOMspider/1.0';
use LWP::Simple qw(get);
$url = "http://some.place/robots.txt";
my $robots_txt = get $url;
$robotsrules->parse($url, $robots_txt);
$url = "http://some.other.place/robots.txt";
my $robots_txt = get $url;
$robotsrules->parse($url, $robots_txt);
# Now we are able to check if a URL is valid for those servers that
# we have obtained and parsed "robots.txt" files for.
if($robotsrules->allowed($url)) {
$c = get $url;
...
}
DESCRIPTION
This module parses a /robots.txt file as specified in "A Standard for Robot Exclusion", described in
<http://info.webcrawler.com/mak/projects/robots/norobots.html> Webmasters can use the /robots.txt file to disallow conforming robots access
to parts of their web site.
The parsed file is kept in the WWW::RobotRules object, and this object provides methods to check if access to a given URL is prohibited.
The same WWW::RobotRules object can parse multiple /robots.txt files.
The following methods are provided:
$rules = WWW::RobotRules->new($robot_name)
This is the constructor for WWW::RobotRules objects. The first argument given to new() is the name of the robot.
$rules->parse($robot_txt_url, $content, $fresh_until)
The parse() method takes as arguments the URL that was used to retrieve the /robots.txt file, and the contents of the file.
$rules->allowed($uri)
Returns TRUE if this robot is allowed to retrieve this URL.
$rules->agent([$name])
Get/set the agent name. NOTE: Changing the agent name will clear the robots.txt rules and expire times out of the cache.
ROBOTS.TXT
The format and semantics of the "/robots.txt" file are as follows (this is an edited abstract of
<http://info.webcrawler.com/mak/projects/robots/norobots.html>):
The file consists of one or more records separated by one or more blank lines. Each record contains lines of the form
<field-name>: <value>
The field name is case insensitive. Text after the '#' character on a line is ignored during parsing. This is used for comments. The
following <field-names> can be used:
User-Agent
The value of this field is the name of the robot the record is describing access policy for. If more than one User-Agent field is
present the record describes an identical access policy for more than one robot. At least one field needs to be present per record. If
the value is '*', the record describes the default access policy for any robot that has not not matched any of the other records.
Disallow
The value of this field specifies a partial URL that is not to be visited. This can be a full path, or a partial path; any URL that
starts with this value will not be retrieved
ROBOTS.TXT EXAMPLES
The following example "/robots.txt" file specifies that no robots should visit any URL starting with "/cyberworld/map/" or "/tmp/":
User-agent: *
Disallow: /cyberworld/map/ # This is an infinite virtual URL space
Disallow: /tmp/ # these will soon disappear
This example "/robots.txt" file specifies that no robots should visit any URL starting with "/cyberworld/map/", except the robot called
"cybermapper":
User-agent: *
Disallow: /cyberworld/map/ # This is an infinite virtual URL space
# Cybermapper knows where to go.
User-agent: cybermapper
Disallow:
This example indicates that no robots should visit this site further:
# go away
User-agent: *
Disallow: /
SEE ALSO
LWP::RobotUA, WWW::RobotRules::AnyDBM_File
libwww-perl-5.65 2001-04-20 WWW::RobotRules(3)