dear friends,
I have a large size file containg two fields data
like this
*** ****
122 222
***** *****
***** *****
232 233
i have file like this.
i want to remove blank lines from file .
i think awk is servive this problem
i wrote a awk command but the error is... (3 Replies)
Dear friends,
I am writing shell script in csh .
i want to make arthimatic operation in csh.
i wrote sysntax like this.
set val = 230
set tmp = `0.1 * $val + 300`
echo $tmp
but it is not working .
anyone please give me syntax. (3 Replies)
Hi there
If i run this command on my Linux box directly, i get the desired result
# ipmitool fru | gawk '!NF{f=0}/mb.net0.fru/{f=1}/Product Serial/&&f{print $NF}'
00:AA:4F:A6:A6:C4
however, if i try to run it from a remote server (using SSH) and populating a variable with the result,... (5 Replies)
Hey guys, i am having a problem in my query statement. I am using Mysql in Netbeans and c++.
What i am trying to do is for the user to enter a certain value and then the program will store the value into the database...
string NewMovie ;
Cout <<" Enter your new movie : " << endl ;
... (1 Reply)
Hi Guys,
When below code is executed in script, I get desired output in output file.
awk 'NR >= $start_line && NR <= 3' master_scriptlist.txt > $driver1/scriptlist.txtBut when i replace 3 with a variable end_line=3, I do not get ouput. See code below. Is there any problem with syntax
awk... (6 Replies)
Hi,
Below is the code I am using. I am trying to list only those numbers which has a + symbol in it
cat num | awk -F"+" '{if (/^$/)
{
}
else
{if ( $0 ~ egrep "^+$" )
{
if ( $0 ~ grep "+" )
{print $0}
}
}}'
I am getting the following error:
awk: 0602-521 There is a... (7 Replies)
I'm calling the following if-else from nawk. But I keep getting an error at the "else". I've tried putting more brackets and ; but still I get complaints about the "else".
Any ideas ?
Thanks,
wbrunc
BEGIN { FS = "," ; OFS = "," }
{ if ( $8 ~ /A/ && $9 == B )
$1="4/29/2013" ;
$2="J.Doe"... (2 Replies)
awk -v sw="lemons|dogs" 'NR>100 && NR<200 BEGIN { c=split(sw,a,""); } { for (w in a) { if ($0 ~ a) d]++; } }
END { for (i in a) { o=o (a"="(d]?d]:0)","); }
sub(",*$","",o); print o;
}' /home/jahitt/data.txt
what am i doing wrong with the above code? im pretty sure the issue is in the... (6 Replies)
Discussion started by: SkySmart
6 Replies
LEARN ABOUT REDHAT
www::robotrules
WWW::RobotRules(3) User Contributed Perl Documentation WWW::RobotRules(3)NAME
WWW::RobotsRules - Parse robots.txt files
SYNOPSIS
require WWW::RobotRules;
my $robotsrules = new WWW::RobotRules 'MOMspider/1.0';
use LWP::Simple qw(get);
$url = "http://some.place/robots.txt";
my $robots_txt = get $url;
$robotsrules->parse($url, $robots_txt);
$url = "http://some.other.place/robots.txt";
my $robots_txt = get $url;
$robotsrules->parse($url, $robots_txt);
# Now we are able to check if a URL is valid for those servers that
# we have obtained and parsed "robots.txt" files for.
if($robotsrules->allowed($url)) {
$c = get $url;
...
}
DESCRIPTION
This module parses a /robots.txt file as specified in "A Standard for Robot Exclusion", described in
<http://info.webcrawler.com/mak/projects/robots/norobots.html> Webmasters can use the /robots.txt file to disallow conforming robots access
to parts of their web site.
The parsed file is kept in the WWW::RobotRules object, and this object provides methods to check if access to a given URL is prohibited.
The same WWW::RobotRules object can parse multiple /robots.txt files.
The following methods are provided:
$rules = WWW::RobotRules->new($robot_name)
This is the constructor for WWW::RobotRules objects. The first argument given to new() is the name of the robot.
$rules->parse($robot_txt_url, $content, $fresh_until)
The parse() method takes as arguments the URL that was used to retrieve the /robots.txt file, and the contents of the file.
$rules->allowed($uri)
Returns TRUE if this robot is allowed to retrieve this URL.
$rules->agent([$name])
Get/set the agent name. NOTE: Changing the agent name will clear the robots.txt rules and expire times out of the cache.
ROBOTS.TXT
The format and semantics of the "/robots.txt" file are as follows (this is an edited abstract of
<http://info.webcrawler.com/mak/projects/robots/norobots.html>):
The file consists of one or more records separated by one or more blank lines. Each record contains lines of the form
<field-name>: <value>
The field name is case insensitive. Text after the '#' character on a line is ignored during parsing. This is used for comments. The
following <field-names> can be used:
User-Agent
The value of this field is the name of the robot the record is describing access policy for. If more than one User-Agent field is
present the record describes an identical access policy for more than one robot. At least one field needs to be present per record. If
the value is '*', the record describes the default access policy for any robot that has not not matched any of the other records.
Disallow
The value of this field specifies a partial URL that is not to be visited. This can be a full path, or a partial path; any URL that
starts with this value will not be retrieved
ROBOTS.TXT EXAMPLES
The following example "/robots.txt" file specifies that no robots should visit any URL starting with "/cyberworld/map/" or "/tmp/":
User-agent: *
Disallow: /cyberworld/map/ # This is an infinite virtual URL space
Disallow: /tmp/ # these will soon disappear
This example "/robots.txt" file specifies that no robots should visit any URL starting with "/cyberworld/map/", except the robot called
"cybermapper":
User-agent: *
Disallow: /cyberworld/map/ # This is an infinite virtual URL space
# Cybermapper knows where to go.
User-agent: cybermapper
Disallow:
This example indicates that no robots should visit this site further:
# go away
User-agent: *
Disallow: /
SEE ALSO
LWP::RobotUA, WWW::RobotRules::AnyDBM_File
libwww-perl-5.65 2001-04-20 WWW::RobotRules(3)