Sponsored Content
Full Discussion: Search text pattern
Top Forums Shell Programming and Scripting Search text pattern Post 302424557 by learnbash on Tuesday 25th of May 2010 03:48:46 PM
Old 05-25-2010
sorry for bothering again, if i want to add robots.txt with kf4sd.php what changes are require, so it not search robots.txt and kf4sd.php and search other txt and php files.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

search a pattern and if pattern found insert new pattern at the begining

I am trying to do some thing like this .. In a file , if pattern found insert new pattern at the begining of the line containing the pattern. example: in a file I have this. gtrow0unit1/gctunit_crrownorth_stage5_outnet_feedthru_pin if i find feedthru_pin want to insert !! at the... (7 Replies)
Discussion started by: pitagi
7 Replies

2. Shell Programming and Scripting

Print a pattern between the xml tags based on a search pattern

Hi all, I am trying to extract the values ( text between the xml tags) based on the Order Number. here is the sample input <?xml version="1.0" encoding="UTF-8"?> <NJCustomer> <Header> <MessageIdentifier>Y504173382</MessageIdentifier> ... (13 Replies)
Discussion started by: oky
13 Replies

3. Shell Programming and Scripting

Need one liner to search pattern and print everything expect 6 lines from where pattern match made

i need to search for a pattern from a big file and print everything expect the next 6 lines from where the pattern match was made. (8 Replies)
Discussion started by: chidori
8 Replies

4. Shell Programming and Scripting

bash: need to have egrep to return a text string if the search pattern has NOT been found

Hello all, after spending hours of searching the web I decided to create an account here. This is my first post and I hope one of the experts can help. I need to resolve a grep / sed / xargs / awk problem. My input file is just like this: ----------------------------------... (6 Replies)
Discussion started by: bash4ever
6 Replies

5. Shell Programming and Scripting

To Search for a pattern and substring text in a file

I have the following data in a text file. "A",1,"MyTextfile.CSV","200","This is ,line one" "B","EFG",23,"MyTextfile1.csv","5621",562,"This is ,line two" I want to extract the fileNames MyTextfile.CSV and MyTextfile1.csv. The problem is not all the lines are delimited with "," There are... (3 Replies)
Discussion started by: AshTrak
3 Replies

6. Shell Programming and Scripting

Search for a pattern in a String file and count the occurance of each pattern

I am trying to search a file for a patterns ERR- in a file and return a count for each of the error reported Input file is a free flowing file without any format example of output ERR-00001=5 .... ERR-01010=10 ..... ERR-99999=10 (4 Replies)
Discussion started by: swayam123
4 Replies

7. Shell Programming and Scripting

How to use sed to search a particular pattern in a file backward after a pattern is matched.?

Hi, I have two files file1.txt and file2.txt. Please see the attachments. In file2.txt (which actually is a diff output between two versions of file1.txt.), I extract the pattern corresponding to 1172c1172. Now ,In file1.txt I have to search for this pattern 1172c1172 and if found, I have to... (9 Replies)
Discussion started by: saurabh kumar
9 Replies

8. Shell Programming and Scripting

Search and Replace a text if the line contains a pattern

My text file looks like below . . . abcdefghi jklmnop $Bad_ptrq_GTS=rcrd_ip.txt $Bad_abcd_REJ=rcrd_op.txt ghijklm $Bad_abcd_TYHS=rcrd_op.txt abcgd abcdefghi jklmnop $Bad_ptrq_GTS=rcrd_ip.txt (2 Replies)
Discussion started by: machomaddy
2 Replies

9. Shell Programming and Scripting

Read in search strings from text file, search for string in second text file and output to CSV

Hi guys, I have a text file named file1.txt that is formatted like this: 001 , ID , 20000 002 , Name , Brandon 003 , Phone_Number , 616-234-1999 004 , SSNumber , 234-23-234 005 , Model , Toyota 007 , Engine ,V8 008 , GPS , OFF and I have file2.txt formatted like this: ... (2 Replies)
Discussion started by: An0mander
2 Replies

10. UNIX for Beginners Questions & Answers

Grep/awk using a begin search pattern and end search pattern

I have this fileA TEST FILE ABC this file contains ABC; TEST FILE DGHT this file contains DGHT; TEST FILE 123 this file contains ABC, this file contains DEF, this file contains XYZ, this file contains KLM ; I want to have a fileZ that has only (begin search pattern for will be... (2 Replies)
Discussion started by: vbabz
2 Replies
LWP::RobotUA(3) 					User Contributed Perl Documentation					   LWP::RobotUA(3)

NAME
LWP::RobotUA - a class for well-behaved Web robots SYNOPSIS
use LWP::RobotUA; my $ua = LWP::RobotUA->new('my-robot/0.1', 'me@foo.com'); $ua->delay(10); # be very nice -- max one hit every ten minutes! ... # Then just use it just like a normal LWP::UserAgent: my $response = $ua->get('http://whatever.int/...'); ... DESCRIPTION
This class implements a user agent that is suitable for robot applications. Robots should be nice to the servers they visit. They should consult the /robots.txt file to ensure that they are welcomed and they should not make requests too frequently. But before you consider writing a robot, take a look at <URL:http://www.robotstxt.org/>. When you use a LWP::RobotUA object as your user agent, then you do not really have to think about these things yourself; "robots.txt" files are automatically consulted and obeyed, the server isn't queried too rapidly, and so on. Just send requests as you do when you are using a normal LWP::UserAgent object (using "$ua->get(...)", "$ua->head(...)", "$ua->request(...)", etc.), and this special agent will make sure you are nice. METHODS
The LWP::RobotUA is a sub-class of LWP::UserAgent and implements the same methods. In addition the following methods are provided: $ua = LWP::RobotUA->new( %options ) $ua = LWP::RobotUA->new( $agent, $from ) $ua = LWP::RobotUA->new( $agent, $from, $rules ) The LWP::UserAgent options "agent" and "from" are mandatory. The options "delay", "use_sleep" and "rules" initialize attributes private to the RobotUA. If "rules" are not provided, then "WWW::RobotRules" is instantiated providing an internal database of robots.txt. It is also possible to just pass the value of "agent", "from" and optionally "rules" as plain positional arguments. $ua->delay $ua->delay( $minutes ) Get/set the minimum delay between requests to the same server, in minutes. The default is 1 minute. Note that this number doesn't have to be an integer; for example, this sets the delay to 10 seconds: $ua->delay(10/60); $ua->use_sleep $ua->use_sleep( $boolean ) Get/set a value indicating whether the UA should sleep() if requests arrive too fast, defined as $ua->delay minutes not passed since last request to the given server. The default is TRUE. If this value is FALSE then an internal SERVICE_UNAVAILABLE response will be generated. It will have an Retry-After header that indicates when it is OK to send another request to this server. $ua->rules $ua->rules( $rules ) Set/get which WWW::RobotRules object to use. $ua->no_visits( $netloc ) Returns the number of documents fetched from this server host. Yeah I know, this method should probably have been named num_visits() or something like that. :-( $ua->host_wait( $netloc ) Returns the number of seconds (from now) you must wait before you can make a new request to this host. $ua->as_string Returns a string that describes the state of the UA. Mainly useful for debugging. SEE ALSO
LWP::UserAgent, WWW::RobotRules COPYRIGHT
Copyright 1996-2004 Gisle Aas. This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.18.2 2012-02-11 LWP::RobotUA(3)
All times are GMT -4. The time now is 11:47 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy