Sponsored Content
Top Forums Shell Programming and Scripting Awk - find string, search lines below string for other strings Post 302509044 by Mbohmer on Tuesday 29th of March 2011 05:00:25 PM
Old 03-29-2011
Awk - find string, search lines below string for other strings

What's the easiest way to search a file for a specific string and then look for other instances after that? I want to search for all Virtual Hosts and print out the Server Name and Document Root (if it has that info), while discarding the rest of the info.

Basically my file looks like this:
Code:
...
<VirtualHost *:>
        ServerName server.name1.com
        ServerAlias svrname
        Redirect Permanent / http://www.sitename1.gov
# Disable HTTP TRACE Support (see http://www.kb.cert.org/vuls/id/867593)
        RewriteEngine On
        RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK)
        RewriteRule .* - [F]
</VirtualHost>

<VirtualHost *:>
        ServerName www.servername2
        ServerAlias srvname2
        ServerAlias srvname.2.com
        DocumentRoot /location/of/document/root
        DirectoryIndex index.shtml
        ScriptAlias /cgi-bin/ /tst/cgi-bin/
        ErrorDocument 403 http://www.error.com/errors/error403.html
#       ErrorDocument 404 http://www.error.com/errors/error404.html
# Disable HTTP TRACE Support (see http://www.kb.cert.org/vuls/id/867593)
        RewriteEngine On
        RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK)
        RewriteRule .* - [F]
</VirtualHost>
...

And I want it to look like this:
Code:
...
<VirtualHost *:>
                 ServerName server.name1.com
<VirtualHost *:>
                 ServerName www.servername2
                 DocumentRoot /location/of/document/root
...

There a simple or not so simple awk script for this? Thanks.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Search multiple strings on a file and copy the string next to it

I tried awk for this, but failed <or my code is not correct? I dont know>. Can anyone help me on this? ---------- Post updated at 08:34 PM ---------- Previous update was at 08:29 PM ---------- my working file looks like this: <empty> <empty> <empty> NAME :ABC AGE :15 GENDER... (6 Replies)
Discussion started by: kingpeejay
6 Replies

2. Shell Programming and Scripting

search string in a file and retrieve 10 lines including string line

Hi Guys, I am trying to write a perl script to search a string "Name" in the file "FILE" and also want to create a new file and push the searched string Name line along with 10 lines following the same. can anyone of you please let me know how to go about it ? (8 Replies)
Discussion started by: sukrish
8 Replies

3. Shell Programming and Scripting

Find a string in textfile, erase $num lines after that string

I have a textfile containing text similar to the following pattern: STRING1 UNIQUE_STRING1 STRING2 STRING3 STRING4 STRING5 STRING1 UNIQUE_STRING2 STRING2 STRING3 STRING4 STRING5 STRING1 UNIQUE_STRING3 STRING2 STRING3 (6 Replies)
Discussion started by: ilcsfe
6 Replies

4. Solaris

Search string between the strings

File name : Sample.txt <ownername>Oracle< ownername> I am new to unix world , i would like to search string and return back to another sh script. bascially i want to read file Sample.txt find the string between <ownername> Sample.txt < ownername> . Gerneric way to find the string... (15 Replies)
Discussion started by: balajikalai
15 Replies

5. Shell Programming and Scripting

awk find a string, print the line 2 lines below it

I am parsing a nagios config, searching for a string, and then printing the line 2 lines later (the "members" string). Here's the data: define hostgroup{ hostgroup_name chat-dev alias chat-dev members thisisahostname } define hostgroup{ ... (1 Reply)
Discussion started by: mglenney
1 Replies

6. Shell Programming and Scripting

awk search an output string between two strings

I would like to search for strings stored in searchstringfile.txt in inputfiles. searchstringfile.txt J./F. Gls. Wal F. Towerinput1.txt What is needed is J./F. 12 var Gls. Wal 16 interp. Tower 12 input2.txt Awk shall search for F. 16 pt. J./F. 22 output.txt input1.txt J./F. = 12 var... (3 Replies)
Discussion started by: sdf
3 Replies

7. Shell Programming and Scripting

Search a string and replace the same string above two lines?.

I need to search this "<CardinalMPI>" string and replace the same string above two lines of the string. Nov 16, 2012 12:58:34 PM INFO: Centinel LookUp ResponseXML : <CardinalMPI> <ReasonCode></ReasonCode> <ErrorDesc></ErrorDesc> <MerchantReferenceNumber></MerchantReferenceNumber>... (4 Replies)
Discussion started by: laknar
4 Replies

8. Shell Programming and Scripting

Search several string and convert into a single line for each search string using awk command AIX?.

I need to search the file using strings "Request Type" , " Request Method" , "Response Type" and by using result set find the xml tags and convert into a single line?. below are the scenarios. Cat test Nov 10, 2012 5:17:53 AM INFO: Request Type Line 1.... (5 Replies)
Discussion started by: laknar
5 Replies

9. Shell Programming and Scripting

Search String and extract few lines under the searched string

Need Assistance in shell programming... I have a huge file which has multiple stations and i wanted to search particular station and extract few lines from it and the rest is not needed Bold letters are the stations . The whole file has multiple stations . Below example i wanted to search... (4 Replies)
Discussion started by: ajayram_arya
4 Replies

10. Shell Programming and Scripting

Find a string and print all lines upto another string

Ok I would like to do the following file test contains the following lines. between the lines ABC there may be any amount of lines up to the next ABC entry. I want to grep for the filename.txt entry and print the lines in between (and including that line) up to and including the last line... (3 Replies)
Discussion started by: revaroo
3 Replies
Dancer::Deployment(3pm) 				User Contributed Perl Documentation				   Dancer::Deployment(3pm)

NAME
Dancer::Deployment - common ways to put your Dancer app into use DESCRIPTION
Dancer has been designed to be flexible, and this flexibility extends to your choices when deploying your Dancer app. Running as a cgi-script (or fast-cgi) In providing ultimate flexibility in terms of deployment, your Dancer app can be run as a simple cgi-script out-of-the-box. No additional web-server configuration needed. Your web server should recognize .cgi files and be able to serve Perl scripts. The Perl module Plack::Runner is required. Start by adding the following to your apache configuration (httpd.conf or sites-available/*site*): <VirtualHost *:80> ServerName www.example.com DocumentRoot /srv/www.example.com/public ServerAdmin you@example.com <Directory "/srv/www.example.com/public"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all AddHandler cgi-script .cgi </Directory> RewriteEngine On RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ /dispatch.cgi/$1 [QSA,L] ErrorLog /var/log/apache2/www.example.com-error.log CustomLog /var/log/apache2/www.example.com-access_log common </VirtualHost> Note that when using fast-cgi your rewrite rule should be: RewriteRule ^(.*)$ /dispatch.fcgi$1 [QSA,L] Here, the mod_rewrite magic for Pretty-URLs is directly put in Apache's configuration. But if your web server supports .htaccess files, you can drop those lines in a .htaccess file. To check if your server supports mod_rewrite type "apache2 -l" to list modules. To enable mod_rewrite (Debian), run "a2enmod rewrite". Place following code in a file called .htaccess in your application's root folder: # BEGIN dancer application htaccess RewriteEngine On RewriteCond %{SCRIPT_FILENAME} !-d RewriteCond %{SCRIPT_FILENAME} !-f RewriteRule (.*) /dispatch.cgi$1 [L] # END dancer application htaccess Now you can access your dancer application URLs as if you were using the embedded web server. http://localhost/ This option is a no-brainer, easy to setup, low maintenance but serves requests slower than all other options. You can use the same technique to deploy with FastCGI, by just changing the line: AddHandler cgi-script .cgi By: AddHandler fastcgi-script .fcgi Of course remember to update your rewrite rules, if you have set any: RewriteRule (.*) /dispatch.fcgi$1 [L] Running stand-alone At the simplest, your Dancer app can run standalone, operating as its own webserver using HTTP::Server::Simple::PSGI. Simply fire up your app: $ perl bin/app.pl >> Listening on 0.0.0.0:3000 == Entering the dance floor ... Point your browser at it, and away you go! This option can be useful for small personal web apps or internal apps, but if you want to make your app available to the world, it probably won't suit you. Running on Perl webservers with plackup A number of Perl web servers supporting PSGI are available on cpan: Starman "Starman" is a high performance web server, with support for preforking, signals and more. Twiggy "Twiggy" is an "AnyEvent" web server, it's light and fast. Corona "Corona" is a "Coro" based web server. To start your application, just run plackup (see Plack and specific servers above for all available options): $ plackup bin/app.pl $ plackup -E deployment -s Starman --workers=10 -p 5001 -a bin/app.pl As you can see, the scaffolded Perl script for your app can be used as a PSGI startup file. Enabling content compression Content compression (gzip, deflate) can be easily enabled via a Plack middleware (see Plack#Plack::Middleware): Plack::Middleware::Deflater. It's a middleware to encode the response body in gzip or deflate, based on Accept-Encoding HTTP request header. Enable it as you would enable any Plack middleware. First you need to install Plack::Middleware::Deflater, then in the configuration file (usually environments/development.yml), add these lines: plack_middlewares: - - Plack::Middleware::Deflater - ... These lines tell Dancer to add Plack::Middleware::Deflater to the list of middlewares to pass to Plack::Builder, when wrapping the Dancer app. The syntax is : o as key: the name of the Plack middleware to use o as value: the options to pass it as a list. In our case, there is no option to specify to Plack::Middleware::Deflater, so we use an empty YAML list. To test if content compression works, trace the HTTP request and response before and after enabling this middleware. Among other things, you should notice that the response is gzip or deflate encoded, and contains a header "Content-Encoding" set to "gzip" or "deflate" Running multiple apps with Plack::Builder You can use Plack::Builder to mount multiple Dancer applications on a PSGI webserver like Starman. Start by creating a simple app.psgi file: use Dancer ':syntax'; use Plack::Builder; setting apphandler => 'PSGI'; my $app1 = sub { my $env = shift; local $ENV{DANCER_APPDIR} = '/Users/franck/tmp/app1'; setting appdir => '/Users/franck/tmp/app1'; load_app "app1"; Dancer::App->set_running_app('app1'); Dancer::Config->load; my $request = Dancer::Request->new( env => $env ); Dancer->dance($request); }; my $app2 = sub { my $env = shift; local $ENV{DANCER_APPDIR} = '/Users/franck/tmp/app2'; setting appdir => '/Users/franck/tmp/app2'; load_app "app2"; Dancer::App->set_running_app('app2'); Dancer::Config->load; my $request = Dancer::Request->new( env => $env ); Dancer->dance($request); }; builder { mount "/app1" => builder {$app1}; mount "/app2" => builder {$app2}; }; and now use Starman plackup -a app.psgi -s Starman Hosting on DotCloud The simplest way to achieve this is to push your main application directory to dotcloud with your "bin/app.pl" file copied to (or symlinked from) "app.psgi". Beware that the dotcloud service enforces one environment only, named "deployment". So instead of having "environments/development.yml" or "environments/production.yml" you must have a file named "environments/deployment.yml". Also make sure that your "Makefile.PL" (or other dependency mechanism) includes both Dancer and Plack::Request. The default in-memory session handler won't work, and instead you should switch to something persistent. Edit "config.yml" to change "session: 'Simple'" to (for example) "session: 'YAML'". In case you have issues with Template::Toolkit on Dotcloud If you use the Template::Toolkit and its "INCLUDE" or "PROCESS" directives, you might need to add the search path of your view files to the config. This is probably going to be something like "INCLUDE_PATH: '/home/dotcloud/current/views'" in "config.yml". An alternative implementation is to use a variation of the above Plack::Builder template: use Plack::Builder; use Dancer ':syntax'; use Dancer::Handler; use lib 'lib'; my $app1 = sub { setting appdir => '/home/dotcloud/current'; load_app "My::App"; Dancer::App->set_running_app("My::App"); my $env = shift; Dancer::Handler->init_request_headers($env); my $req = Dancer::Request->new(env => $env); Dancer->dance($req); }; builder { mount "/app1" => $app1; }; This also supports hosting multiple apps, but you probably also need to specify the specific Environment configuration to use in your application. When mounting under a path on dotcloud, as in the above example, always create links using the "uri_for()" method for Dancer routes, and a "uri_base" variable for static content as shown in Dancer::Cookbook. This means whatever base path your app is mounted under, links and form submissions will continue to work. Creating a service You can turn your app into proper service running in background using one of the following examples: Using Ubic Ubic is an extensible perlish service manager. You can use it to start and stop any services, automatically start them on reboots or daemon failures, and implement custom status checks. A basic PSGI service description (usually in /etc/ubic/service/application): use parent qw(Ubic::Service::Plack); # if your application is not installed in @INC path: sub start { my $self = shift; $ENV{PERL5LIB} = '/path/to/your/application/lib'; $self->SUPER::start(@_); } __PACKAGE__->new( server => 'Starman', app => '/path/to/your/application/app.pl', port => 5000, user => 'www-data', ); Run "ubic start application" to start the service. Using daemontools daemontools is a collection of tools for managing UNIX services. You can use it to easily start/restart/stop services. A basic script to start an application: (in /service/application/run) #!/bin/sh # if your application is not installed in @INC path: export PERL5LIB='/path/to/your/application/lib' exec 2>&1 /usr/local/bin/plackup -s Starman -a /path/to/your/application/app.pl -p 5000 Running stand-alone behind a proxy / load balancer Another option would be to run your app stand-alone as described above, but then use a proxy or load balancer to accept incoming requests (on the standard port 80, say) and feed them to your Dancer app. This could be achieved using various software; examples would include: Using Apache's mod_proxy You could set up a VirtualHost for your web app, and proxy all requests through to it: <VirtualHost mywebapp.example.com:80> ProxyPass / http://localhost:3000/ ProxyPassReverse / http://localhost:3000/ </VirtualHost> Or, if you want your webapp to share an existing VirtualHost, you could have it under a specified dir: ProxyPass /mywebapp/ http://localhost:3000/ ProxyPassReverse /mywebapp/ http://localhost:3000/ It is important for you to note that the Apache2 modules mod_proxy and mod_proxy_http must be enabled. a2enmod proxy a2enmod proxy_http It is also important to set permissions for proxying for security purposes, below is an example. <Proxy *> Order allow,deny Allow from all </Proxy> Using perlbal "Perlbal" is a single-threaded event-based server written in Perl supporting HTTP load balancing, web serving, and a mix of the two, available from <http://www.danga.com/perlbal/> It processes hundreds of millions of requests a day just for LiveJournal, Vox and TypePad and dozens of other "Web 2.0" applications. It can also provide a management interface to let you see various information on requests handled etc. It could easily be used to handle requests for your Dancer apps, too. It can be easily installed from CPAN: perl -MCPAN -e 'install Perlbal' Once installed, you'll need to write a configuration file. See the examples provided with perlbal, but you'll probably want something like: CREATE POOL my_dancers POOL my_dancers ADD 10.0.0.10:3030 POOL my_dancers ADD 10.0.0.11:3030 POOL my_dancers ADD 10.0.0.12:3030 POOL my_dancers ADD 10.0.0.13:3030 CREATE SERVICE my_webapp SET listen = 0.0.0.0:80 SET role = reverse_proxy SET pool = my_dancers SET persist_client = on SET persist_backend = on SET verify_backend = on ENABLE balancer Using balance "balance" is a simple load-balancer from Inlab Software, available from <http://www.inlab.de/balance.html>. It could be used simply to hand requests to a standalone Dancer app. You could even run several instances of your Dancer app, on the same machine or on several machines, and use a machine running balance to distribute the requests between them, for some serious heavy traffic handling! To listen on port 80, and send requests to a Dancer app on port 3000: balance http localhost:3000 To listen on a specified IP only on port 80, and distribute requests between multiple Dancer apps on multiple other machines: balance -b 10.0.0.1 80 10.0.0.2:3000 10.0.0.3:3000 10.0.0.4:3000 Using Lighttpd You can use Lighttp's mod_proxy: $HTTP["url"] =~ "/application" { proxy.server = ( "/" => ( "application" => ( "host" => "127.0.0.1", "port" => 3000 ) ) ) } This configuration will proxy all request to the /application path to the path / on localhost:3000. Using Nginx with Nginx: upstream backendurl { server unix:THE_PATH_OF_YOUR_PLACKUP_SOCKET_HERE.sock; } server { listen 80; server_name YOUR_HOST_HERE; access_log /var/log/YOUR_ACCESS_LOG_HERE.log; error_log /var/log/YOUR_ERROR_LOG_HERE.log info; root YOUR_ROOT_PROJECT/public; location / { try_files $uri @proxy; access_log off; expires max; } location @proxy { proxy_set_header Host $http_host; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://backendurl; } } You will need plackup to start a worker listening on a socket : cd YOUR_PROJECT_PATH sudo -u www plackup -E production -s Starman --workers=2 -l THE_PATH_OF_YOUR_PLACKUP_SOCKET_HERE.sock -a bin/app.pl A good way to start this is to use "daemontools" and place this line with all environments variables in the "run" file. Running from Apache You can run your Dancer app from Apache using the following examples: Running from Apache with Plack You can run your app from Apache using PSGI (Plack), with a config like the following: <VirtualHost myapp.example.com> ServerName www.myapp.example.com ServerAlias myapp.example.com DocumentRoot /websites/myapp.example.com <Directory /home/myapp/myapp> AllowOverride None Order allow,deny Allow from all </Directory> <Location /> SetHandler perl-script PerlHandler Plack::Handler::Apache2 PerlSetVar psgi_app /websites/myapp.example.com/app.pl </Location> ErrorLog /websites/myapp.example.com/logs/error_log CustomLog /websites/myapp.example.com/logs/access_log common </VirtualHost> To set the environment you want to use for your application (production or development), you can set it this way: <VirtualHost> ... SetEnv DANCER_ENVIRONMENT "production" ... </VirtualHost> NOTE: Only a single Dancer application can be deployed using the "Plack::Handler::Apache2" method. Multiple Dancer applications will not work properly (The routes will be mixed-up between the applications). It's recommended to start each app with "plackup" using your favorite server (Starman, for example) and then put a web server (Apache, Nginx, Perlbal, etc.) as a frontend server for both apps using reverse proxy (HTTP based, no fastcgi). Running from Apache under appdir If you want to deploy multiple applications under the same VirtualHost, using one application per directory for example, you can do the following. This example uses the FastCGI dispatcher that comes with Dancer, but you should be able to adapt this to use any other way of deployment described in this guide. The only purpose of this example is to show how to deploy multiple applications under the same base directory/virtualhost. <VirtualHost *:80> ServerName localhost DocumentRoot "/path/to/rootdir" RewriteEngine On RewriteCond %{REQUEST_FILENAME} !-f <Directory "/path/to/rootdir"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all AddHandler fastcgi-script .fcgi </Directory> RewriteRule /App1(.*)$ /App1/public/dispatch.fcgi$1 [QSA,L] RewriteRule /App2(.*)$ /App2/public/dispatch.fcgi$1 [QSA,L] ... RewriteRule /AppN(.*)$ /AppN/public/dispatch.fcgi$1 [QSA,L] </VirtualHost> Of course, if your Apache configuration allows that, you can put the RewriteRules in a .htaccess file directly within the application's directory, which lets you add a new application without changing the Apache configuration. Running on lighttpd (CGI) To run as a CGI app on lighttpd, just create a soft link to the dispatch.cgi script (created when you run dancer -a MyApp) inside your system's cgi-bin folder. Make sure mod_cgi is enabled. ln -s /path/to/MyApp/public/dispatch.cgi /usr/lib/cgi-bin/mycoolapp.cgi Running on lighttpd (FastCGI) Make sure mod_fcgi is enabled. You also must have FCGI installed. This example configuration uses TCP/IP: $HTTP["url"] == "^/app" { fastcgi.server += ( "/app" => ( "" => ( "host" => "127.0.0.1", "port" => "5000", "check-local" => "disable", ) ) ) } Launch your application: plackup -s FCGI --port 5000 bin/app.pl This example configuration uses a socket: $HTTP["url"] =~ "^/app" { fastcgi.server += ( "/app" => ( "" => ( "socket" => "/tmp/fcgi.sock", "check-local" => "disable", ) ) ) } Launch your application: plackup -s FCGI --listen /tmp/fcgi.sock bin/app.pl perl v5.14.2 2012-03-31 Dancer::Deployment(3pm)
All times are GMT -4. The time now is 07:38 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy