Sponsored Content
Top Forums Shell Programming and Scripting awk script to find time difference between HTTP PUT and HTTP DELETE requests in access.log Post 302976768 by Juha on Wednesday 6th of July 2016 02:51:16 AM
Old 07-06-2016
awk script to find time difference between HTTP PUT and HTTP DELETE requests in access.log

Hi,

I'm trying to write a script to determine the time gap between HTTP PUT and HTTP DELETE requests in the HTTP Servers access log.

Normally client will do HTTP PUT to push content e.g. file_1.txt and 21 seconds later it will do HTTP DELETE, but sometimes the time varies causing some issues on the server side.

The format of the log is as below:

Code:
2016-07-06 11:09:04 [127.0.0.2] [PUT http://127.0.0.1:80/abc/bce/cde/file_30.txt HTTP/1.1] [201]
2016-07-06 11:09:04 [127.0.0.2] [DELETE http://127.0.0.1:80/abc/bce/cde/file_9.txt HTTP/1.1] [404]
2016-07-06 11:09:05 [127.0.0.2] [PUT http://127.0.0.1:80/abc/bce/cde/file_31.txt HTTP/1.1] [201]
2016-07-06 11:09:05 [127.0.0.2] [DELETE http://127.0.0.1:80/abc/bce/cde/file_10.txt HTTP/1.1] [404]
...
...
2016-07-06 11:09:25 [127.0.0.2] [PUT http://127.0.0.1:80/abc/bce/cde/file_51.txt HTTP/1.1] [201]
2016-07-06 11:09:25 [127.0.0.2] [DELETE http://127.0.0.1:80/abc/bce/cde/file_30.txt HTTP/1.1] [404]
2016-07-06 11:09:26 [127.0.0.2] [PUT http://127.0.0.1:80/abc/bce/cde/file_52.txt HTTP/1.1] [201]
2016-07-06 11:09:26 [127.0.0.2] [DELETE http://127.0.0.1:80/abc/bce/cde/file_31.txt HTTP/1.1] [404]

So from above I'd need to first find the time stamp e.g. when file_30.txt was pushed in, then find the HTTP DELETE for the file_30.txt and then determine the time difference. (file_30.txt just and example as this would need to be done to every file pushed in)

So it would be the 2 following lines
2016-07-06 11:09:04 [127.0.0.2] [PUT Revive Adserver HTTP/1.1] [201]
2016-07-06 11:09:25 [127.0.0.2] [DELETE Revive Adserver HTTP/1.1] [404]

So time difference would be: 11:09:25 - 11:09:04 = 21seconds

And output would be something like:
File: file_30.txt, difference: 21sec

I'm trying to use awk script to do this but I'm not very familiar with it so I've just started with the following(which might be totally stupid) so if anyone would have ideas on how to go about writing the script to achieve my goal it would be great:

Code:
BEGIN {
    FS=" ";
}

{
    if (/file_/)
    {
      time = $2
      httpmethod = $4
      file = $5
      sub(/.*\//,"",file)
      httpresult = $6
    }
	# Need to find the line with HTTP DELETE with the file from above
	# and then determine the time difference
}

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

turning off certain http requests

On a sparc solaris 8 host running sunone webserver 6 I would like to limit the http requests that can be used when port 80 is accessed. We currently have http/1.0 enabled. For example I would like to remove the http request DELETE. Regards, BLP (1 Reply)
Discussion started by: blp001
1 Replies

2. UNIX for Dummies Questions & Answers

Need to log http requests

Hi folks, I am trying to build this GUI application that will perform some http requests to a specific server. Basically I will use curl to "pretend" that it is a browser a sending an http request from a form. My http knowledge is very low, and the site is tricky, but I think if I could... (3 Replies)
Discussion started by: fundidor
3 Replies

3. Forum Support Area for Unregistered Users & Account Problems

Access to https://www.unix.com/source?

Folks, Hope this is not a newbie question, but it might end up being that. I wanted to retrieve the source code for unix_linux_bench as directed on https://www.unix.com/linux-benchmarks/11175-instructions-linux-benchmarks.html When one clicks on the links specified for download, the browser... (1 Reply)
Discussion started by: wagdalule
1 Replies

4. Solaris

what is the difference between http & Tomcat web server

I do not know the difference between the apache-http and the apche-Tomcat.Is they are differentiated on their version or on their features.:confused: (2 Replies)
Discussion started by: jayaprakash
2 Replies

5. Programming

sending http url through http socket programming..

hi am senthil am developing a software to send and receive SMS using HTTP connection first of all am forming a URL and sending that URL to a remote server using my Client Program i send that url through Socket(using Send() Function) if i send more than one URL one by one using the same... (0 Replies)
Discussion started by: senkerth
0 Replies

6. Shell Programming and Scripting

sending http url through http socket programming..

hi am senthil am developing a software to send and receive SMS using HTTP connection first of all am forming a URL and sending that URL to a remote server using my Client Program i send that url through Socket(using Send() Function) if i send more than one URL one by one using the same... (4 Replies)
Discussion started by: senkerth
4 Replies

7. OS X (Apple)

HbbTV client for OS X or Linux? Access via HTTP possible?

Hi Is there a HbbTV client for OS X or Linux? Access via HTTP possible? https://en.wikipedia.org/wiki/Hybrid_Broadcast_Broadband_TV (0 Replies)
Discussion started by: slashdotweenie
0 Replies

8. Shell Programming and Scripting

A script needs to be created to collect all HTTP GET requests containing a particular string

Hi friends, A script needs to be created to collect all HTTP GET requests containing a particular string say abcd.gif in the url path along with the IP address of the client that issued the request. The source of this data is the web server logs. Also Each script execution should extract... (4 Replies)
Discussion started by: skumar391
4 Replies

9. Web Development

HTTP Headers Reference: HTTP Status-Codes

Hypertext Transfer Protocol -- HTTP/1.1 for Reference - HTTP Headers 10 Status Code Definitions Each Status-Code is described below, including a description of which method(s) it can follow and any metainformation required in the response. (1 Reply)
Discussion started by: Neo
1 Replies

10. Programming

Java HTTP PUT Request/JSON Not Working But Using Python It Does ?

I have some code that I have been playing around with learning Java: import java.io.BufferedReader; import java.io.IOException; import java.io.InputStreamReader; import java.io.OutputStream; import java.net.HttpURLConnection; import java.net.MalformedURLException; import java.net.URL; ... (1 Reply)
Discussion started by: metallica1973
1 Replies
Plack::Test(3pm)					User Contributed Perl Documentation					  Plack::Test(3pm)

NAME
Plack::Test - Test PSGI applications with various backends SYNOPSIS
use Plack::Test; # named params test_psgi app => sub { my $env = shift; return [ 200, [ 'Content-Type' => 'text/plain' ], [ "Hello World" ] ], }, client => sub { my $cb = shift; my $req = HTTP::Request->new(GET => "http://localhost/hello"); my $res = $cb->($req); like $res->content, qr/Hello World/; }; use HTTP::Request::Common; # positional params (app, client) my $app = sub { return [ 200, [], [ "Hello "] ] }; test_psgi $app, sub { my $cb = shift; my $res = $cb->(GET "/"); is $res->content, "Hello"; }; DESCRIPTION
Plack::Test is a unified interface to test PSGI applications using HTTP::Request and HTTP::Response objects. It also allows you to run PSGI applications in various ways. The default backend is "Plack::Test::MockHTTP", but you may also use any Plack::Handler implementation to run live HTTP requests against at web server FUNCTIONS
test_psgi test_psgi $app, $client; test_psgi app => $app, client => $client; Runs the client test code $client against a PSGI application $app. The client callback gets one argument $cb, a callback that accepts an "HTTP::Request" object and returns an "HTTP::Response" object. Use HTTP::Request::Common to import shortcuts for creating requests for "GET", "POST", "DELETE", and "PUT" operations. For your convenience, the "HTTP::Request" given to the callback automatically uses the HTTP protocol and the localhost (127.0.0.1 by default), so the following code just works: use HTTP::Request::Common; test_psgi $app, sub { my $cb = shift; my $res = $cb->(GET "/hello"); }; Note that however, it is not a good idea to pass an arbitrary (i.e. user-input) string to "GET" or even "HTTP::Request->new" by assuming that it always represents a path, because: my $req = GET "//foo/bar"; would represent a request for a URL that has no scheme, has a hostname foo and a path /bar, instead of a path //foo/bar which you might actually want. OPTIONS
Specify the Plack::Test backend using the environment variable "PLACK_TEST_IMPL" or $Plack::Test::Impl package variable. The available values for the backend are: MockHTTP (Default) Creates a PSGI env hash out of HTTP::Request object, runs the PSGI application in-process and returns HTTP::Response. Server Runs one of Plack::Handler backends ("Standalone" by default) and sends live HTTP requests to test. ExternalServer Runs tests against an external server specified in the "PLACK_TEST_EXTERNALSERVER_URI" environment variable instead of spawning the application in a server locally. For instance, test your application with the "HTTP::Server::ServerSimple" server backend with: > env PLACK_TEST_IMPL=Server PLACK_SERVER=HTTP::Server::ServerSimple prove -l t/test.t AUTHOR
Tatsuhiko Miyagawa perl v5.14.2 2011-09-20 Plack::Test(3pm)
All times are GMT -4. The time now is 10:12 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy