Sponsored Content
Full Discussion: awk
Top Forums Shell Programming and Scripting awk Post 302980869 by finn on Sunday 4th of September 2016 04:30:29 AM
Old 09-04-2016
awk - read only part of a field

I have an awk command which reads a file and counts the frequency of each "date/time" column and outputs the results to a log

Code:
# if field7 is "input_message" in the input log
# count frequency of the field4 date and time field then print out totals to log 
awk ' $7=="input_message"{A[$4]++}END {for(i in A)print i","A[i]","}' $IN_FILE >> $OUTPUT_LOG ;

The $IN_FILE input file looks like this
Code:
31.44.217.2 - A [02/Sep/2016:09:44:52 +0100] "POST input_message HTTP" 200 -  
31.44.217.2 - A [02/Sep/2016:09:44:52 +0100] "POST input_message HTTP" 200 -  
31.44.217.2 - A [02/Sep/2016:09:44:53 +0100] "POST input_message HTTP" 200 -  
31.44.217.2 - A [02/Sep/2016:09:45:01 +0100] "POST input_message HTTP" 200 -  
31.44.218.2 - A [02/Sep/2016:09:50:52 +0100] "POST input_message HTTP" 200 -

The $OUTPUT_LOG output file looks like this and gives a total count of the frequency of each date/time field
Code:
[02/Sep/2016:09:44:52,2,
[02/Sep/2016:09:44:53,1,
[02/Sep/2016:09:45:01,1,
[02/Sep/2016:09:50:52,1,

I would like to change it so it counts 10 minute time frames e.g instead of reading $4 as [02/Sep/2016:09:44:52 it reads it as [02/Sep/2016:09:4 (ignore 4:52 at the end).

This would then output counts for the 10 minute period as opposed to counts of every second. I'm trying to figure out how to ignore the last 4 characters of $4 to do this.

Any help appreciated.

Last edited by finn; 09-04-2016 at 05:40 AM.. Reason: better title
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Awk problem: How to express the single quote(') by using awk print function

Actually I got a list of file end with *.txt I want to use the same command apply to all the *.txt Thus I try to find out the fastest way to write those same command in a script and then want to let them run automatics. For example: I got the file below: file1.txt file2.txt file3.txt... (4 Replies)
Discussion started by: patrick87
4 Replies

2. Shell Programming and Scripting

scripting/awk help : awk sum output is not comming in regular format. Pls advise.

Hi Experts, I am adding a column of numbers with awk , however not getting correct output: # awk '{sum+=$1} END {print sum}' datafile 2.15291e+06 How can I getthe output like : 2152910 Thank you.. # awk '{sum+=$1} END {print sum}' datafile 2.15079e+06 (3 Replies)
Discussion started by: rveri
3 Replies

3. Shell Programming and Scripting

awk: assign variable with -v didn't work in awk filter

I want to filter 2nd column = 2 using awk $ cat t 1 2 2 4 $ VAR=2 #variable worked in print $ cat t | awk -v ID=$VAR ' { print ID}' 2 2 # but variable didn't work in awk filter $ cat t | awk -v ID=$VAR '$2~/ID/ { print $0}' (2 Replies)
Discussion started by: honglus
2 Replies

4. Shell Programming and Scripting

Problem with awk awk: program limit exceeded: sprintf buffer size=1020

Hi I have many problems with a script. I have a script that formats a text file but always prints the same error when i try to execute it The code is that: { if (NF==17){ print $0 }else{ fields=NF; all=$0; while... (2 Replies)
Discussion started by: fate
2 Replies

5. Shell Programming and Scripting

Comparison and editing of files using awk.(And also a possible bug in awk for loop?)

I have two files which I would like to compare and then manipulate in a way. File1: pictures.txt 1.1 1.3 dance.txt 1.2 1.4 treehouse.txt 1.3 1.5 File2: pictures.txt 1.5 ref2313 1.4 ref2345 1.3 ref5432 1.2 ref4244 dance.txt 1.6 ref2342 1.5 ref2352 1.4 ref0695 1.3 ref5738 1.2... (1 Reply)
Discussion started by: linuxkid
1 Replies

6. Shell Programming and Scripting

awk command to compare a file with set of files in a directory using 'awk'

Hi, I have a situation to compare one file, say file1.txt with a set of files in directory.The directory contains more than 100 files. To be more precise, the requirement is to compare the first field of file1.txt with the first field in all the files in the directory.The files in the... (10 Replies)
Discussion started by: anandek
10 Replies

7. Shell Programming and Scripting

HELP with AWK one-liner. Need to employ an If condition inside AWK to check for array variable ?

Hello experts, I'm stuck with this script for three days now. Here's what i need. I need to split a large delimited (,) file into 2 files based on the value present in the last field. Samp: Something.csv bca,adc,asdf,123,12C bca,adc,asdf,123,13C def,adc,asdf,123,12A I need this split... (6 Replies)
Discussion started by: shell_boy23
6 Replies

8. Shell Programming and Scripting

Passing awk variable argument to a script which is being called inside awk

consider the script below sh /opt/hqe/hqapi1-client-5.0.0/bin/hqapi.sh alert list --host=localhost --port=7443 --user=hqadmin --password=hqadmin --secure=true >/tmp/alerts.xml awk -F'' '{for(i=1;i<=NF;i++){ if($i=="Alert id") { if(id!="") if(dt!=""){ cmd="sh someScript.sh... (2 Replies)
Discussion started by: vivek d r
2 Replies

9. Shell Programming and Scripting

Pass awk field to a command line executed within awk

Hi, I am trying to pass awk field to a command line executed within awk (need to convert a timestamp into formatted date). All my attempts failed this far. Here's an example. It works fine with timestamp hard-codded into the command echo "1381653229 something" |awk 'BEGIN{cmd="date -d... (4 Replies)
Discussion started by: tuxer
4 Replies

10. Shell Programming and Scripting

awk output yields error: awk:can't open job_name (Autosys)

Good evening, Im newbie at unix specially with awk From an scheduler program called Autosys i want to extract some data reading an inputfile that comprises jobs names, then formating the output to columns for example 1. This is the inputfile: $ more MapaRep.txt ds_extra_nikira_usuarios... (18 Replies)
Discussion started by: alexcol
18 Replies
curl_formget(3) 						  libcurl Manual						   curl_formget(3)

NAME
curl_formget - serialize a previously built multipart/formdata HTTP POST chain SYNOPSIS
#include <curl/curl.h> int curl_formget(struct curl_httppost * form, void *userp, curl_formget_callback append ); DESCRIPTION
curl_formget() is used to serialize data previously built/appended with curl_formadd(3). Accepts a void pointer as second argument named userp which will be passed as the first argument to the curl_formget_callback function. typedef size_t (*curl_formget_callback)(void *userp, const char *buf, size_t len); The curl_formget_callback will be executed for each part of the HTTP POST chain. The character buffer passed to the callback must not be freed. The callback should return the buffer length passed to it on success. If the CURLFORM_STREAM option is used in the formpost, it will prevent curl_formget(3) from working until you've performed the actual HTTP request as only then will libcurl get the actual read callback to use! RETURN VALUE
0 means everything was ok, non-zero means an error occurred EXAMPLE
size_t print_httppost_callback(void *arg, const char *buf, size_t len) { fwrite(buf, len, 1, stdout); (*(size_t *) arg) += len; return len; } size_t print_httppost(struct curl_httppost *post) { size_t total_size = 0; if(curl_formget(post, &total_size, print_httppost_callback)) { return (size_t) -1; } return total_size; } AVAILABILITY
This function was added in libcurl 7.15.5 SEE ALSO
curl_formadd(3) libcurl 7.54.0 February 03, 2016 curl_formget(3)
All times are GMT -4. The time now is 05:40 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy