Convert vnstat log to csv


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Convert vnstat log to csv
# 1  
Old 08-20-2015
Convert vnstat log to csv

My goal is to log daily network activity (rx/tx) from each ethernet interface.

I installed vnstat, and while it reports daily and monthly stats, it doesn't offer a nice way to convert to csv to import into a database. Maybe there's an alternative to vnstat that I'm not aware of you can suggest that already offers it. If not, I would like to be able to convert the example output to a csv file.

I notice that the output will include prior days, which is okay. I can have MySQL ignore inserts on duplicate rows. I would also like to ignore the current date row as well, usually the last dated row.

I run
Code:
vnstat -i eth0 -d

via cron.

Code:
 eth0  /  daily

         day         rx      |     tx      |    total    |   avg. rate
     ------------------------+-------------+-------------+---------------
     08/11/2015     1.83 MiB |    2.31 MiB |    4.15 MiB |    0.39 kbit/s
     08/12/2015    12.35 MiB |   15.40 MiB |   27.75 MiB |    2.63 kbit/s
     08/13/2015    16.31 MiB |   18.36 MiB |   34.67 MiB |    3.29 kbit/s
     08/14/2015    11.45 MiB |   15.11 MiB |   26.56 MiB |    2.52 kbit/s
     08/15/2015    38.65 MiB |   87.17 MiB |  125.81 MiB |   11.93 kbit/s
     08/16/2015    12.08 MiB |   15.76 MiB |   27.84 MiB |    2.64 kbit/s
     08/17/2015    29.86 MiB |   40.78 MiB |   70.65 MiB |    6.70 kbit/s
     08/18/2015    43.67 MiB |   58.62 MiB |  102.29 MiB |    9.70 kbit/s
     08/19/2015    27.08 MiB |   42.74 MiB |   69.82 MiB |    6.62 kbit/s
     08/20/2015      300 KiB |     298 KiB |     598 KiB |    0.14 kbit/s
     ------------------------+-------------+-------------+---------------
     estimated        --     |      --     |      --     |

The filename looks like YYYY-MM-DD_hostname_interface_net.log

So far I wrote this:

Code:
#!/bin/bash

# change directory
cd /var/log/netlogs/

for logFile in *_net.log
do
  awk '
    BEGIN ( OFS = "," ) FNR == 1 {
      # get date, hostname, interface
      split(FILENAME,F,/_/)
      d=F[1]
      h=F[2]
      i=F[3]
    }
    # Output Format
    # date,hostname,interface,rx,tx,tot,avg
    {
      print d,h,i 
    }
  ' $logfile > /var/log/netlogs/$logfile.csv
done

# 2  
Old 08-20-2015
Did you consider the various options of vnstat, like --dumpdb or --oneline?
# 3  
Old 08-20-2015
Quote:
Originally Posted by RudiC
Did you consider the various options of vnstat, like --dumpdb or --oneline?
I've tried vnstat -i eth0 --oneline before but it only printed today and the current month. Is there a way to print the prior day?
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Convert listner.log to csv format with comma seperated

Hi All, I am new to shell scripting i am trying to convert the listner.log to csv which can be inturn converted to excel for easy reading. i used this command awk '/SID=/ && /HOST=/ && /PORT=/ && /USER=/ { i=match($0,"SID="); i=i+RLENGTH; h0=substr($0,i); i=match(h0,")");... (6 Replies)
Discussion started by: skoshekay
6 Replies

2. Shell Programming and Scripting

How to convert xml to csv ?

I am in need of converting billions of XML into csv file to load data to DB, i have found the below code in perl but not sure why it's not working properly. CODE: #!/usr/bin/perl # Script to illustrate how to parse a simple XML file # and pick out all the values for a specific element, in... (1 Reply)
Discussion started by: rspwilliam
1 Replies

3. UNIX for Dummies Questions & Answers

Convert csv to excel

Hi All, I have a csv file in unix and I need to convert it into excel formate. Please help me out (1 Reply)
Discussion started by: Abhisrajput
1 Replies

4. Shell Programming and Scripting

Convert text to CSV

Hi Gurus I need urgent help to convert a flat log file into csv format to load into database. Log looks like: a=1 b=2 c=3 a=4 b=5 c=6 Only the values at right side of = will come into csv and it should create a new line once it receives "a" field. (8 Replies)
Discussion started by: sandipjee
8 Replies

5. Shell Programming and Scripting

Convert xml to csv

I need to convert below xml code to csv. I searched other posts as well but this post (_https://www.unix.com/shell-programming-scripting/174417-extract-parse-xml-data-statistic-value-csv.html) gives "sed command garbled" error. As of now I have written a long script to do it, but can it be done with... (7 Replies)
Discussion started by: dineshydv
7 Replies

6. Shell Programming and Scripting

Convert txt to csv

Hi - I am looking to convert the following text to csv. The columns may not always have data in them and they may have varying spaces but I still need to have a comma there anyway: Sample Data: ~~~~~~~ Name Email Location Phone Tom... (4 Replies)
Discussion started by: JPBovaird
4 Replies

7. Shell Programming and Scripting

Txt to csv convert

Hi, I was trying some split command to pull out values like "uid=abc,ou=INTERNAL,ou=PEOPLE" into a csv file. However because of erratic nature of occurrance of rows made me stopped. Could someone help me in this? and if someone has a one liner for this? The text file contain pattern like this... (14 Replies)
Discussion started by: john_prince
14 Replies

8. Shell Programming and Scripting

how to convert .xls to .csv

Hi, I have problem..How to convert .xls file to .csv.. Plz help me for this problem.. (1 Reply)
Discussion started by: varma457
1 Replies

9. Shell Programming and Scripting

Help to convert XML to CSV

Apologies if this has already been covered in this site somewhere, I did try looking but without any success. I am new to the whole XML thing, very late starter, and have a requirement to convert an XML fiule to a CSV fomat. I am crrently working on a Solaris OS. Does anyone have any suggestions,... (2 Replies)
Discussion started by: rossingi_33
2 Replies
Login or Register to Ask a Question