Zabbix item for last line of a log file


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
# 1  
Zabbix item for last line of a log file

Dear all,
Code:
Zabbix version     : 2.4 (yes, I know, upgrading soon - honest)
Server OS version  : CentOS 6, 64-bit (CentOS 7 with the Zabbix upgrade)

I've got a large log file that I would like to read by an external process. It's basically the same as reading the item value on a web-page. I have tried to create an item for
Code:
vfs.file.contents[/path/to/log]

but it fails because the log is too large. I know that there are log item types and vfs.file.regex where I can specify the start & end lines, but being a log file I won't know the line count to set the start line value. I'm going round in circles with documentation, probably because I'm misunderstanding what it is telling me. Smilie

It is likely that a key of
Code:
log[/path/to/log,something,or,other]

may help. I don't want to keep a lot, just the last line every now and then. I've tried with
Code:
log[/path/to/log,,,,skip]

and other variations without success. My process to pick it up just comes back with unknown item. Trying to set the start line to $ of -1doesn't seem to help me.

Can anyone send me in the right direction to get the last line of a simple text file recorded as a Zabbix item. I don't want to graph it because it will be text such as a timestamp and message. I just want to be able to use Zabbix to collect it. The item would be in a template and I can write something to read the items in rather than have to set up SSH keys all over the place and collect them by shell script. I'd rather not have to set up all sorts of extra spaghetti each time we create a new server, just give it the Zabbix template and extend the list of servers in my script that knows which servers should have the log file. We need to keep track that processing is running normally and last run timestamp, status, messages etc. The process that creates the log file is not available to us, else I would get it to "append log" all messages as it does now and overwrite log each messages to a separate file so that there would be just a single record in the second file.


Pass the dunce's hat. Smilie


Thanks, in advance,
Robin
# 2  
When I want the last line of a file I use the very simple tail command.

Smilie

Maybe I am too simple?

What am I missing from your requirement?
# 3  
Maybe I wasn't clear on the initial post. Sorry about that. The process producing the display doesn't have direct access to the file.

We have multiple CentOS servers running various positions of our application testing programme and I'm being asked to get a display together which shows how they're all doing, batch state etc.. We destroy and re-create them all the time, but the Zabbix agent is in our VMWare template already so that is the mechanism of choice to get the data. It's shouldn't be too hard, but I'm stuck.

The server showing the display doesn't have access to the log file directly (NFS, SMB or an SSH password-less connection defined) If we did have, I could simply tail -1 /path/to/log but because we a frequently rebuilding, I'd rather not have to add another few steps and more complexity between them. There's lots to do already when we build them. With Zabbix, I can get the file contents but only for a small file, and these log files I want to read are very large.

We don't dare have Zabbix remote commands enabled because that is an entry point to attack the server. I know I can define the Zabbix item type as log with a key of log[/path/to/file], but then I don't know how to get at the content/last line.

The files will get quite large, but I can set the history low, because after that we just want the current position.

I'm happy to define the type as log if I could get that to work, but all I get back is a large number. Smilie
Naturally anything I can define in Zabbix will be added to the existing template so I don't have to set that up every time too.


I'm sure I'm missing something simple,
Robin (feeling like a fool Smilie)
# 4  
The following is probably more a workaround then a solution:

You said that you have no problems with small files but ony big files. In addition, i get from your wording that you don't need real-time exactness because you will poll the data only once in a while. So, why not set up a small cron job that copies the last line of the big log into a small file, each time overwriting the old one, like this sketch script:

Code:
tail -n 1 /path/to/big.log > /path/to/last.logline

Then you can query this new file with your Zabbix methods because it always contain one line only.

I hope this helps.

bakunin
# 5  
Got it at last!! It's quite convoluted, so why not share and see if someone else might benefit or suggest how to improve it:-
  1. Create the Zabbix item in a template for the file as type log and key logrt[/path/to/log] with a history of 1 day and a refresh of 3 seconds (don't want old logs and need it reasonably up to date without hammering the servers) The logrt key allows it to cope with logs that are rotated.
  2. Assign the template to the required hosts, although in my case the template was already on several servers so it appears in various places straight away.
  3. Wait a few minutes for the data to be picked up, ensuring that some new data is actually written to the log.
  4. Determine the Zabbix item number for the log file on the particular host I'm interested in
  5. Use a web-request to get the log file from the Zabbix server
  6. Get the first record (stored so newest is at the top)
  7. Drop the log reference number (first item, tab separated)
  8. Hey presto, I have the message that I can display.
Quite a palaver. I'm also not sure if we have bespoke tools that do things like working out the item number or if having a webserver that can share the log files is default.

I hope that this helps someone. Does anyone think I need to add more detail on each step?



Robin
These 3 Users Gave Thanks to rbatte1 For This Post:
# 6  
Quote:
Originally Posted by rbatte1
Does anyone think I need to add more detail on each step?
Not now, but be prepared to be pestered withquestions as in my new project i probably have to work more with Zabbix. :-))

Great find, though, and thanks for sharing.

bakunin
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Science: Computers
Difficulty: Easy
The Windows 7 operating system has six main editions.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Read a lis, find items in a file from the list, change each item

Hello, I have some tab delimited text data, file: final_temp1 aname val NAME;r'(1,) 3.28584 r'(2,)<tab> NAME;r'(3,) 6.13003 NAME;r'(4,) 4.18037 r'(5,)<tab> You can see that the data is incomplete in some cases. There is a trailing tab after the first column for each incomplete row. I... (2 Replies)
Discussion started by: LMHmedchem
2 Replies

2. Shell Programming and Scripting

Reading line by line from live log file using while loop and considering only those lines start from

Hi, I want to read a live log file line by line and considering those line which start from time stamp; Below code I am using, which read line but throws an exception when comparing line that does not contain error code tail -F /logs/COMMON-ERROR.log | while read myline; do... (2 Replies)
Discussion started by: ketanraut
2 Replies

3. Shell Programming and Scripting

HELP: Shell Script to read a Log file line by line and extract Info based on KEYWORDS matching

I have a LOG file which looks like this Import started at: Mon Jul 23 02:13:01 EDT 2012 Initialization completed in 2.146 seconds. -------------------------------------------------------------------------------- -- Import summary for Import item: PolicyInformation... (8 Replies)
Discussion started by: biztank
8 Replies

4. Shell Programming and Scripting

How to process log file line by line?

Greetings, I'm new to this forum, also new to shell script I have done some simple shell script before, like backup linux machine using rsync and crontab, but now I need to do some log analyzing, which is beyond my ability... so I'm going to seek for help in this forum, hope someone could give... (5 Replies)
Discussion started by: lunaticdawn
5 Replies

5. Shell Programming and Scripting

Print new item in file with symbol

Dear all, I have encountered some problem here. I prompt the user for input and store it into a data file, eg. key in name and marks so the data file will look like this andrew 80 ben 75 and the next input is carine 90. So the problem here is i want to print... (2 Replies)
Discussion started by: branred
2 Replies

6. Shell Programming and Scripting

Reading each item from a formatted file

Hi, I have a file generated like this - 1. Fire SQL and store the formatted output in a temp file echo "select path, empid, age from emp_tbl" | /usr/sql emp_db 2 > count_file | grep vol > tempFile 2. The tempFile looks like this after the above statement /vol/emp1 0732 ... (9 Replies)
Discussion started by: angshuman_ag
9 Replies

7. Shell Programming and Scripting

Searching for file and stopping at first item found

Hello, I try to write a shell script that would list all files on a directory and stop when it finds the first item specified on a find or ls command. How can I tell to the find or ls command to stop when it finds the first ".doc" file for example ? Thank you (7 Replies)
Discussion started by: davchris
7 Replies

8. Shell Programming and Scripting

shell script to read a line in gps receiver log file and append that line to new file

Hi, I have gps receiver log..its giving readings .like below Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. GPSD,R=1 $GPGSV,3,1,11,08,16,328,40,11,36,127,00,28,33,283,39,20,11,165,00*71... (3 Replies)
Discussion started by: gudivada213
3 Replies

9. Infrastructure Monitoring

A Review of Zabbix - Zabbix Rules! (Part 2)

Tim Bass 03-23-2009 03:45 PM Source... (0 Replies)
Discussion started by: Linux Bot
0 Replies

10. Infrastructure Monitoring

A Review of Zabbix - Zabbix Rules! (Part 1)

Tim Bass 03-20-2009 07:02 AM Source... (0 Replies)
Discussion started by: Linux Bot
0 Replies

Featured Tech Videos