Shell script newbie- how to generate service log from shell script


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Shell script newbie- how to generate service log from shell script
# 1  
Old 03-02-2019
Shell script newbie- how to generate service log from shell script

Hi,

I am totally a newbie to any programming languages and I just started an entry level job in an IT company. One of my recent tasks is to create a script that is able to show the log file of linux service (i.e. ntpd service)
lets say, if I run my script ./test.sh, the output should be similar to :
Code:
2019-02-26 09:15:00 +1100,Success,ntpd,restart,Service Successfully Restart

I dont know if I didnt get the right requirement or something, I found a lot of scripts online but none of them worked for my case. One thing that I don't understand is, do I firstly need to have an existing .log file that has all the ntpd service history? I check var/log/ntpstats but there is no log files.

I would really appreciate if anyone could give me some ideas, if you need any clarifications please let me know.

Thanks very much

xiao

Last edited by vbe; 03-02-2019 at 04:42 AM.. Reason: code tags
# 2  
Old 03-02-2019
Let us take a step back: programming is about automating tasks and the problem is NOT to write the program but to define as succinctly as possible WHAT it is you want to automate. Running faster will get you nowhere if the direction you run to is wrong.

So, let us think:

1) what do you want your log to look like? You already showed a sample line and, agreed, this might be a good start. But where will the entries come from and how will they be organised? By that i mean:

Code:
2019-02-26 09:15:00 +1100,Success,ntpd,restart,Service Successfully Restart

The first part, the date and time is clear. But "Success,ntpd,restart" and "Service Successfully Restart" is essentially the same information, no? Suppose you have this (and maybe many other) scripts running and they all produce 5000 entries a day. You don't want to read them all but may want a script to "read" them, filter out all the "normal" stuff and only tell you about problems. For such a script to be easily written you want to log entries in a homgenous format which lends itself well to be read by a program. Like this:

Code:
$(date and time in fixed format): $(success status), $(service name), $(activity)

We already had date and time, "$(success status)" could be one of "success", "error", "warning", then "$(service name) could be "ntpd" for your script or something else for a similar script and finally "$(activity)" would be whatever it is your script is logging at that time. A possible log would then look like:

Code:
2019-02-26 09:15:00 +1100,Success,ntpd,restart
2019-02-26 09:16:00 +1100,Error,ntpd,other-activity
2019-02-26 09:17:00 +1100,Warning,ntpd,something else
2019-02-26 09:18:00 +1100,Success,ntpd,restart
...

2) The "fields" i did describe here are just for example. You need to define yourself what you need to log for each process. The better your definition is the easier it will be to apply that to all sorts of services to be logged and always end up with the same format. Ideas for which information might (or might not) be interesting to log is:

Code:
$(date and time in fixed format)
$(success status)
$(service name)
$(activity)
$(exact command issued)
$(return code/output of that command)

But you can easily extend this list and might identify - depending on your environment - many other possibly interesting information.

3) single log versus multiple logs
That brings us to the next question: if you log many services there are two possible strategies. You write one log for every service (i.e. ntpd.log, ftpd.log, sshd.log, etc. or you write one log for all. If you write separate logs for each service you do not need to log the service name in every entry because i.e. what else than "ntpd" as a service name is supposed to stand in the file ntpd.log? On the other hand it might be a good idea to have only one log file where all services are logged. You only have one place to look at and if one service fails and affects other services this is easier to see in correlation because the entries are already sorted by time. What is "better", single or separate logs, depends on your environment and your goals. There is no generally right or wrong answer. Again, analyse carefully your environment and apply good reasoning.

4) separate error log
A final point: it sometimes makes sense to write a separate log for log entries of the "error" condition. That is: "success"-type entries go to the normal log, "warning"-type entries also go to the normal log and "error"-type entries go to the normal AND to a separate error-log. You might then have (if you have separate logs for each service) a ntpd.log and a ntpd.errlog, a ftpd.log and a ftpd.errlog, etc., or if you have a single log you have a system.log and a system.errlog. The log writing becomes a bit more complicated but if errors are usually rare (again, what is considered an "error" depends on your environment, so you will have to analyse carefully) you can just check the size of the error log and if it is empty you know at a glance that no error has happened. You may need to investigate more closely only if the error-log is not empty.

I suggest you try to come up with a plan on what you want to do once you thought it through thoroghly and then we discuss how to implement your plan.

I hope this helps.

bakunin
# 3  
Old 03-02-2019
Hey bakunin,


Many thanks for your reply!


the following is my plan and let me know if anything doesnt make sense :


1. Ideally the script should print out the log file in the following format:
2019-02-26 09:15:00 +1100,Success,ntpd,restart 2019-02-26 09:16:00 +1100,Error,ntpd,other-activity

But if I am really stuck,as long as the "$(success status),$(service name),$(activity)" are in the output then it will still be fine, someone will extend the script later on..
2. and the fields that I want in the log should include, the last three fields are the minimum and the date is optional depending on the complexity:

$(date and time in fixed format), $(success status), $(service name), $(activity)



3. single log vs multiple logs: I shouldve clarified earlier that the only service I need to work on is ntpd. So a single log would be fine for me.


4. I am not sure if it will be more complicated to write a separate error log since I only need to focus on one service which is ntpd.



Let me know if it makes sense to move on to the next step. thanks again for your help!
# 4  
Old 03-03-2019
First off: i apologise for dragging this out a bit. Once you are experienced you could probably do all the reasoning we are going through here in a few seconds of thought. But since you lack this experience and i want to show you how you tackle this sort of problems i show you and discuss every point applicable to any monitoring script. Some of what i will tell you will not be applicable to NTP because it is a very simply service. Others are more complicated and you will need this kind of reasoning, though.

So, bear with me - you might have come asking for a meal, i show you how to cook yourself.

Quote:
Originally Posted by xiaogeji
the following is my plan and let me know if anything doesnt make sense Smilie
All makes sense to me, save for one point:

Quote:
Originally Posted by xiaogeji
But if I am really stuck,as long as the "$(success status),$(service name),$(activity)" are in the output then it will still be fine
Quote:
Originally Posted by xiaogeji
I shouldve clarified earlier that the only service I need to work on is ntpd.
Compare these two quotes. If there is only one service you want to monitor then why repeat the name of that service in every line? Do you really need that? Because the file will already be named "ntpd.log" or something such, so one would know that the messsages in there are about NTP or ntpd respectively, no?

Now, having identified the fields (that is: the type of information you want to log) you need to identify how this information will be comprised in detail. What do i mean by that?

Well, take "success status", for instance: Th simplest model would be to have to possible values: "success" or "failure" (or "error" or whatever you want to name it). You could also have three possible outcomes, "success", "warning" and "failure". You could even have more. Have a look at the documentation of syslog to get ideas. Basically syslog has "severity levels" which map to what i called "success status" here, seven of them. You specify a "threshhold severity" and everything "above" that severity will be logged, everything else will not.

Quote:
Originally Posted by xiaogeji
the date is optional depending on the complexity:
Really? If you leave out the time stamp you have a message and don't know when that was. It is easy to do that but i wonder if it makes sense to do it this way.

OK, go over your list a seond time, plan on how many success status you want to have (and how they are called) and how you want to define them: What constitutes "success", what constitutes a "failure"? And how much has to go "quite not right" when you label the outcome a "warning". Once you do this we finally get around to writing the script.

I hope this helps.


bakunin

Last edited by bakunin; 03-04-2019 at 03:27 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Generate documentation for a shell script

Hi, I've written a shell script with proper intentation and commenting structure. However, I would like to generate documentation for the shell which I have written. Is there any tool as such to generate it like we have javagen/docgen ? Please help. Thanks, Arjun (0 Replies)
Discussion started by: arjun_arippa
0 Replies

2. Shell Programming and Scripting

Shell script to compare and generate a new file

Requirement is I have two files their format is File1 - input_file ----- tmp_value|3|number|| tmp_value1|3|alpha|| tmp_value2|6|alpha|AA AA| tmp_value3|15|number|000000005| tmp_value4|15|number|000000000000000| tmp_value5|11|alpha|bbbbbbbbbbb| tmp_value6|11|alpha|bb bb| input_file ... (4 Replies)
Discussion started by: greenworld123
4 Replies

3. Shell Programming and Scripting

Shell script newbie, what is problem with my script?

Hello, Ubuntu server 11.10 can anybody help what is problem with my shell script? #!/bin/bash #script to find out currently logged on user is root or not. if ] then echo "You are super" else echo "You are awesome!" fi When I run script, I get following output ./uid: line 3: I... (4 Replies)
Discussion started by: kaustubh
4 Replies

4. Shell Programming and Scripting

generate logfile in a shell script

Unix Gurus, I have a shell script which has few "echo" statements. I am trying to create a logfile where all the outputs of the echo statement sare stored. I will have to add this as the final step in the existing script so that everytime the script runs, a logfile is generated with all the... (1 Reply)
Discussion started by: shankar1dada
1 Replies

5. Shell Programming and Scripting

Shell Script for a newbie

Hello all, I want to write a shell script to list the contents of a directory and number them and write them to a file. For example, if I have a directory temp and the contents of the directory are alpha, beta and gamma. I want to write these filenames to a file "test" in a numbered manner. ... (7 Replies)
Discussion started by: grajp002
7 Replies

6. Infrastructure Monitoring

Shell Script - Generate SNMP Traps

Good morning to you all Im kinda of a noob to scripting, and my knowledge is still very basic: anyway, Ive developed a small .sh script with the following purpose: - it will check a result file, checking if it has any values, or if its empty - if its empty it will send an email What... (0 Replies)
Discussion started by: zarahel
0 Replies

7. Shell Programming and Scripting

Shell Script Help -I'm a newbie

Can someone help me write this shell script? I am completely new to shell and as a fun task my uncle has challenged me a problem (out of all other people). Basically, all he wants me to do is to create backup file in a folder that is named “disables.” This is what he said: create a shell script... (0 Replies)
Discussion started by: hotcutiepie05
0 Replies

8. Shell Programming and Scripting

how do i generate random integer using only shell script

Hi All, I need to generate 4 digit random no using only shell script. Please help in this ASAP. Thanks in advance... Regards, sridhar. (1 Reply)
Discussion started by: sridhusha
1 Replies

9. Shell Programming and Scripting

Help - shell script newbie

My problem looks like it should have a simple solution but it seems that after many days of research I cannot find a good solution. What I have is an input file that contains lines of information. What I need is to extract specific information from that file. What I know is that somewhere in the... (2 Replies)
Discussion started by: eback
2 Replies

10. UNIX for Dummies Questions & Answers

generate xml from a shell script

Hello! I would like to generate an xml file from the output of various commands generated from within a shell script (some will be in CDATA). At the moment the only solution I have come up with is echoing xml tags around the commands eg. echo "<bitism>" >> outputfile /usr/sbin/prtconf... (1 Reply)
Discussion started by: speedieB
1 Replies
Login or Register to Ask a Question

Featured Tech Videos