Hey guys, I am working on a script that needs to grab variables from a log file. The script will run morning 9 to 5pm and save variables for each run every hour, these results I will be aggregating at the end of the day.
I am thinking I will be placing the date on each entry in the log every hour, so I think I can grep each date and then go from there.
Can anyone help me with the following?
I am in the train right now so when I get home I will add more ideas.
P.s. If anyone is confused I need to conduct calculations on the enteries on each of the log dates. I will
#1 get variables from a db and assign to a shell variables.
#2 post those variables to logs
#3 every hour the script runs it saves logs
#4 these logs will go to a log file for each day running during business hours
#5 Now the tricky part
I have to for each business day the do calculations like percentage calculations on variables collected in logs for each hour during the day at the end of the day. and eventually a summary of all data points collected during the month until the month.
The datapoints I collect to variables are only pure calculations from system performance, like response times, so only numbers.
I will have many variables per hour max like 5 varibles per hour.
-these variable will be needed to collected for calculations over time.
1: aggregation mathematics for speed of the variables.
2: these variables will be collected to do aggregation, I would prefer to save them to logs like after the Vars I can do a overall calculation...
vartotal will be my tool to use for the percentage calculations or any necessary calculations I need going forward.
More info:
I am collecting system performance info and response times, aggregates will be calculated to see how our system performs over time...
The datapoints are 100% needed to be aggregated for our needs.
-how do you guys suggest I grab variables saved to logs by date? and how do I point logs to search datapoints from the logs by saying to search date hour.
#1 possibilty I think that is better for me.
1st script writes the logs file and does the system speed gatherring for those wonderring still its a combination of DB + linux reponse times saved to variables and >> to log and saved by the hour during business hours.
Anyways the logs will be like I explained one will be for my maintainence chekcs and other to save datapoitns like this
#2 The logs collected will have TOTAL column and other calculation columns...
so that I can use the points over time and do calculations.
I am searching for things I need to get this done anyone want to add in
#2 the second script: grabs total saved calculations for the logs for the month and then produces my aggregations.
one thing confusing me is how to I add to the script a way to search and collect data by date, as over time I may need to modify it to lets save give me a summary of months 1 through 3.
I will need too separate the functionality as the more I think I need too do calculations like over custom times as business needs require.
Any help I can get mates would be appreciated.
Anyone need more information from me like what I have done so far please wait a few hours as I am deploying something I got help making from here yesterday
to help explain what I am trying to accomplish! was on train getting home! Thanks for the patience guys
Last edited by Don Cragun; 08-03-2014 at 07:03 AM..
Reason: Fix CODE tags.
I have two computers with dynamic IP addresses and am using dyndns so that they are identifiable as the same computer even if their IPs change (we'll call them host1.dyndns.com and host2.dyndns.com). I also have a remote server which I would like to store my computers' IP addresses on. There is a... (9 Replies)
I am executing a stored proc and sending the results in a log file. I then want to grab one result from the output parameters (bolded below, 2) so that I can store it in a variable which will then be called in another script. There are more details that get printed in the beginning of the log file,... (3 Replies)
Hi,
I am trying to write a script which would go search and get the info from the logs based on yesterday timestamp and write yesterday logs in new file. The log file format is as follows:
""""""""""""""""""""""""""... (3 Replies)
Hi everyone,
Need your help creating a script that downloads a website after entering the URL e.g. Google, and parses its contents for URL's and appends the output to a text file in this format:
http://grabbed-url-from-downloaded-website.com/
http://another-url-from-downloaded-website.com/
... (3 Replies)
Hello all I hope someone can help me. I am trying to convert something I wrote in C to bash.
But how do I go about reading more than one item at a time in a for loop?
i have been using this format for the loops in the bash script i have been building.
e.g.
for word in `cat -s... (4 Replies)
I have a process that is running locally on the machine.
When you telnet to the process: telnet IP port, it automatically returns a string which shows the status of that process.
Something like this:
# telnet IP Port
Trying 127.0.0.1...
Connected to localhost (127.0.0.1).
Escape... (3 Replies)
Hi folks;
I have a temperature sensor that configured with an IP address so we can open a browser to see the temperatures in our 3 labs all showing in one screen.
I'm trying to write a script to call this link then email me the results/output so i don't have to open a browser manually every... (3 Replies)
ok, so a script i wrote spits out an output like the below:
2,JABABA,BV=114,CV=1,DF=-113,PCT=99.1228%
as you can see, each field is separated by a comma. now, how can I get rid of the first field and ONLY show the rest of the fields.
meaning,
i want to get rid of the "2,", and... (3 Replies)
I have several .csv files containing data like this:
field_1;field_2;date;comment;amount;
I want to extract the 3 last fields and load them in a database.
my input_file = "/dir/file.csv";
my output_file = "/dir/file.sql";
open my $csv_file, '<', $input_file
or die "Can't... (1 Reply)
Appreciate help for the below issue.
Im using below code.....I dont want to attach the logs when I ran the perl twice...I just want to take backup with today date and generate new logs...What I need to do for the below scirpt..............
1)if logs exist it should move the logs with extention... (1 Reply)
08-02-2014
