Sponsored Content
Top Forums UNIX for Advanced & Expert Users Nearly Random, Uncorrelated Server Load Average Spikes Post 303044275 by Neo on Monday 17th of February 2020 09:00:11 PM
Old 02-17-2020
Update:

I have confirmed 100% the source of the these spikes were very aggressive, rogue, unidentified bots originating on Chinese networks. After blocking the resident networks of these bots, all spikes have stopped, completely.

This is a "huge success story", going from unknown, uncorrelated performance hits / spikes due to nearly random spikes in performance to cause identification and total resolution. As you can see from the graph over the last 24 hours, there have been zero spikes.

Nearly Random, Uncorrelated Server Load Average Spikes-screen-shot-2020-02-18-84642-amjpg


I will keep the same MQTT and Node-RED instrumentation in place (which I am very pleased with) and will also keep all "spike trapping" instrumentation and DB logging in place, so if other spikes appear, which I am fairly confident more of these "pesky" bots will appear sooner or later, I will trap them, identify the source and block their resident networks.

Success!

MQTT and Node-RED did not "solve the problem". MQTT and Node-RED provided a very powerful and flexible way for me to quickly instrument custom sensors and logging, which helped me identify the problem.

I highly, recommend MQTT and Node-RED. These tools are free. Thank you very much MQTT and Node-RED developers!
This User Gave Thanks to Neo For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

load average

we have an unix system which has load average normally about 20. but while i am running a particular unix batch which performs heavy operations on filesystem and database average load reduces to 15. how can we explain this situation? while running that batch idle cpu time is about %60-65... (0 Replies)
Discussion started by: gfhgfnhhn
0 Replies

2. UNIX for Dummies Questions & Answers

Load Average

Hello all, I have a question about load averages. I've read the man pages for the uptime and w command for two or three different flavors of Unix (Red Hat, Tru64, Solaris). All of them agree that in the output of the 2 aforementioned commands, you are given the load average for the box, but... (3 Replies)
Discussion started by: Heathe_Kyle
3 Replies

3. UNIX for Dummies Questions & Answers

top - Load average

Hello, Here is the output of top command. My understanding here is, the load average 0.03 in last 1 min, 0.02 is in last 5 min, 0.00 is in last 15 min. By seeing this load average, When can we say that, the system load averge is too high? When can we say that, load average is medium/low??... (8 Replies)
Discussion started by: govindts
8 Replies

4. Solaris

load average query.

Hi, i have installed solaris 10 on t-5120 sparc enterprise. I am little surprised to see load average of 2 or around on this OS. when checked with ps command following process is using highest CPU. looks like it is running for long time and does not want to stop, but I do not know... (5 Replies)
Discussion started by: upengan78
5 Replies

5. UNIX for Dummies Questions & Answers

Please Help me in my load average

Hello AlL,.. I want from experts to help me as my load average is increased and i dont know where is the problem !! this is my top result : root@a4s # top top - 11:30:38 up 40 min, 1 user, load average: 3.06, 2.49, 4.66 Mem: 8168788k total, 2889596k used, 5279192k free, 47792k... (3 Replies)
Discussion started by: black-code
3 Replies

6. UNIX for Advanced & Expert Users

Load average in UNIX

Hi , I am using 48 CPU sunOS server at my work. The application has facility to check the current load average before starting a new process to control the load. Right now it is configured as 48. So it does mean that each CPU can take maximum one proces and no processe is waiting. ... (2 Replies)
Discussion started by: kumaran_5555
2 Replies

7. Solaris

Load Average and Lwps

NPROC USERNAME SWAP RSS MEMORY TIME CPU 320 oracle 23G 22G 69% 582:55:11 85% 47 root 148M 101M 0.3% 99:29:40 0.3% 53 rafmsdb 38M 60M 0.2% 0:46:17 0.1% 1 smmsp 1296K 5440K 0.0% 0:00:08 0.0% 7 daemon ... (2 Replies)
Discussion started by: snjksh
2 Replies

8. UNIX for Dummies Questions & Answers

Load average spikes once an hour

Hi, I am getting a high load average, around 7, once an hour. It last for about 4 minutes and makes things fairly unusable for this time. How do I find out what is using this. Looking at top the only thing running at the time is md5sum. I have looked at the crontab and there is nothing... (10 Replies)
Discussion started by: sm9ai
10 Replies

9. UNIX for Dummies Questions & Answers

Help with load average?

how load average is calculated and what exactly is it difference between cpu% and load average (9 Replies)
Discussion started by: robo
9 Replies

10. Programming

ESP32 (ESP-WROOM-32) as an MQTT Client Subscribed to Linux Server Load Average Messages

Here we go.... Preface: ..... so in a galaxy far, far, far away from commercial, data sharing corporations..... For this project, I used the ESP-WROOM-32 as an MQTT (publish / subscribe) client which receives Linux server "load averages" as messages published as MQTT pub/sub messages.... (6 Replies)
Discussion started by: Neo
6 Replies
EVENTCOUNTER(1) 					      General Commands Manual						   EVENTCOUNTER(1)

NAME
eventcounter, eventgenerator, eventlogger, eventselect, eventsink, eventsource, viewevents - utility applications which can be used together with other MUSIC-aware applications in multi-simulations SYNOPSIS
eventcounter [-tbmh ] n_units prefix [ suffix ] eventgenerator [-tbfmih ] n_units eventlogger [-tlbmih ] eventselect [-th ] n_units units eventsink [-tmih ] n_units prefix [ suffix ] eventsource [-tbmih ] n_units prefix [ suffix ] viewevents [-tshT ] configfile DESCRIPTION
This manual page documents briefly a set of MUSIC-aware utility applications provided together with MUSIC library. eventcounter receives spikes through a MUSIC input port, counts all spikes for each index and writes the frequencies to a set of files with names prefix rank suffix eventgenerator generates spikes from a Poisson distribution. eventlogger logs spikes from a MUSIC port. eventselect receives events from an input port of width n_units and sends events for the subset of id:s specified in the file units eventsink receives spikes through a MUSIC input port and writes these to a set of files with names prefix rank suffix eventsource reads spikes from a set of files with names prefix rank suffix and propagates these spikes through a MUSIC output port. viewevents reads spikes from a MUSIC input port and displays them as a 3D graphical representation. OPTIONS
The utilities follow the usual GNU command line syntax, with long options starting with two dashes (`-'). A summary of options is included below. -b ticks, --maxbuffered ticks Specify maximal amount of data buffered. -f freq, --frequency Specify average frequency (default 10 Hz). -h, --help Show usage information. -i, --indextype Select global (default) or local indices. (Used for testing and benchmarking purposes.) -l latency, --acclatency latency Specify acceptable data latency (s). -m type, --imaptype type Select linear (default) or roundrobin index map. -s scaling, --scaletime scaling Specify real time to simulated time scale factor (s). If omitted, the visualization runs at full speed. -t timestep, --timestep timestep Specify time between tick() calls (default 0.01 s). -T title, --title title Specify window title. SEE ALSO
music(1) AUTHOR
MUSIC was written by Mikael Djurfeldt and Orjan Ekeberg for INCF. viewevents was written by Johannes Hjorth for INCF. This manual page was written by Mikael Djurfeldt <mdj@debian.org>, for the Debian project (but may be used by others). March 5, 2009 EVENTCOUNTER(1)
All times are GMT -4. The time now is 07:13 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy