Sponsored Content
Top Forums Programming ESP32 (ESP-WROOM-32) as an MQTT Client Subscribed to Linux Server Load Average Messages Post 303043610 by Neo on Sunday 2nd of February 2020 04:50:26 AM
Old 02-02-2020
ESP32 (ESP-WROOM-32) as an MQTT Client Subscribed to Linux Server Load Average Messages

Here we go....

Preface:

Quote:
Rant: After a "very interesting" experience back-and-forth with Blynk yesterday over their "pull money out of your pocket for every tiny widget" business model when I was publishing a quick public service app to help folks in China regarding the Wuhan coronavirus, in also combination with the Blynk "terms of service" which permits Blynk to share all of a user's network data, every message, all user mined behavioral analytical "raw material" and share that data with any and all third parties; I decided to build my own messaging network to test these ESP32.......
..... so in a galaxy far, far, far away from commercial, data sharing corporations.....

For this project, I used the ESP-WROOM-32 as an MQTT (publish / subscribe) client which receives Linux server "load averages" as messages published as MQTT pub/sub messages. Here is the Arduino IDE code I wrote for the MQTT client running on the ESP32:

ESP32 (ESP-WROOM-32) as an MQTT Client Subscribed to Linux Server Load Average Messages-s20200201_012jpg



ESP32 (ESP-WROOM-32) as an MQTT Client Subscribed to Linux Server Load Average Messages-img_9116jpg


Code:
/*****************************************************************
 *  ESP32 (ESP-WROOM-32) MQTT Client Test  v0.1
 *  The ESP32 subscribes to MQTT messages from an MQTT server
 *  published by a Linux server. In this example, server load 
 *  average messages.  The MQTT server requires username and password
 *  for basic authentication security.
 * 
 *  Neo  www.unix.com  v0.1  2 Feb 2020
 *  You are free to use this code anyway imaginable
 *  for your ESP32 MQTT projects.
 *****************************************************************/

#include <WiFi.h>
#include <PubSubClient.h>
#define BAUD_RATE 115200
#define DEBUG false
const char *ssid = "YOUR_WIFI_SSID";
const char *password = "YOUR_WIFI_PASSWORD";
const char *mqttServer = "YOUR_MQTT_SERVER";
const int mqttPort = 1883;                      // YOUR_MQTT_PORT
const char *mqttUser = "YOUR_MQTT_SERVER_USER"; // Basic user and password authentication
const char *mqttPassword = YOUR_MQTT_SERVER_USER_PASSWORD "; // Basic user and password authentication
    const int lengthTopic = 16; //required for the if-elseif message decoding based on topic
WiFiClient espClient;
PubSubClient client(espClient);

// The code below is self documenting, I think

void callback(char *topic, byte *payload, unsigned int length)
{
    float loadvar, loadvar0, loadvar5, loadvar15;
    if (DEBUG)
    {
        Serial.print("Message arrived in topic: ");
        Serial.println(topic);
        Serial.print("Message: ");
    }
    String w;
    String receivedTopic = String(topic);
    if (DEBUG)
    {
        int len = receivedTopic.length();
        Serial.print("Length :");
        Serial.println(String(len));
    }
    for (int i = 0; i < length; i++)
    {
        w = w + String((char)payload[i]);
    }

    if (receivedTopic.indexOf("loads/l1") > 0 and receivedTopic.length() < lengthTopic)
    {
        loadvar0 = w.toFloat();
        Serial.printf("LOAD  1 ... %s \n", w);
    }
    else if (receivedTopic.indexOf("loads/l5") > 0 and receivedTopic.length() < lengthTopic)
    {
        loadvar0 = w.toFloat();
        Serial.printf("LOAD  5 ... %s \n", w);
    }
    else if (receivedTopic.indexOf("loads/l15") > 0)
    {
        loadvar0 = w.toFloat();
        Serial.printf("LOAD 15 ... %s \n", w);
    }
    else
    {
        loadvar = w.toFloat();
        Serial.printf("Message: %s \n", w);
    }

    if (DEBUG)
    {
        Serial.println("-----------------------");
    }
}

void setup()
{

    Serial.begin(BAUD_RATE);
    WiFi.begin(ssid, password);

    while (WiFi.status() != WL_CONNECTED)
    {
        delay(500);
        Serial.printf("Connecting to WiFi: %s \n", ssid);
    }

    Serial.printf("Connected to the WiFi network: %s \n", ssid);
    client.setServer(mqttServer, mqttPort);
    client.setCallback(callback);

    while (!client.connected())
    {
        Serial.printf("Connecting to MQTT Server: %s:%d\n", mqttServer, mqttPort);

        if (client.connect("ESP32Client", mqttUser, mqttPassword))
        {
            Serial.println("Connected!");
        }
        else
        {
            Serial.print("failed with state ");
            Serial.print(client.state());
            delay(2000);
        }
    }

    client.subscribe("server/loads");
    client.subscribe("server/loads/l1");
    client.subscribe("server/loads/l5");
    client.subscribe("server/loads/l15");
}

void loop()
{
    client.loop();
}

Crontab entry on Linux:

Code:
* * * * * /usr/bin/nice -n 19 php /usr/local/bin/loads_mqtt.php > /dev/null 2>&1

Here is the crontab PHP file which publishes the server load average messages:

Code:
<?php
/********************************************************
 * Linux PHP Script to Publish Server Load Averges v0.1
 * Simple Script using MQTT to publish messages used
 * in ESP32 test.
 *
 * Neo www.unix.com 2 Feb 2020
 * You are free to use this as you find useful.
 ********************************************************/

$load = shell_exec('cat /proc/loadavg');
$pieces = explode(" ", $load);
define('SEND_TEXT', false);

if (SEND_TEXT) {
    $l1 = "LOAD 1: " . $pieces[0];
    $l5 = "LOAD 5: " . $pieces[1];
    $l15 = "LOAD 15: " . $pieces[2];
} else {
    $l1 = $pieces[0];
    $l5 = $pieces[1];
    $l15 = $pieces[2];
}

$c1 = '/usr/bin/mosquitto_pub -t server/loads/l1 -m "' . $l1 . '" -q 1 -u YOUR_USER -P YOUR_USER_PASSWORD';
$c2 = '/usr/bin/mosquitto_pub -t server/loads/l5 -m "' . $l5 . '" -q 1 -u YOUR_USER -P YOUR_USER_PASSWORD';
$c3 = '/usr/bin/mosquitto_pub -t server/loads/l15 -m "' . $l15 . '" -q 1 -u YOUR_USER -P YOUR_USER_PASSWORD';
echo $c1 . "\n";
$output = shell_exec($c1);
sleep(1);
$output = shell_exec($c2);
sleep(1);
$output = shell_exec($c3);

Here is the sample output in the Arduino IDE serial monitor:

Code:
16:08:05.726 -> Connecting to WiFi: YOUR_WIFI_SSID
16:08:06.226 -> Connecting to WiFi: YOUR_WIFI_SSID
16:08:06.739 -> Connecting to WiFi: YOUR_WIFI_SSID
16:08:07.244 -> Connecting to WiFi: YOUR_WIFI_SSID
16:08:07.724 -> Connecting to WiFi: YOUR_WIFI_SSID
16:08:08.226 -> Connecting to WiFi: YOUR_WIFI_SSID
16:08:08.226 -> Connected to the WiFi network: TIM_WIFI 
16:08:08.263 -> Connecting to MQTT Server: www.your_mqtt_server.com:1883
16:08:09.826 -> connected
16:09:02.238 -> LOAD  1 ... 1.31 
16:09:03.162 -> LOAD  5 ... 3.74 
16:09:04.189 -> LOAD 15 ... 3.78 
16:10:02.252 -> LOAD  1 ... 0.96 
16:10:03.262 -> LOAD  5 ... 3.20 
16:10:04.211 -> LOAD 15 ... 3.59 
16:11:04.808 -> LOAD  1 ... 0.99 
16:11:04.808 -> LOAD  5 ... 2.80 
16:11:04.808 -> LOAD 15 ... 3.42 
16:12:09.545 -> LOAD  1 ... 1.01 
16:12:09.545 -> LOAD  5 ... 2.47 
16:12:09.545 -> LOAD 15 ... 3.27 
16:13:12.122 -> LOAD  1 ... 0.89 
16:13:12.122 -> LOAD  5 ... 2.18 
16:13:12.122 -> LOAD 15 ... 3.12 
16:14:02.504 -> LOAD  1 ... 0.90 
16:14:02.689 -> LOAD  5 ... 1.95 
16:14:04.953 -> LOAD 15 ... 2.98 
16:15:08.289 -> LOAD  1 ... 0.71 
16:15:08.289 -> LOAD  5 ... 1.71 
16:15:08.289 -> LOAD 15 ... 2.83 
16:16:05.209 -> LOAD  1 ... 0.48 
16:16:05.209 -> LOAD  5 ... 1.47 
16:16:05.209 -> LOAD 15 ... 2.68 
16:17:11.766 -> LOAD  1 ... 0.43 
16:17:11.766 -> LOAD  5 ... 1.27 
16:17:11.766 -> LOAD 15 ... 2.53 
16:18:01.881 -> LOAD  1 ... 0.41 
16:18:02.928 -> LOAD  5 ... 1.11 
16:18:07.767 -> LOAD 15 ... 2.39 
16:19:13.309 -> LOAD  1 ... 0.41 
16:19:13.309 -> LOAD  5 ... 0.98 
16:19:13.309 -> LOAD 15 ... 2.27 
16:20:05.301 -> LOAD  1 ... 0.69 
16:20:05.301 -> LOAD  5 ... 0.98 
16:20:05.301 -> LOAD 15 ... 2.18 
16:21:05.305 -> LOAD  1 ... 0.78 
16:21:05.305 -> LOAD  5 ... 0.95 
16:21:05.305 -> LOAD 15 ... 2.10 
16:22:02.641 -> LOAD  1 ... 0.69 
16:22:02.853 -> LOAD  5 ... 0.91 
16:22:03.877 -> LOAD 15 ... 2.01 
16:23:07.344 -> LOAD  1 ... 0.90 
16:23:07.344 -> LOAD  5 ... 0.94 
16:23:07.344 -> LOAD 15 ... 1.95 
16:24:01.310 -> LOAD  1 ... 0.70 
16:24:02.331 -> LOAD  5 ... 0.87 
16:24:03.306 -> LOAD 15 ... 1.87 
16:25:01.924 -> LOAD  1 ... 0.66 
16:25:02.765 -> LOAD  5 ... 0.81 
16:25:03.785 -> LOAD 15 ... 1.78 
16:26:09.173 -> LOAD  1 ... 0.69 
16:26:09.173 -> LOAD  5 ... 0.80 
16:26:09.173 -> LOAD 15 ... 1.72 
16:27:01.644 -> LOAD  1 ... 0.71 
16:27:04.816 -> LOAD  5 ... 0.79 
16:27:04.816 -> LOAD 15 ... 1.65 
16:28:04.723 -> LOAD  1 ... 0.90 
16:28:04.723 -> LOAD  5 ... 0.82 
16:28:04.723 -> LOAD 15 ... 1.60 
16:29:08.297 -> LOAD  1 ... 0.89 
16:29:08.297 -> LOAD  5 ... 0.82 
16:29:08.297 -> LOAD 15 ... 1.55 
16:30:02.891 -> LOAD  1 ... 0.61 
16:30:02.891 -> LOAD  5 ... 0.75 
16:30:03.875 -> LOAD 15 ... 1.48

So far, no joy in finding a good IOS app for the iPhone to display this data in a nice dashboard.

I did find an OK "MQTT Terminal App" for the iPhone and I did confirm I can subscribe to these messages with MQTT on the iPhone. Unfortunately, all the "pretty dashboard apps for MQTT" I tried in the app store has some problems and none worked well "out of the box". However, the simple MQTT terminal apps work fine (but are not "pretty")

ESP32 (ESP-WROOM-32) as an MQTT Client Subscribed to Linux Server Load Average Messages-mqtt_terminal_appjpeg


The MQTT terminal app I tested, which managed to work, does not add line breaks at the end of a message, and when I tried to send line breaks over MQTT, the clients stopped receiving the messages, which is expected since these pub/sub client/servers are not really meant for formatted data. Maybe I'm missing something small?

Also refer to:

Install MQTT Broker on Ubuntu 18.04 & 14.04

to install the MQTT broker on Ubuntu...

I'm still searching for a "pretty dashboard" app for IOS which will display the MQTT data, but so far, nothing has worked well.

Will update this thread if I find an MQTT dashboard app for IOS which "works" OK Smilie
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Server load (Unix/Linux, Redhat, CPanel)

Hello, I'm facing a big problem with my hosting server (Dual Xeon 2.4GHz), I'm having a load in the CPU usage and the memory (maybe it's related) ALSO mySQL: Server Load 5.34 (2 cpus) (to 22 sometime) Memory Used 68.4 % (to 70% sometime) When I go to 'CPU/Memory/MySQL Usage' I found: ... (3 Replies)
Discussion started by: Kh@lid
3 Replies

2. Cybersecurity

linux prog for server space, load etc

Hi all what are the ways by which we can know and generate a report of the space remaining, memory(ram) used and the load on the server over a period of time. (1 Reply)
Discussion started by: arlan
1 Replies

3. UNIX for Advanced & Expert Users

load average

we have an unix system which has load average normally about 20. but while i am running a particular unix batch which performs heavy operations on filesystem and database average load reduces to 15. how can we explain this situation? while running that batch idle cpu time is about %60-65... (0 Replies)
Discussion started by: gfhgfnhhn
0 Replies

4. UNIX for Dummies Questions & Answers

Load Average

Hello all, I have a question about load averages. I've read the man pages for the uptime and w command for two or three different flavors of Unix (Red Hat, Tru64, Solaris). All of them agree that in the output of the 2 aforementioned commands, you are given the load average for the box, but... (3 Replies)
Discussion started by: Heathe_Kyle
3 Replies

5. Linux

how to setup a virtual IP to control 2 server load for linux

Hi anyone know how to setup a setup a virtual IP to control 2 server load for linux? i only have 2 server, i don want to buy another just for the load balance... is there a way to do it? Sumemr (0 Replies)
Discussion started by: summerpeh
0 Replies

6. UNIX for Dummies Questions & Answers

Please Help me in my load average

Hello AlL,.. I want from experts to help me as my load average is increased and i dont know where is the problem !! this is my top result : root@a4s # top top - 11:30:38 up 40 min, 1 user, load average: 3.06, 2.49, 4.66 Mem: 8168788k total, 2889596k used, 5279192k free, 47792k... (3 Replies)
Discussion started by: black-code
3 Replies

7. UNIX for Dummies Questions & Answers

linux tomcat load average issue

Hello I am running a liferay application in tomcat on one of my hosted linux machine having 4GB of ram and 1 CPU. I get the tomcat response really slow and with much digging I found that the cpu might be loaded and can be tracked with top command. The following is the output of top command:... (0 Replies)
Discussion started by: dhavaln
0 Replies

8. UNIX for Dummies Questions & Answers

Help with load average?

how load average is calculated and what exactly is it difference between cpu% and load average (9 Replies)
Discussion started by: robo
9 Replies

9. Programming

A Slightly Better NTP Client for the ESP32 (ESPWROOM32) on macOS Catalina

Currently have two ESP8266 modules testing some Blynk apps, whereI'm not so happy with the Blynk business model for developers, but that's another story. So, with two of my ESP8266s currently "busy", I decided to work on the ESP32, and in particular the ESPWROOM32. I installed the... (0 Replies)
Discussion started by: Neo
0 Replies

10. UNIX for Advanced & Expert Users

Nearly Random, Uncorrelated Server Load Average Spikes

I have been wrangling with a small problem on a Ubuntu server which runs a LAMP application. Linux ubuntu 4.15.0-33-generic #36-Ubuntu SMP Wed Aug 15 16:00:05 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux This server runs fine, basically: ubuntu:/var/www# uptime 20:17:13 up 105 days,... (45 Replies)
Discussion started by: Neo
45 Replies
All times are GMT -4. The time now is 01:06 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy