Telegram Object Detection Bot using Node-RED and TensorFlow Object Detection on macOS


 
Thread Tools Search this Thread
Top Forums Programming Telegram Object Detection Bot using Node-RED and TensorFlow Object Detection on macOS
# 1  
Old 03-06-2020
Telegram Object Detection Bot using Node-RED and TensorFlow Object Detection on macOS

Fun project!

The GitHub project page for Node-RED node with a TensorFlow.js Object Detection model cautions that their demo code only works with the Raspberry PI. However, today I got it working (quite easily) with macOS.

It's fairly accurate too! However, LOL, there was no airplane in our condo today!!!; but maybe that reading happened when I was moving the camera around, haha. Here is a quick screen grab from Telegram on the desktop.

Telegram Object Detection Bot using Node-RED and TensorFlow Object Detection on macOS-screen-shot-2020-03-06-45648-pmjpg


Here is my Node-RED flow for this caper:


Telegram Object Detection Bot using Node-RED and TensorFlow Object Detection on macOS-screen-shot-2020-03-06-45556-pmjpg


To build this, just follow the README.md file on the GitHub page above and here (again):

Code:
https://github.com/IBM/node-red-tensorflowjs

It might also help to watch the YT Video:

Creating A Machine Learning IoT App on Raspberry Pi with Node-RED and TensorFlow.js. Jan 7, 2020

The instructions are easy to follow, and so I will not repeat them here. However, here are the steps to get it to work on macOS:

teps:
  1. Install rpi-gpio into Node-RED (so we can disable those nodes easily when we import the flow from the demo code);
  2. Install node-red-tensorflowjs in Node-RED
  3. brew install imagesnap;
  4. audio node in Node-RED: change aplay to afplay;
  5. adjust full paths as needed;

I modified most of the flow to create this logic, at a high level
  • Turn On and Off the WebCam using Telegram
  • Turn On and Off the "nothing detected" message with Telegram (debugging when away)
  • Retooled the detection logic.

New detection logic (function) as follows:

Code:
var onoff = global.get("onoff") || "on";
let newMsg = {};
var chat = "your_telegram_chat_id_here";
var type = "message";
var message = "";
var count = 0;
var score = 0.0;
var onoi = ["chair", "couch", "bench"];   // onoi = "objects not of interest"

if (onoff == "on" || onoff == "all") {
  for (let object of msg.payload) {
    if (object.class.length > 2) {
      if (onoi.includes(object.class)) {
      } else {
        count++;
        score = (100 * object.score).toFixed(1);
        message =
          'TF CamBot detected a "' +
          object.class +
          '" with ' +
          score +
          "% confidence.";
        newMsg = { payload: { chatId: chat, type: type, content: message } };
        //console.log(newMsg.payload);
        node.send([[newMsg]]);
      }
    }
  }

  if (count < 1 && onoff == "all") {
    message = "TF CamBot detected nothing!";
    newMsg = { payload: { chatId: chat, type: type, content: message } };
    node.send([[newMsg]]);
  }
}

Here is the Node-RED flow:

Code:
[{"id":"21dff30d.cdd83c","type":"tab","label":"NEO DETECT","disabled":false,"info":""},{"id":"940a9a8d.367588","type":"inject","z":"21dff30d.cdd83c","name":"Take Photo","topic":"","payload":"","payloadType":"date","repeat":"15","crontab":"","once":true,"onceDelay":0.1,"x":130,"y":180,"wires":[["f2a273c5.fc849"]]},{"id":"71a92361.dbad0c","type":"debug","z":"21dff30d.cdd83c","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":1000,"y":280,"wires":[]},{"id":"339bc42f.3b72fc","type":"tfjs-object-detection","z":"21dff30d.cdd83c","name":"","modelUrl":"","x":760,"y":180,"wires":[["71a92361.dbad0c","5ff0aa2f.76ba54"]]},{"id":"4442c40d.9bc1dc","type":"usbcamera","z":"21dff30d.cdd83c","filemode":"0","filename":"image01.jpg","filedefpath":"1","filepath":"/Users/Tim","fileformat":"jpeg","resolution":"2","name":"","x":530,"y":180,"wires":[["339bc42f.3b72fc","518f1480.b2223c"]]},{"id":"518f1480.b2223c","type":"image","z":"21dff30d.cdd83c","name":"","width":"240","data":"payload","dataType":"msg","thumbnail":true,"active":false,"x":760,"y":280,"wires":[]},{"id":"cddbc39f.fb82c","type":"comment","z":"21dff30d.cdd83c","name":"Check detected classes for specific object.","info":"","x":210,"y":280,"wires":[]},{"id":"6dc534c0.7946ac","type":"telegram sender","z":"21dff30d.cdd83c","name":"CamBot Send","bot":"995ad6d2.ff8ae8","x":1240,"y":100,"wires":[[]]},{"id":"5ff0aa2f.76ba54","type":"function","z":"21dff30d.cdd83c","name":"Detection Logic","func":"var onoff =global.get('onoff') || 'on';\nlet personDetected = false;\nlet newMsg = {};\nvar chat = 727630977;\nvar type = \"message\";\nvar message = \"\";\nvar count = 0;\nvar score = 0.0;\nvar onoi = [\"chair\", \"couch\", \"bench\"];\n\nif(onoff ==\"on\" ||onoff == \"all\")\n{\nfor (let object of msg.payload) {\n    if (object.class.length >  2 ) {\n        if (onoi.includes(object.class)) {\n        }\n        else {\n        count++;\n        score = (100*object.score).toFixed(1);\n         message = 'TF CamBot detected a \"'+ object.class+'\" with '+score+'% confidence.';\n         newMsg = {payload:{chatId:chat,type:type,content:message}};\n        //console.log(newMsg.payload);\n        node.send([ [ newMsg] ]);\n        }\n    }\n}\n\nif(count < 1 &&  onoff == \"all\"){\n    message = 'TF CamBot detected nothing!';\n       newMsg = {payload:{chatId:chat,type:type,content:message}};\n       node.send([ [ newMsg] ]);\n}\n}\n","outputs":1,"noerr":0,"x":1010,"y":180,"wires":[["6dc534c0.7946ac"]]},{"id":"492a7004.57b96","type":"telegram receiver","z":"21dff30d.cdd83c","name":"CamBot In","bot":"995ad6d2.ff8ae8","saveDataDir":"","x":110,"y":100,"wires":[["f2c0ca3b.61eac8"],[]]},{"id":"f2c0ca3b.61eac8","type":"change","z":"21dff30d.cdd83c","name":"Global: On/Off/All ","rules":[{"t":"set","p":"onoff","pt":"global","to":"payload.content","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":330,"y":100,"wires":[["168875ba.43431a"]]},{"id":"168875ba.43431a","type":"function","z":"21dff30d.cdd83c","name":"Confirm C2","func":"let newMsg = {};\nvar chat = 111111111;\nvar type = \"message\";\nmessage = \"TF CamBot is \"+msg.payload.content;\nnewMsg = {payload:{chatId:chat,type:type,content:message}};\nnode.send([ [ newMsg] ]);\n","outputs":1,"noerr":0,"x":530,"y":100,"wires":[["6dc534c0.7946ac"]]},{"id":"f2a273c5.fc849","type":"function","z":"21dff30d.cdd83c","name":"On || Off ?","func":"var onoff =global.get('onoff') || 'on';\nif(onoff =='on' || onoff =='all')\n{\nreturn msg;\n}\n\n","outputs":1,"noerr":0,"x":310,"y":180,"wires":[["4442c40d.9bc1dc"]]},{"id":"995ad6d2.ff8ae8","type":"telegram bot","z":"","botname":"YourCoolCambot","usernames":"","chatids":"","baseapiurl":"","updatemode":"polling","pollinterval":"300","usesocks":false,"sockshost":"","socksport":"6667","socksusername":"anonymous","sockspassword":"","bothost":"","localbotport":"8443","publicbotport":"8443","privatekey":"","certificate":"","useselfsignedcertificate":false,"sslterminated":false,"verboselogging":false}]

Works like a charm so far, just needs some fine tuning.

Enjoy!

My plan is to try to detect the seagulls which sometimes fly into our condo since the window are open most of the time! Those bird go a bit crazy when they fly into the condo (normally only one at a time) looking for food, and cannot fly out so easily!

Well, it's a good life if that is the main problem we want to detect, for sure!
These 3 Users Gave Thanks to Neo For This Post:
# 2  
Old 03-07-2020
Now that all is working "OK"... I moved this Node-RED image detection app to a MacBook Air, using the built-in web cam, and am now trying to detect the seagulls when they fly into my "balcony bonsai tree", LOL. USB cord for the Logitech HD cam was not long enough from the trash can mac on my desktop.

So far, no birds! Waiting..... waiting...... cyberstalking ..... tensorflowing.....

If it works, first we will try playing some "qwaaawk... qwaaaak " or "brraaapp... braaapp" audio clip loudly ... haha

Telegram Object Detection Bot using Node-RED and TensorFlow Object Detection on macOS-img_9213jpg
# 3  
Old 03-07-2020
This is funny... the sun is setting over the ocean, and "Mr. Smart TensorFlow Muppet Bot"... id's this as "oven"... LOL

Telegram Object Detection Bot using Node-RED and TensorFlow Object Detection on macOS-img_9217cjpg


Telegram Object Detection Bot using Node-RED and TensorFlow Object Detection on macOS-img_9216jpg


So, my wife bursts out laughing tell me "it's trying to tell you it's hot, but does not have the words..... " so funny around here ... hahaha
# 4  
Old 03-08-2020
Had this Node-RED, COCO-SSD, TensorFlow contraption out on the balcony hunting for birds today. Then was busy on some other project, and got this incoming.

So, not sure if the gizmo I made actually identified an airplane or it is confusing birds with planes! LOL

Anyway, I have heard so many "bad stories" about these AI algos are used for evil; but today it was really fun detecting various things and laughing at what my MacBook Air thought was a "airplane" or a "flower pot" or some other crazy idea from an image detection bot.....

"It's a bird! It's a plane! No! It's MuppetBot!"

Telegram Object Detection Bot using Node-RED and TensorFlow Object Detection on macOS-img_9220jpg
This User Gave Thanks to Neo For This Post:
# 5  
Old 03-12-2020
Success!

After running for 4 days (LOL) and detecting the sun as "oven" and moving leaves on a bonsai tree as "airplane" and "keyboard", a small bird flew in and perched itself right on the balcony rail, and my MuppetBot detected "bird".... LOL

My wife was watching the coronavirus updates on the local news and excitedly yelled "A bird is in your camera view! Did you get it!"... hahah. I checked, and sure enough, "bird". LOL

Little MuppetBot is certainly far from perfect (I need to install a more robust (large, slower) TF object detection model, but have been busy on the test migration project....).

Check it out Smilie

Telegram Object Detection Bot using Node-RED and TensorFlow Object Detection on macOS-img_72895d753aba-1jpeg
This User Gave Thanks to Neo For This Post:
# 6  
Old 03-12-2020
FYI,

Here is the current way "MuppetBot" is situated....


Telegram Object Detection Bot using Node-RED and TensorFlow Object Detection on macOS-img_9227jpg
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Programming

Programming a Telegram Bot Using Node-RED, PHP, and MySQL

Yesterday, I announced the first release of a "Telegram Computer Trivia Bot" in this post: Play Computer Trivia on Telegram In this post, I will provide a high level overview of the application. The main components in this application are: Node-RED LAMP (Linux, Apache2, MySQL, PHP)... (7 Replies)
Discussion started by: Neo
7 Replies

2. What is on Your Mind?

MQTT, Node-RED, Linux, Apache2, MySQL, PHP, Telegram, ESP32, ESP8266, Arduino

I have just completed the first phase of integrating all these devices and technologies: MQTT, Node-RED, Linux, Apache2, MySQL, PHP, Telegram, ESP32, ESP8266, and the Arduino Uno The glue that binds all this together is MQTT. In fact, MQTT makes this kind of integration nearly trivial to... (1 Reply)
Discussion started by: Neo
1 Replies

3. What is on Your Mind?

Telegram Bots - Bot Code Examples

I'm currently looking into ways to integrate the Telegram API into the forums: Telegram Bots - Bot Code Examples I'm thinking, first off, to use the Telegram API to get forum alerts and notifications (to Bot or Not?). Second, I thinking of ways to more deeply integrate Telegram into the... (5 Replies)
Discussion started by: Neo
5 Replies

4. Programming

How to initialize an object with another object of different class?

How to initialize an object of class say "A", with an object of type say "B". The following code give the error message "error: conversion from âAâ to non-scalar type âBâ requested" #include <iostream> using namespace std; class B; class A{ public: A() { cout <<"\nA()" << endl; } ... (1 Reply)
Discussion started by: techmonk
1 Replies

5. Programming

Parallel Processing Detection and Program Return Value Detection

Hey, for the purpose of a research project I need to know if a specific type of parallel processing is being utilized by any user-run programs. Is there a way to detect whether a program either returns a value to another program at the end of execution, or just utilizes any form of parallel... (4 Replies)
Discussion started by: azar.zorn
4 Replies

6. UNIX for Dummies Questions & Answers

Object reference not set to an instance of an object

I am new to PHP and UNIX. I am using Apache to do my testing on a Windows Vista machine. I am getting this error when I am trying to connect to a web service. I did a search and did not see any posts that pertain to this. Here is my function: <?php function TRECSend($a, $b, $c, $d,... (0 Replies)
Discussion started by: EddiRae
0 Replies

7. IP Networking

modem detection

How to get information that where is my modem configured in /dev. I have two modems configured in my device .. one is USB and other is PCI modem.. USB is detected as /dev/USB0. but how to see about PCI modem? (0 Replies)
Discussion started by: s123.radha
0 Replies

8. UNIX for Dummies Questions & Answers

virus detection

IS there virus software for unix? I worked in a Solaris environment and dont remember having anything. I also ask because the current enviroment i am working on is Microsoft and they argue that they do not use unix because of virus detection. Any input would be greatly appreciated. (1 Reply)
Discussion started by: pbonilla
1 Replies
Login or Register to Ask a Question