SLAMBOX is designed for use metod simultaneous localization and mapping (SLAM) in education, experiments, research and development by Node-based user interface. This is a box with tools that you can quickly and conveniently experiment with separate SLAM nodes.
[!NOTE]
You can watch demo via Vimeo link here: Demo video.
In computing, a visual programming language (VPL) or block coding is a programming language that lets users create programs by manipulating program elements graphically rather than by specifying them textually. Visual programming allows programming with visual expressions, spatial arrangements of text and graphic symbols, used either as elements of syntax or secondary notation. For example, many VPLs (known as diagrammatic programming) are based on the idea of “boxes and arrows”, where boxes or other screen objects are treated as entities, connected by arrows, lines or arcs which represent relations.
The development of robotics generates a request for recognition and control systems for data received from sensory devices. At present, development of Computer Vision systems requires developers to have knowledge of programming languages and a deep understanding of mathematics. It was like the development of computer graphics: at the beginning, only scientists and researchers were engaged in computer graphics, later applied tools (Presented by such programs as Nuke, Houdini, Blender) were developed for use by less trained users. Over time, the development of computer vision systems should shift to the use of visual, graphical interfaces, such as Node-based UI, so that more ordinary users can access computer vision technologies.
The computer vision systems can be controlled not only by classical programming tools (write text code, which in itself narrows the scope of computer vision technologies), in the architecture of graph nodes it is possible to analyze and modify video streams, data from LIDAR, stereo cameras, acoustic sensors through visual programming, which expands the scope of technologies.
SLAM is the computational problem of constructing or updating a map of an unknown environment while simultaneously keeping track of an agent’s location within it. While this initially appears to be a chicken or the egg problem, there are several algorithms known to solve it in, at least approximately, tractable time for certain environments. Popular approximate solution methods include the particle filter, extended Kalman filter, covariance intersection, and GraphSLAM. SLAM algorithms are based on concepts in computational geometry and computer vision, and are used in robot navigation, robotic mapping and odometry for virtual reality or augmented reality.
Feature-based visual SLAM typically tracks points of interest through successive camera frames to triangulate the 3D position of the camera, this information is then used to build a 3D map.
The basic graph for SLAM in SLAMBOX consists of the following nodes: Camera, DetectorDescriptor, MatchPoints, Triangulate, Open3DMap. There are also nodes for optimization and elimination of erroneous feature points: DNNMask, GeneralGraphOptimization, LineModelOptimization, KalmanFilterOptimization.
OpenCV (Open Source Computer Vision Library) is a library of programming functions mainly aimed at real-time computer vision. Originally developed by Intel, it was later supported by Willow Garage then Itseez (which was later acquired by Intel). The library is cross-platform and free for use under the open-source Apache 2 License. Starting with 2011, OpenCV features GPU acceleration for real-time operations.
NumPy is a library for the Python programming language, adding support for large, multi-dimensional arrays and matrices, along with a large collection of high-level mathematical functions to operate on these arrays.
g2o is an open-source C++ framework for optimizing graph-based nonlinear error functions. g2o has been designed to be easily extensible to a wide range of problems and a new problem typically can be specified in a few lines of code. The current implementation provides solutions to several variants of SLAM and BA.
scikit-image Image processing in Python is a collection of algorithms for image processing.
SciPy (pronounced “Sigh Pie”) is an open-source software for mathematics, science, and engineering.
Open3D is an open-source library that supports rapid development of software that deals with 3D data. The Open3D frontend exposes a set of carefully selected data structures and algorithms in both C++ and Python.
Qt is cross-platform software for creating graphical user interfaces as well as cross-platform applications that run on various software and hardware platforms such as Linux, Windows, macOS, Android or embedded systems with little or no change in the underlying codebase while still being a native application with native capabilities and speed.
NodeGraphQt a node graph UI framework written in python that can be implemented and re-purposed into applications supporting PySide2.
FFmpeg is a free and open-source software project consisting of a suite of libraries and programs for handling video, audio, and other multimedia files and streams.
Fedora
cd ~/your_fav_code_directory
git clone https://github.com/shrimo/SLAMBox.git
cd SLAMBox
pip install -r requirements.txt
dnf install ffmpeg
Ubuntu
...
sudo apt update
sudo apt install ffmpeg
[!NOTE]
Please install latest opencv-pythonpython3 -m pip install --upgrade opencv-python
g2o framework for Python can also be build from source code, also add path to the compiled library in file config.py, see the g2opy_path variable.
SLAMBOX is distributed in the hope that it will be useful, but there is no guarantee that it will work perfectly. There are no warranty as to its quality or suitability for a particular purpose. Our primary development platform is Linux and Python 3.10 (Fedora Linux 36-39, Ubuntu 22). Has been tested on Mac OS X 10.15 (Only the AKAZE descriptor works; the ORB detector still works with errors.)
cd SLAMBox
bash slambox.sh
Or you can specify a custom version of Python
bash slambox.sh 3.10
examples/slambox_base_flask.json
python build_graph.py FlaskMS
python node_graph.py WebStreaming
Create a class named SimpleNode, which will inherit the properties and methods from the RootNode class. Receives node parameters from the client side when the object is initialized.
"""Example of a simple node (backend)"""
import cv2
import numpy as np
from boxes import RootNode
class SimpleNode(RootNode):
"""A simple node that draws
a circle with a specific color
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.size = self.param["size"]
self.color = self.color_reversed(self.param["picker"])
def out_frame(self):
frame = self.get_frame(0)
if self.disabled:
return frame
height, width, channels = frame.shape
cv2.circle(
frame,
(np.int32(width * 0.5), np.int32(height * 0.5)),
self.size,
self.color,
-1,
)
return frame
def update(self, param):
self.disabled = param["disabled"]
self.size = param["size"]
self.color = self.color_reversed(param["picker"])
In the client part of the node, we describe the parameters that are passed to the server part. __identifier__
attribute indicates whether the node belongs to a type, in this case it is Draw. Also the color of the node type is set in set_color
"""Example of a simple node (frontend)"""
from Qt import QtCore, QtWidgets
from plugins_ui.main_gui_nodes import NodeColorStyle
from NodeGraphQt import BaseNode
from NodeGraphQt.constants import (
NODE_PROP_QLABEL,
NODE_PROP_COLORPICKER,
NODE_PROP_SLIDER,
)
ncs = NodeColorStyle()
ncs.set_value(15)
class SimpleNode(BaseNode):
__identifier__ = "nodes.Draw"
NODE_NAME = "SimpleNode"
def __init__(self):
super().__init__()
self.add_input("in", color=(180, 80, 180))
self.add_output("out")
self.create_property("label_color", "Color", widget_type=NODE_PROP_QLABEL)
self.create_property("picker", (0, 255, 0), widget_type=NODE_PROP_COLORPICKER)
self.create_property("label_size", "Size", widget_type=NODE_PROP_QLABEL)
self.create_property("size", 100, range=(1, 300), widget_type=NODE_PROP_SLIDER)
self.set_color(*ncs.Draw)
Example of a simple node working
Ready modules (nodes) must be placed in the following directories: plugins_ui/ for the client part, boxes/plugins/ for the server part. SLAMBOX will automatically upload them to existing nodes.
:rocket: