SLAMBox

logo

SLAMBOX is designed for use metod simultaneous localization and mapping (SLAM) in education, experiments, research and development by Node-based user interface. This is a box with tools that you can quickly and conveniently experiment with separate SLAM nodes.

Screenshot01 examples/slambox_base.json

[!NOTE]
You can watch demo via Vimeo link here: Demo video.

Introduction

In computing, a visual programming language (VPL) or block coding is a programming language that lets users create programs by manipulating program elements graphically rather than by specifying them textually. Visual programming allows programming with visual expressions, spatial arrangements of text and graphic symbols, used either as elements of syntax or secondary notation. For example, many VPLs (known as diagrammatic programming) are based on the idea of “boxes and arrows”, where boxes or other screen objects are treated as entities, connected by arrows, lines or arcs which represent relations.

The development of robotics generates a request for recognition and control systems for data received from sensory devices. At present, development of Computer Vision systems requires developers to have knowledge of programming languages and a deep understanding of mathematics. It was like the development of computer graphics: at the beginning, only scientists and researchers were engaged in computer graphics, later applied tools (Presented by such programs as Nuke, Houdini, Blender) were developed for use by less trained users. Over time, the development of computer vision systems should shift to the use of visual, graphical interfaces, such as Node-based UI, so that more ordinary users can access computer vision technologies.

The computer vision systems can be controlled not only by classical programming tools (write text code, which in itself narrows the scope of computer vision technologies), in the architecture of graph nodes it is possible to analyze and modify video streams, data from LIDAR, stereo cameras, acoustic sensors through visual programming, which expands the scope of technologies.

Simultaneous localization and mapping

SLAM is the computational problem of constructing or updating a map of an unknown environment while simultaneously keeping track of an agent’s location within it. While this initially appears to be a chicken or the egg problem, there are several algorithms known to solve it in, at least approximately, tractable time for certain environments. Popular approximate solution methods include the particle filter, extended Kalman filter, covariance intersection, and GraphSLAM. SLAM algorithms are based on concepts in computational geometry and computer vision, and are used in robot navigation, robotic mapping and odometry for virtual reality or augmented reality.


Blender Blender

Design and main components of SLAM pipeline

Feature-based visual SLAM typically tracks points of interest through successive camera frames to triangulate the 3D position of the camera, this information is then used to build a 3D map.

The basic graph for SLAM in SLAMBOX consists of the following nodes: Camera, DetectorDescriptor, MatchPoints, Triangulate, Open3DMap. There are also nodes for optimization and elimination of erroneous feature points: DNNMask, GeneralGraphOptimization, LineModelOptimization, KalmanFilterOptimization.

Camera

DetectorDescriptor

MatchPoints

Triangulate

Open3DMap

DNNMask

GeneralGraphOptimization


Screenshot03 examples/slambox_dnn.json

The following libraries are used in development



Screenshot04 examples/slambox_g2o.json

Dependent libraries


g2o framework for Python can also be build from source code, also add path to the compiled library in file config.py, see the g2opy_path variable.

SLAMBOX is distributed in the hope that it will be useful, but there is no guarantee that it will work perfectly. There are no warranty as to its quality or suitability for a particular purpose. Our primary development platform is Linux and Python 3.10 (Fedora Linux 36-39, Ubuntu 22). Has been tested on Mac OS X 10.15 (Only the AKAZE descriptor works; the ORB detector still works with errors.)

Installation and launch

Or you can specify a custom version of Python


Screenshot05 examples/slambox_base_flask.json

Launch Flask version


Custom node development

Node for server part (backend)

Create a class named SimpleNode, which will inherit the properties and methods from the RootNode class. Receives node parameters from the client side when the object is initialized.

Class Methods

Class attributes


"""Example of a simple node (backend)"""

import cv2
import numpy as np
from boxes import RootNode


class SimpleNode(RootNode):
    """A simple node that draws
    a circle with a specific color
    """

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.size = self.param["size"]
        self.color = self.color_reversed(self.param["picker"])

    def out_frame(self):
        frame = self.get_frame(0)
        if self.disabled:
            return frame
        height, width, channels = frame.shape
        cv2.circle(
            frame,
            (np.int32(width * 0.5), np.int32(height * 0.5)),
            self.size,
            self.color,
            -1,
        )
        return frame

    def update(self, param):
        self.disabled = param["disabled"]
        self.size = param["size"]
        self.color = self.color_reversed(param["picker"])

boxes/plugins/simple_node.py

Node for client part (frontend)

In the client part of the node, we describe the parameters that are passed to the server part. __identifier__ attribute indicates whether the node belongs to a type, in this case it is Draw. Also the color of the node type is set in set_color

"""Example of a simple node (frontend)"""

from Qt import QtCore, QtWidgets
from plugins_ui.main_gui_nodes import NodeColorStyle
from NodeGraphQt import BaseNode
from NodeGraphQt.constants import (
    NODE_PROP_QLABEL,
    NODE_PROP_COLORPICKER,
    NODE_PROP_SLIDER,
)

ncs = NodeColorStyle()
ncs.set_value(15)


class SimpleNode(BaseNode):
    __identifier__ = "nodes.Draw"
    NODE_NAME = "SimpleNode"

    def __init__(self):
        super().__init__()
        self.add_input("in", color=(180, 80, 180))
        self.add_output("out")
        self.create_property("label_color", "Color", widget_type=NODE_PROP_QLABEL)
        self.create_property("picker", (0, 255, 0), widget_type=NODE_PROP_COLORPICKER)
        self.create_property("label_size", "Size", widget_type=NODE_PROP_QLABEL)
        self.create_property("size", 100, range=(1, 300), widget_type=NODE_PROP_SLIDER)
        self.set_color(*ncs.Draw)

plugins_ui/simple_gui_node.py

Screenshot06 Example of a simple node working

Ready modules (nodes) must be placed in the following directories: plugins_ui/ for the client part, boxes/plugins/ for the server part. SLAMBOX will automatically upload them to existing nodes.


Based on twitchslam by geohot

:rocket:

This site is open source. Improve this page.