SE4 Inc is introducing a robot operating system that combines virtual reality (VR), Computer Vision (CV) and artificial intelligence (AI) software to enable highly-accelerated remote control of robots.
The system also offers adaptive operational capability for dynamically changing environments at any distance.
Although the tech is designed for robotics in space, the company sees suitable applications here on earth too.
SE4 is initially targeting repetitive tasks such as excavation, with a view towards deployment of robots in outer space, as their software is able to sidestep the speed of light constraints that cause latency issues.
SE4 in construction
The software is targeted at both new and existing robots equipped with the necessary sensor technology and is designed to make robotics more useful in a wider variety of settings, including areas of danger, high latency or inhospitable environments.
SE4 will supply its software to companies engaged in repetitive work such as construction, where autonomous excavators can be told — via a virtual environment —which sector to excavate, to what depth, and where to put the excavated earth.
Once the robot has received its instructions, the operator can walk away while the system uses computer vision, machine learning and a semantic structure to adapt to environmental changes. For example, if the excavator were to come across a boulder, it might contact the operator for additional guidance. So far, SE4 has tested its tech on a scale model of an excavator.
“We are in discussions to test it on real systems within the year,” said Lochlainn Wilson, CEO of SE4. “We are in discussions with a large Japanese equipment manufacturer, among others.”
When SE4 is ready to equip excavators with its system, Wilson explained there won’t be a one-size-fits-all solution. However, OEMs are beginning to include Application Programming Interfaces (APIs) into machinery.
“Currently they are not open APIs, but a time may come where we can use manufacturer-provided interfaces without any retrofits,” Wilson said.
“We are not at this stage considering providing a retrofit service. We aim to make any system that meets our control requirements easy to integrate with our software, however.”
Sematic Control
SE4’s tech, called Semantic Control, eliminates the need for complex programming steps in controlling robots – such as which joint to move and where – and removes the need for the involvement of a vast range of programming teams for installation and continued operation.
Instead, operators use VR in a safe, simulated 3D environment, where they can annotate objects and perform actions on them. Then, AI interprets the actions, and the tasks are organized and sent to the robot as a series of tasks, like a to-do list. At that point, the robot itself can determine the best way, and order, to accomplish each task using local information, adapting intelligently to changes in its workspace.
“The majority of robot teaching programs use 2D methods – such as a touchscreen pendant – to interact with 3D environments,” said Pavel Savkin, chief technology officer at SE4 Inc.
“We control objects in 3D space using VR, which is far more efficient and eliminates the learning curve. It also increases robot and operational safety because tasks are sanity checked first by the simulator. They have to make physical sense before they can be executed.”
Operational flexibility
SE4’s approach manages the significant problem of latency in the instruction process. By uploading a directive to the robot and providing it with operational flexibility, there is no need for a stream of constant instructions and the resulting delays which in outer space can take an average 12.5 minutes one way from Mars to Earth, making direct and real-time control impossible.
“Imagine a robotic construction project on Mars right now – it is like King George telling the colonists in the New World how to build a single structure with a one-month sea delay each way. That’s the status of where we are now,” Savkin said.
“SE4’s solution delivers instructions as a to-do list, with dependencies included for flexibility in execution, so that latency becomes manageable. A general directive can go out to a single robot, or to many devices for simultaneous control, using command queueing and automated delegation.”
Who is SE4?
Based in Tokyo, SE4 is a robotics company specializing in remote robotics using VR simulation to enable remote operation at vast distances, as well as nearby. SE4’s vision is to effectively enable an off-world robot-driven industrial revolution to increase the prosperity and long-term survivability of humanity.
The company is part of the NVIDIA Inception accelerator, which supports AI start-ups with tools and expertise.
