Fujitsu develops behavioural recognition tech, completes world first

Fujitsu Laboratories has developed technology designed to achieve the world's most accurate recognition of complex actions and behaviours based on skeleton data.

The technology utilises deep learning to recognise the positions and connections of adjacent joints in complex movements or behaviour in which multiple joints move in tandem.

This makes it possible to achieve greater accuracy in recognising, for instance, when a person performs a task like removing objects from a box.

This technology successfully achieved the world's highest accuracy against the world standard benchmark in the field of behaviour recognition, with significant gains over the results achieved using conventional technologies that don't make use of information on neighbouring joints.

By leveraging this technology to perform checks of manufacturing procedures or unsafe behaviour in public spaces, Fujitsu states it predominately aims to contribute to significant improvements in public safety and in the workplace.

The development of the technology

In general, human behaviour recognition utilising AI relies on temporal changes in the position of each of the skeletal joints, including in the hands, elbows, and shoulders, as identifying features, which are then linked to simple movement patterns such as standing or sitting.

With time series behaviour-recognition technology developed by Fujitsu Labs, Fujitsu has successfully realised highly-accurate image recognition using a deep learning model that can operate with high-accuracy even for complex behaviours in which multiple joints change in conjunction with each other.

The new technology is based on a AI model that can be trained in advance using the time series data of joints.

The connection strength (weight) with neighbouring joints can be optimised, and effective connection relationships for behaviour recognition can be acquired, Fujitsu states.

With conventional technologies, it was necessary to accurately grasp the individual characteristics of each joint. With an AI model that has already been trained, the combined features of the adjacent joints that are linked can be extracted, making it possible to achieve highly-accurate recognition for complex movements, according to the company.

A world first

This technology was evaluated against the world standard benchmark in the field of behaviour recognition using skeleton data, and in the case of simple behaviours such as standing and sitting in the open data set, the accuracy rate was maintained at the same level as that of conventional technology that does not use information on neighbouring joints.

In the case of complex behaviours like a person unpacking a box or throwing an object, however, the accuracy rate improved greatly, to achieve an overall improvement of more than 7% over the conventional alternative to reach the world's highest recognition accuracy.

Future plans

By adding the newly developed AI model for recognising complex behaviours obtained with this technology to the 100 basic behaviour already accommodated by Fujitsu's behavioural analysis technology "Actlyzer," it will become possible rapidly deploy new, highly-accurate recognition models, the company states.

Fujitsu ultimately aims to leverage this new capability to roll out the system in fiscal year 2021, and contribute to the resolution of real-world issues to deliver safer and more secure workplaces.