Story image

How Sony is using distributed learning to create enterprise AI

19 Nov 2018

Sony announced that by utilising its deep learning development framework "Core Library: Neural Network Libraries" in addition to the AI Bridging Cloud Infrastructure (ABCI), a world-class computing infrastructure for AI processing, it has achieved the fastest deep learning speeds in the world.

Deep learning is a method of machine learning which uses neural networks modelled after the human brain. By harnessing deep learning, image and sound recognition capabilities have seen rapid growth in recent years, even outperforming humans in certain domains. 

However, the size of data used in this learning and model parameters used to improve recognition accuracy has been increasing, causing a subsequent rise in calculation times. 

In some cases, it has taken weeks or even months to conduct a single learning session. Because AI development requires a continuous process of trial-and-error, shortening this learning time is of the utmost importance. 

To this end, distributed learning using multiple GPUs as a means of shortening learning times is emerging as a popular solution.

When increasing the number of GPUs for distributed learning, there are cases where an increase to batch sizes (the amount of data to be processed at one time) halts the learning process and other cases where the learning speed actually decreases due to the processing delays caused by data transmission times between GPUs. 

By utilising technology that can determine the optimal batch sizes and the appropriate number of GPUs based on the current state of the learning process, Sony makes it possible to carry out learning even in large-scale GPU environments such as ABCI, and increased transmission speeds between GPUs through data synchronisation technology optimised for ABCI's system structure. 

These technologies were implemented into the "Neural Network Libraries," and used ABCI computing resources provided by AIST's "ABCI Grand Challenge" to carry out learning. 

As a result, it was able to complete ImageNet/ResNet-50*2 (the general industry benchmark used to measure distributed learning speeds for deep learning) in approximately 3.7 minutes (when using as many as 2,176 GPUs), achieving the world's fastest speeds to date. 

The results of this experiment demonstrate that learning/execution carried out using Neural Network Libraries can achieve world-class speeds and that by utilising the same framework, it is possible to conduct technology development using deep learning with a shorter trial-and-error period. 

Moving forward, Sony will continue development on related technologies and seek to contribute to the development of society using AI technology.

Dell EMC launches interactive AI Experience Zones
The AI Experience Zones are designed to educate visitors about how to start, identify, and implement an AI project.
What NZ can learn from the Baltimore cyberattack
“Businesses must control physical access to their computers and secure their networks."
Infratil seeks clearance to acquire up to 50% stake in Vodafone NZ
The commission will give clearance to a proposed merger if they are satisfied that the merger is unlikely to have the effect of substantially lessening competition in a market.
Hands-on review: MiniTool Power Data Recovery Software
I came across a wee gem of advice when researching the world of data recovery. As soon as you get that sinking feeling and realise you’ve lost a file, stop using your computer.
Deepfakes the 'next wave of concern' - but can law really stomp it out?
Enforcing the existing law will be difficult enough, and it is not clear that any new law would be able to do better. Overseas attempts to draft law for deepfakes have been seriously criticised.
Acquia delivers open source framework for contextual commerce
The framework connects the Drupal open source web content management system with e-commerce platforms from Acquia partners.
Are you all set to ride the new wave of technology disruption?
Why IT professionals are not immune to digital disruption.
Salesforce continues to stumble after critical outage
“To all of our Salesforce customers, please be aware that we are experiencing a major issue with our service and apologise for the impact it is having on you."