IT Brief New Zealand logo
Technology news for New Zealand's largest enterprises
Story image

NVIDIA has launched 8th generation AI software TensorRT 8

By Ryan Morris-Reade
Wed 21 Jul 2021

NVIDIA has launched the eighth generation of the company’s AI software, the TensorRT 8, which now does language queries at half the speed. 

TensorRT 8’s optimisations deliver record-setting speed for language applications, enabling developers to build high-performance search engines, ad recommendations, and chatbots from the cloud to the edge. 

The new software runs BERT Large, a widely used transformer-based model, in 1.2 milliseconds. NVIDIA says companies had to reduce the model size in the past, resulting in less accurate results. It says with TensorRT 8, companies can double or triple their model size to improve accuracy dramatically. 

“AI models are growing exponentially more complex, and worldwide demand is surging for real-time applications that use AI,” says NVIDIA VP of developer programs, Greg Estes.

“That makes it imperative for enterprises to deploy state-of-the-art inferencing solutions. The latest version of TensorRT introduces new capabilities that enable companies to deliver conversational AI applications to their customers with a level of quality and responsiveness never before possible.” 

According to NVIDIA, in five years, more than 350,000 developers across 27,500 companies in wide-ranging areas, including healthcare, automotive, finance and retail, have downloaded TensorRT nearly 2.5 million times. TensorRT applications can be deployed in hyperscale data centers, embedded, or automotive product platforms. 

In addition to transformer optimisations, TensorRT 8’s AI inference is made possible through two other key ways. 

  • Sparsity is a new performance technique in NVIDIA Ampere architecture GPUs to increase efficiency, allowing developers to accelerate their neural networks by reducing computational operations. 
  • Quantisation aware training enables developers to use trained models to run inference in INT8  precision without losing accuracy. This significantly reduces compute and storage overhead for efficient inference on Tensor Cores. 

Hugging Face is an open-source AI company that works with AI service providers across multiple industries. The company is working closely with NVIDIA to introduce new AI services to enable text analysis, neural search and conversational applications at scale. 

“We’re closely collaborating with NVIDIA to deliver the best possible performance for state-of-the-art models,” says Hugging Face product director, Jeff Boudier. 

“The Hugging Face Accelerated Inference API already delivers up to 100x speedup for transformer models powered by NVIDIA GPUs. With TensorRT 8, Hugging Face achieved 1ms inference latency on BERT, and we’re excited to offer this performance to our customers later this year.” 

GE Healthcare, a global medical technology, diagnostics and digital solutions company, uses TensorRT to help accelerate computer vision applications for ultrasounds, a critical tool for the early detection of diseases. This enables clinicians to deliver high-quality care through its healthcare solutions. 

“When it comes to ultrasound, clinicians spend valuable time selecting and measuring images,” says GE Healthcare chief engineer of Cardiovascular Ultrasound, Erik Steen.

“During the R&D project leading up to the Vivid Patient Care Elevated Release, we wanted to make the process more efficient by implementing automated cardiac view detection on our Vivid  E95 scanner. The cardiac view recognition algorithm selects appropriate images for analysis of cardiac wall motion. 

TensorRT, with its real-time inference capabilities, improves the performance of the view detection algorithm, and it also shortened our time to market during the R&D project,” he says.

TensorRT 8 is now generally available and free of charge to members of the NVIDIA Developer program. The latest versions of plug-ins, parsers and samples are also available open-source from the TensorRT GitHub repository.
 

Related stories
Top stories
Story image
Employment
Tech job moves - Forcepoint, Malwarebytes, SolarWinds & VMware
We round up all job appointments from May 13-20, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Migration
Let’s clear the cloud visibility haze with app awareness
Increasingly, organisations are heading for the cloud, initiating new born-in-the-cloud architectures and migrating existing applications via ‘lift and shift’ or refactoring.
Story image
Remote Working
Successful digital transformation in the hybrid work era is about embracing shifting goalposts
As organisations embraced remote working, many discovered they lacked the infrastructure needed to support history’s first global load test of remote work capabilities.
Story image
Vectra AI
Understanding the weight on security leader’s shoulders, and how to shift it
Millions of dollars of government funding and internal budgets are being funnelled into cybersecurity to build resilience against sophisticated threats, indicating how serious this issue has become.
Story image
Cybersecurity
NCSC advisory highlights poor security configurations
The GCSB's National Cyber Security Centre (NCSC) has released a cyber security advisory identifying commonly exploited controls and practices.
Story image
Qualys
Qualys updates Cloud Platform solution with rapid remediation
The new update is designed to enable organisations to fix asset misconfigurations, patch OS and third-party applications, and deploy custom software.
Story image
Lightspeed
Lightspeed launches all-in-one marketing platform in A/NZ
ECommerce provider, Lightspeed has launched a new all-in-one marketing solution, Lightspeed Marketing & Loyalty in Australia and New Zealand.
Story image
Amazon Web Services / AWS
RedShield leverages AWS to scale cybersecurity services
"Working with AWS gives RedShield the ability to mitigate significant application layer DDoS attacks, helping leaders adopt best practices and security architectures."
Story image
Microsoft
PwC NZ unveils new Cloud Security Operations Center
PwC New Zealand has unveiled its new Cloud Security Operations Center for the entire Microsoft technology stack.
Story image
Customer experience
Gartner recognises Okta for abilities in Access Management
Okta has announced it has been recognised as a Customers' Choice for the fourth time in a row in the Gartner Peer Insights "Voice of the Customer" report.
Story image
Digital Transformation
Trading up: It's time to swap core systems for flexible digital applications
This year will see more oranisations planning and commencing high tech renovations that will shake up the way they operate.
Story image
Infrastructure
Nutanix study reveals financial services sector lagging with multicloud adoption
Nutanix has released new research that reveals the financial services sector is lagging behind when it comes to multicloud adoption.
Story image
Ransomware
Cybersecurity starts with education
In 2021, 80% of Australian organisations responding to the Sophos State of Ransomware study reported being hit by ransomware. 
Story image
Digital Transformation
Pluralsight and Ingram Micro Cloud team up on cloud initiative
Pluralsight has teamed with Ingram Micro Cloud to build upon cloud competence and maturity internally, and externally support partners’ capabilities.
Story image
Excel
Could your Excel practices be harming your business?
While Excel has been the de-facto standard for budgeting, planning, and forecasting, is it alone, enough to support organisations in the global marketplace that’s facing rapid changes due to digital transformation?
Story image
Microsoft
New Relic enters multi-year partnership with Microsoft Azure
New Relic has announced a strategic partnership with Microsoft to help enterprises accelerate cloud migration and multi-cloud initiatives. 
Story image
Ransomware
Alarming surge in Conti Ransomware Group activity - report
A new report has identified a 7.6 per cent increase in the number of vulnerabilities tied to ransomware in Q1 2022.
Story image
Wireless
Cradlepoint expands its Cellular Intelligence capabilities
Cradlepoint has announced additional Cellular Intelligence capabilities with its NetCloud service.
Story image
Supply chain
Jetstack promotes better security with supply chain toolkit
The web-based resource is designed to help organisations evaluate and plan the crucial steps they need to establish effective software supply chain security.
Exabeam
Find out how a behavioural analytics-driven approach can transform security operations with the new Exabeam commissioned Forrester study.
Link image
Story image
Cybersecurity
A10 Networks finds over 15 million DDoS weapons in 2021
A10 Networks notes that in the 2H 2021 reporting period, its security research team tracked more than 15.4 million Distributed Denial-of-Service (DDoS) weapons.
Story image
Sisense
Data and analytics could be key to higher selling prices in APAC
Sisense's latest report has found that almost half of data professionals in APAC think customised data and analytics can create better selling prices for their products.
Story image
Sustainability
Power at the edge: the role of data centers in sustainability
The Singaporean moratorium on new data center projects was recently lifted, with one of the conditions being an increased focus on power efficiency and sustainability.
Story image
Apricorn
Data backup plans inadequate, data still at risk - study
The Apricorn 2022 Global IT Security Survey revealed that while the majority organisations have data backup plans in place, data for many are at risk.
Booster
Booster Innovation Fund. A fund of Kiwi ingenuity – for Kiwi investors.
Link image
Story image
Malware
New vulnerabilities found in Nuspire’s Q1 2022 Threat Report
“Threat actors are quickly adjusting their tactics and these exploits tend to get industry attention, but the threat posed by older and attacks still persists."
Story image
Data Center
Preventing downtime costs and damage with Distributed Infrastructure Management
Distributed Infrastructure Management (DIM) can often be a lifeline for many enterprises that work with highly critical ICT infrastructure and power sources.
Story image
Sift
Sift shares crucial advice for preventing serious ATO breaches
Are you or your business struggling with Account Takeover Fraud (ATO)? One of the latest ebooks from Sift can provide readers with the tools and expertise to help launch them into the new era of account security.
Softiron
For every 10PB of storage run on HyperDrive vs. comparable alternatives, an estimated 6,656 tonnes of CO₂ are saved by reduced energy consumption alone over its lifespan. That’s the equivalent of taking nearly 1,500 cars off the road for a year.
Link image
Story image
ChildFund
ChildFund launches new campaign to protect children online
ChildFund says WEB Safe & Wise aims to protect children from sexual exploitation and abuse online while also empowering them to become digitally savvy. 
Story image
Artificial Intelligence
How to ensure ethical deployment of AI implementations
The increase in automation and machine technology such as AI and machine learning has unlocked a whole new level of scale and service to organisations. 
Story image
Nozomi Networks
Nozomi Networks, Siemens reveal software integration
Nozomi Networks and Siemens have extended their partnership by embedding Nozomi Networks’ software into the Siemens Scalance LPE local processing engine.
Story image
Vodafone
Vodafone NZ buys remaining stake in retail joint venture
Vodafone New Zealand has purchased the remaining 50% stake in the specialist joint venture (JV) with private equity company Millennium Corp.
Story image
Cybersecurity
More than 40% of banks worried about cloud security - report
Publicis Sapient's new report finds security and the lack of cloud skills and internal understanding of business benefits are big obstacles for banks moving to the cloud.
Darktrace
Threat actors are exploiting weaknesses in interconnected IT/OT ecosystems. Darktrace illuminates your entire business and takes targeted action to stop emerging attacks.
Link image
Story image
Fonterra
Fonterra to use automated guided vehicles by Dematic
New Zealand's dairy giant Fonterra is upgrading its manufacturing facility in Edendale with new automated guided vehicles (AGVs) from Dematic.
Story image
Digital Transformation
How to modernise legacy apps without compromising security
At a time when digital transformation has become central to business, even the most important applications come with a ‘use-by’ date.
Story image
Sustainability
Grasping the opportunity to rethink the metrics of a sustainable data centre
A data centre traditionally has two distinct operations teams: the Facility Operations team, and the IT Operations team. Collaboration between them is the key to defining, measuring, and delivering long-term efficiency and sustainability improvements.
Story image
Artificial Intelligence
SAS announces new products amid cloud portfolio success
Analytics and AI company SAS is deepening its broad industry portfolio with offerings that support life sciences, energy, and martech.
Story image
Application Security
What are the DDoS attack trend predictions for 2022?
Mitigation and recovery are vital to ensuring brand reputation remains solid in the face of a Distributed Denial of Service (DDoS) attack and that business growth and innovation can continue.
Story image
Data Protection
Barracuda launches new capabilities for API Protection
"Every business needs this type of critical protection against API vulnerabilities and automated bot attacks," Barracuda says.
Story image
Sustainability
Legrand unveils Nexpand, a data center cabinet platform
Legrand has unveiled a new data center cabinet platform, Nexpand, to offer the necessary scalability and future-proof architecture for digital transformation.
Story image
Ransomware
A third of companies paying ransom don’t recover data - report
Veeam's report finds 76% of businesses who are victims of cyberattacks paid the ransom to recover data, but a third were still unable to get their information back.
Story image
Apple
Apple previews new features for users with disabilities
Apple says new software features that offer users with disabilities new tools for navigation, health and communication, are set to come out later this year.