IT Brief New Zealand logo
Technology news for New Zealand's largest enterprises
Story image

Why AI algorithms are only as good as the data they’re fed - Snowflake

By Contributor
Fri 5 Jul 2019
FYI, this story is more than a year old

Article by Snowflake Asia Pacific and Japan vice president Peter O’Connor

Artificial intelligence (AI) has shifted from being a computing novelty into mainstream use much faster than many people expected.

Once the domain of researchers and computer scientists, it’s now powering everything from personal assistants to aircraft and business accounting systems.

However, it must be remembered that AI systems are only as good as the data being fed into them.

If this data is of poor quality or contains bias, this will have a significant impact on subsequent decisions made by the software.

There’s no question that AI has the potential to address some of society’s biggest problems. Yet, time after time, AI-powered projects have failed the ‘fairness’ test as algorithms have amplified gender, ethnic or social biases that lurk within the data sets being used.

Guiding models

For this reason, it’s vital that organisations collating large data sets develop robust models and ethics that encourage transparency, equality, and trust.

This is fundamental to the task of delivering AI systems that produce fair and balanced outcomes.

If these models and guidelines are not in place, biased data sets will never be challenged.

A particular data set might be tainted by the conscious or unconscious biases of the people who collated it, but we would never know.

There is also a tendency to overestimate the limits of the current generation of AI tools.

There have been cases where car drivers have been lulled into a false sense of security by vehicle assistance systems and fail to take back control when something unexpected happens.  

Usage of AI tools in the legal sector is also causing some issues.

A number of judges now seek recommendations from an AI tool before ruling in cases involving things such as bail or parole.

Although AI-provided results are supposed to be only recommendations, it can be tempting to take them as a definitive result.

In the wider community, many people are becoming increasingly uncomfortable with AI and the rate at which it is being developed and adopted.

High-profile businessman Elon Musk gained headlines when he described the technology as society’s “biggest existential threat”.
Indeed, in a recent consumer sentiment survey commissioned by Snowflake, 62% of respondents said AI tools should not be used to make decisions in areas such as criminal justice and healthcare.

Overcoming the challenge

For AI to become a trusted and valuable contributor to society, the organisations making use of it must focus on fairness and transparency from the outset.

Any AI algorithms that can’t be clearly explained and independently assessed should not be trusted with critical decisions.

It might take some extra effort, but tools that give poor results should be tuned to remove any inherent bias.

For example, a facial recognition program can be further trained so that it gives equal weight to both sexes and all skin colours.

Of greatest importance, however, is that organisations focus attention on the data being used. They must have clear guidelines in place about how this data is collected and how it is put to use.

There are two key areas in which attention needs to be focused.

Firstly, all organisations need to have a stated data governance policy that dictates how data is handled.

This is particularly important in an era when cloud-based storage is increasingly used and data sets are regularly combined in different ways.

Second, the workings of the AI tools being used need to be able to be explained.

While machine learning algorithms such as deep learning neural networks might be very powerful, any lack of transparency or understanding may cause regulatory and ethical risks.

To achieve this, it’s worth considering partnering with risk and compliance professionals early in the tool development process.

It’s clear that AI is becoming an increasingly powerful force in many areas of modern society, and its development is happening at an exponential rate.

Any failure to get the required approaches and frameworks in place now could risk unforeseen outcomes in the future.

By taking the time to understand the importance of both data quality and the transparency of algorithms, society will be best placed to reap the significant benefits that AI can deliver.

Related stories
Top stories
Story image
Migration
SNP unveils next generation of CrystalBridge software platform
Data is a key pillar of every customer-centric organisation, as it relies on agile decisions to become increasingly sustainable and intelligent.
Story image
Cybersecurity
FIDO Alliance releases guidelines for optimising UX with FIDO Security Keys
The new guidelines aim to accelerate multi-factor authentication deployment and adoption with FIDO security keys.
Story image
MarTech
Martech experts reveal the “buzz” on personalisation
In the digital age, innovative technology must be leveraged to power an efficient and effective relationship marketing strategy.
Story image
Robotic Process Automation / RPA
Salesforce announces latest generation of MuleSoft
Salesforce has introduced the next generation of MuleSoft, a unified solution for automation, integration and APIs to automate any workflow.
Story image
Hybrid workforce
How organisations can prepare for a post-pandemic workforce
The so-called 'new normal' office looks different to how it did pre-pandemic, and organisations need to take steps to better manage their post-pandemic workforce. 
Story image
Microsoft
TO THE NEW unveils A/NZ Managed Services for Microsoft Azure
TO THE NEW has released Managed Services for Microsoft Azure to meet the growing demand in the A/NZ market and globally.
Story image
Infrastructure
Global investment in data centers more than doubled in 2021
DLA Piper's latest global survey finds the total investment in data center infrastructure worldwide rose from USD $24.4 billion in 2020 to USD $53.8 billion in 2021.
Story image
Internet of Things
Global 5G subscriptions to top one billion by the end of 2022
Global 5G subscriptions are predicted to pass the one billion milestone by the end of 2022, according to a new report.
Story image
Cryptocurrency
NOWPayments launches new service to analyse cryptocurrency fees
NOWPayments has launched a new network fee optimisation solution that analyses current network fees and picks the most profitable option out of the client's payout wallets.
Story image
Web Development
Whitecliffe fosters careers for the future of tech
Do you want a career in Information Technology, Networking, Web Development, Software Development, or are you looking to upskill?
Story image
Metaverse
How the metaverse will change the future of the supply chain
The metaverse is set to significantly change the way we live and work, so what problems can it solve in supply chain management?
Story image
Internet of Things
Domino's Pizza: A blueprint for secure enterprise IoT deployment
Increasingly, organisations are embracing smart technologies to underpin innovations that can enhance safety and productivity in every part of our lives, from industrial systems, utilities, and building management to various forms of business enablement.
Story image
Public Cloud
Public cloud services revenues top $400 billion in 2021
"For the next several years, leading cloud providers will play a critical role in helping enterprises navigate the current storms of disruption."
Story image
Cybersecurity
Video: 10 Minute IT Jams - An update from CrowdStrike
Scott Jarkoff joins us today to discuss current trends in the cyber threat landscape, and the reporting work CrowdStrike is doing to prevent further cyber harm.
Story image
Artificial Intelligence
Accenture shares the benefits of supply chain visibility
It's clear that gaining better visibility into the supply chain will help organisations avoid excess costs, inefficiencies, and complexity to ultimately improve their bottom line.
Story image
Data ownership
Brands must reclaim trust by empowering data ownership
According to Twilio's new State of Personalisation Report 2022, 62% of consumers expect personalisation from brands, and yet only 40% trust brands to use their data responsibly and keep it safe.
Story image
Dark web
Cybercrime in Aotearoa: How does New Zealand law define it?
‘Cybercrime’ is a term we hear all the time, but what exactly is it, and how does New Zealand define it in legal terms?
Story image
BitTitan
How to achieve your monthly recurring revenue goals
Monthly recurring revenue (MRR) is the ultimate goal, the most important issue on which anyone in the IT channel should focus.
Story image
Oracle Cloud
Commvault, Oracle to deliver Metallic Data Management as a Service
"We are excited to partner with Commvault and enable our customers to restore and recover their most mission-critical cloud data."
Story image
Vendor
Forescout reveals top vulnerabilities impacting OT vendors
Forescout’s Vedere Labs has disclosed OT: ICEFALL, naming 56 vulnerabilities affecting devices from 10 operational technology vendors.
Story image
Entelar
How TruSens air purifiers can create healthier workspaces
The pandemic has heightened our awareness of our own and others’ health, and made us all much more conscious of the environments we work in.
Story image
Hybrid Cloud
HPE GreenLake advances hybrid cloud experience with new services
"The innovations unveiled today further build on our vision to provide the market with an unmatched platform to spur innovation and drive transformation.”
Story image
Data
MYOB improves data visibility and user access with Snowflake
"Solutions such as Snowflake allow us to better understand our customers and make evidence-based decisions on what features work best for them."
Story image
Hybrid workforce
Why hybrid working is here to stay and how to ace it
Citrix's new report reveals hybrid workers are more productive and engaged at work than their office and completely remote counterparts.
Story image
Cybersecurity
Unknown connections: How safe is public WiFi in Aotearoa?
If it's not your own household WiFi, then who has control of your data and is your connection actually safe?
Story image
Sustainability
Vertiv releases updates on ESG initiatives, sets sights on future
Vertiv has released its inaugural environmental, social and governance (ESG) report, the company’s first public report of its ESG activities.
Story image
Cybersecurity
Email threats spike 101%, remains a top attack vector
"Each year we see innovation in the threat landscape, but each year email remains a major threat to organisations."
Story image
Great Resignation
New SAP study uncovers impact of 'the great resignation'
Coined in 2021, the phrase 'the great resignation' refers to millions of employees globally leaving their jobs. The phenomenon is real and impacting SMEs.
Story image
Citrix
The best ways to attract young talent during labour shortages
New research from Citrix reveals hybrid working and ventures into the metaverse are top of mind for Gen Z workers.
Story image
Cloudera
Overcoming hybrid and multi-cloud challenges to drive innovation
Driven by improvements in technology, financial services companies have advanced both internal and external systems and processes, with the likes of digitisation, personalisation and risk management redefining the industry.
Story image
Airwallex
How Airwallex helps businesses achieve globalisation success
As markets continue to shift, businesses need to be able to provide the same quality of service for customers regardless of where they are located around the world.
Story image
Revenue
Datacom announces revenue of $1.45 billion, fall in profit
Growing market pressures and border closures saw Datacom place increased focus on talent development initiatives for both existing and future employees.
Story image
Cybersecurity
Vulnerable APIs costing businesses billions every year
Large companies are particularly vulnerable to the security risks associated with exposed or unprotected APIs as they accelerate digital transformation.  
Story image
API
Industry-first comprehensive risk-based API security enhances protection
Application Programming Interfaces (APIs) have become a crucial part of operating web and mobile application businesses and are causing significant economic growth in the digital sector.
Story image
Talend
Forrester names Talend Leader in enterprise data fabric
Forrester has named Talend a leader among enterprise data fabric providers in the Forrester Wave: Enterprise Data Fabric, Q2 2022 report.
Story image
Cybersecurity
Why is NZ lagging behind the world in cybersecurity?
A recent report by TUANZ has revealed that we are ranked 56th in the world when it comes to cybersecurity - a look into why we're so behind and what needs to be done.
Story image
Phishing
Online identity theft is rising in NZ - here’s what to do about it
It may start with a few stolen details online, but it could end with thousands of dollars missing or worse, a reputation down the drain.
Story image
Digital Transformation
Stax and Consegna partner to accelerate modernisation
According to a statement, the new alliance will help both companies expand their reach across the region and realise joint goals.
Story image
New Relic
How to tackle the great brain drain in the tech industry
Attracting and retaining tech talent in Australia and New Zealand is becoming increasingly challenging, with the 2022 Hays Salary Guide showing a startling 91% of employers facing a skills shortage.
Story image
Sustainability
Honeywell launches new carbon energy management software for buildings
The new Carbon & Energy Management service allows building owners to track and optimise energy performance against carbon reduction goals, down to a device or asset level.
Story image
Trend Micro
5G network projects driven by improving security and privacy
Trend Micro's new study reveals the prospect of improved security and privacy capabilities are the main motivations behind private 5G wireless network projects.
Story image
Finance
Airwallex launches global payment services in New Zealand
The launch will enable businesses in New Zealand to tap into Airwallex's global payments services, offering an alternative to traditional banks.
Story image
Samsung
Monitors are an excellent incentive for getting employees back
The pandemic has taught us that hybrid working is a lot easier than we would’ve thought, so how can the office be made to feel as comfortable as home? The answer could be staring you in the face right now.
Story image
eCommerce
Robust digital warehouse management crucial in Asia-Pacific
Thanks to a network of “cloud” stores, grocery and food delivery providers such as Foodpanda can arrange for these commonly requested items to get packed up and sent over in almost no time.