IT Brief New Zealand logo
Technology news for New Zealand's largest enterprises
Partner content
Story image

The deepfake dilemma: How it affects privacy, security & law in Aotearoa

By Sara Barker
Wed 17 Nov 2021

On a YouTube channel called Genuine Fake, a video shows Prime Minister Jacinda Ardern as the character of Maleficent. Her husband Clarke Gayford then appears shortly afterwards. Even National Party leader Judith Collins looks a bit like a forest fairy princess. 

The faces are eerily lifelike, but they're not quite right - they're too smooth, and the eyes barely blink. Also, the voices are wrong - Jacinda Ardern is not a Hollywood actress (as far as we know), and anyone who has seen the film will know that actress Angelina Jolie actually played the lead role of Maleficent. The editing is a bit choppy, but this video is meant for entertainment, not big-budget movies. A quick browse through the same YouTube channel reveals more than 50 other videos making fun of various local and international public figures. Welcome to the world of deepfakes.

'Deepfake' is a portmanteau of two words: 'Deep learning' and 'fake'. A deepfake is generally described as a video clip or image in which a person's face has been replaced with someone else's. The replacement face is usually replaced by a combination of technology and artificial intelligence-based deep learning.  Deepfakes are used for 'entertainment' or politics, such as putting words into the mouths of people like Mark Zuckerberg or Barack Obama.

However, deepfakes are not to be confused with CGI. There is also the term "uncanny valley", which has been around since the 1970s and describes the feeling we get when we see and hear digital characters that look uncomfortably human.

For better or worse, deepfakes are the next step in the evolution of image manipulation - photographers have been editing images for years, from the days of plates to the dawn of digital tools like Photoshop and now, video editing software powered by deep learning. 

The 'deep learning' part of deepfakes can be slightly misleading if you think of them like videos that take massive amounts of processing power to replace things frame by frame. However, much of it is cloud-based, and deepfakes don't even take much skill to create - there are now face swap apps available that can swap out faces at the touch of a button. All of these work through a simple smartphone. 

While it's entertaining to see New Zealand's public figures in entertaining or downright bizarre situations, these deepfakes not only raise questions about morality, legality, privacy, and online harm.

Could someone go to the effort of creating a digital version of your CEO - and replicating their voice - to declare an untrue statement like the company is going into liquidation? Or what if someone created a video of someone committing a crime, only to swap out the offender's face with yours?  During what some call the misinformation age, it could easily be taken as truth - with catastrophic consequences.

Last year Microsoft launched a Video Authenticator tool to detect whether an image or video has been created with AI. Google has also been involved in similar projects, such as the FaceForensics benchmark. These efforts are designed to crack down on deepfakes and separate the 'real' from the 'manipulated' or 'made up'.  But what's the big fuss?

Up for debate: Morality, privacy, and online harm 

Deepfakes are not all about entertainment. There is a nasty side. Netsafe's chief Martin Cocker says deepfakes, like most technologies, exist in a grey area.

"Usually the benefits from new technologies outweigh the harms. It's hard to say that about deepfakes." 

While deepfakes are still reasonably rare, he notes that more occurrences of what he calls 'cheap fakes' can often be faces added to adult images or videos.

Co-author of the Perception inception: Preparing for deepfakes and the synthetic media of tomorrow report, Curtis Barnes says, "Unfortunately, it's just too easy to misuse or abuse synthetic media (including deepfakes), and it's both technically challenging to prevent or mitigate.

"The combination of synthetic media and the web as a platform makes it possible for more people to produce media that makes it look or sound like something happened when it didn't, then share it quickly to other people. It is easy to see the kinds of harm that might occur if this is done maliciously or ignorantly."

Deepfakes: Legal - and legally ambiguous

New Zealand does not have any specific rules or regulations that cover deepfakes. Still, tangential laws such as the Privacy Act, Films, Videos and Publications Classification Act, the Copyright Act, the Human Rights Act, and the Harmful Digital Communications Act offer some protection.

In 2019, the Law Foundation backed a research report into synthetic media, including deepfakes. Report co-authors Curtis Barnes and Tom Barraclough explored how New Zealand law could deal with the existence of created and manipulated forms of media.

In the report, Tom Barraclough noted, "Enforcing the existing law will be difficult enough, and it is not clear that any new law would be able to do better. Overseas attempts to draft law for deepfakes have been seriously criticised."

"It is completely legitimate to call for regulatory intervention. But the merits of any course of action cannot be assessed without specifics. What exactly is being proposed? In the case of harmful synthetic media, even if we all agreed we should ban it or regulate it, how could we realistically do that? What exactly are we looking to prevent?"

When we spoke to Curtis Barnes this year for an update, he mentioned there is a glaring vaguery in the current law, particularly around synthetic media and sexual image abuse.

"For New Zealand, the key policy question is whether this kind of sexual synthetic media is (or should be treated as) an "intimate visual recording" for the purposes of section 216G of the Crimes Act. 

"Much turns on the intended purpose of the existing provision, as well as whether the harms of misusing an actual intimate visual recording of a person are the same as a sexualised but 'fake' representation of them. I think there are several differences between the two phenomena. Nonetheless, sexual synthetic media abuses are still capable of causing kinds of harm that the law should seek to redress and prevent. As such, I think it would be sensible to account for them somewhere else, probably in the Crimes Act. 

Barnes adds, "More important than what I think is the matter of what Parliament thinks, and at the moment they have chosen not to seriously debate the topic. They may soon, as Louisa Wall's private members Bill on revenge pornography has several overlaps. Until Parliament debates the issue and decides one way or another, the legal questions around the status of sexually abusive synthetic media remain an unresolved question in New Zealand law."

Could deepfakes be the next frontier for social engineering and malware?

Security firm Malwarebytes stated in a blog earlier this year that deepfakes could end up taking centre stage as bait for ransomware attacks. Whilst somewhat alarmist, it does acknowledge the dangers that deepfakes present.

"A threat actor scrapes videos and voice samples of their target from publicly-available websites to create a deepfake video—but sprinkling in certain elements inspired from ransomware, such as a countdown timer that lasts for 24-48 hours.

"Deepfake ransomware could also happen this way: A threat actor creates deepfake video of their target. Takes screenshots of this video and, pretending to be a legitimate contact of their target, sends them the screenshots and a link to the supposed video that they can watch themselves if they are in doubt."

However, Curtis Barnes says he is not convinced that synthetic media like deepfakes pose security risks, but it is easy to speculate about how they could be used.

"Most scenarios are already possible without the use of synthetic media. For this reason, most businesses and organisations have already developed systems of verification and trust to avoid being duped. However, where businesses haven't developed these systems, I see no reason to believe that they won't adapt quickly to new threats as they arise - they always do. 

"It is now several years since the emergence of this technology and there are very few cases where it is clear that synthetic media has been used to commit a crime."

Barnes has a point - deepfake attacks are rare, although they have garnered the interest of various security firms and media.

Take business email compromise (BEC) scams, for example. These are ways in which attackers either hijack an executive's email account or pose as the executive. For example, one form of a BEC scam involves a request for a money transfer or invoice, which looks like it's from an executive. Unbeknownst to the person who initiatives the transfer, the request is fake, and the money ends up in a scammer's bank account.

Traditionally these relied on carefully sculpted emails and stolen email signatures, but deepfakes take it to a new level. For example, an attacker can create a video or use audio, using stolen characteristics of the executive's face and voice to add another level of authenticity to their scam.

It seems wild, but it has happened - allegedly. In 2019, the Wall Street Journal posted the story of a BEC scam in real life. A CEO in the United Kingdom unwittingly handed over €220,000 after he thought he was talking to his boss at his firm's parent company. But, unfortunately, he was actually talking to a fraudster who had used AI to spoof his boss's German accent and voice tone.

However, Curtis Barnes says this example has never been properly verified, and it's possible that a deepfake voice was never used. 

"In my opinion, a deepfake voice was probably never used. In truth, the number of false claims of deepfake-crimes far outweighs the actual number. This may hint towards a greater threat - that synthetic media provides plausible deniability for people who commit ordinary crimes, even when it is not used. But frankly, I'm not persuaded that this is likely to create intractable problems."

To Barnes' point, it's not clear how many of these types of deepfake or synthetic media attacks have occurred in New Zealand - CERT NZ's quarterly reports don't yet have an explicit category for deepfakes, but they may well be buried in other categories.

So what's the solution?

Malwarebytes suggests that people should not give cybercriminals the materials they need to conduct attacks - by that, they mean your images, your videos, or your voice. Unfortunately, that can be difficult if you've ever been posted a public image, video, or voice recording on social media or the internet.

Legally, New Zealand, like the rest of the world, has a long path to follow.  Individual countries could ban the use of deepfake technologies, but as Martin Cocker says, "It is possible to regulate deepfakes – but not specifically saying an image has to be real. So for example, if it is an offence to send an image of a person naked – then a deepfake is as much and offence as a real image."

"Governments focus regulation on harms and harmful behaviour. So, for example, if people use technology to harm another person – that should be considered an offence. 

"Companies that build and create deep fakes should ensure that outputs are 'watermarked' so they can be detected and removed. Likewise, platforms that host deepfakes should remove them when they are causing harm, just as with any other harmful content. 

"Content creators should be liable for the harm that their creations cause, and people who watch deepfakes should be educated to recognise the possibility of deepfakes."

Social media platforms like Facebook and YouTube are cracking down on deepfakes by marking them as manipulated content or changing their algorithms to make them less visible, which won't stop them from existing. Viewers and listeners need to be able to tell the difference - but as technology improves, will we be able to, or will we rely on external video authentication tools from the likes of Microsoft and Google to tell us what is real and what isn't?

And what happens if someone finds themselves on the receiving end of a potentially damaging deepfake? Martin Cocker says that anyone who has found online content that appears to use their likeness can contact Netsafe or the Police.

"It really depends on how the likeness is being used. It could be for a scam, or in a way that breaches the Harmful Digital Communications Act 2015. Netsafe has built a network of contacts across the international ICT industry – so we can often facilitate removal of content from major platforms. We can also provide advice on legal options." 

You can report online harm incidents to Netsafe on their website or by phoning 0508 NETSAFE.

Public Interest Journalism Fund logo
Public Interest Journalism funded through NZ On Air.
Related stories
Top stories
Story image
eCommerce
Robust digital warehouse management crucial in Asia-Pacific
Thanks to a network of “cloud” stores, grocery and food delivery providers such as Foodpanda can arrange for these commonly requested items to get packed up and sent over in almost no time.
Story image
Cybersecurity
Unknown connections: How safe is public WiFi in Aotearoa?
If it's not your own household WiFi, then who has control of your data and is your connection actually safe?
Story image
MYOB
MYOB improves data visibility and user access with Snowflake
"Solutions such as Snowflake allow us to better understand our customers and make evidence-based decisions on what features work best for them."
Story image
Christchurch
Enable launches free Wi-Fi in Christchurch city centre
Fibre broadband provider, Enable, and the Christchurch City Council have launched their new Christchurch Free Wi-Fi service in the central city. 
Story image
Cloudera
Overcoming hybrid and multi-cloud challenges to drive innovation
Driven by improvements in technology, financial services companies have advanced both internal and external systems and processes, with the likes of digitisation, personalisation and risk management redefining the industry.
Story image
trust
Consumers want personalisation, but don't trust brands with their data
Customers expect personalisation during every brand interaction but they don't trust brands to keep their personal data secure and to use it responsibly. 
Story image
Cybersecurity
Aqua Security, CIS create software supply chain security guide
Aqua Securityand the Center for Internet Security have together released the industry’s first formal guidelines for software supply chain security.
Story image
Contact Centre
Customer service agents don't want to return to contact centres
A new report has revealed that 85% of customer service agents want to work full-time at home and not return to contact centre offices.
Story image
Cybersecurity
Ingram Micro launches vendor-backed security program
Ingram Micro has unveiled a new program intended to give resellers the effective offerings their customers need to stay safe in the evolving threat landscape.
Story image
Commerce Commission
ComCom puts electronics sector on notice over resale price maintenance
The Commerce Commission has concluded an investigation into allegations that television manufacturers were engaging in illegal resale price maintenance.
Story image
Testing
Video: 10 Minute IT Jams - An update from Tricentis
Tricentis provides software testing automation, and software quality assurance products for enterprise software.
Story image
Citrix
The best ways to attract young talent during labour shortages
New research from Citrix reveals hybrid working and ventures into the metaverse are top of mind for Gen Z workers.
Story image
Partnerships
Microsoft expands APAC Enabler Mentorship Program
"Mentors are the key to success for every professional. A good mentor is a coach, a guide, as well as a vocal advocate."
Story image
Tech job moves
Tech job moves - ActiveCampaign, Arcserve, LogRhythm & Qlik
We round up all job appointments from June 17-22, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Cybersecurity
Threat actors ramp up their social engineering attacks
As people get better at identifying potential threats in their inbox, threat actors must evolve their methods. Their new M.O? Social engineering.
Story image
Shouta
The rise of digital gifting in the workplace
The name itself does most of the explaining; it’s a gift you receive virtually. But a misconception about digital gifts is that they need to be redeemed virtually as well. 
Story image
F5 Networks
Telstra, F5 team up to bolster services and solutions
“This partnership demonstrates our ongoing investment into APAC as we continue delivering high value services and solutions to our partners and customers."
Story image
Digital Transformation
Cybersecurity priorities for digital leaders navigating digital transformation
In recent years, Asia-Pacific has especially been a hotspot for cyberattacks, and as we continue into 2022, it’s evident that the problem is becoming more significant.
Story image
Cybersecurity
Significant security concerns resulting from open source software ubiquity
"The risk is real, and the industry must work closely together in order to move away from poor open source or software supply chain security practices."
Story image
Airwallex
How Airwallex helps businesses achieve globalisation success
As markets continue to shift, businesses need to be able to provide the same quality of service for customers regardless of where they are located around the world.
Story image
Market growth
Salesforce unveils new offerings for consumer goods companies
Salesforce has announced new products for consumer goods companies to help brands navigate increasing market complexity more easily.
Story image
Government
Cyclone selected as NZ MOE software licensing partner
Following a recent Request for Proposal (RFP), Christchurch-based company Cyclone Computer Company Ltd (Cyclone) has been selected as The Ministry of Education’s software licensing partner.
Story image
Cybersecurity
Why is NZ lagging behind the world in cybersecurity?
A recent report by TUANZ has revealed that we are ranked 56th in the world when it comes to cybersecurity - a look into why we're so behind and what needs to be done.
Story image
Phishing
Online identity theft is rising in NZ - here’s what to do about it
It may start with a few stolen details online, but it could end with thousands of dollars missing or worse, a reputation down the drain.
Story image
API
Industry-first comprehensive risk-based API security enhances protection
Application Programming Interfaces (APIs) have become a crucial part of operating web and mobile application businesses and are causing significant economic growth in the digital sector.
Story image
N4L
N4L, Spark, Chorus partner for Hyperfibre school upgrade
Networks for Learning (N4L) has partnered with Spark and Chorus to upgrade Wellington College to Hyperfibre, fostering stronger outcomes for students and teachers.
Story image
Hybrid workforce
Why hybrid working is here to stay and how to ace it
Citrix's new report reveals hybrid workers are more productive and engaged at work than their office and completely remote counterparts.
Story image
Internet of Things
Global 5G subscriptions to top one billion by the end of 2022
Global 5G subscriptions are predicted to pass the one billion milestone by the end of 2022, according to a new report.
Story image
New Relic
How to tackle the great brain drain in the tech industry
Attracting and retaining tech talent in Australia and New Zealand is becoming increasingly challenging, with the 2022 Hays Salary Guide showing a startling 91% of employers facing a skills shortage.
Story image
Microsoft
Volpara, Microsoft project to detect cardiovascular issues
Volpara Health Technologies is working with Microsoft on a research and development project to speed up creating a product that detects and quantifies breast arterial calcifications (BACs).
Story image
Microsoft
TO THE NEW unveils A/NZ Managed Services for Microsoft Azure
TO THE NEW has released Managed Services for Microsoft Azure to meet the growing demand in the A/NZ market and globally.
Story image
Internet of Things
Domino's Pizza: A blueprint for secure enterprise IoT deployment
Increasingly, organisations are embracing smart technologies to underpin innovations that can enhance safety and productivity in every part of our lives, from industrial systems, utilities, and building management to various forms of business enablement.
Story image
SaaS
Sealord partners with Infor to improve sustainability
Sealord has chosen Infor as a strategic partner to implement an operational cloud-based platform that provides day-one functionality and sustainability gains.
Story image
Digital Transformation
Stax and Consegna partner to accelerate modernisation
According to a statement, the new alliance will help both companies expand their reach across the region and realise joint goals.
Story image
Secure access service edge / SASE
Cloudflare adds new capabilities to zero trust SASE platform
New features for Cloudflare One include email security protection, data loss prevention tools, cloud access security broker, and private network discovery.
Story image
PagerDuty
Ready for anything with the PagerDuty Operations Cloud
In a world of digital everything, teams face increasing complexity. Ever-growing dependencies across systems and processes put customer and employee experience, not to mention revenue, at risk.
Story image
Dark web
Cybercrime in Aotearoa: How does New Zealand law define it?
‘Cybercrime’ is a term we hear all the time, but what exactly is it, and how does New Zealand define it in legal terms?
Story image
Gartner
Gartner's top recommendations for security leaders
"Leaders now recognise that major disruption is only one crisis away. We can’t control it, but we can evolve our thinking, philosophy, program and architecture.”
Story image
Artificial Intelligence
Accenture shares the benefits of supply chain visibility
It's clear that gaining better visibility into the supply chain will help organisations avoid excess costs, inefficiencies, and complexity to ultimately improve their bottom line.
Story image
Sustainability
Honeywell launches new carbon energy management software for buildings
The new Carbon & Energy Management service allows building owners to track and optimise energy performance against carbon reduction goals, down to a device or asset level.
Story image
Trend Micro
5G network projects driven by improving security and privacy
Trend Micro's new study reveals the prospect of improved security and privacy capabilities are the main motivations behind private 5G wireless network projects.
Story image
Entelar
How TruSens air purifiers can create healthier workspaces
The pandemic has heightened our awareness of our own and others’ health, and made us all much more conscious of the environments we work in.
Story image
Talend
Forrester names Talend Leader in enterprise data fabric
Forrester has named Talend a leader among enterprise data fabric providers in the Forrester Wave: Enterprise Data Fabric, Q2 2022 report.
Story image
Infrastructure
Global investment in data centers more than doubled in 2021
DLA Piper's latest global survey finds the total investment in data center infrastructure worldwide rose from USD $24.4 billion in 2020 to USD $53.8 billion in 2021.