Exclusive interview: A Fuji Xerox IoT expert on the ethics of ambient intelligence

Dr Lynn Wilcox is the chief technology officer at Fuji Xerox's Palo Alto Laboratory (FXPAL) in California and an expert in the Internet of Things (IoT) and artificial intelligence (AI).

Recently, Wilcox and her team at FXPAL have been working on projects in the ambient intelligence space – that is, the real world, practical application of IoT and AI.

From a smart teleconferencing system that will notify individuals interrupting the meeting to tracking the location of patients in a hospital, as Ambient Intelligence develops - the impact it could have across all industries is exciting.

But what exactly is Ambient Intelligence and why should we care about it?

In anticipation of her upcoming visit to New Zealand, Wilcox breaks down why we should be excited about Ambient Intelligence, what's going on at FXPAL at the moment and what happens when the use of IoT technologies becomes unethical.

What is Ambient Intelligence, and why should enterprises be excited about it?

Ambient Intelligence refers to electronic environments that are sensitive and responsive to people.

It utilises IoT to collect data about people and the environment from various types of sensors. It then analyses and makes inferences from this sensor data using AI to provide personalised and contextual actions.

Ambient Intelligence has the power to make work more efficient - for example, by automatically knowing when you walk into a conference room that you would like to display the agenda and other information about the meeting that has been scheduled for this time, and order preferred beverages for the people scheduled to attend the meeting.

How does FXPAL work, do you develop technology solely for Fuji Xerox, or do you do work with other partners?

FXPAL is a research lab wholly funded by Fuji Xerox.

We develop prototypes to demonstrate technology and if there is interest we deploy the technology outside the lab to test real use cases. Our primary goal is to provide successful technology to Fuji Xerox for commercialisation.

In certain cases where FX is not interested in using the technology for their business, we license it to third parties. We also generate a good amount of Intellectual Property in the form of patents, which Fuji Xerox commercialises.

FXPAL is developing some very exciting AI and IoT technology with the capability to track the location of people in factories and hospitals, could you tell me about what has gone into developing these technologies? 

The work began as a project to help locate co-workers in and out of the office and used a variety of technology such as GPS, device monitoring and cameras. To achieve more precise indoor localization, we began experimenting with BLE beacons.

FX Australia got interested in the technology and asked us to help develop a solution for office space management that could track which offices and meeting rooms were in use and how frequently various types of space was used. After that, the business development team at FXPAL found an application for patient management at a health care facility and we helped them deploy our technology there.

Last year Fuji Xerox Manufacturing executives visited FXPAL and saw a demonstration of our localisation technology and invited us to do a trial at the FX Suzuka factory.

We did the initial deployment this summer and are working with the factory to maximise the utility of the technology.

How often would you scrap a project? Do you ever get right to the final stages of development before you decide the technology won't work?

The typical research cycle is to first describe the concept, usually in an Invention Proposal, then demonstrate the concept to others, then build a prototype to test outside the lab.

There are checks at each of these stages. For example, we review Invention Proposals and if the idea is not sound or has already been described by someone else, research is stopped.

Similarly with demos, if they are not convincing the project is stopped. Finally, if the prototype fails or if no business case can be found we stop the work.

Many of our research projects are never used in products or solutions, sometimes because the technology is just too early for the market.

Could you discuss the FXPAL's development of a secure document viewer that only allows viewing of an electronic document in secure locations? Where are you at in the development of this technology and what would define, or who would define, a secure location?

We are at the demo stage of this technology.

We can demonstrate how a secure viewer would work by using our indoor location technology to determine whether you are inside the secure location or not. To secure a specified location, for example an executive office space, we simply deploy beacons around the office.

So far, we have not developed a location aware document viewer that would be needed for a prototype system.

One of the projects that FXPAL is working on that's really intriguing is the creation of a smart teleconferencing systems that will monitor participants, notifying them if they are interrupting the conversation, or speaking too much. Could you tell me more about it? How does it make decisions such as what's interrupting and what's considered a contribution?

The system analyses the audio and video content of the video stream during the teleconference. It knows when each of the speakers is talking, and so can detect when a speaker begins talking while another person is talking - that's an interruption.

It keeps statistics on how long each person is speaking during the meeting and visualises this by smaller or larger circles depending on how much each speaker is talking, so it is easy to see how your circle compares to others.

When can the use of IoT and smart technologies by enterprises, or any users, cross the line and become unethical? How can this be monitored and prevented, and is there a danger in the power these technologies could give certain users?

That's a good question - the same problems already arise with our online activities. There we encrypt information before sending it over open networks.

The same could and should be done for IoT devices. I think the general rule is that if a technology provides sufficient value to us we overlook the loss of privacy.

A good example is surveillance cameras.

At first we rejected them but now seeing their value in preventing crime and terrorism, everyone accepts them.

What excites you about the future of IoT? What area would you like to see IoT, AI and AR move into?

I'm particularly excited about AR, which uses IoT for localisation and AI for vision.

I'd like to see AR used for providing more information about things and places in the real world. In the digital world, I can learn more about something by doing a search.

With AR, I'd be able to look at something, for example building, and see information about its history or how it was constructed.

This would be possible with an AR wearable device that could sense my location, then recognise the building, search for related information and project it on a wearable display.