itb-nz logo
Story image

Q&A: SnapLogic CTO makes the case for investment in machine learning

08 Oct 2020

SnapLogic field chief technology officer Brad Drysdale discusses the roadblocks to a successful machine learning implementation and the ways they can overcome them.
 

What are the barriers for firms when it comes to machine learning (ML) adoption?

The ML process can seem daunting for many organisations due to its unpredictable and experimental nature. 

When IT teams initially start going through the data to deploy ML algorithms, most probably won’t know the type of data that is required. A lot of exploration must be done before IT decision-makers have an idea of what data will be useful, and which ML algorithms will work best to solve a particular problem. 

Other technical challenges include being able to automate data access. When organisations have formulated clear policies that allow easy access to real-time data, they need to consider how to set up a channel or a pipeline to access the data. 

Organisations also need to ensure that there is availability of constant real-time data. ML models should not be trained on a single fixed set of data, so organisations need to set them up so that they can retrain their models to adapt to the changing behaviour of the data and the systems that they’re working with. 

Additionally, there is a significant talent shortage. While the number of qualified data scientists is growing, there are only a small number of data scientists that we can produce each year. 

How can these barriers be overcome? 

There are concerns about access to certain types of data, particularly when you have different groups of employees or other stakeholders coming in at different times to work on projects. So organisations should consider filtering out any potentially sensitive information from the data first so that the rest can be used to deploy ML algorithms. 

Another issue to overcome is how to fulfil the demand for data scientists. While it’s great to see that more data scientists are emerging in the workplace, a lot of time still goes into training them, so supply is still not keeping up with the rising demand. 

However, more people who have been trained in other areas, such as senior business analysts and software engineers, are increasingly expanding their knowledge of data science and ML, which can help bridge that gap.

Additionally, organisations will have IT business analysts who have experience with handling databases, even if they’re not programmers, they’re still analytically minded, so they can take advantage of ML through self-training too. 

All of these developments are following a positive trend, as tools and platforms are beginning to allow a broader range of users to engage with ML and make it useful for them. 

What are the biggest misconceptions businesses have concerning ML?

I see two main misconceptions about ML that relate to its complexity and capabilities. First, businesses often think that ML is very complex and requires PhD’s to get value out of an implementation. 

Many relatively simple ML algorithms can be applied to business data to provide predictions or classifications.  On the other hand, there is an unrealistic conception that ML is a panacea to all business problems. The sweet spot is understanding the realistic capabilities of different, well-understood ML algorithms and match them with the right business data to derive real value. 

How can businesses address the data science skills gap?

Businesses of all sizes should be working with universities or creating apprenticeship programmes to bring in fresh talent. 

For example, the Computer Science department at the University of San Francisco offers a project course for both undergraduates and graduates for one term, where they typically do small project work with an industry sponsor. This not only allows students to work with a variety of different companies, but it also makes the recruitment process for businesses far easier. 

Another way to help bridge the skills gap is through investment in technology that will lift the burden off IT professionals. 

Low-code/no-code platforms are a prime example of this, as they can enable data tasks to be undertaken by people outside of the IT department working in the lines of business. 

Currently, having the right data to deploy ML algorithms is an incredibly time-consuming process. A lot of time is spent trying to get access to and sift through vast volumes of disorganised data with manual coding, leaving IT professionals little time to focus on higher-value tasks. 

By investing in the right low-code/no-code technology, businesses can easily automate data pipelines, giving all departments regular access to real-time data, and make ML processes as seamless as possible with little to no coding required. 

How will businesses benefit if they invest more in emerging technologies and skills training? 

Businesses can look at how investment in emerging technologies will benefit them in two ways – either to get ahead of their competition or to prevent their organisation from becoming obsolete.

Businesses need to follow and even move ahead of technology trends to not only offer a better experience and more effective utilisation of their resources but continue to provide the services that their users and customers expect. 

Eventually, all organisations will need to adopt ML simply because that will become an expectation, so that applications and services can better anticipate what their users are attempting to do and to provide recommendations or predictions that enable them to achieve their goals more rapidly. 

This doesn’t just apply to just investment in the technologies, but the skills training as well. Businesses need to ensure that people are trained well to utilise these technologies, but also continue to help expand their skill set to harness the full potential of these emerging technologies. 

Link image
The 5G network emulation solution that accelerates device workflow
Here's how to streamline your workflow across test domains including protocol, radio frequency (RF)/radio resource management, and functional and performance testing.More
Story image
Gartner reveals the top strategic tech trends for 2021
“CIOs are striving to adapt to changing conditions to compose the future business - this requires the organisational plasticity to form and reform dynamically. Gartner’s top strategic technology trends for 2021 enable that plasticity.”More
Link image
It’s almost time for StorageCraft’s NZ October Technical Bootcamps
Whether you’re in Hamilton, Auckland, Christchurch of Wellington, StorageCraft’s technical bootcamps will take you through technical updates and deep dives into ShadowProtect, ShadowXafe, and OneXafe. These are not to be missed! More
Story image
Microsoft’s Azure Space – Cloud above the clouds
The company has hired a team to build cloud capabilities that meet the unique needs of space.More
Story image
Video: 10 Minute IT Jams - Who is LogRhythm?
LogRhythm VP of sales for Asia Pacific Simon Howe, who discusses the company's primary offerings and services, what products the company is focused on for the future, and the infrastructure it has in the A/NZ market.More
Story image
How 'data gravity' centres can spell trouble for enterprises
In the not-too-distant past, data was created in a much more centralised place, and users and systems had far less access to it. Now, with digital data from social, analytics, mobile, cloud, IoT and more being created with both simultaneity and omnipresence, so much information is being collected that it’s forming a ‘centre of gravity’.More