How big data can solve big societal problems

New Zealand's youth suicide rates are the highest in the OECD. The numbers are even worse for Maori and Pasifika youth. And this week, statistics and commentary have highlighted the 'epidemic of mental illness' in rural areas of New Zealand.

There is a crisis in mental health services for young people and it's hard to get help.

Recent government statistics reveal that more than a thousand New Zealanders under the age of 18 have had to wait more than two months to see mental health support services, while more than five thousand have had to wait more than three weeks for an appointment. The numbers clearly indicate that the supply of services is inadequate to meet the demand.

The Government has promised a review of the mental health system, but reviews take time. There is a good chance that useful insights that will help address the problem already exist in the huge volumes of data held across a range of ministries and agencies.

It is a digital government that can help solve some of society's most pressing problems by improving decision making to help give vulnerable New Zealanders the support they need.

Reducing infant mortality Here's an example from my time working for the government of Indiana, which faced stubbornly high rates of infant mortality, even as it declined in other states. To change the status quo, we knew we needed a far better grasp, grounded in evidence, of why too many babies in our state were dying.

The project drew together vast amounts of anonymised data from different agencies. It quickly became clear that the biggest issue for the state was lack of access to quality healthcare.

In 2014, Indiana's government issued a report that examined the predictors of infant mortality. It found that the baby of a mother on Medicaid (a federal health programme for low-income Americans) between the age of 15 and 20, and who attended fewer than ten prenatal visits was at the highest risk of mortality. These babies accounted for only 1.6 per cent of the births in Indiana, but almost half the deaths.

Data illuminated the critical role of prenatal care and the state acted by boosting funding to improve prenatal care and by better targeting of advertising to pregnant mothers.

Public sector agencies collect and hold vast quantities of data, but it is held in siloes, invisible to other parts of the same government. Another challenge is often the sheer volume of information it's hard to even know where to start. By anonymising and analysing data, governments can gain valuable insights that drive better decisions for those that need support.

We identified through the Indiana experience four key success factors for deploying data in tackling public policy challenges. First, it takes leadership from the very top to drive the necessary culture shift. Second, it's important to have that all stakeholders agree clearly on the purposes and objectives at stake — there is no room for interagency rivalry. Third, to assuage privacy concerns, the data sharing framework needs to be robust and standardised. Finally, success should be replicable, for instance, the methodology that underpinned the infant mortality project could be applied to other challenges like opioid addiction, mental health or a myriad of other societal problems.

Solving the US opioid crisis Ultimately digital government transformation is all about the problems that need solving, not about the data.

The $1 trillion opioid crisis in the US could be addressed by the same approach, mapping drug overdoses to prevention and treatment programme outcomes.

Bringing together real-time information from a variety of sources can show where drug activity is highest, the number of opioid prescriptions and death tolls.

Emergency services, social service agencies and treatment providers can use the data to identify where overdoses are increasing and correlate with other data such as age and ethnicity. Mapping that data on top of the most effective treatments can provide effective solutions fast.

One US county used the insights garnered to motivate people to get treatment, prevent overdoses and measure response rates. Easy-to-use apps helped people locate nearby drug drop-offs locations, and find treatment and pain management options, complete with directions to the nearest provider.

Better services for vulnerable New Zealanders  Can New Zealand take the lessons learned from the examples above to use data-driven insights to reshape services and policies to deliver better mental health services to young New Zealanders?

It absolutely can and it's not unique in facing these problems. Other countries and states have faced similar issues, which can be solved using these information-driven techniques. Data is merely the tool.

Prime Minister Jacinda Ardern has stated that she wants New Zealand to be the best place to be a child. Having spent some time here recently, I can see why. The question for policymakers is: How can we use the data we have that is currently siloed across agencies to make the best possible reforms to make that ambition a reality?