Story image

Implementing a big data programme in the cloud

04 May 16

Enterprises with limited resources may face technical and infrastructure challenges when implementing big data analytics solutions, according to CenturyLink.

Big data projects don't exist in a vacuum, the company says.

CenturyLink says a company’s big data infrastructure elements should have the flexibility to support multiple approaches to integration with one another, as well as with external elements. 

Martin Hooper, head of business development, Australia, at CenturyLink Cognilytics, says cloud infrastructure provides the resources needed for big data strategies. 

“The data in your Hadoop application comes from many sources and is fed into a variety of applications in order to maximise their value for different constituencies,” Hooper explains.

“In addition, the constant shifting of big data workloads requires your big data infrastructure to scale elastically,” he says.

“Elastic infrastructure means being able to scale up and down at will. Cloud infrastructure is the best option for elasticity.

“The cloud typically gives you this ability, making it relatively simple to add or remove data analytics capacity on demand,” says Hooper. 

Hooper says there are four key steps to take before implementing a big data project in the cloud: 

1. Plan for the big picture 
“Like most IT initiatives, it usually saves time to plan a cloud-based big data program up front,” Hooper says.

He says architectural decisions made early on in the project will influence the success of future phases.

“In planning, it is important to first identify potential specific business use cases, such as customer loyalty, product development, or risk management,” explains Hooper.

2. Overcome cultural obstacles 
According to Hooper, one of the greatest barriers to becoming a data-driven organisation is overcoming a legacy decision-making culture.

“It takes experience and credibility to make a convincing argument for a cloud-based big data strategy, and an effective change management program to succeed in its implementation,” he explains. 

3. Consider compliance and data sovereignty 
Organisations in every industry need to deal with data privacy and security management, while many also have to maintain certain practices to meet regulatory compliance standards and avoid penalties.

Hooper says establishing an effective data governance strategy is an essential step to developing a successful cloud-based big data program. 

4. Meet infrastructure requirements 
“For a big data application to grow in size and capability, it needs to be developed on an architecture that is designed to support its specific needs,” says Hooper.

“This is why it is important to identify the baseline infrastructure and future requirements of a big data project.

“The scalability, performance, reliability, and security of the cloud infrastructure involved in the project should be taken into consideration,” he says.

Disruption in the supply chain: Why IT resilience is a collective responsibility
"A truly resilient organisation will invest in building strong relationships while the sun shines so they can draw on goodwill when it rains."
The disaster recovery-as-a-service market is on the rise
As time progresses and advanced technologies are implemented, the demand for disaster recovery-as-a-service is also expected to increase.
Apax Partners wins bidding war for Trade Me buyout
“We’re confident Trade Me would have a successful standalone future," says Trade Me chairman David Kirk
The key to financial institutions’ path to digital dominance
By 2020, about 1.7 megabytes a second of new information will be created for every human being on the planet.
Proofpoint launches feature to identify most targeted users
“One of the largest security industry misconceptions is that most cyberattacks target top executives and management.”
What disaster recovery will look like in 2019
“With nearly half of all businesses experiencing an unrecoverable data event in the last three years, current backup solutions are no longer fit for purpose."
NVIDIA sets records with their enterprise AI
The new MLPerf benchmark suite measures a wide range of deep learning workloads, aiming to serve as the industry’s first objective AI benchmark suite.
McAfee named Leader in Magic Quadrant an eighth time
The company has been once again named as a Leader in the Gartner Magic Quadrant for Security Information and Event Management.