itb-nz logo
Story image

How Big Data is slowly destroying our environment

06 Jun 2018

Article by David Leichner, CMO of SQream

I started my career around 30 years ago at one of the largest investment banks on Wall Street.

Their data center was huge.

We had mainframes, midrange computers and various types of minicomputers. The cost of running the data center was enormous, in terms of energy, maintenance, real estate and resources to keep the data center operational.

In 1991 I joined Information Builders in the group that released EDA/SQL to market.

You could say this was the beginning of the client/server era as microcomputers decreased in price and increased in power.

Many enterprises moved from centralized data centers to a distributed client/server architecture with departmental servers and fat clients. The mantra was that the mainframe and centralized data centers were dead - long live client/server.

Well, not exactly.

While client/server provided greater flexibility for individual users and more control over specific computing resources, it also complicated the management of IT resources, servers and standalone databases.

In addition, managing these distributed computing networks became a huge challenge, with the multiples of servers taking up lots of space and energy and requiring many more resources to manage them.

Most large companies ended up with a hybrid architecture of the centralized data center working in tandem with the distributed or departmental computing servers and resources.

Fast forward to the 2000s, with the applications running on the web mature enough to take the place of applications developed for CPU-based systems, mini or even midrange-computers.

This advancement together with affordable storage were among the influences that gave way to the initial move into cloud computing.

Now organizations were beginning to change their computing architecture more drastically, with many evolving their client/server applications to the cloud.

While there has been a lot of talk about the move to the cloud, and in fact, organizations have already taken steps to move their applications and/or data to the cloud, many enterprises are not planning on moving to full cloud operations.

Enterprises often create hybrid cloud architectures, merging existing or new data centers with integration into the cloud. This is true of many of the large financial and healthcare organizations who are still running mainframes, albeit smaller ones that require less energy and space. 

The phenomenon of organizations using the hybrid model as a first move to a potentially longer term and fuller cloud implementation is still not clear.

The hybrid model works well for many enterprises, especially for those wishing to keep certain data stores close to the data’s origin or where it will be analyzed.

Regardless of whether an organization decides to move to the cloud, a move to more innovative processing methods can help significantly downsize computing resources, whether within an existing data center or in the cloud.

By embracing new GPU computing technology from NVIDIA, with massive parallel processing, enterprises are able to significantly reduce their number of servers, which in turn reduces costs associated with energy, space, maintenance and more.

NVIDA has developed its GPU processor which breaks tasks into small problems, solving them in parallel processing on the GPU. This has proven much more efficient that the traditional CPU which works in a linear way, processing one task at a time.

Not only do GPUs perform up to 100x faster than CPUs in many tasks, they are also much smaller. A 16 GPU server is akin to a 1,000 CPU cluster.

A GPU can enable near real-time data exploration and very fast data ingest, which is simply not possible with a traditional CPU. That said, the CPU is still a very important piece of the puzzle for creating high-performance applications.

For example, the architecture of the GPU makes it less suitable for random text operations.

As the thousands of GPU cores process data in a strictly parallel way for high throughput, having contents that will cause the code to behave differently will harm performance.

We are seeing more and more organizations taking a green approach to computing these days. While it would be nice to say that these organizations are self-sacrificing when it comes to the environment, this may be true for a small minority. Regardless of what their reasons are, organizations can benefit greatly from going green.

The responsibility for data center costs is often not in the hands of IT, but the responsibility falls in the hands of the department that managers the organization's facilities.

IT management is often not involved with budgeting data center energy costs, real estate, and other facility-oriented costs. Data centers realize great benefits from going green including significant energy cost reductions, much lower real estate costs and more.

So looking back thirty years, anyone who ever predicted the death of the mainframe has been proved wrong. As it appears now, those conglomerates who are using mainframes will probably continue to do so, especially as they become smaller and more energy efficient.

Seemingly, the days of client/server are coming to an end and are being replaced by a hybrid of more cost effective, lower resource intensive servers based on the most innovative efficient parallel processing available in GPU processors together with the continuing movement into the cloud.

While applications continue to evolve, data is increasing and will continue to grow at unparalleled rates. Enterprises need to adapt innovative technologies that will enable them to store and analyze these growing data stores in a rapid and comprehensive way.

Expenditure on managing and analyzing big data will grow to billions of dollars annually and will only provide enterprises with competitive advantage if they are able to effectively and rapidly analyze the data and extract actionable intelligence from it.

By using the most advanced processors available for computing, such as GPU-based servers and databases, much less hardware is needed, making maintenance significantly easier and less time-consuming.

In addition, the ability to respond to business needs and advanced analytics is much greater. So not only are IT investments reduced and overall costs decreased, but the business benefits overall.

In fact, many enterprises are already adapting GPU servers and databases to realize dramatic gains in energy efficiency, while increasing the performance of their applications and data analytics.

This adoption of GPU technology, either stand-alone or in a hybrid model, appears to be the next stage in the ever-evolving paradigm of computing.

Download image
Remote workforces drive new risk management strategies
remote and onsite workers have access to multiple collaboration tools, leading to fresh risk management strategies in the post COVID world.More
Link image
Scale-out multi-protocol storage & data management with Hedvig
The advantages of data growth are clear if they are harnessed properly: Agility, scalability, and lower costs are but a few of those advantages. Find out how you can make these a reality with Hedvig.Register Here
Story image
Commvault partners with Microsoft to deliver SaaS and cloud technology
“Today’s announcement combines two of the best enterprise cloud technologies to simplify and accelerate a customer’s journey to the cloud."More
Story image
Why DX is not complete without a transformed security architecture
Secure Access Services Edge (SASE) is the process by which core WAN edge capabilities like SD-WAN, routing, and WAN optimisation at branch locations are integrated with cloud-based security services like secure web gateways, firewall-as-a-service, cloud access security brokers, and more.More
Download image
Ultimate security: The best authentication just got better
Cloud applications can hold sensitive data, and top-notch authentication is key. But having separate tools for separate applications can be cumbersome - here's how to overcome that.More
Story image
Unstructured data: Making sense of the mass and the mess
A modern data experience requires infrastructure that can bridge silos and meet demands for performance, agility and simplicity, without complexity or compromise. This calls for a data hub approach. More