Most organisations still use trucks for data migration, and over half fail

WANdisco has released a survey finding most companies migrate data to the cloud using trucks and other outdated methods while ironically looking to modernise their organisations.

Software company WANdisco has announced the launch of its annual Hadoop-to-Cloud Migration Benchmark Report, which finds most data migrations still done using outdated methods. With 56% using trucks to ship data or using tools not designed for on-premises to cloud migration, such as DistCp (48%).

The result has meant more than 54% of migration projects not going according to plan, with 24% not completed on time, 11% not within budget, and 20% missing the mark on both.

"These manual migration tools strain resources, add complexity, and ultimately increase risks to the business," says WANdisco SVP marketing, Van Diamandakis.

"This DBTA survey shows that companies are not taking advantage of technologies that can ease the transition and mitigate risks in moving to the cloud. As we look to the future, the next wave of data migrations will be larger and more complex as companies seek to implement hybrid and multi-cloud data infrastructure.

"The outdated nature of these migration techniques puts companies data and business at risk and will soon become obsolete," he adds.

The survey finds many IT leaders are looking to cloud migration for the future. The top three Hadoop-to-cloud migration drivers being, data modernisation initiatives (78%), cloud-scale analytics (61%), and adopting scalable cloud storage (49%), which points to a desire for more agile capabilities, despite using outdated technologies to get to that stage.

Many respondents say they expect to be using hybrid environments for the foreseeable future, with 42% planning to maintain a hybrid environment for one to three years and another 36% planning to maintain on-premises Hadoop environments indefinitely.

Some key findings from the survey include:

• Over 73% of respondents intend to migrate on-prem Hadoop to the cloud
• Over half of these have not yet started
• Companies with 5,001-10,000 employees show the highest intent to migrate (48.3%)
• 21% claim to have fully completed the process
• Companies with over 10,000 employees have the highest completed percentage (27.9%)
• 52% expect their on-premises Hadoop data volumes to grow over next year
• 6% expect significant increases
• Only 24% expect decreases
• Of companies with the highest intent to migrate, 46% have more than 5PB of data to migrate

The survey questioned over 200 technical professionals from the Global 2000 in May 2021 who currently use Hadoop or previously migrated Hadoop data lakes to the cloud. Respondents included C-level employees (e.g. CIOs, CTOs), cloud and data architects, IT and analytics directors or managers, and others within their organisation's data, development, or management roles.