Prior to the availability of cloud, storing data was a relatively simple proposition for organisations – either it was hosted locally or remotely in their data centres. However, in today’s cloud enabled world, IT teams are running applications in a combination of public, private or hybrid cloud environments.
Developers may opt to spin up test environments for applications in one public cloud but deploy on another to link into their enterprise suites, and may use a third cloud provider to develop proof of concept for an application that will run only in a private environment. Alternatively, an organisation’s analytics tools might live in its private cloud but the data those tools process is created and stored in other platforms.
This trend is repeated and amplified as organisations use more cloud environments. With cloud users running applications in an average of 1.8 public clouds and 2.3 private clouds1 , data fragmentation, is becoming a major challenge for organisations looking to keep their data agile.
There are several problematic aspects of data fragmentation. First, the sprawling of data can create security and compliance concerns. If organisations don’t have policies in place covering what data can and cannot be moved to which environments/systems, or how the data stored on them should be secured and protected, they open themselves up to security risks and regulatory action.
There is a high chance of data duplication when several platforms are involved which can lead to an increase in operational overheads. This is due to storage wastage and over-provisioning, driving up volumes and costs. It can also potentially lead to data integrity issues when users assume that they are accessing the latest data, but are instead referring to previous copies, causing unnecessary errors.
Typically, when organisations run into storage limits they either buy more physical hardware, transfer some data sets to the public cloud or find another location. However, organisations need to ask themselves how agile their data is, and consider the complexity of moving their data around from one provider to another.
Driving data to a new platform enables better insights for applications such as machine learning, big data analytics or CRM systems. Data is the intellectual property that helps differentiate an organisation from its competition, and helps identify insights that deliver better customer experiences. Data is also increasingly reported against the bottom line as a commodity in its own right. However, moving data from one service provider to another has a direct impact on the value derived from this information.
Ultimately, data needs to have maximum agility, and be accessible and centralised to become useful. If analytics or transfers become prohibitively expensive or cumbersome, then an organisation’s agility - the speed and ability at which it can respond to changing market and customer demands for a competitive advantage - is diminished.
The Queensland Building and Construction Commission (QBCC) is one such organisation that was keen to better utilise their data to adopt an insights driven approach to its work. “That may include things such as helping to identify companies that are at future risk of insolvency or being able to identify top building defects throughout the state, for example.” Says QBCC Chief Operating Office, Ben Ward. “To get those types of insights, and also insights around how we can better regulate, we needed to have to the right platforms in place to allow us to be innovative.”
Centralised virtual storage solutions enable organisations to overcome this challenge. They decouple the storage and compute functions so enterprises can keep control, move environments seamlessly as required, and reduce costs. For example, developers can make instant copies of data for use in test environments and only consume a fraction of the original data footprint without having to take an application out of service.
The best virtual storage options are those that are located near major cloud service providers, and linked with reliable high speed and low latency connectivity. They empower organisations to take advantage of the flexibility of public cloud and drive value from their data assets, while ensuring the data remains secure.
As such, virtual storage becomes a key component in a modern hybrid cloud architecture and IT and migration managers should consider how it can make a difference to their deployments.
“When platforms are more reliable, and simply work, we tend to hear very few complaints.” says Ward.
For more information on how Telstra is helping organisations get the most from hybrid cloud, read our blueprint for best practice here.
1 - Rightscale, State of the Cloud 2017