No items found.

Unlocking knowledge from internal data silos without drowning in a data lake

Unlocking knowledge from internal data silos without drowning in a data lake
Unlocking knowledge from internal data silos without drowning in a data lake
September 13, 2018
Nick Halstead

A data silo is an isolated dataset that cannot be analysed with other datasets held by an organisation. This fragmentation, means that it’s not possible to run analysis across the entire business and gain a true picture of customer behaviour.

Flying half blind has a dramatic impact on business success and enables those who have a complete picture to succeed.

“Any enterprise CEO really ought to be able to ask a question that involves connecting data across the organization, be able to run a company effectively, and especially to be able to respond to unexpected events. Most organizations are missing this ability to connect all the data together.”
Tim Berners Lee*

Where do these data silos come from?

The most common example of siloed datasets is where it is inherent in the business structure. With so many companies now diversifying into other products, it is natural that these data silos would appear. For example, some of the larger supermarkets in the UK have spread their wings into loyalty programmes, banking, insurance, and other retail ventures. It is common that the customer data for these different divisions is held separately.

With this customer data being held separately, it is not possible for the group level leadership to quickly and easily run analysis across their entire customer base. They may be able to quickly take a look at their supermarket customers, but can’t then easily determine if a large proportion of those customers also bank with them, and therefore what sort of cross-promotional campaigns they could be running.

Why don’t they simply consolidate their data silos?

Consolidating siloed datasets has been one option for those looking to get a complete understanding of their customers. However, the increased focus on data protection in GDPR makes this increasingly less attractive for a variety of reasons. First, the additional processing of personal data requires a solid legal basis, perhaps bringing together several data sets themselves run on different grounds. Secondly, the creation of a single data pool is an additional point of risk, with one exploit potentially leading to a much bigger breach than if it occured in one silo. Finally, there may be solid business reasons for keeping the data from the different divisions separate and not having entanglements which would, for example, prevent rapid re-organisations or spin offs in the future.

So, continuing with the above example, while the supermarket would likely have a clear legal basis to merge its customer data from its grocery and financial services divisions, the risks and logistics of creating a single data pool are problematic.

The second option, that has been more popular, is to use a Business Intelligence Platform, that brings together various datasets and provides a complete picture of the data. However, the majority of these platforms require the raw data to be physically moved (through the cloud) and combined.

So what’s the answer?

Share knowledge, not data

The answer sounds simple, but the technology to achieve it is far from it. The answer is to take anonymised statistics from one dataset and make a statistical comparison with another. Our Quantum Technology allows data to be analysed across completely disparate datasets, without sharing or pooling the raw data. This will allow any business to run an aggregate analysis across any dataset within their company, unlocking knowledge that can help them to:

  • Improve customer knowledge across all aspects of your business
  • Enhance customer service
  • Understand the wider implications and impact of marketing activity
  • Increase productivity and reduce operational costs

*Source Q&A with Tim Berners-Lee - Bloomberg

Subscribe to the InfoSum Newsletter