Are OLAP Cubes an Outdated Technology?

OLAP cubes – a technology that was introduced in the 70’s for the purposes of storing multi-dimensional data in a format that could be easily queried by reporting technologies. It had a major impact on the way that we thought of databases and rewrote expectations with regards to how easy and quick data could be joined together.

It was a key innovation at the time.

Sadly for OLAP technologies, the timeline of history has seen many innovations – some greater than others. And more often than not, they all have a time where they get replaced for something better, simply because technology is always evolving.

If it didn’t, we would never improve on the things that we built yesterday.

The Pros and Cons of OLAP Cubes

The main advantages of OLAP cubes as already mentioned were the relative ease and speed at which programmers could pull data together from the data warehouse into one repository.

Whilst this was impressive, it also had its drawbacks. The reason OLAP performed so well is that all of the data joined within an OLAP cube was already aggregated meaning the view that analysts got of their data was limited and didn’t show the full picture.

This was a problem, as the questions raised from looking at a report built using OLAP cubes were often left unanswered because the analyst was not able to see the underlying data contributing to the aggregated values.

At this point, SQL was needed in order to provide the drill-down capabilities of OLAP-based business intelligence. Changing between the two technologies to see the raw data in SQL and the finished output in OLAP added an extra, unnecessary step to the whole process and proved to be both clunky and time-consuming.

In addition, these sorts of technologies were mainly the tools of the highly technical and thus the business user (who was and still is often the end user of report outputs) had little influence over the creation of the reports.

Surely, there’s a better way? One that combines both approaches and provides business users an easier way to access and present information without the restrictions imposed by rigid OLAP structures defined by IT.

What architecture will take its place?

Many technologies can probably lay their own claim in replacing the now obsolete OLAP cube technologies. For example, the likes of in-memory analytic technologies immediately spring to mind.

However, from our own slightly biased view, we believe that another technology has risen up to take its crown: search-engine technology. And we believe we are best placed to talk about it and advise having started the movement within the world of business intelligence with our solution CXAIR.

Search-engine technology is a relatively new technology, coming around 30 years or so after the first OLAP-based tool was created, and has therefore been specifically designed with today’s and the near-future’s technology advancements in mind.

In-memory technology came to prominence as users started to see the restrictions of using OLAP, where they needed access to data at a more granular level with reduced loading times. By taking advantage of the development of 64-bit architectures and the advancements made in building hardware with more RAM, BI vendors were able to apply in-memory technology to their business intelligence applications.

The problem with in-memory technologies is that it pre-loads data into system memory, restricting the hardware available to the user in running the business intelligence application, often requiring more expensive hardware for larger amounts of RAM.

Search-Based Analytics – The Future

All data within a search-engine is stored at document/transaction level and is not pre-aggregated like it would be when contained in an OLAP cube or an in-memory technology application. Furthermore, search-engines are not limited to defining the fields and values made available for reporting and lend themselves to fast data retrieval like these other technologies.

This means that users are able to have full access to their raw data, whether structured or unstructured, and can create the aggregations themselves on the fly, all in a super-fast data environment.

Think about the web browser search-engines you use every day, how often do you have to wait longer than 1 second for your results to be returned? If I’m a betting man I would certainly say it’s not often!

Utilising this fast data-retrieval technology in analytics seems a no-brainer to me, and you shouldn’t be put off by the relative new introduction of this technology. Instead, you should have feelings of excitement and optimism as the technology continues to develop and moves to the forefront of business intelligence application architecture served to improve your own business intelligence environments.