The death of BI

It’s obvious by now that everyone is struggling to gain insight into data. Can the issues be solved with a new form of business intelligence (BI)? Or is BI doomed? Our managing partner, Wim Debreuck, shares his insights in this article.

Too little, too late

Let’s take a step back. In the beginning...there was little data. Too little data. The data that you did have, you had to store at a sky-high price. Not to mention the way data was captured. Think back to how people used to manually draw lines, so to speak, to count incoming people. Fortunately, this outdated way of working was soon solved with sensors, among others. Data was finally quickly and easily captured. Too easily.

Databases became completely overwhelmed in no time. Previously, when using BI, you could barely study user behaviour because there was far too little data. Suddenly, though, we were confronted with an overload of available data that we could no longer analyse. What happened: we were dumping everything en masse into data lakes for ‘later’ analysis. Terabytes and terabytes of data to look at ‘later’... It soon became ‘too late’, of course.

Real-Time Data Processing

Making a sensible report on the incoming information about your business is a hopeless task when you go about things like that. Not only are you usually way too late, but the ocean of data often contains useless and/or corrupt data. Just imagine getting started. The answer: real-time data processing. We don’t crunch endless data anymore because you can’t see trends of analyse patterns in that manner.

What if we could dissect data while it is being created, though? In other words: what if we sift through everything before it is being dropped into a lake? Then it is nicely stored indeed, and it doesn’t have to be dug out again afterwards. The philosophy behind the world of real-time and streaming architecture? Let’s see if we can predict the bottlenecks on the spot instead of afterwards.

Standing on the shoulders of giants

True, such an answer isn’t ground-breaking since you keep on building on what is already there. However, you use it smarter and more efficiently. We are part of a hybrid system as we speak, where you look at something like fraud detection in a real-time process. A comparative quarterly report can be moved up, though, and isn’t necessarily required immediately. It isn’t an all-or-nothing change but a gradual improvement this way. Subsequently, classical BI is being replaced, at least partially, by what we call hot analytics. In short: the transformation to a streaming landscape is inevitable.

Of course, it all depends on speed. Companies are growing incredibly fast these days, switch businesses rapidly, and shoot themselves to the top. The same applies to your company: you must be able to respond to change on time or you’re doomed to fail. On the spot analytics can therefore be responsible for a competitive advantage over competitors. Companies that recognise this are the most agile and have the best chance of survival.

Turn your database inside out

Streaming integration architecture is therefore the way to go, if you ask us. It is the next step in being able to analyse business behaviour and provide answers to problems from previous technologies. Before, the concept of ‘time’ wasn’t a core element of data. Now, it is part of an ‘Event-Driven’ data world. Classical BI doesn’t consider the time of an important occurrence. However, the online shopping world, for instance, has long seen the moment when someone deletes or adds items to his or her shopping cart. Based on such information, you can anticipate customer behaviour much better than if you only look at what someone eventually bought, after the facts.

Let’s be real: Business Intelligence on its own is worthless. The transformation to streaming is driven by several problems; the main one being that one monolithic system doesn’t work. In such a system, it’s often too difficult to analyse how users interact with different functionalities of apps and departments.

By streaming data, you can follow up on business events, or business process elements, way better and look at their lifecycles. Where does it all end up and what is being done with it in the extensive app landscape? Something to consider as a new and interesting perspective for decision makers, wouldn’t you say?

Inspired by this topic or could you use support on the future of BI for your business? Do not hesitate to contact us; we are happy to help you.