How recommendation engines could change the future of BI

The best piece I read this month was about how to build a recommendation engine in R by Data-Mania. It’s a little bit tangential to the BI space but it was interesting to me personally because I’m not a data scientist.

The article explains the fundamentals of what a recommendation engine is and how it works.

Personalizing information is key to user adoption

While reading this article I had a light bulb moment – the core problem in the way large enterprises deploy analytics is in how they curate content. It’s very easy to build a generic sales or marketing dashboard but people aren’t generic. Dashboards are created and managed by committees who second-guess what the organization wants rather than taking a step back and thinking about what every unique individual needs.

As a result, people are getting generic BI analytics rather than something that truly resonates with them. This is one of the core reasons I think the BI industry has such low adoption rates. This article helped me realize that if we can curate content for the individual it will help solve the problem of adoption.

It’s up to vendors to solve the problem of curation

As vendors of BI software, we need to ensure that the right people have access to the right data. Through a curated view of the world, business users will get the content they need to do their jobs effectively regardless of their role or level.

You can curate personalized information by doing it manually or you can automate it. The article talks about different ways that you can solve recommendation problems like user-based filtering or by clustering similar types of content. We also need to find ways to automate analytics so that we can surface appropriate information to individual users at the right time.

This is not something that traditional BI tools can solve for because they’re just not nimble enough. As the article points out, you need something that collects and analyzes data fast. It needs to understand and change content based on a person’s role or how they interact with the content.

This is a new problem that vendors have to solve. I’ve never heard a customer say “we must have better ways to recommend content to the end user”. The current mindset is about building better dashboards because no one has really thought about an alternative solution. We’ve been trying to build better mousetraps rather than asking whether we really need a mouse trap.

Reading this article helped me think about whether there was a way to curate personalized information for users efficiently and effectively. If we can do that it will disrupt the way analytics is used within organizations.