A Signal for Change in Analytics

There is so much talk about the potential of Artificial Intelligence (AI) and Machine Learning (ML), especially in the Business Intelligence (BI) and Analytics arena.

The most powerful examples revolve around the ability of machines to spot patterns and trends within huge volumes of data. CIO Magazine spotted the trend towards better data signals last year, saying: “Many BI tools are bringing in more, and better, data signals to produce more accurate, insightful reports that blur the distinctions that traditionally separated BI from more advanced analytics. As such, BI vendors will need to advance, or risk losing out in the market.”

If you visit any event in the BI space, every leading vendor is talking about their approach to AI and ML. At Yellowfin, we’ve made it possible to cut through the deluge of data by using well-understood algorithms along with newer machine learning elements to surface insights and deliver practical benefits. Whatever tool you look at, you need to consider how users across areas such as sales, marketing, and human resources can maximise their BI to uncover valuable business information without the need for deep analytical skills.

Dashboard limits

For many organisations, the dashboard is what most people associate with their Business Intelligence tool. Charts and graphics alongside a traditional ‘traffic light’ approach offer a variety of methods for visualising your data specifically for monitoring a known set of metrics over time. However, business users increasingly need to discover new insights, rather than just monitor existing ones, and this is where dashboards tend to fail.

Dashboards were originally designed as a monitoring tool and not as a tool for data discovery. And with the increasing amount of data that sits in silos within every business, and the ever increasing number of applications available now, it is almost impossible to condense it all on to one page. As a result, the data is aggregated. For the business user, the dashboard offers essential top line numbers but gives little visibility into what's happening in the underlying business. In order to get into the detail, users need to feel confident at using filters and drill-down. So dashboards put the onus of discovery back onto the user by making them slice and dice the data themselves to find insights. However, this gives rise to several problems.

The discovery of insights become less predictable and requires the user to master the skill of manually drilling down into data and having the time to do so. As the data may not change every day, the user has to repeat the process of data discovery to check for changes. Not only is this a waste of time, but it can lead to ‘analysis fatigue’ with the result that users may miss a change when something significant actually happens.

Workbook workaround

To counter this, businesses often start to use workbook solutions where users load data onto the platform, and then automation helps them build a better dashboard. Often the data is loaded manually so it’s not live and if a business user doesn’t have access or knowledge of the backend, then they must rely on data analysts to keep the underlying information updated. Once the data is loaded, the platform then runs basic algorithms that make layout suggestions based on the data. However, the platform only looks at the data at that particular point in time which means that analysts must constantly refresh the data.

More advanced solutions have started to allow users to ask a question and the application then analyses the data and tries to find an answer. However, this assumes that the user knows what questions to ask the platform – if a user has seen an outlier or a spike it may trigger them to question it. The platform doesn’t do the heavy lifting of finding outliers for the user. Instead, the platform just automates the existing data discovery process, but the onus of understanding what questions to ask still sits with the user. That’s why this approach is used mostly by data analysts rather than business users.

Automate to innovate

Yellowfin takes automated analytics much further. Yellowfin Signals applies machine learning along with more advanced algorithms to discover critical variances in business data. It runs continually in the background and surfaces statistically significant changes, alerting people of the changes relevant to their role. When it finds something, it analyses other data sources to find correlated information, then sends an alert highlighting the pattern so all recipients can determine if the data is actionable.

Signals runs on live data, so analysts don’t have to manually load data into a workbook before analysis can be performed. The underlying algorithms also tailor the notifications of changes to users’ roles, using information from each user’s interactions, and the interactions of those who hold similar roles, with previous signals. Something that may be insignificant to the CEO could be very important to someone further down the organisation. This level of nuance is effectively a learning process that gets more powerful as the system runs within an organisation. As it’s automated, it can never suffer from analysis fatigue or human bias, which are common issues within traditional dashboard uses.

Skilled analysis

Automation technology is not just making it easier for users to gain more insights, faster; it is ultimately helping to alleviate the skills shortage in the industry. Data analysts and scientists command premium salaries and there simply aren’t enough people to fill demand. 

The adoption of new technologies like Signals is at the start of the curve but growing rapidly. For organisations that have reached the limits of the usefulness of dashboards or that are struggling with hiring and retaining a growing army of data analysts, a change is coming.