Education analytics: Benefits, barriers and state-of-play

Applying Business Intelligence (BI) and analytics to educational or academic challenges can yield some truly worthwhile results.

Academic institutions can harness the capabilities and insights offered by analytics to answer complex questions that positively alter the world in which we live.

Additionally, there are many possibilities to improve the quality of education and student learning when academic organizations apply reporting and analytics to assess their own strategies and operations.

It is this second potential application that was the subject of a detailed explorative study by EDUCAUSE – a nonprofit association and community of IT professionals committed to advancing Higher Education via the application of IT techniques and technologies.

The study – Analytics in Higher Education: Benefits, Barriers, Progress, and Recommendations – drew three main conclusions:

1. Education analytics can help institutions advance crucial areas such as “resource allocation, student success, and finance”
2. Higher Education institutions are collecting more data than ever before, but most is used to satisfy mandatory reporting requirements rather than to address strategic questions for competitive advantage
3. Institutions should view analytics as an investment, not a ‘cost’

The importance of analytics

Intriguingly, the study unearthed mixed results with regards to the perceived importance of reporting and analytics initiatives. Whilst just six percent of the 339 Higher Education business and IT decision-makers surveyed reported that analytics was not a priority, only 28% said that analytics was a major priority for the entire organization. However, 69% stated that reporting and analytics endeavors were considered to be a major priority for at least some departments or faculties. In fact, the study found that almost two-thirds of institutions (62%) are already using Data Warehouse and BI systems “as a way of integrating, organizing and summarizing” data sets.

It seems that most Higher Education institutions are aware of the growing importance of analytics, as it relates to education, with 84% of survey participants agreeing with the statement that “compared to two years ago, analytics is more important for Higher Education success”. Fifteen percent said that it was “just as important [as two years ago], while just 0.6% said that it was now “less important”.

Targets of analytics

The vast majority of respondent institutions are collecting data across 17 identified areas, including:

• Enrollment management
• Finance and budgeting
• Student progress
• Instructional management
• Central IT
• Student learning
• Progress of strategic planning
• Alumni
• Research administration
• Library
• Cost to complete degree
• Human resources
• Facilities
• Faculty promotion and tenure
• Faculty teaching performance
• Procurement
• Faculty research and performance

However, the survey found that there were only three areas – enrolment management; finance and budgeting; and student progress – which were assessed via analytics by 40% or more of respondent organizations. The majority (nine) of the identified areas were evaluated via analytics by less than 20% of respondent organizations.

These findings suggest there remains significant scope to increase the use of reporting and analytics within most Higher Education institutions.

Benefits of analytics

The report revealed that most Higher Education institutions perceive analytics to be of the greatest use when applied to operational endeavors directly affecting students, such as student performance, optimization of resources, recruitment and retention.

However, respondents listed a large range of other current or potential applications, leading the EDUCAUSE report to conclude: “What these results suggest is that analytics can be used in many areas and that institutions need to map their strategic initiatives to the data at hand to identify which key areas to target.”

Analytical challenges

Implementing analytics programs in large and complex environments, such as Higher Education institutions, often come with notable challenges. EDUCAUSE survey respondents identified four major challenges:

• Affordability and resources: Almost 50% of survey participants cited affordability of analytics as a major concern – both in terms of the cost and availability of internal resources to harness and manage analytical capabilities. In combating this concern, the report suggested that recent budget cuts, and increased ferocity of competition between institutions for student dollars, would actually help to justify the efficiency outcomes able to be generated from the deployment of analytic solutions. In fact, the following sentiment was expressed in multiple EDUCAUSE focus groups, conducted as part of this research: “Using the cost of tools as a reason for postponing an analytics program is counterintuitive when one considers that analytics can cut costs in the long run”. It seems that securing adequate funding remains the biggest obstacle to analytic implementations, with fewer than 20% of respondents agreeing that they currently had sufficient funding or personnel.
• Data quality, ownership, access and standardization: Almost one-third of respondents identified inaccurate data as a significant concern, citing data quality, ownership, access and standardization as potential barriers to successful analytics. Whilst subsequent focus group based research indicated that many respondents were pragmatic, and understood that attaining perfect data quality prior to analytical implementation was unrealistic, the need to “de-silo” disparate data was identified as a prevalent issue. The report suggested that unwillingness to publicize and share faculty-specific information more broadly, for the purpose of reporting and analytics, should be overcome via a ‘top down’ approach. Respondents also agreed that analytics initiatives needed to be supported, or even ‘championed’, by respected senior leaders to “encourage the sharing, standardization and federation of data”. Many participants also stated that this approach was essential, or very useful, for establishing a single uniform data repository necessary for creating a consistent enterprise-wide view of organizational data assets.
• Culture (leadership and ‘visible wins’): The report revealed that some Higher Education institutions harbor a fear or mistrust of organizational data and / or are concerned about the potential to distort and misuse analytical information in an educational context. As a result, the study concluded that it was important for organizations to establish a data-driven culture – where data-based decision-making was viewed not as a threat, but as the basis for providing better academic offerings and student services. Again, respondent feedback suggested that instilling a culture of information sharing and fact-based action was dependent upon executive leadership driving that change. To bolster this agenda, many participants also underscored the importance of publicizing ‘wins’ – instances where analytics had been used to achieve better organizational, academic or student outcomes.
• Lack of executive understanding: The fourth major challenge outlined was that, too often, organizational executives do not understand the time and expertise required to establish and manage a successful analytics program. To combat this lack of understanding, the report suggested that Higher Education institutions need to establish partnerships and clear lines of communication, between business analysts and members of the executive team, to help determine an analytics agenda and reasonable project timelines. To aid this communications process, and democratize data as an organizational asset, respondents to the report argued that IT should support, but not ‘own’, analytics programs. The idea is to ensure that analytics remains a business-driven initiative, so that:
  • It remains relevant and tied to specific business objects
  • Its benefits, or ‘wins’, remain highly visible from outside of the IT department

Importantly, over 80% of participants said that they already had a “data-oriented” leadership team, which was aware of the importance of using analytic means and methodologies to make fact-based decisions.

Education analytics: Current state-of-play

While some Higher Education institutions are already embracing BI and analytics, and others are aware of its usefulness and potential benefits, many educational organizations have room to make significant improvements.

EDUCAUSE established a list of ‘maturity themes’ via their survey and focus group research methodologies. These maturity themes were then fed back to respondents. Respondents had to agree or disagree that each of these themes was in place within their organization. The below graph presents those findings.

Encouragingly, the senior leadership teams within an overwhelming majority of institutions (87%) appear to understand the potential paybacks of BI and analytics. Pleasingly, a majority (58%) even believe that they currently possess the “right data” to embark upon an analytics project.

However, significant work is required to secure adequate funding and personnel. Only 12% of respondents believe that their institution already has the right business analysts on hand, while just 19% agree that they have sufficient financial backing to succeed in their analytic ambitions.

Recommendations for analytics success

The EDUCAUSE report outlines many methods for overcoming potential challenges and fostering successful analytics programs, and should be read in full HERE >

However, its key recommendations can be summarized as such:

• Identify the processes you want to improve, objectives you want to meet and strategic questions you want answered before embarking upon an analytics project. From there, let those goals drive your analytic efforts – not the readily available data.
• Demonstrate the value of BI and analytics by publicizing a good example of an analytics-based outcome. EDUCAUSE suggests target areas within the institutions’ enrolment processes as a good place to start.
• Invest in people and training first – invest in BI and analytics solutions once the right personnel and knowledge has been acquired.
• While developing an analytic culture and ensuring acceptable data quality are both integral, don’t wait for perfection before instigating your analytics program.
• Form partnerships between – and delivery teams made up of – IT, analysts, leaders of affected departments and members of the executive team, in order to communicate the value of:
  • Accessible and shareable data
  • Your analytics project
  • A culture of fact-based decision-making in general
• Plan for infrastructure that supports analytics use across the institution.
• Develop a plan that clearly outlines the processes and resources available to IT to help them support the analytics program – including training materials.