librarians-faculty-communication

Universities find a collaborative way to measure and obtain success


An Amazon Web Services infrastructure has allowed for a predictive analytics framework to determine college and university at-risk students.

PAR-data-institutionsBy using student data from partner colleges and universities across the country, it can identify more than 75 percent of an institution’s at-risk students, with the ability to save millions of dollars in lost tuition revenue  – as much as  $7 million at a campus with 10,000 students  by identifying students twice as likely to drop out and intervening with them. It can also help identify, measure and prescribe intervention services.

It’s called the Predictive Analytics Reporting (PAR) Framework, a non-profit, analytics-as-a-service provider, and its success comes not only from a revolutionary infrastructure from Amazon, but from the 25 million cross-institutional course-level records from over 2.5 million students.

“With more than 5o member institutions spanning over 351 unique campuses, all using common data definitions and insight tools, [PAR Framework] is the only national multi-institutional lens for examining dimensions of student success from both unified and contextual perspectives,” said Beth Davis, CEO of the PAR. “PAR member institutions collaborate on identifying points of student loss and to find effective practices that improve student retention in U.S. higher education.”

PAR is also being covered by Gartner, an IT research and advisory firm, which calls PAR a “major step forward” for higher education.

“…In this complex endeavor we recommend a ‘learning by doing’ approach and joining or at least studying the PAR Framework project experience. This is the most advanced openly available information in higher education to our knowledge,” noted Jan-Martin Lowendahl, VP distinguished analyst for Gartner in a 2014 report.

“We’re innovative because we’re unlocking the potential of scale,” emphasized Davis. “Using comparative data with a common language among institutions allows any institution—large or small, community or state, traditional or progressive—to crack the code on what works and what doesn’t in student successprograms.”

(Next page: How PAR works; the power of transparency)

Leveraging Amazon’s infrastructure

PAR Framework originally began as a Gates Foundation-funded project in 2011, with support from the Amazon Web Services (AWS) Education Grants Program.

AWS in Education allows educators and academic researchers to apply to obtain free usage credits to tap into the on-demand infrastructure of the Amazon Web Services cloud.

Often, large research projects require extensive compute power and storage infrastructure to complete. Instead of purchasing a large amount of hardware, researchers can get started by opening an AWS account, eliminating much of the heavy lifting of provisioning and configuring open-source software frameworks for processing very large data sets.

“We needed to build a data warehouse able to analyze and compare numerous, diverse data sets,” Davis explained. “We require a scalable backend, able to grow and adapt, as well as provide high-performance access. We also needed an infrastructure that’s as security-conservative as higher-ed institutions. Really, we needed best of breed.”

After PAR secured a hosting grant during their research phase, when PAR was ready to emerge into its current operations as an independent 501c3 in January 2015, AWS provided the flexibility that allowed the non-profit to shift all the existing infrastructure to its private accounts, maintaining the infrastructure’s scalability and security. Operations went on uninterrupted for PAR and its member institutions.

Protecting student privacy

PAR gathers a rich set of student-level course and demographic data from members, which is then used to develop institutional-, program-, course- and student-level descriptive analytics and predictive insights. The PAR ecosystem of reports, predictive models, tools and frameworks is geared at improving student success, retention and credential completion.

Therefore, said Davis, preparing and securely transmitting anonymized student-level data is a key part of the process.

“Institutional partners gather data from their local systems according to the PAR Framework common data definitions and a detailed file specification,” Davis explained. “As a last step before data submission, they remove any personally identifiable data, including date of birth, social security number and local student ID number and replace those items with a PAR student ID. Institutions maintain a translation table of their internal ID to PAR Student ID which will be used to easily re-identify those students after the data has been analyzed by PAR. PAR never receives, or has access to, student-level identifiable data, adding a layer of privacy and security for student records.”

Why PAR works

1. The first way PAR helps institutions is through its AWS-hosted Student Success Matrix (SSMx), which gives institutions a means to inventory, organize, and conceptualize interventions aimed at improving student outcomes.

“Often there’s no single version of the truth about student success programs on a campus, since issues with student retention could come down to multiple departments or programs,” said Davis. “Until an institution has a comprehensive picture of campus intervention systems and student success programs, it’s hard to know what to leverage.”

According to Davis, the SSMx essentially inventories and categorizes all interventions on campus, with unlimited access to faculty, staff, and administration to interact with the intervention data.

“This allows a campus to quickly identify overlaps and gaps,” she noted.

PAR also creates “Student Watch List” reports, generated from predictive models custom to each member institution. The watch lists evaluate every student in the institutional data set, assigning risk scores, and revealing the individual student characteristics that contribute to risk of the student not passing key Obstacle Courses [courses with high drop-out or fail rates] or not being retained at the institution. These watch lists can be downloaded by members, and using translation tables, PAR members can restore the identities of the students in the watch list, enabling the institution to take action on the insights revealed through predictive modeling while also not putting student privacy at risk.

(Next page: Closing the loop; data for non-traditional learners)

2. “The most innovative function of PAR is what we like to call solving the ‘Now what?’ problem,” said Davis. “It’s not enough to identify which students are at-risk and which courses are obstacle courses. Colleges and universities need to know what to do about it. We call this closing the loop.”

PAR’s data is able to provide institution-specific, statistically-validated recommendations on which intervention strategies and/or programs need work, as well as what other interventions the institution may want to try.

“We can show our member institutions ‘here are your  available interventions, and here’s what other interventions from partner institutions are successful for similar campus goals,” she said.

From providing PAR student data to receiving predictive and intervention insights, the entire process takes weeks, not months.

Helping colleges and universities determine which courses are the most challenging for students, as well as providing action plans, can produce compelling results. One PAR member realized a 25 percent increase in pass rates when it tackled issues in a PAR-identified obstacle course. Applied across all the obstacle courses on that campus, that institution has the potential to recoup more than $3 million a year, says PAR.

“Institutions can report data back to us after implementing our data-backed recommendations, so that’s how we came to these numbers,” said Davis.

3. Transparency in how PAR uses institutional data, we well as its common definitions, provides institutions what Davis calls open, common language that unlocks the potential for scale.

“Sometimes with predictive analytics solutions, results come across as if from a black box shrouded in mystery as to how the data is analyzed. Not with PAR,” she emphasized. “We have openly reveal how student characteristics put them at risk all of which is based on readily available data and publicly available definitions, which also provide a common language among institutions.”

“PAR’s project’s release of the data definitions as a Creative Commons license is a major step forward and a possible competitive advantage over commercial competitors such as Civitas Learning,” wrote Lowendahl.

The data definitions were published in 2013 using the Data Cookbook, a collaborative data dictionary and data management tool for higher education built by iData Inc.

Using the online Data Cookbook “was a very beneficial way to develop the overall parameters of the individual data variables,” said Vincent Maruggi, director of institutional research at Broward College. “The iterative, cooperative process unearthed a number of variables that had multiple interpretation options that needed rational and consistent consideration, especially considering their application across the diverse set of institutions involved in PAR.”

Maruggi attested that working with the other institutions within the project was a positive experience.

“The dialogue was highly productive and forced me to investigate deeper into Broward College’s student data system for elements that were not previously utilized in a like manner, while also giving us a new conceptual structure for future reporting.”

Into the future

Recently, PAR announced a partnership with the American Institutes for Research (AIR) to extend higher education outcomes metrics and measurement efforts at the national policy level.

This partnership will feature the collaboration of more than 11 U.S. higher-ed institutions focused on serving non-traditional student populations coming from the for-profit and not-for profit sectors.

“Today’s metrics used to analyze the performance of institutions don’t often reflect the growing number of non-traditional students, instructional methods, business models, and data resources that distinguish contemporary higher education,” noted Davis. She believes that this collaboration with AIR will inform benchmarking development in higher ed, as well as provide initial insight into performance of for-profit and alternative delivery models including online learning and competency-based education.

PAR’s website also noted that the effort will aim to “identify potential improvements to federal data collections, statutory disclosure and reporting requirements, especially with regards to transfer students and adult learners.”

For more information on PAR Framework, as well as how to become a member, click here.

Sign up for our newsletter

Newsletter: Innovations in K12 Education
By submitting your information, you agree to our Terms & Conditions and Privacy Policy.

Oops! We could not locate your form.

Sign up for our newsletter

Newsletter: Innovations in K12 Education
By submitting your information, you agree to our Terms & Conditions and Privacy Policy.