The 5 phases of a campus-wide technology pilot

How a grassroots movement at the George Washington University led to a successful pilot of a campus-wide ePortfolio program.

ePortfolio-committee-GWUThanks to the grassroots effort of an ad hoc committee at the George Washington University (GWU), students and faculty part of a campus-wide pilot now have access to a vetted ePortfolio program–an education-enhancing program, say the University’s researchers, that’s possible thanks to a strategic five-phase process.

According to a recent research report released by the University, ePortfolios are quickly becoming an important academic toolkit component for institutions interested in enabling students, faculty, and administrators to curate evidence of learning in creative ways that are not possible with typical paper-based methods. Not only can ePortfolios allow students to demonstrate, reflect on, and share multimedia, the technology can also be used to assess complex, higher-order student competencies, as well as help students promote their achievements to employers.

“The adoption of ePortfolios at the institutional level is growing,” emphasizes the report. According to the Association for Authentic, Experiential and Evidence-Based Learning (AAEEBL), “reports of institution-wide initiatives increased in 2013 to exceed the percent of participants reporting either a course or program/department-level focus.”

Thanks to this increasing national trend, as well as a growing dissatisfaction from GWU faculty and staff with course-based ePortfolio technology that was either free online or part of the University’s LMS, the University’s central Teaching and Learning Center (TLC) reached out to schools and departments across the institution to gauge interest in the need for a new ePortfolio technology.

The response was overwhelmingly positive, and interested faculty and admin were invited to join an ad hoc committee whose goal was to address the diverse ePortfolio needs of administrative and academic groups at the University. The result was a five-phase process.

Key Concepts Pre-Process

The report notes that the ad hoc committee adhered to research-based practices on change management within higher education before drafting the five-phase ePortfolio pilot process.

For example, according to research by Balaban, Divjak, and Mu, an institution must define its purpose(s) for using the technology, and propose a meta-model that considers all stakeholders within three levels: individual (student and faculty), institutional and employer. The research also recommends introducing ePortfolios at three levels: Strategic level (consistent with the institution’s mission and vision), tactical level (defining teaching and learning processes the technology is intended to support), and the operational level (hardware and software infrastructure, and user acceptance).

Another concept taken into consideration by the committee was of successful implementation’s reliance on adoption by the end user. Using research by Housego and Parker, the committee determined that students require both technical support and educational support, with educational support including reinforcement of the value of the ePortfolio, of the competencies expected, and of how they map to the curriculum. Faculty members and administrators also require additional support. [More detail on the types of support needed can be found in GWU’s report.]

Five Phases to a Campus-Wide ePortfolio Program

Representing five schools and three administrative units, the ad hoc committee created a five-phase program:

Phase I: Completing a needs analysis

The committee defined a common set of goals and needs for ePortfolios and explored platform options, with the short-term goal of pilot-testing one or two options. “Given the diversity of needs and the potential that one platform might not meet all of these needs, each committee member was then asked to prioritize the specific functions each required in an ePortfolio program,” says the report.

Phase II: Selecting a platform for pilot testing

Based on the needs and goals identified, five ePortfolio solutions were chosen for analysis: D2L, PebblePad, Digication, Pathbrite, and TaskStream. Using criterion-weighting software called Comparion, members were able to weight the importance of different features and functions and evaluate the platforms anonymously. Some of the weighted features and functions included: “Support student reflection throughout their degree programs and other learning experiences to help students make sense of their learning,” and “Demonstrate and capture program outcomes and competencies to support accreditation and broader assessment activities.” [More goals and needs can be reviewed in the report.]

Phase III: Usability testing

Following vendor demonstrations, the committee selected Digication, PebblePad, and Pathbrite for further analysis through hands-on usability testing. 25 students from undergraduate and graduate programs participated in a 30-minute testing period in the campus computer lab. The students were asked to perform 11 basic web publishing tasks using the same pre-made web content. If needed, students were given basic help documentation for each ePortfolio platform. Student feedback was collected organically throughout the 30-minute testing period, as well as within a brief end-of-use survey. [Specific student tasks, as well as the surveys used can be seen in the report.] Although no single platform was favored by a clear majority of participants, 92 percent of students completed tasks with little to no assistance, and 70 percent recommended the committee pursue using ePortfolios.

Phase IV: Pilot testing

Based on the usability testing results, qualitative comments and committee discussion, Digication was selected for pilot testing, due to its “capacity to support creativity in the ePortfolio creation process, its robust assessment features, and the ability to link ePortfolios to competencies for accreditation,” says the report.” The committee received an internal grant from the Office of the Provost to fund “inter-professional and inter-disciplinary collaboration and innovation, one of the pillars of the newly developed University Strategic Plan.”. The grant enabled the committee to guarantee funding for 1,000 seats for one year and to guarantee that all students participating in the pilot could keep their ePortfolios for the duration of their enrollment at the University. The grant also provided additional funding for the vendor to provide onsite training to admin, faculty and students. Also included in the funding was support for the committee to undertake outreach activities to promote the pilot, manifesting in a planned ePortfolio showcase day.

The pilot test began in September 2014 and concluded in July 2015. Nine pilots were conducted in various departments. For instance, the Milken Institute School of Public Health integrated ePortfolios and reflective practice into a competency-based hybrid Executive Master’s program in Health Administration. The portfolio will serve as an academic record of program competency attainments over the program’s duration and as a location to house research, data, and other evidence relating to a year-long health systems quality and performance improvement capstone project. [More information on the other eight pilots can be found in the report].

Phase V: Evaluation

A total of 555 students and 141 faculty and admin have established accounts with Digication and have participated in the pilot project. The committee identified common themes for evaluation (based on the weighted needs and goals defined in Phase II), as well as those needed to determine whether or not the solution is an appropriate long-term solution that the University should sponsor. Also, the committee evaluated whether the process “encourages faculty and staff to work across programs and disciplines to use ePortfolios to support student success before and after graduation,” states the report. Student and faculty surveys were designed to be administered before and after using the Digication platform [these can be reviewed in the report]. The committee plans to summarize the results shortly, as well as compile recommendations for University leadership. If the evaluation supports adoption of the ePortfolio platform, the goal is to request funding for an institution-wide adoption in the upcoming fiscal year.

GWU researchers emphasize that while the five-phase ePortfolio program is institution-specific, and the user sample size is relatively small, “grassroots technology initiatives like the one presented…can positively impact the broader university mission in many ways.”

For instance, multiple departments with diverse needs came together and identified commonalities to successfully launch an important campus-wide initiative. Also, funding support from university leadership and a centralized development process organized through the University TLC, “have enabled a robust implementation and evaluation process of a single technology platform that will hopefully meet the long-term needs of this very diverse community. Moreover, the committee has evolved into a learning community that has enhanced the knowledge and technical skills of its members,” concludes the report. “By bringing this expertise and information back to their home units through the pilot project, these early adopter ePortfolio champions have planted the seeds for a significant and sustainable educational innovation.”

Read the full report, “Developing a Pathway for an Institution Wide ePortfolio Program.”