Hard Questions About ePortfolio Evidence
What is Valid and Reliable ePortfolio Evidence?
As AAEEBL places new emphasis on the “evidence-based learning” part of AAEEBL’s name in AAEEBL conferences this year, the question naturally arises – at least if one adopts a critical perspective – “what is valid and reliable eportfolio evidence?”
ePortfolio literature is rich in accounts, based on various research approaches, about the impact of eportfolios. It is rich in institutional stories, assuring readers – rightly – that an eportfolio initiative produces many valuable outcomes. But the literature is not rich about the question posed above about valid and reliable eportfolio evidence.
In The International Journal of ePortfolio archives – IJeP at http://www.theijep.com -- it is hard to find any description in any published article that describes eportfolio evidence, what it is, what guidelines are offered to learners, or what consensus exists in the field of eportfolio studies about the question of evidence – what is valid and reliable eportfolio evidence?
Do Guidelines for Evidence Exist?
On individual campuses, such guidelines may exist, perhaps embedded in introductory courses about using eportfolios, or in capstone eportfolio courses. Faculty members using eportfolios in their courses may set standards for eportfolio evidence, or service learning or coop learning or internship offices may also promote certain approaches and standards for collecting evidence and for certification of that evidence.
The eportfolio field – the field of evidence-based learning or of “folio-thinking,” a Stanford University phrase – is at the point of needing to begin developing consensus on eportfolio evidence – a taxonomy, a theory, guidelines, and other initiatives. What does a person need as eportfolio evidence in different fields study, at different points in a learner’s development, and for different purposes?
Badges as Evidence
One form of evidence – digital badges – that can be linked to from an eportfolio, defines the criteria behind the awarding of the badge. Digital badges reside within a national infrastructure for badges and have a strong following in the eportfolio community. Clicking on the badge reveals not only the criteria behind the badge but information about who awarded the badge and what pedigree the badge has.
The knee-jerk reaction among those educators new to badges is to assume that badges are certified by teachers, but that quick reaction imposes a preconception – that learning is only important if it is certified by a teacher – that misses the point of badges. Badges are a form of micro-credentialing of a specific skill that often is awarded by peers in a working team or others in a supervisory position outside of academia. If used in this broader curricular and non-curricular manner, badges may be among the most valid and reliable forms of evidence if those viewing the badge feel they can trust the issuer of the badge. A collection of badges in one field of study can add up to a strong statement of competence in that field.
Going Beyond ePortfolio Owner Testimony
But, those who search the web for samples of individual learning eportfolios generally find that “evidence” in an eportfolio is often simply testimony by the learner that they did something. If I was hiring someone or admitting someone to a graduate program, I would want better evidence than just the written testimony of the applicant.
Certain professions do have standards for eportfolio evidence, notably education, nursing, physical therapy and other “clinical practice” fields. High-impact educational practices such as undergraduate research or first-year seminars and others may also have standards for valid and reliable evidence of achievement.
Alverno College, a pioneer in new forms of assessment and in eportfolios, says this on their website about the Alverno portfolio:
This first-of-its-kind, web-based system was implemented in 1999 at Alverno College. The DDP enables Alverno students — anyplace, anytime — to follow their learning progress throughout their years of study. It helps students process the feedback they receive from faculty, external assessors and peers. It also enables them to look for patterns in their academic work so they can take more control of their own development and become more autonomous learners.
The Diagnostic Digital Portfolio (DDP) is built on Alverno's student assessment-as-learning process. It makes the process more transparent to students and others who seek to understand this important educational program. It also provides actual, accessible performance data with which graduates can create an electronic resume for potential employers or for graduate schools. (http://ddp.alverno.edu/)
The evidence in the Alverno portfolio may, however, show strong evidence of being able to perform academically and may do this reliably, but is it valid evidence of being able to do something outside of an academic learning model? The workplace success of Alverno graduates may prove the validity of the eportfolio evidence in the Alverno portfolio.
Nicole Buzzetto-More advocated the inclusion of a “Commentary on Evidence” section in eportfolios so the eportfolio owner could strengthen the claims made in the eportfolio. This “Commentary on Evidence” might be likened to an annotated bibliography attached to an article: being able to be objective about your own evidence is a sign of intellectual maturity, pretty strong evidence in itself. (The E-portfolio Paradigm: Informing, Educating, Assessing, and Managing with E-portfolios 2010)
An article published by two researchers at Curtin University in Australia makes the case for collecting evidence of “work-integrated learning” (WIL). Does evidence from the workplace better convince potential employers than evidence from the classroom?
There is substantial evidence that eportfolios provide an ideal platform to evidence program-wide WIL [work-integrated learning] achievements (Edwards & Burnham, 2009). It provides a setting for students to clearly demonstrate that they not only ‘know’ discipline content but that they can ‘do’ by applying the knowledge in a professional context. This opens up authentic assessment task options in preference to traditional assessment methodologies thus providing rich contextual demonstrations outside the limitations of customary assessment paradigms. Employers have expressed interest in this mode of assessment (Cai, 2012) as they have more confidence in the overt demonstration of the achievement of authentic learning experiences which transcend the classroom environment. http://www.apjce.org/files/APJCE_15_3_269_280.pdf
“ePortfolios as evidence of standards and outcomes in work-integrated learning.” Asia-Pacific Journal of Cooperative Education, Special Issue, 2014, 15(3), 269-280.
Sonia Ferns and Jude Comfort, Curtin University, Australia.
Motivations of ePortfolio Researchers and the Needs of the Field
ePortfolios are in their second decade of national attention and broad-scale adoption but the movement is still faced with the root question of “what is an eportfolio” and “why should we adopt eportfolios.” But, maybe, instead of continuing to answer these questions, the field needs to move toward aggregating wisdom. The Connect to Learning Project at LaGuardia Community College did aggregate wisdom from 24 campuses between 2010 and 2014 and produced the Catalyst for Learning site at http://c2l.mcnrc.org/ to share the fruits of their labors.
In particular, we need to aggregate wisdom about the nature of eportfolio evidence. I say this as myself but also as a leader of AAEEBL, uncomfortably aware of what agenda I may be setting for AAEEBL as a professional association in the field of eportfolios.