STLHE 2015 has ended
Achieving Harmony: Tuning into Practice
Wednesday, June 17 • 4:00pm - 5:30pm
POSTER.40 - Evidence-based recommendations to improve the accuracy of peer-evaluation of written assignments.

Sign up or log in to save this to your schedule and see who's attending!

Peer-evaluation is used frequently in higher education, as both a supplement to instructor grading (Sho, Schunn & Charney, 2006), and as a replacement to instructor grading (Harris, 2011). Many factors likely contribute to the success of any peer-review activity (skill level, number of assessments etc.), however these are not well defined (Topping, 2010). The accuracy and reliability of peer-evaluations is an important consideration, as research has shown examples at both ends of the spectrum (Sho et al., 2006, Yankulov & Couto, 2012). The objective of this study was to systematically determine which factors (course, assignment and student related) have the strongest influence on the accuracy of peer-assessment. We identified 17 variables and ranked their correlations with accuracy and reliability across three courses from different disciplines that used peer-assessment on written assignments (research proposals, term papers) over four years (>1000 peer-reviews). We then altered the single most significant variable in one course to confirm our prediction. We demonstrate that the number of reviews completed per reviewer has the greatest influence on the accuracy of peer-assessment. Our calculations suggest that six reviews must be completed per reviewer to achieve quantitative peer assessment that is no different from the instructor. Effective training, previous experience and strong academic abilities in the reviewers may reduce this number. This poster will provide evidence based suggestions for instructors to encourage accurate peer-evaluations in their classrooms. Importantly, it will also indentify factors that don’t seem to influence the success of peer-evaluation, allowing flexibility for instructors._x000D_
1. Cho, K., Schunn, C. D., & Charney, D. (2006). Commenting on writing: typology and perceived helpfulness of comments from novice peer reviewers and subject matter experts. Written Communication 23 (3), 260-294._x000D_
2. Harris, J. R. (2011). Peer assessment in large undergraduate classes: and evaluation of a procedure for marking laboratory reports and a review of related practices. Advances in Physiology Education 35, 178-187._x000D_
3. Topping, K. J. (2010). Methodological quandaries in studying process and outcomes in peer assessment. Learning and Instruction 20, 339-343._x000D_
4. Yankulov K, Couto R. (2012). Peer Review in Class: Metrics and Variations in a Senior Course. Biochemistry and Molecular Biology Education, 40 (3), 161-168.

Wednesday June 17, 2015 4:00pm - 5:30pm
Bayshore Foyer

Attendees (0)