Evaluating online learning involves evaluating educational design, technology use, and “online instructional practices that integrate technology appropriately for developing and delivering quality online courses” (Bangert, 2008, p. 28). Research shows that classroom-based evaluation tools do not adequately capture the relationships between content, pedagogy, and technology in online courses (Berk, 2013). In response to this evidence, the Faculty of Arts and Science, which offers a large variety of online courses and programs, sought an alternative to the collective agreement-endorsed, university-wide evaluation tool, which is designed for classroom learning. An instrument developed by Bangert (2004) to assess constructivist-compatible online teaching practices, the Student Evaluation of Online Teaching Effectiveness (SEOTE), was adapted and piloted in 2014. The evaluation tool, like a harmonic chord, has different meanings in different contexts, some of which are discordant. This presentation explores those contexts, and explains the process by which consonance was achieved over two pilot phases. Using Stobart's (2009) validity framework as a guide, this process included negotiations with the university’s faculty association, statistical analyses of the data, and student and instructor focus groups to ascertain the tool’s usefulness. Audience members will be stimulated to consider factors that differentiate effective evaluation of online courses from classroom-based teaching. They will gain insight into strategies for introducing a new evaluation tool in a unionized environment, and methodologies for evaluating the tool itself. During the presentation, the audience will be invited to brainstorm items on which an online instructor/course could be assessed, which will then be compared to the SEOTE.
References:
Bangert, A. W. (2008). The development and validation of the student evaluation of online teaching effectiveness. Computers in the Schools, 25 (1-2), 25–47.
Bangert, A. W. (2004). The seven principles of of good practice: A framework for evaluating online teaching. The Internet and Higher Education, 7(3), 217–232.
Berk, R. A. (2013). Face-to-Face versus Online Course Evaluations: A “Consumer's Guide" to Seven Strategies. Journal of Online Learning & Teaching, 9(1). http://jolt.merlot.org/vol9no1/berk_0313.htm
Stobart, G. (2009). Determining validity in national curriculum assessments. Educational Research, 51(2), 161–179. doi:10.1080/00131880902891305
