Enabling best practice for outcome-based assessment facilitating interoperability based on standards and specifications

In the current e-learning landscape there are a variety of standards and specifications related to the assessment process; among them, the IMS QTI specification stands out as the de facto standard. But it is not as widely used as expected, and when it is implemented, several approaches could be followed deriving into interoperability problems among assessment systems. However, there is a lack of recommendations or guidelines to implement the assessment specifications and even fewer when dealing with outcome-based assessment.

Recommendations

1. To use IMS QTI 1.2 when developing a new assessment system or upgrading an existing one. Particularly the profile defined in the IMS CC 1.0 because it is intended for interoperability.
2. To mask the complexity of assessment standards and specifications.
3. Formats used in assessment processes should support the quick provision of relevant formative feedback to students.
4. To use intended learning outcome information to enrich assessment resources in an open federated repository, taking into account that the learning outcome information must be complete.
5. Adopting of integrated environments for outcome-based assessment.
6. Sharing and re-using of assessment resources across the organization to leverage teachers' workload, saving time and effort.
7. ICOPER recommends HE management to use assessment concepts from standards and specifications in the institutional assessment system.
8. It is highly recommended that training faculty members use systems that implement assessment concepts from standards and specifications in the institutional assessment system.

How did we get there?

An in-depth analysis of the assessment and evaluation standards and specification has been carried out. This analysis consisted of two parts: a) a theoretical analysis of the specifications from an outcome-based perspective; and b) a study of their usage in European higher education institutions. An emphasis was put on studying the interoperability issues derived from the usage of the standards and specifications, including a pilot experience demonstrating the exchange of questions and tests among different learning management systems.

Out of the previous analysis and literature review, several contributions have been provided to the ICOPER Reference Model (IRM): a data schema for outcome-based assessment resources interoperability and several processes related to learner assessment and course evaluation. These models were disseminated and discussed in several international events involving different stakeholder groups.

In order to validate these contributions, an application prototype has been developed for the .LRN learning management system demonstrating the use of the data schema and the assessment for the different stages of the assessment process. The application helps teachers and course developers to author and manage assessment resources in an outcome-based format, facilitating the development and deployment of the assessment process in outcome-based education. The application also provides functionalities to update the learner's profile with the achieved learning outcomes during a course. This application was successfully evaluated in several training events, ranging from a series of hands-on workshops to interviews with teachers and course developers.

These contributions have been produced and supported by empirical evidences gathered from our work and constitute the main outcomes of the project. They address issues concerning different stakeholders: higher education management and faculty, technology providers and standardization bodies.

Visit

IMS QTI:
http://training.icoper.org/moodle/mod/resource/view.php?id=293

.LRN application documentation:
http://training.icoper.org/moodle/mod/resource/view.php?id=238

Online demonstration:
http://mozart.gast.it.uc3m.es:9000

Reports:
http://www.icoper.org/results/deliverables