Thursday 10 November 2011

eRaUI Testing and Prototype Evaluation Plan - Work Package 4

As part of our Work Package 4 deliverables we require a means of testing and evaluating eRaUI to try to quantify the effectiveness of the prototype which has been developed so far. Although we cannot yet gain interactive user feedback in regard to the finished software, it would be useful to test the efficacy of the current model in regard to enhancing user experience and productivity on the NaCTeM website. To this end, we propose to do the following:

1) Assemble a test group of subjects, perhaps only 4 persons for the initial test. These could be staff, students or Ph.D. students, although it would be preferable not to involve persons who have direct experience with eRaUI already.

2) Decide upon a set of three tasks which could be performed on the NaCTeM website. These would vary in complexity and could for instance involve:
  • Sending some feedback via the feedback form.
  • Looking for a paper using the text mining tools on the NaCTeM website.
  • Downloading U-Compare and learning how to use it to complete a simple text-mining task.
3) Divide the test group into two halves. One half would be asked to complete the three tasks on a NaCTeM clone website without the widget enabled, and the other would be asked to complete the task on a version of the site with the widget.

4) Using the recordings produced from the test user interaction along with times taken and verbal feedback, we could assess to what extend the eRaUI widget improves the experience of interaction with the NaCTeM website, and to what extent it improves the times taken and the correctness of results obtained using the site alone as opposed to using the widget-enabled version.

This simple method of testing should hopefully enable us to glean some insights into the initial reactions to eRaUI and will form a vital part of our Work Package 4 deliverables. Later as the software develops we can expand this testing methodology so as to incorporate a larger test group and a wider range of tasks.

No comments:

Post a Comment