WP 2 Meeting - December 2nd 2011

On the 2nd December 2011 Farhi and Eamonn met to discuss progress within Work Package 2 and to discuss targets which have been achieved as well as set milestones for the upcoming weeks.

First of all, here are WP2 Work package deliverables which have been achieved already:

1. Implementation of the eRaUI widget (Portable Content Search Widget) in such a way that it is portable and deployable upon any web application or interface. The widget enables content search which intelligently adapts to the content of the interface on which it is deployed.
2. Implementation of click / mouse movement heat maps which provide rich analytics based upon users' interaction with the interface upon which eRaUI is deployed.
3. Implementation of user Ontology, a catalogue of keywords and strings which are matched to weightings that indicate their relevance to the various user categories defined for the interface. The nearest neighbour algorithm is used in conjunction with this to determine the categories of new users.
4. Text Mining which filters user selection from the e-Research tool services has been implemented at the point of data collection, which occurs in real-time.
5. Implementation of the Machine Learning-Nearest Neighbour- algorithms within eRaUI - so as to match users' patterns of behaviour to categories as defined in the site category meta-dictionary.

The following WP2 deliverables have yet to be achieved; they have been enumerated below along with their respective deadlines:

1. Breadth Search - featuring the use of window panes so as to allow users to preview other relevant e-Research tool services related to those he / she searched for. We intend to implement this before the middle of January 2012.
2. Workflow Capabilities - We intend to implement a means by which we can guide users between content using workflow-based logic. We aim to have this completed by the middle of January 2012.

We also discussed the possibility of implementing a separate meta-explorer interface which would be separate from the eRaUI widget and based upon the data collected within the analytics and text-mining components of the software.

WP1 Meeting 2nd December 2011

On the 2nd December 2011 Eamonn and Farhi met to discuss WP1 deliverables and progress:

• We discussed moving eRaUI and all associated systems to the university server. We have allocated up to two weeks to complete this fully and ensure that everything is working properly.
• Christmas meeting - we decided that we should try to arrange a meeting before the end of the term in order to review progress. This would take place somewhere outside of the university.
• We are also proud to unveil the new official eRaUI logo, which sports the colours of London Metropolitan University in addition to aspects of the university logo. Here it is:

Testing and Evaluation of initial eRaUI Prototype - WP4

On Monday the 28th November 2011 we undertook the first formal testing of an eRaUI protoype using MSc students at the London Metropolitan University Faculty of Computing. This was a very small initial test and involved 4 students, none of whom had any prior experience of eRaUI or the concepts related to this project.

The testing process consisted of assigning a set of 8 questions to each of the 4 participants. The participants answered a subset of these questions using a proxy version of the NaCTeM website on which the eRaUI widget was embeded, and another subset consisting of a proxy of the website without the widget. eRaUI recording and analysis tools were deployed on both proxy websites. The questions were divided up so that each participant answered 4 questions using the website with the widget and 4 questions on the version without the widget. Users were given a maximum of 2 minutes to complete each question, and the results and completeness of users' answers were recorded so as to allow comparison of the efficacy of embeding the eRaUI widget on the NaCTeM interface as compared to using the interface without the widget.

Upon completion of the 8 questions, participants were asked to complete a quick questionnaire asking questions as to which version of the website facilitated a better perception of user experience. The participants were then invited to give feedback by means of a brief informal spoken interview.

This intitial testing process has yielded some very useful feedback and data which can be analysised so as to best know how to go forward with the research and development of the eRaUI widget tool. Results of this analysis, which will include the recordings produced by the eRaUI analysis tools, are pending and will be disseminated in a forthcoming blog post.

WP2 Meeting - 24th Nov 2011

On the 24th November 2011 Eamonn and Farhi met to discuss work package 2 deliverables and progress. We discussed the implementation of Nearest Neighbour algorithms within eRaUI to match content to particular users based on automatic user categorisations. We also showed how text mining has been implemented in such a way that we are able to extract information from user sessions and structure the information so as to inform automatic categorisation of users. The process of extracting and structuring information is undertaken in realtime as users navigate the interface on which eRaUI has been deployed.

We also discussed testing plans for the prototype eRaUI widget. We shall be testing the widget next week.

WP2 Meeting 16th November

Farhi and Eamonn met up on the 16th November to discuss Work Package 2 deliverables and progress. First of all we discussed the proposed eRaUI logo which is currently under development. We have decided to adopt an arrow design using some elements from the London Metropolitan University logo, including the colours. This logo will be unveiled shortly once final considerations have been addressed.

The next point of discussion centred around the design and implementation of Nearest Neighbour algorithms within eRaUI. We discussed a current model which is now operational, and also considered a new model which will involve attributing a weighting to each link within a web application. These weights will then be summed wherever there is a 1 to 1 match between a user's click history and each previous existing use case. This will form the kernel of a sophisticated algorithm which should enable us to classify users most effectively.

Work Packages 1 and 2 - Meeting

Today (10th November) Farhi and Eamonn met to discuss Work Package 1 and 2 deliverables and progress. Primarily our conversation centered on the following two areas:

• Configuration of the university server to host the NaCTeM codebase. Difficulties have been encountered in enabling PHP short tags so as to allow the current codebase to execute correctly. This problem has now been resolved and the code is running according to expectations.
• Implementation of nearest neighbour algorithms within eRaUI in order to classify users into the correct level. Work has been done on this already, and we decided that we should store in the database a historical record of users matched to url clicks and categories. This can be used to generate a case-base which will then be used to categorise users as novice, student, expert etc.

We have also proposed a test plan for the eRaUI widget (see previous blog post) which forms a component of our WP4 deliverables.

eRaUI Testing and Prototype Evaluation Plan - Work Package 4

As part of our Work Package 4 deliverables we require a means of testing and evaluating eRaUI to try to quantify the effectiveness of the prototype which has been developed so far. Although we cannot yet gain interactive user feedback in regard to the finished software, it would be useful to test the efficacy of the current model in regard to enhancing user experience and productivity on the NaCTeM website. To this end, we propose to do the following:

1) Assemble a test group of subjects, perhaps only 4 persons for the initial test. These could be staff, students or Ph.D. students, although it would be preferable not to involve persons who have direct experience with eRaUI already.

2) Decide upon a set of three tasks which could be performed on the NaCTeM website. These would vary in complexity and could for instance involve:

• Sending some feedback via the feedback form.
• Looking for a paper using the text mining tools on the NaCTeM website.
• Downloading U-Compare and learning how to use it to complete a simple text-mining task.

3) Divide the test group into two halves. One half would be asked to complete the three tasks on a NaCTeM clone website without the widget enabled, and the other would be asked to complete the task on a version of the site with the widget.

4) Using the recordings produced from the test user interaction along with times taken and verbal feedback, we could assess to what extend the eRaUI widget improves the experience of interaction with the NaCTeM website, and to what extent it improves the times taken and the correctness of results obtained using the site alone as opposed to using the widget-enabled version.

This simple method of testing should hopefully enable us to glean some insights into the initial reactions to eRaUI and will form a vital part of our Work Package 4 deliverables. Later as the software develops we can expand this testing methodology so as to incorporate a larger test group and a wider range of tasks.