CAP 4104 Human and Technology Interaction

Assignment 2

This is the final version of Assignment 2 that was formally assigned in class on Feb. 7, 2012.
The purpose of this assignment is to provide students with experience in performing user testing of the usability of an existing system.
It is also intended to provide students with experience in finding information beyond that presented in class on how to apply usability methods.

Students will use the e-Commerce Web site that they used for Assignment 1 and evaluate it via user testing.

NOTE: If you have not completed Assignment 1, you will be unable to complete step 5 d below.

Steps in completing this assignment:

  1. Plan your user testing
    1. your test goals,
    2. how you will conduct the user tests
    3. how you will collect and evaluate the results, including                                                              
      1.   performance measures
      2. observing the use
      3. follow up questions

NOTE 1: This should involve your finding out more about user testing on your own, beyond what is presented in the class and the text. The instructor will not provide references - it is part of the assignment for the students to identify suitable sources of information on user testing.

NOTE 2: If you have any other specific questions about this assignment, be sure to ask them as soon as possible.

  1. Identify and develop descriptions of what you will be testing, this will involve
    1. a minimum of 3 different tasks that a user could perform on the web site

NOTE: It is ideal for you to evaluate as many different tasks as possible, but while it would be nice to include more scenarios, it is important that all scenarios can be completed in a maximum of between 30 and 45 minutes of user testing.

    1. a minimum of 5 distinctly different scenarios (which are what you actually will be testing)
    2. there must be at least two distinct scenarios for at least one of the tasks.

NOTE 1: The descriptions of the scenarios should focus on what needs to be done rather than how to do it, so that you can give these scenarios to test subjects and use them to see how they try and accomplish them with the web site.

NOTE 2: For scenarios to be of use, you need to identify in advance each of the logical steps that must be accomplished in order to successfully accomplish the scenario, whether or not you choose to provide this information to your test subjects.

NOTE 3: Any scenario that involves attempting to purchase a product should not actually involve submitting a bogus or real order to the owner of the web site, but it has to involve sufficient interactions up to the point of submitting an order to allow sufficient testing to determine relevant usability issues.

  1. Recruit a minimum of 4 individuals to test the web site for you. You need to obtain informed consent from each of them to participate. This informed consent should include your explaining:
    1. The purpose of your user test and the procedure that you will be using
    2. Any potential benefits and risks that they might receive from their participation
    3. Their right to withdraw from the user test at any point during the test
    4. How you will protect their identity (e.g. confidentiality of any published results)

NOTE 1: You should select your participants to represent a wide range of possible user characteristics. You will be expected to report the characteristics that apply to each user, but not the names or other personally identifying information about them. You should also consider how these characteristics relate to results in your analysis of results.

NOTE 2: You may not use students from this class as participants in your usability tests.

  1. Individually conduct the user tests according to your plans.

NOTE 1: Do not wait too long to complete this, there is still a lot to do after the tests are completed.

NOTE 2: If you do not get useful results from an individual test subject, you should recruit an additional test subject and conduct an additional test so that you have 4 sets of useful results.

  1. Develop a well-structured and well-written report of your user testing, with the following sections:
    1. The planning and conducting of the testing, including:
      1. Your plan for the testing (from step 1 above)
NOTE: You should formally acknowledge, with suitable academic references, the sources of information that helped you with constructing this plan

      1. Descriptions of the tasks and scenarios (from step 2 above) and discussions of how you will obtain the measures you chose to observe / measure during the testing (including any that will be asked in a pre-testing and/or a follow-up interview/questionnaire).
NOTE: This should include the actual wording of the scenarios that you provided to your research participants and any testing or interview scripts / questionnaires used

      1. Brief descriptions of the characteristics of each of your research participants, without actually identifying them. (from step 3 above)
      2. A brief discussion of how the testing was conducted and any changes to your initial plans that you had to make.
      3. A brief discussion of how you analyzed / evaluated the results.
    1. The results of the testing. Results are objective statements of what measures you obtained from the testing. They should be presented on their own without any personal discussions from the person doing the testing. 
    2. The analysis of the results. Analysis of results where you comment on what you think the results mean and what should be done about them.
    3. What you have learned about the differences between user testing and expert evaluations. This should start with a comparison of your results from this user testing with the results from the expert evaluation you conducted in Assignment 1. But it needs to go beyond presenting this comparison of results to include an analysis of what this means to you.
  1. Submit your assignment electronically via WebCourses by 12:00 noon on Monday Feb. 27.

NOTE: This is a time consuming assignment. Do not leave this too long and do not expect any extensions. If necessary it is better to submit an incomplete assignment than not to submit anything.

Marking Structure

In order to be eligible to receive full marks:
 The following marking structure will be used:
NOTE 1: for each of these marks of {5, 4, 3, 2, 1, or 0} will be assigned depending on the quality and completeness (in terms of content) of what you have produced.
NOTE 2: the marks on this section of the report are not subject to being diminished based on the actual testing involving less than the required minimums.
NOTE: the marks on this section of the report are subject to being diminished based on the actual testing involving less than the required minimums.
NOTE 1: for each of these marks of {5, 4, 3, 2, 1, or 0} will be assigned depending on the quality and completeness (in terms of content) of what you have produced.
NOTE 2: the marks on this section of the report are subject to being diminished based on the actual testing involving less than the required minimums.
NOTE: each of the significant items in this section of the report will get {2 or 1 or 0 marks}
2 marks for very good to excellent quality work
1 mark for adequate quality work
0 marks for unacceptable quality work