SRS 200

Goals

• To use the results from your first user study to redesign specific parts of your stove and your prototype
• To use videotaping and coding as a new means of gathering and analyzing data
• To practice with either between-subject testing or within-subject testing
• To perform comparative analysis of your redesign vs. your original design
• To gain more experience in researching related work, writing, and group work

Your Mission

What to do

1. Your team should pick one aspect of your stove to redesign and concentrate on just that one. Be as specific as possible. Some examples of things you might choose include:
   • whether your timer should be analog or digital
   • whether a particular control should be a knob or a slider
   • whether the feedback of a particular knob should be changed
   • whether the placement of a burner should be changed with respect to the placement of the control for that burner

   Do not pick something too general. For example, choosing the "timer" as a whole to redesign would be inappropriate if it involves altering many things at once such as how one sets the timer, cancels the timer, set/modifies delays, etc. In other words, do not try to fix the entire stove at once. Complex interfaces have facets that interact with each other in ways you cannot foresee. If you change many things at once, you will undoubtedly fix some problems, introduce new ones, and still not be able to isolate the good from the bad usability elements. By testing one element at a time, you will be able to see how it alone contributes to or detracts from the usability of the artifact as a whole.

   We recommend making a list of two or three possible features that you may want to redesign, and then searching for related work (see #2 below) to see which design feature yields the most promising outside resources.

2. Start researching related work early in the redesign process to see if there are other possibilities you have not yet considered. Unlike the previous project, concentrating on just one feature of the stove will allow you to perform a more focused literature search. This, hopefully, will yield publications which will speak more directly to the issues you are facing. Note that you can now perform research on the failed design from Assignment 2 too! This may also yield fruitful insights. And don't forget about using design heuristics for additional inspiration!

3. Once your team settles on a new design, you should alter your prototype so that you can easily switch between the original design of your chosen feature and the new design. (You may need to do some ungluing, poster board cutting, or other surgery to your prototype. That's ok. Low resolution prototypes are destined for change.) Remember, don't change the prototype more than you have to, and make it easy to switch back and forth between the two designs. You may need to do this switcheroo during subject testing.

4. Design and execute a new usability study. You will probably want to reuse some of the tasks from the first study, but only the ones that relate to your redesigned aspect. You will want to come up with new tasks too. You are required to videotape your subjects as they use your artifact. Try to capture both their facial expressions and their actions. As before, all tests should be carried out in LESS, and you should test 3-4 subjects. Also as before, it is your job to determine what protocol to use (think aloud, retrospective think aloud, etc.), whether you should use between-subject testing or within-subject testing, what to measure, and how to measure it. The discussions we have in class about videotaping should give you fresh ideas about data measurement. Feel free to use pre- and/or post-test questionnaires if you wish.

5. The analysis of your data should include the coding of the videotape, which you will turn in as part of your final report. You should also make comparisons between the results you obtained from this study versus the results obtained from the previous study. Are there conclusions you can draw from such a comparison?

What to turn in

• A typed report, written by your team, based on the template that we gave you in class. Your paper must be between 8-10 pages: single-spaced, 2-column format, including the list of references. The hard part will be keeping your paper under the 10-page limit, since you will have more to say than you will have room for. That's where your skills in research writing will come in handy. Good organization, multiple edits, and collaborative revising are a must. Here are some further details:
  • Your paper should include all of the sections that we have previously discussed.

  • If your team used the "book boss" model that was described in class, someone else who has not done it before should be "book boss" for this project.

  • You are allowed to reuse some of your paper from the previous project for this one, but no more than 15% may be reused (one and one-half pages for every ten pages of text). For example, you may have a particularly good motivating example and related work reference as to why you placed the burners in their original configuration. That is something that is ok to reuse. But at least 85% must be original writing.

  • Even though a given section (like Related Work) may be originally written by a single team member, all members should be reading it and suggesting edits. In addition, for each section, at least two team members should sit down at a computer together and collaboratively revise. In your journal, you are required to record:
    1. the sections that you originally wrote
    2. the sections that you collaboratively revised with someone else (and who the other person was).

• A presentation, presented by at most two (2) members of your team, that lets the rest of the class know what you learned and did. For each team, at least one member who did not present for Assignment 2 should present for this project. Each team will have 10 minutes plus 2 minutes for questions and answers. You should not bring your artifact to the presentation, but your presentation may include pictures/video clips of the artifact in use.

Journaling

When all the presentations are done, you should again write a quick (1-2 page, handwritten) response to the experience. What were your expectations and actual outcomes of implementing your new design? How did it compare to other groups? What are your reflections on the videotaping and coding techniques that you used? Feel free to again comment on team dynamics too. And don't forget to record which report sections you wrote and which you collaboratively revised, as mentioned above.

We will collect your journals on Wed, Oct. 11.

Grading

• Grades for the group project and presentation will again be based on your ability to effectively answer "yes" to the questions posed in the Grading section of the previous project. Be sure to review them. Because this project is giving you more time to allocate to related work, careful data measurement, and comparative analysis, we will be putting more emphasis on those aspects of your work than we did in Assignment 2. Be sure your report and presentation can also answer "yes" to these questions:
  • Did your group effectively use the videotape and codes to provide meaningful data?
  • Did your group effectively compare results from the two usability studies?
  • Did your group draw conclusions about your redesigned aspect, and was evidence given to support these conclusions?

  Be sure your report is written for the correct audience (the usability science community) and stays under the 10-page limit.

• In your journal, we will be looking for evidence that you have been engaged in the work of the group, that you have been trying different ideas, that you have brainstormed, analyzed, observed, and measured. In the response at the end of the journal, we will be looking for evidence that you are truly reflecting on the experience, in particular on the methods you and your team used to produce your results. The journal will be worth 3 points of the 10 total for the term.