|
|
AUI Usability Tabulated Test Results |
Tabulated results from AUI usability tests. For a discussion of test results see Athena User Interface Usability Test Results . To see the results of the card sorting exercise, see AUI Card Sorting Results.
Table 1 shows whether testers were able to successfully complete a given task.
Y= Yes, the task was successfully completed by the tester
X=No, the tester was unable to complete the task (see linked explanation)
0=The tester was able to complete the task but with help from the facilitator (see linked explanation).
Table 2 shows how long it took each tester to complete each task. Grayed cells highlight where the task to more than 10 minutes to complete (see linked explanation).
Table 3 shows results of the follow-up survey.
|
Completion success
|
Total
|
Percent correct
|
|||||||
|
Y
|
Y
|
Y
|
Y
|
Y
|
Y
|
Y
|
7
|
100%
|
|
|
Y
|
Y
|
Y
|
Y
|
Y
|
Y
|
Y
|
7
|
100%
|
|
|
Y
|
X
|
X
|
Y
|
0
|
X
|
Y
|
3
|
43%
|
|
|
Y
|
X
|
X
|
Y
|
0
|
X
|
Y
|
3
|
43%
|
|
|
Y
|
Y
|
Y
|
Y
|
Y
|
Y
|
Y
|
7
|
100%
|
|
|
Y
|
Y
|
X
|
0
|
X
|
Y
|
X
|
3
|
43%
|
|
|
Y
|
Y
|
Y
|
Y
|
Y
|
Y
|
Y
|
7
|
100%
|
|
|
Y
|
Y
|
Y
|
Y
|
Y
|
Y
|
Y
|
7
|
100%
|
|
|
Totals
|
8
|
6
|
5
|
7
|
5
|
6
|
6
|
43
|
|
|
Percent
|
100%
|
75%
|
62.5%
|
87.5%
|
62.5%
|
75%
|
75%
|
77%
|
|
Time to complete
|
Total
|
Average Time
|
|||||||
|
Task 1
|
1
|
1
|
0.5
|
1
|
1
|
1.5
|
1
|
6.5
|
1.3
|
|
2
|
1
|
0.5
|
1
|
1
|
1.5
|
0.5
|
3
|
8.5
|
1.7
|
|
3
|
3
|
5
|
25
|
6
|
13
|
16
|
33
|
101
|
20.2
|
|
4
|
3
|
3
|
4
|
1
|
1
|
7
|
5
|
24
|
4.8
|
|
5
|
2
|
1.5
|
1
|
1
|
10
|
3
|
5
|
23.5
|
4.7
|
|
6
|
2.5
|
5
|
14
|
15
|
15
|
2
|
5
|
58.5
|
11.7
|
|
7
|
1
|
1
|
1
|
1
|
1
|
2
|
1
|
8
|
1.6
|
|
8
|
1
|
1
|
3
|
1
|
2
|
2
|
1
|
11
|
2.2
|
|
TOTALS
|
14.5
|
17.5
|
50
|
27
|
45
|
33
|
54
|
241
|
48.2
|
| 1. If you have previous Athena experience, do you feel this version of the system is a change for the better?Why? |
Yes The new environment appears to include a lot more menus which include all sorts of programs I didn't know were there because they don't appear to be on the Dash or I don't know how to find them in terminal window. |
Yes This version lets me see all my processes on the taskbar, and lets me access all my application from a "start menu". |
Yes Uses WWW directly |
Yes for people used to using icons - but it's just different enough to need some training |
Yes The new version of AUI is much easier because it utilizes
a more "Mainstream" system setup (i.e. Microsoft Windows.) I could find
items much easier in the AUI than in XWindows. |
Yes anything is better than the exceptionally counterintuitive character-based system |
|
| 2. If you have previous Athena experience, are there aspects of the system that you feel have changed for the worse? Why? |
No During this test, I didn't see any differences I thought were necessarily worse. |
No | No |
|
No those who are used to the command prompt way of working will continue. |
No | No |
| 3. If you have previous Athena experience, are there aspects of the system that you feel have changed, but have neither really improved or gotten worse? Why? |
No The few programs I actually use are all still there,
and they aren't any harder to get at. I usually run them from a terminal
window anyway. |
Yes Minor items such as showing the time etc., have not changed.
|
No more like PC use, w/which I'm more familiar |
No Actually, there is some improvement for people who like to use gui's. |
No | ||
| 4. Was your previous experience using a Windows or Macintosh system useful in helping you complete these task? | Yes | Yes | Yes | Yes | Yes | Yes | |
| 5. Were you able to use the knowledge from previous computer experience to navigate and maneuver in this environment? | Yes | Yes | Yes | Yes | Yes | Yes | |
| 6. What would have made the experience easier? | It seemed pretty easy to me, but I don't know if I would have been able to complete this as easily before MIT forced me into using UNIX-based operating systems. | instructions at the beginning. "welcome to athena" |
|
The file manager looks too much like a browser - I got confused about which was netscape (by the way - a new version I had not yet used) and the file manager. | When I clicked on the file manager in the menu bar I was expecting to have a list of all my files appear. However, I didn't know that I needed to double click on the "gtest's Home" icon to actually OPEN the file manager. | use the standardized menu items, icons and navigation systems from Windows and Mac. Add prominent titles to each window to let the user know where he or she is in the system. | |
|
7. On a scale of 1-5, how does the prototype compare to: |
where 1 means the prototype system is much worse than some other system and 5 is much better. NA means you have no applicable experience to compare. | ||||||
|
Current Athena
|
4
|
4
|
5
|
|
5
|
4
|
|
|
Windows 95/98/2000
|
4
|
5
|
3
|
|
4
|
2
|
2
|
|
Macintosh |
NA
|
NA
|
4
|
|
NA
|
1
|
2
|
|
Other UNIX Specify: Other Specify |
NA |
4 (Redhat 6.2)
|
NA
|
|
NA
|
NA
|
|
| 8. What would you change about the prototype, if anything? | I don't think I've used it enough just yet. | pop up help at first time startup | some features of easel (services) crash easel itself easel--> edit--> customization- -> backgrounds crashed | See LONG tape |
|
better location information, commands (menu items etc.) standardized to PC/Mac versions, File Manager is critical; needs a bit work on navigation side. This is a brave attempt! | |
| Gender: | M | M | F/F | F | F | M | F |
| Affiliation with MIT: | Student | Student | GradStudent MIT GradStudent NonMIT |
Staff | Staff - Library | Support | Faculty |
| Class | 2004 | 2004 | |
|
|
||
| Previous computer experience |
|
|
|||||
|
general
|
extensive
|
extensive
|
extensive
|
|
extensive
|
extensive
|
extensive
|
|
Athena
|
little
|
little
|
little
|
|
moderate
|
little
|
little
|
|
windows:
|
admin
|
extensive
|
extensive
|
|
extensive
|
extensive
|
extensive
|
|
mac
|
little
|
little
|
moderate
|
|
|
extensive
|
moderate
|
|
Linux UNIX
|
little
|
moderate
|
|
|
|
little
|
moderate
|
| Finally, please let us know what interested you in participating in this experiment? | I like new toys. | the email that I got while in my seminar | personal connection w/usability testers | I'm doing usability for the Libraries DSpace project. Wanted to get experience with testing. | My love for technology. | I need to use it in the classroom, and cannot use the current system effectively. I am also interested in helping out with improving the environment for new students, many of whom have no Unix experience. | |
Created 01/22/2001 by sbjones@mit.edu
Modified February 27, 2001
Return to AUI Test Home Page