Ph.D. Theses

A testing methdology and architecture for computer supported cooperative work software

By Robert F. Dugan, Jr.
Advisor: Ephraim Glinert
May 26, 2000

Despite enormous potential, CSCW software is still immature. In particular, leading researchers in both the CSCW and testing fields have noted CSCW testing tools are non-existent. This thesis contributes a methodology and architecture for execution based testing of CSCW software.

The CSCW Application MEthodoLOgy for Testing (CAMELOT) provides an organized set of specific techniques that can be used for technological evaluation. The evaluation is organized into two phases: single user and multi-user. Single user evaluation is subdivided further into general computing and human computer interaction. General computing examines software components that provide basic application capabilities. Human computer interaction focuses on the interface between the user and the software application. Multi-user evaluation examines distributed computing and human-human interaction. Distributed computing scrutinizes components responsible for multitasking and multiprocessing in the application at the thread, process, processor and machine level. Human-human interaction focuses on how the software facilitates interaction between users during application use.

Rebecca, our testing architecture contributes to both general and multiuser testing systems. In the area of general testing Rebecca:

Provides an extensible component and event model that allows the record/playback of non-GUI events
Allows selective event recording through record filtration
Promotes the integration of the test system into the development environment
Outputs test scripts in the developer's native language
Reduces re-recording using component-centric events and runtime component resolution
Simplifies the test process using a simple VCR-like interface

In the area of multiuser testing Rebecca:

Integrates live users into a test session with triggers that playback virtual user behavior based on user interface, state change, timer, or user customized events
Provides runtime configuration of triggers via the threshold models
Simplifies virtual user synchronization with deadlock detection and recovery
Simplifies multiuser script editing via a global clipboard
Maintains IPC independence, but allows IPC to be recorded
Scales well with a resource conserving architecture

Our architecture was implemented in Java as a working system called Rebecca-J. The methodology, architecture, and working system were evaluated by testing a mature CSCW application. The evaluation uncovered several dozen problems with the CSCW system. In addition to validating our approach, the evaluation prompted immediate improvements to the architecture and implementation, and provided important ideas for future enhancements.

Return to main PhD Theses page