CrossWeaver

CrossWeaver is a user interface designer's application for planning a multimodal application. It allows the designer to informally prototype multimodal user interfaces, akin to paper prototyping. User interface designs are drawn as storyboards with a tablet computer and are left in sketched, unrecognized form. This enables quick exploration of multimodal scenarios.

Typical usage: a designer sits down, draws some screens in the application, draws the transitions among the scenes, and then tests out the application. This usage scenario is shown with screenshots from the CrossWeaver application.

CrossWeaver Design Mode

The CrossWeaver design mode’s left pane contains the storyboard, which is made up of scenes and input transitions.  The right pane contains the drawing area for the currently selected scene.

A scene in the storyboard contains (a) a thumbnail of the drawing, (b) device targets and text to speech audio output, (c) input transitions showing the natural inputs necessary to move from scene to scene, and (d) a title.

Here we specify an input region, a dashed green area in the pane, in which linked gesture commands must happen to follow the transitions.

Grouping two scenes and creating an “Operation.” Based on the difference between the scenes, this operation is the addition of a building to a scene, triggered by any of the three input modes in the transition joining the two scenes.

A global transition points back to the scene from which it started.  A keyboard press of ‘h’ or a gesture of ‘h’ or a spoken ‘home’ on any scene will take the system back to this starting scene of the storyboard.

A bitmap image of a map has been inserted into the scene.  A typed text label has also been added.  These images co-exist with strokes, providing combined formal and informal elements in the scene.  Images can be imported from the designer’s past work or an image repository.

CrossWeaver Test Mode

Clicking on the “Run Test…” button in design mode brings up the Test Mode Browser, which accepts mouse, keyboard, and pen input.  (Top) In the first sequence, the end user gestures ‘s’ on the scene and the scene moves to the appropriate scene in the storyboard.  (Bottom) In the second sequence, the user accesses the ‘add building’ operation, adding buildings to the scene.  This is occurring in the standalone browser running on device PDA #0, as identified by the ID in the title bar of the window.  Pen recognition and speech recognition results come into the browser from separate participating agents.

CrossWeaver Analysis Mode

CrossWeaver Analysis Mode running timeline of all of the scenes that were shown across all of the devices.  It also displays the interaction that triggered changes in the state machine.  The red outline represents the current state in the replay routine, which steps through the timeline step-by-step replaying the scenes and inputs across devices.