Markpad is an gestural interaction technique working on laptop's trackpads. It enables rapid gestural shortcuts to trigger a large variety of commands on a laptop to open applications, trigger keyboard shortcuts, open bookmarks on the web or the file explorer, and more. The strength of Markpad is to be user-tailored: users can customize their gestures and map them to the actions they want. Markpad's gestures consist in simple strokes starting from the bezels of the trackpad, thus avoiding to conflict with common touch interactions without requiring extra delimiting actions (e.g. pressing a key to input a gesture). Through three experiments, we show how the design of Markpad avoids conflicts with common touch interactions, and demonstrate how it enables large gesture sets using visual and tactile aids superposed on the trackpad. Finally, we demonstrate the advantages of the approach through a fully functional system running on macOS X.
2017
Interaction DesignDesign ToolOpen Source
The Markpad interaction design
Markpad enables rapid gestural shortcuts on trackpads. A Markpad gesture consists of a stroke movement that starts from one area of the trackpad and ends in another. Multiple gestures can start from the same area, hence be part of the same menu. To avoid conflicts with common touch interactions, starting areas (or menus) are positioned next to the trackpad's bezels (see orange areas on the right-hand pictures).
After initiating a movement from a starting area (e.g., the bottom-centered on the right-hand picture), the user can finish it in several ending areas (depicted in blue), thus completing a gesture and triggering an action (e.g., opening an application).
Using visual and tactile aids to enable large gesture sets
A powerful implication of the Markpad's interaction design is to enable large gesture sets by its simplicity. For users to perform these gestures accurately, however, they need to be guided with visual aids. These aids depict landmarks that reveal the positions of the underlying Markpad areas. Depending on the size of these underlying areas, visual aids are more or less important (more details below).
We propose a very simple method to add visual aids on off-the-shelf trackpads: adding a thin plastic layer on top of them with visual marks drawn with a permanent marker, or paper stickers glued on it. In addition, users can create other types of aids to help them perform Markpad gestures like using transparent tape to not alter the trackpad look.
Supporting novice and expert users
Novice users need to learn the gestures before they can use them rapidly and efficiently. To help them learning, Markpad provides a novice mode that depicts the available starting areas or menus when pressing option key as well as the possible ending areas to trigger actions once a Markpad gesture is initiated. To enter this mode, users must stay idle on a starting area for 300ms.
Once they know the gestures, users can perform them quickly without waiting, hence trigger an action without displaying the Markpad graphical interface.
Designing Markpad gestures
We built a simple design tool to let users tailor the Markpad gestures to their needs. Users can define the starting areas (i.e., menus) and the ending areas. They must map an action to an ending area. Markpad provides default actions like opening an application or changing the volume, but users can define new actions using Applescript or shell scripts (see the pictures below).
It is important to note that different users would likely have different configurations: they might not need to trigger the same actions, and they might not use the laptop's trackpad the same way. For instance, a left-handed user might use mostly the bottom right corner of the trackpad while a right-handed user would use the bottom left corner more often. Therefore, such a design tool is particularly important for users to adapt the gestural interactions to their behaviors and preferences.
Assessing the usability of Markpad
We performed three user studies to assess the usability of Markpad.
The first two were lab experiments focused on the efficiency of Markpad gestures, i.e., do users perform these gestures accurately and do they benefit from visual or tactile aids? We asked the 28 participants to perform a set of Markpad gestures with different layouts of visual and tactile aids. We found the aids have a significant impact on the user accuracy in a dense configuration (a total of 680 possible gestures) but that tactile aids did not yield better results than visual ones. More details can be found in the paper.
The third experiment was a longitudinal study assessing whether Markpad gestures conflict with common touch interactions. We recorded the touch behaviors of 12 right-handed Macbook users for a week. We then assessed how many Markpad gestures would be triggered unvoluntarily based on these movements and various Markpad configurations. This allowed us eventually to optimize the height and width of starting areas to avoid false activations. More details can be found in the paper.
MarkPad: Augmenting Touchpads for Command Selection Bruno Fruchard, Eric Lecolinet, Olivier Chapuis CHI‘17: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, ACM (2017), pp.5630-5642
