Sketching in 3D -Towards a fluid space for mind and body

Woongki Sung

Final project for the MIT class 4.566 Advanced Project in Digital Media
Development: September-December, 2012
Instructor: Prof. Takehiko Nagakura

3D sketching prototype demonstration

This animation was created as part of the final class presentation at MIT to demonstrate the gesture-based interactions of the 3D sketching prototype.

Movie info: mp4, 51 seconds

Full version including 4 segments (120 seconds) is available here at YouTube

Project Overview

Sketching, as a process, has been actively used in the design process; sketching is a cognitive process where designers use physical means in order to explore and develop design ideas. However, when designers deal with three-dimensional ideas, e.g. volumetric designs and spatial properties, current sketching methodologies have limitations such as an indirect manipulation and rigidly-structured representations. The aim of this project is to propose a sketching application for volumetric designs that provides designers with intuitive body-engaging interaction modes and computational functionalities for exploring different design options.

Fig 1. Project framework

From reviewing related literature and case studies, a set of design criteria and prospective techniques for a 3D sketching tool was established.

1) Intuitive interaction: tangible interface, gesture-based interaction
2) Direct manipulation: 3D sketching space, body-inclusive manipulation, augmented reality, physical guide structure
3) Flexible representation: Marching Cubes, point-based representation

Fig. 1 illustrates the confituration of the software prototype developed in this project.

Fig 2. Pottery Making

The first prototype was developed to examine the integration of the virtual object and a gesture-based interface. In this prototype, the virtual object constantly rotates and a user can manipulate this object by pushing and pulling with the hands. A simple mesh structure was used for the geometric representation.

Fig 3. Sculpting

A user can perform a subtraction operation of a virtual object just like working with clay or carving wood. Even though the program only provides a subtraction function, once the scope of interaction is expanded to gestures, users unconsciously started to apply rich physical knowledge, such as drilling and cutting, to the interaction..

Fig 4. Drawing

From an empty space, users can draw or fill the space by moving their hands. The sizes of the modeling space and hand strokes are adjustable. In this mode, the right and left hand were assigned drawing and erasing functions, respectively.

Fig 5. Live material

A simple cellular automata function was embedded into the virtual object. As such, the object autonomously changes its form over time. The aim of this experiment was to investigate the potential that only a tool mediating a virtual object and physical gestures can provide. This subject will be further explored.

Further development

Throughout the prototype and experiments, several drawbacks emerged, such as confusion arising from physical references and insufficient KINECT scanning resolution. Further development will focus on addressing these issues and also investigating possible unique functionalities in this sketching environment and different representation techniques.