Figure 1: General Data Flow and Timing
One Measure of Happiness
An interactive Video Installation
Content
Content
Objectives
Concept
1. Installation
2. The Video
3. System Architecture
4. Setup
5. Project state (January 2003)
6. Who we are
Figure 1: General Data Flow and Timing
2. High level Roles & Responsibilities
2.1 Sensation TBD
2.2 Perception TBD
2.3 Cognition
2.4 Emotion
2.5 Sprite Manager
Table 1: Sprite Manager Selectors
Table 2: Sprite Manager Events
3. Detailed Roles & Responsibilities
3.1 Sensation Details TBD
3.2 Perception Details TBD
3.3 Cognition Details
3.4 Emotion Details TBD
3.5 Sprite Manager Details
3.5.1 Scene Selection
Table 3 - Story Skeleton Phases
Table 4 - Story Scene representation
Table 5 - Sequence representation
3.5.2 History
Table 6 - History
Table 7 ñ Memory Clips
1. To tell a coherent story interactively, fluently and cinematically through the interplay of three actors: a database-driven story model, an autonomous synthetic character, and an involved and active interactor.
2. To create a compelling experience of intimacy between an interactor and a synthetic video character using familiar gestures through an affordable touch-screen interface.
The installation is situated in a physical four-wall compartment. Interactors enter the compartment one at a time through a path of obstacles. Each obstacle involves a prohibition that must be transgressed in order to proceed. At the end of the path, a touch screen is embedded into a wall, with a prohibitive ìDonít Touch!î sign above it.
The Video is composed of two composite layers: Foreground and Background. In the foreground layer we see a Close-Up shot of the protagonist, a young woman whose, face we can touch. The background layer is a back-projection of the protagonistís thoughts, memories and feelings.
The protagonist invites us to touch her face. The way we touch, the location, manner and duration determines the level of intimacy and confidence we inspire in the protagonist. This, in turn, affects the way we progress through the story.
The foreground and background layers are generated in real time by Macromedia Director, from an external clip database. The clips are assembled on the fly according to the program algorithm, which is comprised of three elements: A narrative and presentation manager, which is in charge of maintaining narrative progression, coherence and presentation; the protagonist, with her autonomous goals, emotions and behaviour; and an interaction manager, which monitors and interprets the interactorís activity.
5. Project State (January 2003)
Software:
We are progressing in a modular mode. Weíve already completed the overall system architecture and algorithms and are now writing the first modules of the Sprite Manager in Macromedia Director Lingo. Cuncurrently, we are writing the pseudo-code of other modules, to be executed later.
Creative:
We have completed a synopsis and are now in an advanced stage of adapting it into a 30-page treatment. We expect to be casting by March 2003, in production by April 2003 and in a gallery near you by May 2003.
Investigators, producers and creative directors:
Noam Knoller & Udi Ben-Arie
Research Supervisor:
Amnon Dekel
Story and screenplay:
Galit Reuchman and Nelle Schaeffer
Architecture Engineer:
Maya Lotan
Software Manager:
Mirit Tal
Lingo Team Manager:
Gal Tushia
Lingo Team:
Udi Ben Arie & Noam Knoller
Figure 1: General Data Flow and Timing
2. High level Roles & Responsibilities
Evaluate and react to a gesture.
Evaluates the user interactivity level, maintains the mood of the character, its attitude towards the user and the level of intimacy between them.
All selectors are responsible for collecting all necessary data for their selection process.
Table 1: Sprite Manager Selectors
|
Memory Sprite |
Character Sprite |
Reaction Sprite |
|
BG |
FG |
UFG |
|
Select Memory & Speech Scene |
||
|
Select Reaction Clip |
||
|
Select Reaction Clip |
||
|
Select Memory Scene |
SelecMood Clip |
Select Reaction Clip |
|
Select Mood Clip |
||
|
Select Mood Clip |
||
|
Select Mood Clip |
Select Reaction Clip |
|
|
Select Mood Clip |
||
|
Select Mood Clip |
||
Table 2: Sprite Manager Events
|
Memory Sprite |
Character Sprite |
Reaction Sprite |
User Activity |
|
|
BG |
FG |
UFG |
||
|
a scene with out a FG |
Memory Scene |
Mood clips |
- |
User Activity = No (from Emotion) |
|
Reaction Clip Regular Opacity<100% |
Reaction (Type) = Regular (from Cognition) |
|||
|
Interrupt |
Reaction Clip: Extreme Opacity=100% |
Reaction (Type) = Extreme (from Cognition) |
||
|
a scene with a FG |
Memory Scene |
Speech Scene |
- |
User Activity = No (from Emotion) |
|
Reaction Clip Regular Positive Opacity<100% |
Reaction (Type) = Regular, Reaction (Tag) = Positive (from Cognition) |
|||
|
Reaction Clip Harassed reaction (u r bugging me!) Opacity=100% |
Reaction (Type) = Regular, Reaction (Tag) = Negative/Impertinent (from Cognition) |
|||
|
Interrupt |
Interrupt |
Reaction Clip Extreme reaction Opacity=100% |
Reaction (Type) = Extreme (from Cognition) |
3. Detailed Roles & Responsibilities
3.3 Cognition Details
TBD reaction Connotation is the GestureArea + neg/pos
Scenes are selected according to all the below, each weighted ñ the weight of each aspect can be changed by us.
Table 3 - Story Skeleton Phases
|
Story Phase |
Description |
In |
out |
K |
|
Introduction |
Single Intro scene to the system played in loop. |
user enters system |
user touches start. |
K1 |
|
Exposition |
Several scenes exposing 4 aspects of the character and the story in no specific order. Selected out of several scenes, each covering at least one aspect. |
end of Introduction |
all aspects are covered exactly once or eject |
K5 |
|
Plan |
The plan scene of the story. Selected out of several scenes. |
end of Exposition |
scene ends or eject. |
K6 |
|
Body |
Several interlaced sequences. Each sequences is composed of a procession of stages (crisis, local peak etc.) Each stage is represented by a selected scene.
Several sequences may be played in parallel. There is an incentive to close open sequences for the sake of coherence. |
end of Plan |
eject |
K? |
|
Peak & Relaxation |
The peak and relaxation sequence of the story and interaction. The scene is selected according to the K level and preceding scenes played, the mood and the Attitude Level towards the user. |
Eject |
Sprite watches
K ñ User knowledge of the character (story information) ñ not each scene contributes to the K level (e.g. scenes that open a scene-loop ñ contribute nothing, while closing scenes contribute the whole-lot).
I - Intimacy (Level)
Eject ñ triggered when either:
Table 4 - Story Scene representation
|
Property |
Description |
Value |
example |
|
ID |
number |
||
|
Phase |
story phase of the scene |
phases described above |
|
|
Depth |
How much intimate knowledge about the character does this scene provide? |
number |
|
|
Precedent |
Specifies which other scenes must be presented before the scene can be presented (only for peak scene) |
list of number (scene ID) |
12 |
|
StartMood(Level) |
number (same as Mood(Level) of Emotion) |
2 |
|
|
EndMood(Level) |
number (same as Mood(Level) of Emotion) |
-2 |
|
|
XXX Area Connotation (XXX = face area from clip map) |
The connotations of this area of the face in the scene |
connotation |
positive negative, neutral (or null) |
|
XXX Area Connotation Score |
weight number (1-?) |
2 |
|
|
Association |
A list of the association of a scene with other scenes. Each has a weight. |
list of associations (keyword, weight) weight number (1-?) |
mother, 4 |
Table 5 - Sequence representation
|
Property |
Description |
Value |
example |
|
ID |
number |
||
|
Sequence skeleton |
The stages within a sequence |
List of strings |
|
|
Sequence Order |
the order of the stages within the equence |
List of strings |
2
|
|
Sequence stage scenes |
The possible scenes in every stage |
List of numbers |
A sequence is selected indirectly when its 1st scene is selected.
A sequence may be played in more than one order of stages, but only once (except for Brooding Mood scenes).
A stage within a specific order of a scene loop is filled by a specific scene, selected according to mood.
History maintains the order and frequency of the scenes, and monitors the incentives to close sequences.
|
scene.played |
The number of times a scene has been played through |
Variable |
|
scene.repeat |
the number of times a scene may be repeated. |
Variable |
|
Last scene |
The last scene played |
Number (of scene.ID) |
|
open sequences |
which sequencesí 1st scene or more has been played |
List of Numbers (of sequence.ids) |
|
Open sequence last scene |
Which scene was last played in a sequence |
Number (of scene.ID) |
|
Open sequence incentive |
How many scenes have been played since open sequence last scene |
Number |
General clip ñ composing the scenes (not reaction clips) - remember clip for more than one scene TBD
|
Property |
Description |
Value |
example |
|
ID |
|||
|
Scene |