<head> (WG 4 refinement)
Movement of the head offset from posture and gaze direction. Types include nodding, shaking, tilting. Supports rhythmic movement.
| Attribute | Type | Use | Default | Description |
|---|---|---|---|---|
| action | name | required | ROTATION | [ROTATION, ORIENTATION] |
| rotation | name | *required* | Category of head movement [X (NOD), Y (SHAKE), Z (TILT)]. Need to decide whether to use X, NOD or either. In core, we assume only 1 axis in an element. | |
| repeats | int | optional | 1 | Number of times the basic head motion is repeated. May use “repetitions” instead as per style guidelines for all behaviors. By the way, repetitve tilts are possible with this spec which we may not want. We would want to support synchronization of the repeats, which implies we would use the same mechanism of generating the synch points that gesture repeats use. |
| amount | float | optional | 0.5 | The extent of the movement here 1.0 is fully extended and 0.0 is the least extended, and -1.0 is fully extended starting in the opposite direction. Need to define what is positive direction. |
| orientation | name | *required* | FRONT | [FRONT,RIGHT,LEFT,UP,DOWN,UP_RIGHT,UP_LEFT,DOWN_RIGHT,DOWN_RIGHT, POLAR_ANGLE] Note we are slipping in a polar angle alternative here - which is consistent with the previous specification but is a typing violation. This should be changed based on style standards. |
| offset | angle/ float/ name | optional | Need to decide in general whether this should be an angle or a magnitude specified as a float or name. Note also as currently specified this does not support jutting or retracting of the head - which it could if we specified the orientation and offset differently, e.g., by x, y , z. |
Here we use *required* as a temporary hack to signify the “required” only holds if the action is the same type (ROTATION actions require a rotation attribute). In general the above syntax will need to change once style standards are specified. Orientation can also be used to specify starting and ending location of nods/shakes. Head movements are assumed to exploit standard synchronization points - so they can be used as a form of gesture, for example.
<face> (WG 4 refinement)
Movement of facial muscles to form certain expressions. Types include eyebrow, eyelid and expressive mouth movements.
| Attribute | Type | Use | Default | Description |
|---|---|---|---|---|
| type | name | required | The part of the face being controlled. CORE: [EYEBROW, MOUTH, BLINK, FACS]. BLINK only requires an amplitude, with negative opening the eye more, while EYEBROW and EYELID require the part and side of the face being specified. BEYOND CORE: [EYELID,CHEEK,NOSE,CHIN,JAW] | |
| amount | float | optional | 0.5 | [-1 to +1], Amount of movement, where 0 is no movement (or closed position), +1 is the highest (or most open) position), and -1 is the greatest possible movement in the opposite direction , if applicable. The physical meaning of these magnitudes are realizer and character specific (e.g., part of the gestuary). So 1.0 is the maximum movement allowed by the realizer for that character. Also the interpretation is somewhat feature specific (for example do we want eyelids to be closed at 0 and then more tightly closed at negative numbers or simply not allow negative numbers) |
| side | name | optional | BOTH | [LEFT,RIGHT,BOTH] Specifies the side of the face being controlled, so for example left and right eyebrows can be controlled separately |
| direction | char | required | [X,Y,Z] X is sideways, Y is up/down and Z is away/inwards from the idealized plane of the feature. | |
| EYEBROW_PART | ||||
| eyebrow_part | name | optional | BOTH | [INNER,OUTER,BOTH] The part of the eyebrow. Note outer can only move in in Y direction (up/down), while inner can move in X (sideways) and Y directions. See picture below. |
| MOUTH_PART | ||||
| mouth_part | name | optional | CORNERS | Core: [CORNERS], where CORNERS refers to the side corners of the mouth. CORNERS can move in both Y direction (raise/lower), and X direction (e.g. stretching the mouth). Beyond Core: [UPPER, LOWER, MIDPOINT]. UPPER is the upper center of the lips. LOWER is the lower center of the lips. These can move in the Y direction (e.g. for lip rounding), and also in the Z direction (e.g. for lip protrusion or retraction). MIDPOINT is one of the two points on the upper lip midway between the upper center and the corner, used for an asymmetrical sneer, where only one side of the upper lip is raised. To specify which side is raised, use the <side> attribute. Note that visemes are covered in a separate section, outside of <face>. |
| FACS | ||||
| au | int | optional | 0 | (FACS only) The Action Unit (AU) reference number for a Facial Action Coding System (FACS) expression. Arguably beyond core but there is a debate on that. If we decide to put it into core, we should only require those AUs that “roughly” correspond to core elements in the above “POINT” approach: such as AU1: inner brow raiser, AU2: outer brow raiser, AU4: brow lowerer, AU 12: Zygomatic Major (upturned lip/mouth corners), AU15: lip corner depressor |
| EYELID_PART | ||||
| eyelid_part | name | optional | BOTH | (Beyond Core) Whether both upper and lower eyelids are affected [BOTH, UPPER, LOWER]. |
The mouth has the following control points:
Facial features, such as the eyebrows, are assumed to follow the standard synchronization points. Specifically, for the eyebrows we envision two types of curves: An attack (or onset), sustain and release (or offset) and an attack, decay, sustain, release. The first models a simple onset period to a peak amplitude that is sustained until the release. The second models also incorporates an initial attack that overshoots and then decays to the sustain level. See pictures below. A key question is how the standard synch point align with these phases. We are considering two options, depending on whether the sustain is to be considered to be part of the post stroke hold or part of the stroke and also whether the stroke-start is associated with start of the entire behavior or the end of the attack. (Should we bother to give the specific variants at this point).
Beyond Core: CHEEK (amplitude,direction), NOSE (amplitude, direction: where X flares or pinches nostril and Y can move the tip of the nose up or down), CHIN (amplitude, which can raise the chin), JAW (amplitude,direction: where Y can raise/lower, X can move it sideways and Z can jut or retract)
<gaze> (WG 4 refinement)
Coordinated multimodal movement indicating where the character is looking. (Note no updates were made of this element.)
| Attribute | Type | Use | Default | Description |
|---|---|---|---|---|
| type | name | required | Type of gaze behavior, either AT or AVERT (direct towards or avoid looking at a target respectively) | |
| target | ID | optional | The world ID of the reference target | |
| offsetangle | angle | optional | 0.0 | Adds an angle degrees offset to gaze direction relative to the target in the direction specified in the offsetdirection |
| offsetdirection | direction | optional | RIGHT | Direction of the offsetdirection angle [RIGHT, LEFT, UP, DOWN, UPRIGHT, UPLEFT, DOWNLEFT, DOWNRIGHT, POLAR angle] |
| modality | name set | optional | Some sub-set of EYES, HEAD, NECK, TORSO and LEGS |
WORLD and CHARACTER. This <target> sub-element should also be used by gaze and pointing gestures.
Locomotion also defines target proximity. What is the equivalent here?
