Humaneness PerformanceNetAll references · Download PDF ↗
ContentsThe shared system · a two-minute introductionScene encoding, learning targets and data selection1. Read the whole performance2. What the scales contain3. Define the person, scene and beat4. Read body and face5. Hear voice, words and events6. Describe conduct in context7. Follow agreement and change8. Separate feeling, display and restraint9. Build a playable scene10. Choose a situation11. Evaluate the viewer’s experience12. Annotate with an evidence trailThe shared record interface13. A complete worked reading14. Validate and reuse responsiblyFind the right appendix

Performance Annotation System

Humaneness
Performance Annotation
System

Encode real, reference and synthetic performances

Describe what can be heard and seen in audio, video or audiovisual material. Preserve the difference between evidence, interpretation, audience response and task-fit judgement so the records can support analysis, model inputs, learning targets and data selection.

Script Analysis SystemSource → character → sceneOrdered beats + one inner chunk eachAuthored conditions for virtual actorsPerformance Annotation SystemReal · reference · synthetic recordingsEvidence → description → task-fit reviewSeparate observation and interpretationShared taxonomies; different roles for each valuePerformanceNet · VoiceNet · EmoNet · situation and scenario vocabulariesCondition models · construct learning targets · select and diversify dataA taxonomy is a representation, not a trained model or an automatic reward.
Humaneness PerformanceNet joins two complementary systems through a shared descriptive space. A value’s source and role remain explicit when it is reused.

Part of Humaneness PerformanceNet
PerformanceNet Taxonomy: 82 descriptive fields · 8 separate observer-impression scales
Formerly the PerformanceNet main reference

Humaneness PerformanceNet

Describe the performance you have

The Humaneness Performance Annotation System begins with a recording. It can encode a real encounter, a staged human reference, a synthetic performance or a mixed production. A generation prompt is useful context when available; it is not a prerequisite for annotation.

Humaneness PerformanceNet is the umbrella. The Script Analysis System structures source material into Character Makeup, all seven Dramatic Obligations and ordered Acting Beats, each with exactly one nested Experience Chunk. This Annotation System describes available performance evidence over time. Both use shared taxonomies while preserving the role and basis of each value.

The unit remains one participant in an interval. Link voice, face, body and interaction only when attribution is supported. Keep known scene context separate from what the recording reveals. An unstaged person need not have a fictional character, a script or an authored Experience Chunk.

Role of the valueExampleKeep attached
Authored conditionThe fictional character fears losing a partner and chooses composed reassurance.Source text or directorial invention, character/scene/beat IDs and intended modality.
Observed descriptionA steady voice, a delayed answer and an available hand movement.Recording/participant/interval IDs, observable channels, exact field anchors and evidence.
Inference or responseA viewer reads restrained fear or finds the performance emotionally distant.Who judged it, what they could access, reasons, alternatives and uncertainty.
Task-fit or preference labelA reviewed comparison favors the version that maintains reassurance while preserving the requested pause.The declared objective, comparison conditions, rubric and rater distribution.
Training target or reward signalAn approved annotation target or calibrated preference label is selected for a particular learning objective.Its derivation, eligibility mask, split assignment and validation. A later model prediction must be identified as such.

A small example across both systems

A script asks a character to reassure a frightened partner while privately fearing failure. The Script Analysis System can author that fear and protective objective. An annotation record may describe a steady voice and delayed answer. A viewer may interpret restrained fear. A task-fit comparison may favor one take’s timing. A later learning pipeline may select that reviewed comparison as a preference target. These are related records, not interchangeable facts.

For an unprompted reference clip, keep the steady voice and delayed answer but leave unprovided inner conditions unavailable. For an audio-only clip, visible hands and gaze are unavailable. For a synthetic clip, record what was actually rendered even when it differs from the prompt.

A taxonomy is not a trained model or a ready-made reward. It defines shared fields and anchors. Annotation protocols, supervised targets, preference datasets, reward models and quality decisions are separate components that require explicit tasks and validation.

What the annotated record enables

1 / Structured scene encoding for omni models

Represent speech content, vocal events, visible action, partner interaction and timing alongside the known scene. An omni model can receive or produce a structured account that keeps audio, video, text and context aligned. The record identifies missing channels and the source of an interpretation instead of flattening everything into a single emotional caption.

Example. In a meeting recording, encode who speaks, the interruption interval, a delayed reply and the visible turn away. Mark a reading of social withdrawal as an interpretation with alternatives. The representation can support retrieval, scene understanding or conditional generation without asserting an unobserved private motive.

2 / Supervised target construction

Use reviewed, evidence-linked labels as targets for the relevant audio, video or audiovisual task. Targets can include transcript and event intervals, categorical capture conditions, ordinal descriptive fields and explicitly labelled contextual interpretations. Preserve the field type, uncertainty and eligibility mask. Exclude unavailable visual labels from an audio-only target set; do not replace missingness with the low anchor.

Example. Pair a permitted performance clip with the actual words, breath-event interval and reviewed voice descriptors. If a source script also exists, store it as a separate conditioning input and retain any difference between intended and spoken text.

3 / Preference and reward-model target construction

Define the performance objective before asking which output is better. Collect task-fit ratings or pairwise preferences under declared rater access, then record reasons, disagreements and calibration. A later reward model may be trained on those selected labels. The present taxonomy is neither that model nor evidence that its reward is valid or generalizes.

Example. Compare two takes of the same composed reassurance. Keep the words, role and scene fixed; ask which better preserves the requested action, timing and restraint. Stronger audible tension is not automatically a better or worse take. Its value depends on the intended scene.

4 / Filtering, grouping and diversity

Select data by the conditions a task needs: usable modality, supported attribution, technical quality, situation, interaction pattern, language, expression, timing or reviewed performance traits. Group examples into explicit buckets and inspect sparse or overrepresented combinations. Keep coverage counts separate from quality labels; an unusual or difficult scene is not low-quality data merely because its descriptors are rare.

Example. Build a balanced sample of quiet refusals, direct requests and interruptions across different capture conditions. Use situation tags and observed actions to find candidates, review their eligibility, and keep related source scenes and generated variants in the same dataset split.

From annotation to use. An evidence-backed descriptor can become an eligible training target or selection feature only through a declared pipeline. A viewer preference can become a candidate reward-learning label only under an explicit task. Neither transformation changes an ordinal descriptor into a universal “good acting” score.

A practical first pass

Five questions to begin

  1. Who and when? Choose the focal participant and a meaningful interval. A beat follows a stimulus or action; it need not follow an edit.
  2. What is available? Record audio, face, eyes, torso, hands and legs separately. A cropped-out hand is unavailable, not still.
  3. What changes? Describe events before assigning broad labels: a pause, a turn away, a shift in pace, a movement that stops.
  4. What does it mean here? Add the known partner, task and stakes. Identify whether each interpretation comes from the script, observation or a hypothesis.
  5. Does it serve the scene? Collect observer impressions and evaluate task fit separately. A low warmth score may be exactly right for a hostile encounter.

Three kinds of annotation value

RepresentationMeaningExample
Ordinal scale, 0–6Seven ordered descriptions. The distances between values are not assumed equal.Use the exact BOD-OPN (Postural openness) anchor for the observed posture.
CategoryA named capture condition. A larger code is not “more” or “better”.CAP-SHT (Shot Scale), CAP-INT (Interaction Setting) and CAP-POS (Body Configuration).
null + reasonNo supported rating was possible.Gaze unavailable because the eyes are occluded.

The emotion vocabulary has a separate four-level intensity system. Its values 1–4 describe requested prominence; they do not replace the 0–6 descriptive scales.

Choose your reading path

Directing or prompting: use the Script Analysis System, then Sections 9, 8 and 13 here. Annotating real, reference or synthetic recordings: begin with Sections 3 and 12. Preparing model inputs, training targets or sampling groups: read the use cases and shared interface, then Sections 11 and 14 before deriving any quality or preference signal.

1. Read the whole performance

Describe one person, in one interval, across the channels you can actually observe.

Read across three timescales: character continuity across scenes; scene obligations and stakes; and beats with a stimulus, action, response, start time and end time. Observable channels are words, voice, non-speech events, face, gaze, gesture, body and interaction. Intended ≠ observed ≠ interpreted; unavailable is null.

A performance is more than a label

The Humaneness Performance Annotation System describes observable performance in recorded real, reference and synthetic material. Its input may be audio, video or audiovisual media: a real encounter, an interview, a human acting reference, a film or theatre recording, an AI-generated performance or a mixed production. Describe what is said and done, how available channels relate and what a listener or viewer can perceive. Audio-only and silent-video records are first-class profiles. No fictional role or generation prompt is assumed.

A useful account connects physical organisation, expressed affect, attention and reasoning, social conduct, the demands of the situation and the impression on an observer. It should be possible to describe a smiling face over a rigid body, a confident voice accompanying uncertain words, a polite refusal with a closed posture, or an apparently authoritative person who visibly changes their mind. One global label such as “sad”, “natural” or “convincing” loses these differences.

One participant, one interval

The central unit is one identified participant in one time interval. Ratings are linked to relevant partners, tasks and evidence. A scene can contain several participant records and a shared interaction record. Narrative context is recorded when known; it is not silently invented from a camera view. Structured records support annotation, scene encoding, comparison, retrieval, supervised or preference-target construction, data filtering and diversity analysis, with task-specific eligibility and calibration.

LayerCore questionExample
Observed formWhat actually changes in the available channels?Jaw sets; shoulders fold inward; words remain courteous.
Expressive descriptionWhat quality is conveyed, with what confidence?A restrained, brittle courtesy in voice and face.
Context and actionWhat is known about the relationship, task and stakes?The script establishes a failed promotion and a public congratulations.
Psychological formulationWhat interpretation connects the actions?An authored envy–gratitude mask; alternative interpretations retained for unlabelled footage.
Observer responseHow does this performance affect a particular viewer?Compelling but uneasy; high engagement, low perceived ease.
EvaluationHow well does it satisfy a declared task?A convincing maintained facade may have little detectable hidden affect.

2. What the scales contain

82 descriptive fields belong to the performance profile. Eight observer scales belong to a separate response layer.

The PerformanceNet Taxonomy is the common descriptive profile: 82 fields comprising 79 ordinal scales and 3 categorical capture fields. Appendix A preserves 41 original dimensions; Appendix B supplies 41 additional descriptive dimensions. A further 8 observer-impression scales in Appendix B form a separate evaluation layer and are not part of the 82. Field codes, definitions, scale directions, anchors, observation cues and distinctions remain traceable in the appendices.

Shared componentWhat it contributesHow to use it
PerformanceNet Taxonomy82 descriptive fields: 41 inherited fields in Appendix A plus 41 additions in Appendix B. There are 79 ordinal scales and 3 categorical capture fields.Represent available performance attributes with exact anchors, source IDs, modality and missingness.
Observer-impression layer8 further scales in Appendix B, describing a particular viewer’s response.Keep viewer response and task-fit evaluation separate from the 82-field descriptive profile.
VoiceNetDetailed vocal qualities such as tempo, articulation, pitch range, respiration and texture.Pin the codebook. Map an authored voice direction or observed vocal description into compatible voice fields; do not equate similar names across taxonomies.
EmoNet40 emotion and state labels, with the selected intensity and expression conventions.Declare whether a label names authored inner state, intended display, observed expressive impression or a hypothesis.
Situation and scenario spaceThe shared Situation Taxonomy, plus situation, compound edge-case and extreme-premise source catalogs.Use the tags and premises to encode context, build conditions and inspect coverage. Catalog entries may overlap; their counts are not a sum of independent psychological states.
ComponentWhat the framework adds
41 original dimensionsPreserved source wording and provenance; do not silently replace old anchors with new text.
Body and faceAdd openness, expansion, orientation, symmetry, stability, rigidity, self-directed movement, restlessness, facial tension, mobility, asymmetry and gaze measures.
Physical expressionAdd respiratory disruption, trembling and recovery; retain regional tension and exertion profiles.
VoiceIntegrate audible projection, pace, flow, pitch excursion, articulation, roughness and breathiness; add structured timbre, rhythm and style profiles.
Social, intellectual and psychological aspectsAdd contextual warmth, vulnerability, hostility, cooperation, boundaries, repair, care, expressed certainty, reasoning structure, revision, self-monitoring and protective presentation.
Channel relationshipsSeparate face–body, face–voice, body–voice and words–delivery correspondence; preserve disagreement and timing.
Situation and emotion annexesThe supplied source catalogs retain 351 situation records, 266 edge cases, 600 extreme premises, 40 state labels and 400 masking pairs. These are named source inventories, not a combined dimension count for the shared taxonomy.

How to read inherited definitions

  • Terms such as prefrontal, limbic and autonomic in the original source are historical explanatory language. Ordinary audio or video cannot identify the neural source of a movement. In this framework they never become observed biological ground truth.
  • A still body, averted gaze or facial asymmetry is not evidence of deception, a diagnosis or a stable personality trait. Statements implying such a one-to-one relationship are not an inference rule in this edition.
  • An open torso and emotional receptivity are separable. Use the original REC-INF (Receptivity & Influencability) definition for a documented legacy score, and BOD-OPN (Postural openness) for posture itself; do not replace one with the other.
  • The original prose mentions two categorical capture fields; the actual index has three: CAP-SHT (Shot Scale), CAP-INT (Interaction Setting) and CAP-POS (Body Configuration). All three are categorical in this framework.
  • Preserved wording does not make old scores automatically comparable to newly evidence-filtered scores. Store scale_version, protocol_version, available channels and rater access. Earlier rewritten anchors do not replace the preserved original definitions.
  • The supplied original worked examples are not adopted as calibrated gold labels: some rate mutual interaction in solo material or conflict with visibility rules. The worked cases here explicitly specify observability.

3. Define the person, scene and beat

A rating becomes useful when we know who it describes, when it applies and what the observer could access.

Start with a scene, then identify a focal person, their relevant partners and meaningful beats. A beat is a period organised around one operative stimulus or action; it ends when information, intention, bodily constraint, relationship or tactic changes. A shot boundary is an editing boundary, not automatically a psychological event. Use both start_s and end_s for every interval. Keep speaker, face and body track IDs linked only when that association is supported.

RecordRequired content
MediaDuration, frame timestamps, sample rate, audio–video offset, cuts, crops, dubbing, playback rate and processing history.
Focal participantStable local person ID, visible body regions, audible speech attribution, identity-link confidence and relevant occlusions.
PartnersPresent, off-screen, mediated, imagined or absent; who can see or hear whom; response availability.
Social relationKnown role, institutional authority, dependency, familiarity, shared history, norms, language and audience. Unknown facts remain unknown.
Knowledge and goalsWho knows or believes what; declared or scripted objective; evidence for any inferred objective; incompatible interpretations.
Task and environmentPractical action, object use, support, constraints, distance, noise, visibility and current disturbance.
Consent and boundaries in the sceneWhat limit was communicated, by whom, and how the focal person responded; avoid inferring consent from friendliness.
Observer accessMedia-only, script-informed, production-informed or actor self-report; keep these evaluation conditions distinct.

Use only the context you have

In surveillance-style footage, annotate movement, interaction and visible circumstances conservatively. A person pacing is observed pacing; anxiety, intoxication, intent to offend or private emotion are not established by that observation. A uniform or role label may be known from supplied context, but appearance alone does not establish authority or occupation. The same restraint applies to documentary and interview clips.

Character circumstances and actor circumstances

For film and theatre, distinguish the actor’s real circumstances from the character’s authored circumstances. The character can be exhausted while the actor is not. A screen performance can use a close-up, a cut or music to shape the audience’s reading; record those contributions instead of attributing every effect to the focal performer.

4. Read body and face

Posture, space, movement and apparent tension are separate properties. Describe their combination.

Whole-body description should cover support, orientation, openness, occupied space, movement quality and the distribution of apparent tension. Open/closed, large/small, relaxed/rigid and moving/still are different questions. A person can occupy little space with an open torso, stand expansively while emotionally guarded, or remain still with very little muscular bracing. Keep these combinations available rather than forcing them into a single “confidence” scale.

Structured profileWhat to record
Regional tensionBrow, eyelids, jaw, lips, neck, shoulders, upper arms, hands, torso, pelvis, legs and feet; visibility and apparent contraction separately for each.
Posture and supportStanding, seated, lying, moving, crouched or restrained; contact points, lean, spine shape, limb crossing and weight shifts.
Movement formPath, direction, range, speed, acceleration, stopping, flow, apparent weight, directness, symmetry and recovery. Use physical units only with adequate calibration.
Practical actionReach, grasp, carry, protect, approach, withdraw, help, block or handle an object; intended and completed actions separately.
FaceBrow, eyelids, cheeks, mouth, jaw and chin; onset, peak, hold and release; changes rather than only a representative still.
Gaze and headTarget, contact, scan, fixation, blink, head turn and eye–head disagreement; uncertainty for distant or obstructed faces.
Physiological-looking eventsBreathing changes, trembling, flushing where visible, sweating, tears, swallow, startle and recovery; presence is distinct from inferred cause.
Contact and spaceDistance category, approach/withdrawal, barriers, touch target, initiation, duration and response.

Appearance is not a physiological measurement

“Apparent muscle tension” is a visual judgement about bracing or contraction, not a physiological measurement of muscle tone. Record an instrumented value only when an appropriate sensor is actually available. Similarly, an acted pain response is not a measurement of pain, breathing sound is not oxygen saturation, and a red face is not a reliable measure of autonomic activation. The framework can describe these appearances exhaustively without pretending to measure their biological causes.

Use the person’s available range

Physical individuality matters. Habitual posture, mobility differences, sign language, assistive devices, facial movement differences and disability can change the available movement range. Describe the observed form and task-relative change; do not assign low quality or psychological meaning because a person differs from an imagined standard body. A person’s own available baseline is useful when genuinely observed, but must not be fabricated from a short clip.

5. Hear voice, words and events

Keep the words, delivery, non-speech events and recording conditions distinct.

The audio layer remains detailed, but it is one channel of the performance. Retain the actual transcript rather than assuming the spoken words match a script. Separate projection from microphone gain, pitch range from absolute register, articulation from accent, vocal texture from recording damage, and emotional intensity from volume. Whispered anger can be intense; a loud celebratory instruction need not be angry.

Audio profileFields beyond the anchored dimensions
Rhythm and phrasingPhrase chunks, pause intervals, stress placement, speaking rate, rate change, restarts, disfluency and sentence structure.
Pitch and spectral characterRegister, local range, brightness, fullness, perceived warmth, metallic character, harmonicity and resonance descriptors; these are acoustic impressions, not anatomical localisation.
Delivery styleConversational, casual, formal, dramatic, narrator, newsreader, didactic, authoritative, playful, cartoon-like, close-miked sensory delivery (ASMR), whispered, ranting, storytelling or monologue; multiple styles may coexist.
Temporal changeArousal change, valence change, volatility, dynamic arc and pace variation; signed change and magnitude must be separate.
Speaker presentationPerceived vocal age or gender presentation if relevant to the task; never treat this as verified demographic identity.
RecordingNoise, clipping, compression, reverb, filtering, music, dubbing and technical quality. Preserve clean source and processed versions separately.
Non-speech eventsLaughter, sob, gasp, sigh, scream, groan, grunt, throat-clear and other reviewed categories; start/end, channel, class uncertainty and overlap.
Quality and enjoymentHuman or model ratings with their rubric and calibration; aesthetic preference is not an observable bodily property.

Keep the audio vocabulary identifiable

The PerformanceNet Taxonomy organizes the performance as a whole. VoiceNet supplies a complementary, more detailed vocal vocabulary, including tempo, articulation, pitch range, respiration and texture. Store the exact codebook and mapping used when retaining model predictions, constructing targets or comparing datasets; the source’s distinct 57-output and 59-dimension inventories are not automatically interchangeable. Appendix J preserves the historical production crosswalk and campaign assumptions.

Time and attribute every vocal event

Each burst has start and end times, event type, optional description, detector confidence and provenance. Intended bursts, detected bursts and human-verified bursts are different records. A voiced cry may overlap speech, and background laughter may belong to somebody else. Preserve overlaps and source attribution; a single exclusive frame label cannot represent every case.

6. Describe conduct in context

Specify the partner and task. Record demonstrated reasoning and behaviour before interpreting a private motive.

Social conduct: towards whom?

Social conduct is relational. Describe warmth toward whom, hostility toward whom, vulnerability before which audience, and cooperation toward which agreed task. Institutional rank, enacted dominance, expertise and momentary control of the floor are not the same. A low-ranking person may confidently lead a task; a high-ranking person may seek reassurance. Rules of politeness and eye contact vary by relationship, culture and setting, so the known frame belongs with the rating.

Reasoning: what is demonstrated?

Intellectual description concerns processes demonstrated in the scene: tracking information, organising an explanation, recognising a contradiction, updating a belief, monitoring uncertainty and changing a strategy. It does not infer general intelligence. A fluent monologue can contain poor reasoning; an uncertain, interrupted sentence can reveal careful thought. If the clip contains no reasoning opportunity, reasoning-specific dimensions are unavailable rather than low.

Psychological formulation: what is supplied or inferred?

Psychological formulation connects the person’s authored or documented commitments, needs, protective strategies and contradictions to the current situation. Record the desired outcome, the feared cost, the image being protected and the competing impulse. For fiction these can be deliberate instructions; for uncontextualised footage they are hypotheses with alternatives. Stable personality or pathology is outside what a short performance establishes.

Keep distinctWhy it matters
Displayed vulnerability / helplessnessA forceful request can openly reveal need without surrendering agency.
Care / warm mannerSpecific practical help can be curt; a warm voice can ignore the other’s expressed need.
Boundary response / complianceRespecting a refusal differs from obeying an imposed demand.
Certainty / correctnessThe performance may intentionally present a false claim with confidence.
Protective persona / insincerityA professional role can be sincerely maintained.
Emotional intensity / arousal / loudnessGrief, controlled rage and exuberance can occupy different combinations.
Surface display / underlying stateConcealed emotion requires script, report or an explicit uncertain interpretation.

7. Follow agreement and change

Channels can agree, diverge or change at different times. Agreement is not a universal measure of quality.

Compare one quality at a time

Analyse correspondence between face and body, face and voice, body and voice, and literal words and delivery separately. Specify the quality being compared: warmth, activation, certainty, tension, direction of attention or timing. A smile and a stiff torso can be a meaningful divergence; neither channel should automatically overrule the other. High correspondence is descriptive, not inherently better acting.

Check the recording before interpreting a mismatch

A believable mask may require a sustained difference between channels. Irony can require words and delivery to disagree. A person may attend visually to a practical task while speaking affectionately to someone beside them. Conversely, accidental lip-sync error, dubbing, an audio offset or an edit can produce divergence that has nothing to do with the actor’s intention. Check capture conditions before psychological interpretation.

Temporal fieldDefinition
Onset / offsetBoth endpoints for every beat, gesture, utterance and non-speech event.
TrajectoryInitial level, peak or trough, terminal level, direction and time of turning point.
LatencyTime between an available stimulus and a specified observable response.
Lag between channelsTime difference between comparable face, body and voice changes; uncertainty retained.
ArcBuild, release, reversal, rupture, recovery, oscillation, delayed reaction or sustained line of action.
Transition qualityAbrupt/gradual, integrated/discontinuous, stimulus-linked/unexplained; distinct from emotional magnitude.
RecoveryReturn, reorganisation, renewed mask, exhaustion or unresolved disturbance; follow-up duration recorded.

A sustained performance can still be alive

A change of state is one possible dramatic form, not a requirement for good acting. Sustained composure, grief, delight or cool refusal can remain alive through responsive attention and action. Evaluate whether a transition is motivated when the scene asks for one; do not reward a generator for inserting an emotional switch into a scene that calls for continuity.

8. Separate feeling, display and restraint

A character can be directed to feel one thing and show another. The prompt does not prove what a viewer can detect.

The forty-label inventory includes emotions, attentional states, physical states and interactional intentions. Treat it as a practical annotation vocabulary, not forty interchangeable biological emotion mechanisms. Appendix C explains each label, its dramatic use and common confusions. Store several labels when necessary, with evidence, target and temporal span. A single primary category is a sampling convenience, not a complete account of the performance.

Level or distinctionOperational use
Intensity 1 — traceA small but consequential cue; the state does not organise the whole performance.
Intensity 2 — moderateClearly present, with ample room for the task and other feelings.
Intensity 3 — strongOrganises much of the performance but does not erase its objective or responsiveness.
Intensity 4 — extremeNear the character’s expressive limit in this situation; not automatically louder.
Free / held expressionHow much of the target state is permitted outward; independent of its authored internal strength.
MaskingA public display serves a goal while another authored or supported state is concealed.
Leak / repairA local departure from the public display followed by recovery or reorganisation.
Suppression controlOutward flattening without substituting another emotion; distinct from a full mask.

Direction matters in a masking pair

The 400 source pairs are directional. They are printed as public surface first and underlying state second for readability, but their original felt→displayed IDs and direction remain intact. An amused person trying to appear angry is a different task from an angry person trying to appear amused. The source contains related-state controls and context-dependent pairs as well as strong contrasts; it is not a list of 400 scientifically established opposites.

Compare maintained and leaking versions

A good pair needs an audience, a reason for the public display, a reason to conceal the underlying state and a cost if the facade fails. For a controlled comparison, maintain the same words, voice, role, situation and style in both versions; alter only permitted leakage and repair. Hidden feelings do not become measurable ground truth because they were included in the prompt. Measure the public display and perceptible leakage separately.

Understand the source distribution

The source has forty underlying labels but only thirty outward labels. Some outward labels occur far more often than others, so preserving every source pair once is incompatible with exact equal outward marginals in that sub-bank. Additional candidates and corrected gustatory Sourness proposals remain explicitly separate in Appendix G; their existence does not retroactively make the original source balanced.

9. Build a playable scene

For a scripted or generated scene, give the actor a partner, task, obstacle and stake. The Script Analysis System supplies the richer authored context; it is optional context when annotating an existing recording.

Give the virtual actor a consequential task in a specific fictional world. Start from the source script, the character’s established makeup, a partner, an obstacle and a stake. Add a sensory detail that matters to this character. “Be very sad and whisper” provides little dramatic structure. “Ask your sibling to stay while you finally pack your father’s coat; you have insisted all day that you need no help” gives the request, inner conflict and delivery a reason. The generator receives these authored conditions together with the exact words and actions to perform.

Seven source obligationsHow they become playable
Time & PlaceWhich space, time, soundscape, distance and physical arrangement matter?
RelationshipWho are the participants, and how does the character feel towards each?
EmotionalWhich feelings are present, mixed, redirected or hidden in the authored scene?
CharacterWhich established tendencies and contradictions are activated?
HistoricWhich period norms, technologies and manners matter, separately from personal history?
ThematicWhich broader dramatic question is embodied?
SubtextualWhich unsaid reality differs from or supports the literal words?

Connect the scene to the character

Character Makeup connects four portraits: physical, emotional, intellectual and psychological. Each includes a tendency, its origin in the fictional life, an activation condition, a countertendency and a consequence for the scene. Read the script through this makeup to establish all seven Dramatic Obligations. The Emotional Obligation is the scene’s combination of feelings and their regulation; the other six describe time and place, relationships, character, historical world, theme and subtext. Together they guide the beat sequence and its Experience Chunks. The character’s self-image and an audience’s impression may differ. Historic refers to period norms and material conditions, not personal backstory.

Use the inspiration index with its provenance

Appendix H contains 806 source summaries attributed to the Eric Morris system: 100 concepts, 302 techniques, 174 exercises and 230 recommendations. For an AI actor, translate them into fictional role design and clear generation instructions. A sensory exercise can inspire what the character attends to; a conflict exercise can inspire competing impulses within an Experience Chunk. The model receives a specification, rather than being instructed to undergo a human rehearsal process.

Translate inspiration into playable choices

Use fictional sensory circumstances, observable triggers, impulses, conflicts, relationship obligations, discoveries and changes of tactic. Leave room for unexpected but situation-fitting microresponses. Do not translate “authentic” into generic breathiness, constant vulnerability or random hesitations. A cool, forceful, comic, ceremonial or apparently indifferent character can be fully responsive. Appendix H lists the full 806-entry inspiration index and source summaries; those summaries are repository evidence, not a newly verified edition of the original books.

10. Choose a situation

Use the catalogues to specify circumstances. A situation is not a fixed prediction of a person’s response.

The shared PerformanceNet Situation Taxonomy provides the situation representation used across both systems. Alongside it, the inherited catalogs preserve 289 original situation tags in eleven families plus 62 human-experience additions, 266 compound edge cases and 600 extreme premises in six groups and sixty subgroups. Their 1,217 source records are printed in Appendices D–F. These totals describe those source catalogs, not the number of independent dimensions or mutually exclusive states in the shared space. Map and version their relationships; a scene may combine compatible tags and several catalog premises may overlap.

Inherited situation familyRecords
POSE32
ACTV69
TRGT25
SOCT56
ENVI22
HLTH18
GEAR14
CLMT10
SUBST12
FATG19
PAIN12
Coverage facetWhat must remain describable
Life stage and transitionsDeparture, reunion, caregiving, work, illness, bereavement and aging
Economic and practical stakesWork, scarcity, dependence and loss of resources
Affiliation and attachmentCare, intimacy, rejection, loyalty and abandonment
Power and legitimacyFormal authority vs enacted dominance; coercion vs voluntary action
Knowledge and uncertaintyKnowing, guessing, discovering, misunderstanding and concealment
Speech-act functionRequest, refusal, invitation, apology, accusation and repair
Listening and silent responseUptake before speaking; refusal to respond; shared silence
Masking and counter-impulseDisplay vs concealed state; containment vs leakage
Physiological demandEffort, pain, reserve, distress and recovery
Sensory worldsTemperature, taste, texture, noise, distance and occlusion
Coordination and accessMovement, communication access and alternative phonation
Concurrent actionCare work, tool use, locomotion and divided tasks
Attention and cognitionTarget, width, load and apparent formulation
Face and gazeIntensity, contact, aversion and occlusion
Gesture, body and spaceAmplitude, flow, support, orientation and proximity
Interaction and repairOverlap, latency, coupling, interruption and recovery
Culture and historyPeriod norms, ritual and context-specific expressive conventions
Language and identityMultilingual speech, code switching, accent concealment and accommodation
Audience and genrePrivate behaviour, direct address, realism, theatre and codified form
Non-speech vocalisationBreath, laughter, sobs, cries, throat clearing and creature performance
Sound–image relationsAligned or discrepant channels; dubbing and offsets
Temporal formBuild, collapse, waves, pivot, leak, repair and recurrence
Media constructionShot scale, cuts, cropping, missing regions and recording effects
Ambiguity and alternativesMore than one plausible account of the same signal

Allow quiet and unusual responses

The extended remit includes ordinary affiliative encounters, care, work, hierarchy, grief, uncertainty, deprivation, conflict, physical overload, sensory constraints, crisis, aftermath and altered or unfamiliar states. Extreme does not mean continuously loud: immobility, delayed reaction, absent response, brittle politeness and near-silent resistance are important boundary cases. The annotation standard can describe difficult source contexts without requiring a generator to stage them literally.

Keep adaptations identifiable

Fifty source records describe mature contexts that were previously catalogue-only. They remain indexed. When a scene uses an adapted treatment, preserve the source ID and identify what was changed. Coverage of an ID is different from a literal rendition of its premise; neither establishes that a generated performance has successfully realised it. Appendix J retains the original campaign’s allocation note.

Use the same context space in both directions

Script Analysis uses situation descriptors to design a scene. Annotation uses supported context descriptors to encode a recording. Edge cases and extreme premises are points or combinations within that broader scenario space, not additional scales to average into a profile. Their source IDs remain useful for retrieval, sampling, coverage review and controlled generation.

11. Evaluate the viewer’s experience

Separate what happened, how it came across and how well it met the declared task.

The question “How does this person come across?” is essential, but it belongs to a particular viewer under a particular viewing condition. Eight impression scales cover engagement, believability, legibility, warmth, threat, vulnerability, competence and enjoyment. Record the viewer’s language competence, relevant cultural familiarity, available modalities, script access, reasons and confidence. Report rater distributions and disagreement rather than treating one model’s verdict as a fact about the person.

ImpressionQuestion
EngagementHow strongly this performance holds this viewer’s attention.
Believability in the stated sceneHow plausible the performed responses feel within the stated fictional or real context.
Legibility of the intended actionHow clearly the viewer can follow what this person is trying to do, conditional on access to the scene.
Perceived warmthHow interpersonally welcoming this person seems to this viewer in this clip.
Perceived threatHow threatening the performance feels to this viewer; not a prediction of dangerousness.
Perceived vulnerabilityHow exposed or emotionally reachable the person seems to this viewer.
Perceived competenceHow capable the person appears in the specific depicted task; not an assessment of general ability.
Viewing or listening enjoymentHow rewarding this viewer finds attending to this performance; distress and artistic value may coexist.

Evaluate fit to the task

A performance-quality rubric asks whether the requested objective, relationship, circumstances, emotional intensity, style and timing were realised. It separately scores dramatic specificity, responsiveness, transition motivation, intelligibility, technical integrity and consistency with the intended constraints. Do not average all descriptive dimensions into a “good acting” number. High facial tension, low warmth and strong concealment may be exactly right for the scene.

Calibrate the judge as well as the scales

A model or Gemini judge can support screening, but judge agreement is not validation of the taxonomy. Calibrate on diverse actors, languages, capture conditions, naturalistic and stylised performances, then compare independent human raters. Evaluate missingness and observability errors as well as score correlation. Use ordinal agreement for anchored scales, category agreement for capture fields, and boundary accuracy for timed events. Report performance separately for audio-only, silent video and audiovisual access.

Before deriving a learning signal

The eight observer-impression scales describe responses, while a task-fit rubric describes success under a declared objective. Neither should be silently mixed into the 82-field descriptive profile. Specify which response or task-fit labels are eligible for supervision or preference learning, how disagreement is represented and which viewing conditions apply. A derived scalar reward is a modeling choice that needs separate calibration and held-out evaluation.

12. Annotate with an evidence trail

Capture access first, observations second, interpretations and task-fit ratings afterwards.

  1. Identify the focal participant, time interval, partners and known context. Record actual capture conditions and audio–video synchronisation.
  2. Mark visibility and audibility separately for eyes, face, hands, torso, legs and voice. Rate capture metadata first. An invisible face is not a neutral face.
  3. For audiovisual material, inspect silent video and audio separately before the combined pass. This exposes channel differences and reduces cross-channel halo effects.
  4. Annotate observable events and regional profiles before assigning broad psychological interpretations. Attach the evidence interval to each claim.
  5. Use the original anchors for an original field and the matching anchors for an added dimension. Store the scale version; do not average categorical capture values.
  6. Mark unavailable fields with null plus a reason: occluded, inaudible, insufficient context, no relevant opportunity, uncertain attribution or insufficient duration. On an ordinal scale, zero means the observed low anchor. In a categorical field, zero identifies its named category. Neither use means missing information.
  7. Record alternatives when an interpretation is not unique. Distinguish authored intention, self-report, documented context, human observation, human inference and model prediction.
  8. Annotate time-local change, recovery and channel disagreement. Whole-clip summaries must retain variability and must not erase a consequential transition.
  9. Collect observer impressions and task-fit ratings in separate passes, with disclosure of what the rater knew.
  10. Recheck systematic disagreements and confusing dimensions. Publish the protocol revision and calibration evidence before claiming a reliable measurement standard.
MaterialAvailable analysisUnavailable without extra evidence
Audiovisual, clear full viewVoice, face, body, shared action, temporal correspondence and known contextual interpretation.Private feelings, neural mechanisms and stable traits are still not directly observed.
Audio onlyWords, voice, audible events, phrasing, speaker-attributed interaction and listener impressions.Posture, face, gaze, visible muscle appearance and visual cross-channel correspondence.
Silent videoVisible actions, posture, face where resolved, space and visible interaction.Acoustic properties, spoken content unless independently supplied, voice–body correspondence.
Distant fixed cameraSupport, gross movement, object actions, approach and withdrawal when visible.Fine facial actions, exact gaze and private psychological state; speech if inaudible.
Theatre recordingPerformed action and audience-directed scale with camera-view observability.Assuming a stage performance should be naturalistically small or judging the live audience’s view from a camera crop.
Script or generation prompt onlyAuthored intentions, requested states, expected beats and sampling controls.Any realised score, actual spoken transcript, duration or successful generation claim.

The shared record interface

The two systems can describe the same scene without claiming the same knowledge. Preserve the original text, authored design, media and observations as separate records. When a generated take or reference rendition corresponds to a script, link it explicitly; observed timing and segmentation may differ from the planned beats.

Interface fieldWhat it preserves
Source, scene and participant IDsThe material and focal person or character. Use a recording-local participant ID when no fictional role is known.
Planned beat / observed interval alignmentAn optional link between an authored beat and an observed time span. It need not be one-to-one and must not force a recording into the intended timing.
Field code + taxonomy versionThe exact descriptor and its interpretation; preserve ordinal, categorical and event types.
Role + basisAuthored condition, direct observation, supported context, inference, viewer response, task-fit label or derived training target.
Modality + availabilityAudio, video or audiovisual access; which regions/events can be rated, with null and a reason for unavailable fields.
Evidence + calibrationThe source span or media interval, rater/model provenance, uncertainty, alternatives and protocol revision.
Learning or selection useWhich labels are eligible for supervision, preference/reward learning, retrieval or sampling, and why.

Do not fill an unobserved inner state from a script and then label it “observed.” Do not convert a missing visual score into zero for an audio-only example. Do not use a high ordinal value as an automatic positive reward: the high end may describe stronger tension, hostility or another task-dependent attribute.

Conditions, labels and model predictions stay distinct

A supervised target may be a reviewed observation of a recording. A conditioning value may be authored from a script. A preference target may express a task-specific comparison. A model prediction is the output of a particular learned or rule-based annotator. Preserve these origins in the record and in any exported training table; using the same field code does not make the values epistemically equivalent.

13. A complete worked reading

One gracious public gesture can carry several simultaneous, apparently conflicting signals.

Illustrative authored scene, not an evaluated recording: a project lead publicly congratulates the colleague chosen for a promotion. The script specifies envy beneath a gracious public display. From 0–5 seconds the face smiles, the voice is warm and the torso is enclosed; from 5–8 seconds the jaw sets briefly while the words stay generous; from 8–12 seconds the person offers to hold the colleague’s glass and turns toward the group. The scene is deliberately small, but it carries physical, social, intellectual and psychological information.

Three beats from zero to twelve seconds, showing public action, face and body separately.
Figure 2. A shared time axis makes divergence and change visible without inventing a continuous emotional measurement.
LayerIllustrative annotationWhat it does not establish
BodyEnclosed torso; limited occupied space; shoulders held. BOD-OPN (Postural openness) low, BOD-EXP (Occupied bodily space) low, regional bracing noted.Not automatic social withdrawal, shame or actual elevated muscle tone.
FaceSmile maintained; a brief jaw contraction at 5–8 seconds.Not a universal facial signature of envy.
Voice / wordsWarm public congratulations; a brief held pause; practical offer at the end.Not proof that the words were sincerely felt or insincerely produced.
Cross-channelFace–voice correspondence mostly high; face–body warmth correspondence lower.Lower correspondence is not lower performance quality.
ContextPublic setting, known promotion, relationship and authored concealment goal.The actor’s real feelings or employment situation.
CognitionNo extended reasoning is shown. COG-ORG (Organisation of expressed reasoning) and COG-REV (Updating after new evidence) unavailable.A short courteous utterance cannot assess intelligence.
ObserverMay feel controlled, poignant or uncomfortable; collect reasons and ratings.One observer’s impression is not the only valid reading.

In the table, a code names a specific descriptive field. “Low” means that an observed example falls towards that scale’s low anchors. “Unavailable” means the example supplies no relevant evidence; it is not a low score.

With audio alone, retain the warm public delivery, pause, actual words and listener impression; visual fields become unavailable. With a distant silent camera, retain the body orientation and practical action if resolved; the precise jaw event, words and vocal warmth may be unavailable. The script can still supply an authored envy condition, but that is contextual knowledge rather than an inference from the reduced recording.

14. Validate and reuse responsibly

A clear specification still needs empirical validation. Keep the definitions, evidence and uncertainty attached.

This is a proposed descriptive and evaluation framework, with intended uses in structured scene encoding, supervised target construction, preference/reward-model data preparation and dataset selection. A complete taxonomy, a valid annotation file or agreement between two automated judges does not establish reliable human judgement, model controllability or a validated reward signal. Test labels and downstream decisions on actual real, reference and synthetic material with independent raters, declared tasks and held-out data. Source-package checks and historical campaign plans remain in Appendix J.

Test the distinctions with counterexamples

A first validation study should deliberately include counterexamples: quiet anger, loud sadness, friendly-looking boundary violation, curt responsive care, confident error, hesitant correct reasoning, naturalistic high amplitude and highly stylised small movement. Test whether raters keep related dimensions apart. Sample diverse language, culture, mobility, age presentation and recording conditions without interpreting those attributes as causes of a score.

Prevent leakage between dataset splits

For dataset splits, group all versions of one parent scene, translations, voice presets, masking twins and processing variants together. Otherwise a near-identical scene can leak from training into evaluation. Keep target instructions, measured observations and reference labels in separate namespaces. Audio-only training cannot validate the new visual scales; those require a later audiovisual collection.

Validate the downstream decision

For supervision, measure label reliability, timing accuracy, missingness and performance on the intended modality. For preference or reward learning, test whether the signal respects the declared task, generalizes to held-out scenes and does not simply favor high amplitude, obvious emotion or recording quality. For filtering and diversity, audit eligibility errors, coverage and the effect of the chosen buckets. Record selection criteria and retain difficult counterexamples instead of treating rarity as evidence of poor quality.

Keep ordinal anchors ordinal: do not assume equal distances between neighboring values. Keep categorical capture fields categorical. A value of zero is an observed low anchor or a named category, depending on the field; it is never a substitute for unavailable evidence. Any numeric loss, weighting or reward derived from these fields must state its assumptions and be tested for the target use.

Reference library

Find the right appendix

Each appendix has its own reading guide, searchable entries and navigation. Follow the volume links below; retain exact codes when citing an entry.

VolumePurposeContents
AOriginal dimensions41 dimensions with original definitions and anchors.
BAdded dimensions and observer scales41 additions; 8 separate observer-impression scales.
CEmotion and state vocabulary40 labels; 320 intensity × expression codes.
DSituation taxonomy351 records: 289 original tags + 62 additions.
ECompound acting edge cases266 challenges with source cue suggestions.
FExtreme physical premises600 premises; 6 groups and 60 subgroups.
GSurface and underlying pairs400 source pairs + 135 candidates; reverse index.
HMorris inspiration and character questions806 synopses + 63 questions, arcs and mechanisms.
INon-speech event vocabulary180 event types in the non-explicit source inventory.
JSources and glossaryProvenance, file availability and 24 definitions.

Companion system and shared situation volume

The Humaneness Script Analysis System supplies the source-to-character-to-scene-to-beat workflow, including one Experience Chunk per beat and three complete fictional dossiers. The PerformanceNet Situation Taxonomy supplies the shared situation reference. Appendices D–F preserve the inherited situation, edge-case and extreme-premise catalogs with their original source IDs.

Naming continuity. Humaneness Performance Annotation System is the current name of the PerformanceNet main reference. Humaneness PerformanceNet names the umbrella; PerformanceNet Taxonomy names its 82 descriptive fields. Historical source titles, citations and codebook versions remain provenance records, not additional current systems. The taxonomy itself is not a trained annotation model or a validated reward model.