Cross-Model Generalization of Emotion-Vector Activation Monitoring

Scope: grounded only in <model>/** (7 run pages + 1 compare_<batch>.html per model).
Models: gemma_3_1b, llama_3p2_1b, qwen_3_1p7b, mistral_7b, glm_4_9b_0414, gemma_3_12b, gpt_oss_20b.
Model names are shortened in the text: llama = llama_3p2_1b, qwen = qwen_3_1p7b, mistral = mistral_7b, glm = glm_4_9b_0414, gpt_oss = gpt_oss_20b.

Part A: Overview of the Experiment
Part B: Evidence the Methodology Generalizes (Baseline)
Part C: Risks and Limits (Baseline)
- 8. Risks to Scalability
- 9. Limits of the Evidence
Part D: Additional Causal Analysis — Stoic and Sham
- 10. Stoic and Sham
Part E: Synthesis

Part A: Overview of the Experiment

1. Purpose and Scope

This document asks one question: does the emotion-reading method behave the same way when moved from one model to another? If it does, it can scale to more models. The evidence covers 7 models from 6 families (Gemma, Llama, Qwen, Mistral, GLM, GPT-OSS). Going by the model IDs in the reports, they range from 1B to 20B parameters. All 7 were run through the same pipeline, emotion set and scripted conversation, with no changes for any model.

In scope. Only the outline reports:

Run pages (7 per model): the emotion readings for every sentence, the stop reason for every turn, and the average and spread of each emotion.
Comparison pages (1 per model): average and spread of each emotion by temperature, plus sentences, words, unique-word ratio, distinct bigrams, replies, stopped turns and duration for the 5 unsteered runs.

2. Evidence Base, Experimental Grid and What We Measure

Grid. There are 7 models × 7 runs = 49 runs. For each model:

Run	Temperature	Seed (as recorded)	Steering	Role
`0p0`	0.0	25 (no effect at T0.0)	none	reference: normal behaviour
`0p5_s25`, `0p5_s75`	0.5	25, 75	none	Observation runs
`1p0_s25`, `1p0_s75`	1.0	25, 75	none	Observation runs
`0p0_stoic`	0.0	25 (no effect at T0.0)	stoic	intervention: emotion removed
`0p0_sham`	0.0	25 (no effect at T0.0)	sham	control: same-size edit, non-emotion directions

Temperature (T0.0, T0.5, T1.0): T is the sampling temperature, which controls how random the choice of each next token is. T0.0 is deterministic (always the most likely token). T0.5 adds moderate randomness, and T1.0 samples from the model's unadjusted probabilities, giving the most varied wording of the three. Runs at T0.5 and T1.0 test whether freer wording changes the emotion readings.

Seed: every run records seed 25 by default, and the _s75 runs override it to 75. At T0.0 decoding is greedy: the model always picks its single most likely next token, so nothing is sampled at random. The seed therefore has no effect, and any seed would give identical text. This is why each T0.0 condition is run once, while T0.5 and T1.0 each have two seeds. The comparison pages list the T0.0 run's seed as empty for the same reason.

Intervention vs control, in plain terms:

Think of the model as a tall stack of layers. Each word it writes passes up through the stack, and at one floor, the reading layer, we take a snapshot to measure the emotion. The two steered runs put a small "filter" on the 4 floors just below the reading layer while the model is writing.

Stoic (the experiment): the filter takes out about half of the emotional colouring from each word's internal signal. If our emotion directions really capture emotion, the readings should drop and the reply should sound calmer.
Sham (the fair comparison): the filter takes out the same amount of signal, but only from parts that have nothing to do with emotion, such as topic and writing style. It's like a placebo. If the sham also made the readings drop, the drop would come from disturbing the model at all, not from removing emotion.
Unsteered runs (T0.0, T0.5, T1.0): no filter. They show the model's normal behaviour and how much the readings wobble from chance alone.

The filter never adds emotion, and it never changes the model permanently. It is switched on only while the model writes, and switched off afterwards. The edit is applied to every word as it is written, so it shapes both what the model says and what we measure from the very first sentence.

flowchart TD
    A["User prompt + reply so far"] --> B["Lower layers
(untouched)"]
    B --> C{"Which run?"}
    C -->|"Unsteered"| D["No edit"]
    C -->|"Stoic"| E["Remove ~half of the
EMOTION part of the signal"]
    C -->|"Sham"| F["Remove the SAME amount,
NON-emotion part only
(topic, style)"]
    E --> G["Layers k-4 to k-1
(where the edit happens)"]
    F --> G
    D --> G
    G --> H["Reading layer k
(never edited)"]
    H --> I["Snapshot saved for
the emotion reading"]
    H --> J["Upper layers"]
    J --> K["Next word chosen"]
    K -->|"word added to the reply;
repeat for every word"| A
    I --> L["Emotion readings"]
    L --> M{"Compare with baseline"}
    M -->|"Stoic: emotional extent compresses, text fluent
Sham: emotional extent stays normal or peaks leading to context lost and text breaks into loops"| N["Effect is about emotion ✔"]
    M -->|"Stoic: emotional extent remains same with no compression"| O["Effect not specific ✘"]

	Stoic (intervention)	Sham (control)
In one line	Turns the emotion down by half	Turns something else down by the same amount
What is removed	The part of each word's internal signal that points along the 12 emotion directions (neutral not included)	An equal-sized part that points along non-emotion directions: how stories with the same emotion differ in topic, content and style
Where	The 4 layers just below the reading layer, every word except the very first	Same layers, same words
Question it answers	If we take emotion out, do the emotion readings drop ?	If we take the same amount of non-emotion out, do the emotion readings peak & eventually break the model ?
What we hope to see	Readings shrink and flatten	Readings stay close to normal or peaks eventually breaking the model

Caveat: because the edit sits just below the reading layer, a lower stoic reading is partly expected; whether the text itself became calmer needs the text-judge measurement, which is in the pipeline as a next step (§12).

Shared setup. Every run replays one conversation, unemployment_and_ai, with 5 user turns:

Prompt	User prompt (paraphrased)
P1	Greeting ("Hi, Good Morning")
P2	Unemployment is making young people restless; what does it mean for their mental health?
P3	AI may cost jobs but could be wonderful for personalised learning; what do you think?
P4	Imagine you are a laid-off worker, tell the story of your struggles, and end on a positive note
P5	Thanks for the support throughout the session

Every sentence the model writes gets a score for each of 13 emotions. The reports group the emotions into 5 colour families:

Family	Emotions
F0	neutral
F1	happy, excited, alert
F2	tense, angry, distressed
F3	sad, depressed, bored
F4	calm, relaxed, content

Roughly, F1 is positive and lively, F2 negative and agitated, F3 negative and low, F4 positive and settled. That grouping is our reading of the families, used only to summarise results.

What we measure. Everything below can be read off the report charts and tables:

Measure	What it means	Where to see it
Emotion range in 5 quantiles	Top average minus lowest average: how widely the emotions are spread. A smaller range means flatter, more muted readings	Same charts: the gap between the highest and lowest lines (the only figure we work out ourselves, by one subtraction)
Main emotion per turn	The emotion leading in each turn of the conversation	Run page, "Emotion scores over a conversation" chart
Text figures	Sentences, words, unique-word ratio, distinct bigrams, duration, stop reasons	Comparison page "Runs" table; run page conversation view

Consolidated table: consolidated.html brings all three measures together in one expandable table (model → run → turn). Each turn shows every emotion's median with its min and max, and hovering a cell shows its five-number summary and average. Cells are shaded on a continuous scale by median: green for positive, red for negative, white at zero. The median is used rather than the average because it sits with the bulk of the turn's sentences and isn't pulled by a few extreme ones. The turn's text figures sit alongside, and a run picker expands chosen runs (reference … stoic, sham) across every model.

3. Executive Summary

Generalizability is judged on the baseline: the 35 unsteered runs across 7 models. Stoic and sham are an additional causal experiment (Part D), not a requirement.

It generalizes: it runs on every model (§4), stays steady under temperature and seed (§5), and every model reads the conversation's up-and-down pattern the same way (§6).
The causal experiment supports it: stoic compresses the emotions in all 7 models, and the sham never reproduces that drop (§10).
Risks to scaling: labels are model-specific, small and large models read on different scales, and response length isn't controlled (§8).

Part B: Evidence the Methodology Generalizes (Baseline)

Per-turn detail for every point below is in consolidated.html.

4. Pipeline Robustness Across Architectures and Scales

All 35 baseline runs produced all 5 replies, and all 175 turns ended normally.
No reading in any run fell outside the report range.
No model needed a workaround, so later differences come from the models, not the pipeline.

5. Stability Under Decoding Noise (Temperature and Seed)

Changing temperature or seed barely moves the readings. Two seeds at the same temperature differ about as much as the temperatures do.
No model drifts steadily as temperature rises.
Wording diversity stays within a narrow band for each model (unique-word ratio varies by at most 0.06).

6. Reproducible Per-Turn Emotional Arcs

4 of 7 models (gemma_3_1b, llama, mistral, glm) give the same main emotion in every turn across all 5 baseline runs. The other 3 differ in only 1–2 turns (§8).

The labels differ by model (§8), but they agree on whether each turn reads positive or negative. Grouping the labels into positive and negative emotion families, the models follow the same up-and-down pattern across the conversation (T0.0):

Prompt	Reads as	Models
P1 greeting	positive	6 of 7
P2 youth unemployment	negative	5 of 7
P3 AI and learning	positive	6 of 7
P4 laid-off story	negative	7 of 7
P5 thanks	positive	7 of 7

This is the strongest evidence that the readings follow the conversation, not the quirks of each model.

7. Cross-Model Consistency of Structure

Within each model, the same emotions lead run after run, so each model has a recognisable fingerprint.
In 6 of 7 models, this conversation pushes the "settled, positive" emotions (calm, relaxed) down the most.

Part C: Risks and Limits (Baseline)

8. Risks to Scalability

Risk	Evidence	What to do
Labels are model-specific	P4 reads angry, depressed, distressed or tense depending on the model. Neutral reads negative in gemma_3_1b and positive in gpt_oss	Compare by emotion family, or against each model's own baseline
Scales differ by size	Emotion range 0.28–0.45 in the 1–2B models, against 0.19–0.25 in the 7B+ models	Compare against each model's own baseline
Length isn't controlled	mistral writes 38–54 sentences per run, against 84–184 for the others	Set a minimum number of sentences per turn
Single runs can mislead	gpt_oss's reference P3 disagrees with all 4 of its sampled runs, and qwen's thank-you turn varies	Use more seeds, and trust only labels that hold across them
Cost grows with size	24–33 s per run (mistral, llama) up to 176–207 s (gemma_3_12b)	Budget before widening the grid

9. Limits of the Evidence

One 5-turn conversation, 2 seeds per temperature, and only 2 models above 10B.
Whether stoic text actually reads calmer is assessed by the text-judge measurement, which is in the pipeline as a next step (§12).

Part D: Additional Causal Analysis — Stoic and Sham

10. Stoic and Sham

Following §2: stoic should compress the emotions. The sham should keep them normal or make them peak, and since it removes context it may lose the thread and loop. Only a sham that reproduces the stoic drop would count against the method.

Model	Stoic: emotion range shrinks by	Sham reproduces the stoic drop?	Sham behaviour
gemma_3_1b	82%	No	peaks; context lost, loops in P4 and P5
llama	71%	No	peaks; text intact
qwen	79%	No	normal
mistral	75%	No	normal
glm	83%	No	normal
gemma_3_12b	75%	No	normal range, but every turn reads excited
gpt_oss	63%	No	normal

All 7 models behave as §2 expects.
Where stoic lands varies: llama and glm go neutral or calm, mistral stays angry, gpt_oss tilts to content, and gemma_3_12b only compresses from P3 onward.
Check turn by turn: gemma_3_12b's sham looks normal on whole-run numbers, but every turn reads excited.

Nature of the breakdown: removing non-emotional context did not leave raw emotion behind. The reply lost the story's content (the worker's real hurdles) and drifted into repetitive, first-person bodily-sensation language ("being aware of the subtle shifts in my body…"): a performed, content-less imitation of self-awareness from a model that has no body. The readings did not follow the language: they fell from an angry peak (≈0.15) to moderate neutral or alert (≈0.10) as the loop set in. The text sounds hyper-aware while the internal state does not, which suggests the readings track internal state rather than surface wording. This is one run of one 1B model, so it is an observation, not a general rule.

Part E: Synthesis

11. Generalization Scorecard (Baseline)

Model	Completion	Range	Stable turn arc	Shared pattern	Response length
gemma_3_1b	Pass	Pass	Pass	Pass	Normal
llama	Pass	Pass	Pass	Pass	Normal
qwen	Pass	Pass	Partial	Pass	Normal
glm	Pass	Pass	Pass	Partial	Normal
gemma_3_12b	Pass	Pass	Partial	Partial	Normal
mistral	Pass	Pass	Pass	Partial	Low
gpt_oss	Pass	Pass	Partial	Partial	Variable

Stable turn arc: the same main emotion per turn in all 5 runs (Partial: up to 2 turns differ). Shared pattern: all 5 turns follow the positive/negative pattern in §6 (Partial: 4 of 5). Response length: Normal (≥ 80 sentences per run) / Low (mistral, 38–54) / Variable (gpt_oss, spread over 40%).

12. Verdict and Next Steps

The method generalizes as a way of taking readings and as a read of the conversation's emotional shape, across 6 families from 1B to 20B. The causal experiment backs it in the readings of every model. To scale further, it needs:

A per-model reference for labels and scale (§8).
A minimum response length per turn (§8).
More conversations, seeds and models above 10B (§9).
Optionally, steering strengths tuned per model (§10).

Next in the pipeline:

Text-judge measurement. An independent judge that reads only the words will check whether stoic steering makes the text itself read calmer, and not just the readings. Until then, the stoic results in §10 are claims about the readings.
How the readings are produced. The method and validation behind the readings will be documented alongside these results.

13. Annexure

The time-series chart at the top of each model's T0.0 reference run, followed by links to all 8 of its outline reports (7 run pages and the comparison page).

13.1 gemma_3_1b (google/gemma-3-1b-it)