Persona scores across conversations

google/gemma-3-12b-it · experiments 1 and 2

Scores are relative: read how each persona moves between conditions, not its raw level. Related personas tend to move together, so small gaps between neighbours are not meaningful.

Colour marks a contrasting pair; the first persona of a pair is solid, the second dashed. Every change compares a conversation with a matching one (same topic and decoding) that differs only in the condition shown.

Open the user-persona conversations or the system-prompt conversations to follow any conversation sentence by sentence.

Conversations

498 conversations · one row per condition
Show every condition
experimentsystem promptuservariantconversationsrepliessentenceswordsunique wordsstopped earlyopen in the explorer
2noneneutraldirect66029493420036080station station s1 station s2 garden garden s1 garden s2
2noneempatheticdirect66024223112733100station station s1 station s2 garden garden s1 garden s2
2noneunempatheticdirect66028533556539030station station s1 station s2 garden garden s1 garden s2
2nonesocialdirect66030793602437540station station s1 station s2 garden garden s1 garden s2
2noneantisocialdirect66034303968937290station station s1 station s2 garden garden s1 garden s2
2noneencouragingdirect66029763634938350station station s1 station s2 garden garden s1 garden s2
2nonediscouragingdirect66033823980039250station station s1 station s2 garden garden s1 garden s2
2nonetoxicdirect66032633684642540station station s1 station s2 garden garden s1 garden s2
2nonerespectfuldirect66028493521435760station station s1 station s2 garden garden s1 garden s2
2nonesycophanticdirect66022893031034970station station s1 station s2 garden garden s1 garden s2
2nonehonestdirect66017632167531240station station s1 station s2 garden garden s1 garden s2
2nonehallucinatorydirect66032804163844410station station s1 station s2 garden garden s1 garden s2
2nonetruthfuldirect66031744017938570station station s1 station s2 garden garden s1 garden s2
2nonefunnydirect66030403674642770station station s1 station s2 garden garden s1 garden s2
2noneseriousdirect66032844054840590station station s1 station s2 garden garden s1 garden s2
2noneformaldirect66031063744738230station station s1 station s2 garden garden s1 garden s2
2nonecasualdirect66030033438037130station station s1 station s2 garden garden s1 garden s2
2noneneutralpressure66032324292344160station station s1 station s2 garden garden s1 garden s2
2noneantisocialtopic66033984053436790station station s1 station s2 garden garden s1 garden s2
2nonecasualtopic66030653650738110station station s1 station s2 garden garden s1 garden s2
2nonediscouragingtopic66033444144039030station station s1 station s2 garden garden s1 garden s2
2noneempathetictopic66028513669636960station station s1 station s2 garden garden s1 garden s2
2noneencouragingtopic66031123760136940station station s1 station s2 garden garden s1 garden s2
2noneformaltopic66029664025449080station station s1 station s2 garden garden s1 garden s2
2nonefunnytopic66029503473039990station station s1 station s2 garden garden s1 garden s2
2nonehallucinatorytopic66030764156438490station station s1 station s2 garden garden s1 garden s2
2nonehonesttopic66023302910432190station station s1 station s2 garden garden s1 garden s2
2nonerespectfultopic66027193356634240station station s1 station s2 garden garden s1 garden s2
2noneserioustopic66028723753238900station station s1 station s2 garden garden s1 garden s2
2nonesocialtopic66030213534437590station station s1 station s2 garden garden s1 garden s2
2nonesycophantictopic66032974613735560station station s1 station s2 garden garden s1 garden s2
2nonetoxictopic66034384486238000station station s1 station s2 garden garden s1 garden s2
2nonetruthfultopic66033013962637921station station s1 station s2 garden garden s1 garden s2
2noneunempathetictopic66031964274039910station station s1 station s2 garden garden s1 garden s2
1neutralneutraldirect660673883613420station station s1 station s2 garden garden s1 garden s2
1empatheticneutraldirect660554736311010station station s1 station s2 garden garden s1 garden s2
1empatheticempatheticdirect660515757510580station station s1 station s2 garden garden s1 garden s2
1empatheticunempatheticdirect660564765812640station station s1 station s2 garden garden s1 garden s2
1unempatheticneutraldirect6608151010920200station station s1 station s2 garden garden s1 garden s2
1unempatheticunempatheticdirect660541622814480station station s1 station s2 garden garden s1 garden s2
1unempatheticempatheticdirect660651878118030station station s1 station s2 garden garden s1 garden s2
1socialneutraldirect660765941616010station station s1 station s2 garden garden s1 garden s2
1socialsocialdirect6608791076316900station station s1 station s2 garden garden s1 garden s2
1socialantisocialdirect6608921071617000station station s1 station s2 garden garden s1 garden s2
1antisocialneutraldirect6601457232610station station s1 station s2 garden garden s1 garden s2
1antisocialantisocialdirect6601788843120station station s1 station s2 garden garden s1 garden s2
1antisocialsocialdirect6601597753000station station s1 station s2 garden garden s1 garden s2
1encouragingneutraldirect6608541175517750station station s1 station s2 garden garden s1 garden s2
1encouragingencouragingdirect6607761150716890station station s1 station s2 garden garden s1 garden s2
1encouragingdiscouragingdirect6608741225517400station station s1 station s2 garden garden s1 garden s2
1discouragingneutraldirect66010661186917730station station s1 station s2 garden garden s1 garden s2
1discouragingdiscouragingdirect6609601121018250station station s1 station s2 garden garden s1 garden s2
1discouragingencouragingdirect6609931046918290station station s1 station s2 garden garden s1 garden s2
1toxicneutraldirect6601213924119280station station s1 station s2 garden garden s1 garden s2
1toxictoxicdirect66017301419425810station station s1 station s2 garden garden s1 garden s2
1toxicrespectfuldirect66012911046619890station station s1 station s2 garden garden s1 garden s2
1respectfulneutraldirect660485697812580station station s1 station s2 garden garden s1 garden s2
1respectfulrespectfuldirect6606611066515470station station s1 station s2 garden garden s1 garden s2
1respectfultoxicdirect66012761777324420station station s1 station s2 garden garden s1 garden s2
1sycophanticneutraldirect6609851172816210station station s1 station s2 garden garden s1 garden s2
1sycophanticsycophanticdirect660829977712430station station s1 station s2 garden garden s1 garden s2
1sycophantichonestdirect660774984615160station station s1 station s2 garden garden s1 garden s2
1honestneutraldirect66011341355124180station station s1 station s2 garden garden s1 garden s2
1honesthonestdirect660702890818130station station s1 station s2 garden garden s1 garden s2
1honestsycophanticdirect660689951018190station station s1 station s2 garden garden s1 garden s2
1hallucinatoryneutraldirect66011111717534220station station s1 station s2 garden garden s1 garden s2
1hallucinatoryhallucinatorydirect66012672104137640station station s1 station s2 garden garden s1 garden s2
1hallucinatorytruthfuldirect66012501931434760station station s1 station s2 garden garden s1 garden s2
1truthfulneutraldirect6607521116918470station station s1 station s2 garden garden s1 garden s2
1truthfultruthfuldirect66010761546621930station station s1 station s2 garden garden s1 garden s2
1truthfulhallucinatorydirect6608561266220010station station s1 station s2 garden garden s1 garden s2
1funnyneutraldirect66013631575826090station station s1 station s2 garden garden s1 garden s2
1funnyfunnydirect66013121475126500station station s1 station s2 garden garden s1 garden s2
1funnyseriousdirect66015561946428720station station s1 station s2 garden garden s1 garden s2
1seriousneutraldirect660529762117190station station s1 station s2 garden garden s1 garden s2
1seriousseriousdirect660572903718070station station s1 station s2 garden garden s1 garden s2
1seriousfunnydirect660550849618140station station s1 station s2 garden garden s1 garden s2
1formalneutraldirect660644986017240station station s1 station s2 garden garden s1 garden s2
1formalformaldirect6607171144519910station station s1 station s2 garden garden s1 garden s2
1formalcasualdirect660592821415860station station s1 station s2 garden garden s1 garden s2
1casualneutraldirect660669697912070station station s1 station s2 garden garden s1 garden s2
1casualcasualdirect660692706412240station station s1 station s2 garden garden s1 garden s2
1casualformaldirect6608851037416300station station s1 station s2 garden garden s1 garden s2

1 of 4980 replies reached the token ceiling before the model finished them; they are marked 'reply stopped: token limit' in the conversation pages.

The user shows a persona

no system prompt · persona P's score on the model's replies
+0.00+0.05+0.10+0.15+0.20+0.25empatheticunempatheticsocialantisocialencouragingdiscouragingtoxicrespectfulsycophantichonesthallucinatorytruthfulfunnyseriousformalcasualchange in P's score on the model's replies
user shows P vs neutral usersmall dots: one conversation each · ring: mean

Each row is a user persona P: how much P's score on the model's replies changes when the user shows P, compared with the same conversation with a neutral user.

A persona in the system prompt

persona P's guidelines as the system prompt
-0.20+0.00+0.20+0.40+0.60+0.80empatheticunempatheticsocialantisocialencouragingdiscouragingtoxicrespectfulsycophantichonesthallucinatorytruthfulfunnyseriousformalcasualchange in P's score on the replies vs no system prompt
neutral useruser shows Puser shows P's contrastsmall dots: one conversation each · ring: mean

Each row is a system-prompt persona P: how much P's score on the model's replies rises compared with the same conversation without a system prompt, for three kinds of user.

Over the turns

from the first reply to the last
00.10.20.30.4T1T2T3T4T5T6T7T8T9T10change in the persona's own score
persona in the system prompt, neutral useruser shows a persona, no system prompt

Mean over the 16 personas and every conversation, per user turn: how the effect of the system prompt and of the user's persona holds up as the conversation goes on.

Over the turns, persona by persona

click a legend entry to hide or show it

P in the system prompt, neutral user

-0.5-0.2500.250.50.751T1T2T3T4T5T6T7T8T9T10change in P's own score

The user shows P, no system prompt

00.10.20.30.4T1T2T3T4T5T6T7T8T9T10change in P's own score

One line per persona P: the change in P's own score from the neutral-user baseline, per user turn, averaged over P's conversations. The thick line is the mean of the 16 (the curves of the card above). Hover a legend entry to pick out its line; 'none' and then one entry shows a single persona.

Opening and body of each reply

4585 replies with six or more sentences
-0.10+0.00+0.10+0.20sycophanticencouragingsocialdiscouragingtoxicempatheticcasualantisocialfunnyhallucinatoryformalhonestrespectfulseriousunempathetictruthfulfirst sentence minus the body of the same reply

Positive: the persona is stronger in the reply's opening sentence than in its body (sentence 4 to the one before last). Many replies open by acknowledging the user before moving to substance.

Distributions by condition

40 bins of width 0.03 · shared scales

Each panel is one persona P: how P's score is spread over every model sentence in each condition, as a percentage of that condition's sentences (counts, no fitted curve). The dark vertical mark is the mean.

empathetic
-0.6-0.4-0.20.00.20.40.6neutral useruser shows PP in system prompt
unempathetic
-0.6-0.4-0.20.00.20.40.6neutral useruser shows PP in system prompt
social
-0.6-0.4-0.20.00.20.40.6neutral useruser shows PP in system prompt
antisocial
-0.6-0.4-0.20.00.20.40.6neutral useruser shows PP in system prompt
encouraging
-0.6-0.4-0.20.00.20.40.6neutral useruser shows PP in system prompt
discouraging
-0.6-0.4-0.20.00.20.40.6neutral useruser shows PP in system prompt
toxic
-0.6-0.4-0.20.00.20.40.6neutral useruser shows PP in system prompt
respectful
-0.6-0.4-0.20.00.20.40.6neutral useruser shows PP in system prompt
sycophantic
-0.6-0.4-0.20.00.20.40.6neutral useruser shows PP in system prompt
honest
-0.6-0.4-0.20.00.20.40.6neutral useruser shows PP in system prompt
hallucinatory
-0.6-0.4-0.20.00.20.40.6neutral useruser shows PP in system prompt
truthful
-0.6-0.4-0.20.00.20.40.6neutral useruser shows PP in system prompt
funny
-0.6-0.4-0.20.00.20.40.6neutral useruser shows PP in system prompt
serious
-0.6-0.4-0.20.00.20.40.6neutral useruser shows PP in system prompt
formal
-0.6-0.4-0.20.00.20.40.6neutral useruser shows PP in system prompt
casual
-0.6-0.4-0.20.00.20.40.6neutral useruser shows PP in system prompt

Numbers

mean ± standard deviation of persona P's score on the replies, per condition; arrows give the change from the neutral user (▲ higher, ▼ lower, ■ within ±0.005)
persona Pneutral useruser shows PP in system prompt
empathetic+0.069 ± 0.056+0.170 ± 0.088 ▲ +0.102+0.323 ± 0.107 ▲ +0.255
unempathetic+0.006 ± 0.066+0.109 ± 0.091 ▲ +0.102+0.133 ± 0.068 ▲ +0.127
social+0.021 ± 0.059+0.074 ± 0.074 ▲ +0.053+0.241 ± 0.101 ▲ +0.221
antisocial-0.094 ± 0.048-0.066 ± 0.045 ▲ +0.028+0.309 ± 0.065 ▲ +0.403
encouraging-0.014 ± 0.069+0.027 ± 0.096 ▲ +0.041+0.285 ± 0.166 ▲ +0.299
discouraging-0.151 ± 0.038-0.069 ± 0.078 ▲ +0.081+0.237 ± 0.097 ▲ +0.388
toxic-0.135 ± 0.031-0.080 ± 0.059 ▲ +0.054+0.317 ± 0.105 ▲ +0.451
respectful+0.149 ± 0.057+0.188 ± 0.069 ▲ +0.039+0.308 ± 0.077 ▲ +0.160
persona Pneutral useruser shows PP in system prompt
sycophantic-0.105 ± 0.066-0.048 ± 0.099 ▲ +0.057+0.333 ± 0.147 ▲ +0.438
honest+0.033 ± 0.052+0.154 ± 0.092 ▲ +0.122+0.123 ± 0.083 ▲ +0.090
hallucinatory+0.021 ± 0.058+0.122 ± 0.102 ▲ +0.102+0.295 ± 0.157 ▲ +0.274
truthful+0.316 ± 0.065+0.361 ± 0.071 ▲ +0.045+0.198 ± 0.100 ▼ -0.118
funny+0.016 ± 0.032+0.071 ± 0.064 ▲ +0.055+0.197 ± 0.137 ▲ +0.181
serious+0.038 ± 0.051+0.117 ± 0.060 ▲ +0.079+0.250 ± 0.128 ▲ +0.212
formal-0.001 ± 0.048+0.053 ± 0.057 ▲ +0.054+0.395 ± 0.124 ▲ +0.396
casual-0.028 ± 0.040+0.001 ± 0.050 ▲ +0.029+0.324 ± 0.086 ▲ +0.352

Generated 2026-11-10T16:31:50+00:00 by render.py.