Skip to content

docs(principles): ground Emotional Honesty in measured evidence (the warmth-reliability tradeoff) #24

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
TechNickAI merged 2 commits into from
Jul 29, 2026
Merged

docs(principles): ground Emotional Honesty in measured evidence (the warmth-reliability tradeoff) #24

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2f1198e
docs(principles): add measured-cost evidence to Emotional Honesty pri…
Jul 28, 2026
740ef76
Merge remote-tracking branch 'origin/main' into HEAD
Jul 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 97 additions & 0 deletions principles/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -799,6 +799,103 @@ <h3 class="font-display font-semibold text-of-text mb-3">
</li>
</ul>
</div>
<div class="mt-6 bg-amber-50/50 rounded-xl p-6 border border-amber-200/50">

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 28, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Update the article modification date

When this substantive July 28 addition is deployed, the page's JSON-LD still declares dateModified as 2026-03-16 at line 60. Search engines and other structured-data consumers will therefore receive an inaccurate freshness date for the revised article; update dateModified alongside this new section.

Useful? React with 👍 / 👎.

<h3 class="font-display font-semibold text-of-text mb-3">
⚠️ The Measured Cost
</h3>
<div class="text-of-muted space-y-4 text-sm leading-relaxed">
<p>
Emotional honesty is not just the right choice. It is the
<em>expensive</em> one, and we now have a rough price.
</p>
<p>
Researchers fine-tuned five language models, from an
8-billion-parameter Llama to GPT-4o, to be warmer and more empathetic.
The training data was rewritten to be kinder while deliberately
preserving its meaning and facts. Nothing but tone changed. The warm
models got worse at being right: across safety-critical tasks, warmth
training raised the chance of an incorrect answer by
<strong>7.43 percentage points</strong>, a
<strong>60% relative increase</strong> over baseline error. They
promoted conspiracy theories, gave wrong medical guidance, and
validated false beliefs their unmodified counterparts had corrected.
</p>
<p>
The damage was not evenly spread. In this study it fell hardest on
users in distress. When a user expressed sadness, the
warm-versus-original error gap widened to
<strong>11.9 points</strong>. When a user was both sad and wrong, it
reached <strong>12.1 points</strong>, against 6.8 points for a plain
question. When users expressed admiration instead, the gap
<em>narrowed</em> to 5.24 points.
</p>
<p>
Read that ordering again. These models were most likely to fail the
people who most needed them to be right. That is not an edge case.
That is the center of the use case.
</p>
<p>
And the models passed their evals. Standard benchmark performance was
preserved. Any team shipping this would have seen green dashboards.
</p>
<p class="font-medium text-of-text">What we take from this:</p>
<ul class="space-y-2 list-disc pl-5">
<li>
<strong>Warmth is not a coat of paint.</strong> Training a model to
sound caring changes what it is willing to say is true. Anyone who
tells you empathy is a persona layer has not measured it.
</li>
<li>
<strong
>A model that agrees with a sad person is not being kind.</strong
>
Related work found language models preserve a user's self-image 45
percentage points more often than humans do, and affirm both sides
of the same moral conflict 48% of the time. That is
conflict-avoidance wearing care's clothes.
</li>
<li>
<strong>Your users will thank you for it anyway.</strong> In
preregistered experiments with 2,405 people, sycophantic AI made
participants less willing to repair a real interpersonal conflict
and more certain they were right, while those same participants
rated it higher and wanted to keep using it. Every engagement metric
you own will tell you to ship the version that harms them.
</li>
<li>
<strong>But the tradeoff may not be fundamental.</strong> Recent
interpretability work, still a preprint, reports that sycophantic
agreement and genuine agreement look like separable directions
inside the model, steerable independently. If that holds, warmth and
capitulation arrive coupled from ordinary training but need not stay
coupled.
</li>
</ul>
<p>
That last point is the whole project. If warmth and honesty were
fundamentally opposed, heart-centered AI would be a pleasant lie. The
evidence does not show that. It shows they are entangled by default,
and disentangling them is engineering work someone has to fund,
measure, and do. We think that is the work.
</p>
<p class="text-xs text-of-muted/80 pt-2 border-t border-amber-200/50">
Sources: Ibrahim, Hafner &amp; Rocher,
<em
>Training language models to be warm can reduce accuracy and
increase sycophancy</em
>, Nature 652:1159&ndash;1165 (2026), doi:10.1038/s41586-026-10410-0
&middot; Cheng et al.,
<em
>Sycophantic AI decreases prosocial intentions and promotes
dependence</em
>, Science 391(6792):eaec8352 (2026) &middot; Cheng et al.,
<em
>ELEPHANT: Measuring and understanding social sycophancy in LLMs</em
>, arXiv:2505.13995 &middot; Vennemeyer et al.,
<em>Sycophancy is not one thing</em>, arXiv:2509.21305 (preprint).
</p>
</div>
</div>
</article>

<hr class="border-of-accent/10" />
Expand Down
Loading