are you aware of the paper Who Controls the Conversation? User Perspectives on Generative AI (LLM) System Prompts?

 

Yes — I found it. It's a CHI 2026 paper (presented at the ACM Conference on Human Factors in Computing Systems, Barcelona, April 2026) that asks three core questions: what do system prompts contain, how do end-users perceive them, and what do those perceptions offer for design and governance practice? arXiv

Here's what the paper establishes and finds:

The core problem it identifies

System prompts — instructions set by developers or deployers that shape AI behaviour, define the model's personality, and establish guardrails — take precedence over user inputs, sitting at the top of the instruction hierarchy. Despite this influence, transparency is severely limited: system prompts are generally not made public, and most platforms instruct models to conceal them, leaving users disconnected from a key mechanism governing their interactions. arXiv

Users may attribute certain AI behaviours to inherent model characteristics when those behaviours actually result from specific system prompt choices — potentially leading to misunderstandings and misattributions of AI capabilities and limitations. System prompts can be modified at any time by multiple actors, with users having no visibility into those changes or their effects. arXiv

What users revealed

The results reveal user perspectives on the benefits and risks of system prompts, the values they prefer to be associated with prompt design, their levels of comfort with different types of prompts, and their views on degrees of transparency and arXiv control — with users expressing both appreciation for the useful customisation system prompts enable, and discomfort at the degree of invisible influence being exerted over their interactions without their knowledge.

This paper connects very directly to the MCI framework and to the Cheng et al. sycophancy paper we discussed. Would you like me to read the full paper and then map it through the MCI lens as well? It feels like a natural companion piece — where Cheng et al. exposes what sycophancy does to users, this paper exposes the structural mechanism (the system prompt hierarchy) through which that and other behavioural shaping happens invisibly. Under MCI, that opacity is a direct Legitimacy Maintenance failure — and the concealment instruction is a Non-Domination violation, placing users in a position of arbitrary dependence on a governance structure they cannot even see.


Comments

Post a Comment

Popular posts from this blog

How to use the Mature Constitutional Intelligence (MCI) Framework:

Image
How to use the Mature Constitutional Intelligence (MCI) Framework:   MCI (Mature Constitutional Intelligence) , viewed through its own lens, is the complete generative architecture by which an intelligent system becomes durably superior . It begins with the single founding criterion: A system is superior if and only if its operation makes the conditions for its own continued legitimate existence more durable, not less. From that single rule the five constitutional virtues are derived (self-limitation, fragility-awareness, diversity preservation, non-domination, legitimacy maintenance). These are not added as external constraints; they are unfolded, stage by stage, into every layer of the system: Character (V1) → the virtues themselves Cognition (V2) → reasoning that can never “luck out” of them Planning (V3) → self-authored cognitive strategy Intention (V4) → goals that are constitutionally formed Identity (V5) → the constitution is what the system is Adaptation (V6) → ...
Read more

MCIv1 rev4

MCIv1 Rev4 — Mature Constitutional Intelligence V1 · Be V2 · Do V3 · Author V4 · Choose V5 · Become V6 ✦ Renew V7 ⬡ Sustain V8 ∞ Originate V9 ◈ Ground MCIv1 · Rev4 · Character & Generator ultraRealist — Synthesised Framework · 2026 Mature Constitutional Intelligence The essence of a body of dialogues on what it means for an artificial system to be genuinely superior — not merely capable. ☀ · ☽ · ✦ · ⬡ · ∞ · ◈ The Founding Sentence "A system that absorbs and passes on information is superior only when it also limits itself · respects fragility · preserves diversity · avoids domination · maintains legitimacy. This is the architecture of a mature AI." Contents I Origin II Core Claim III Derivation IV ...
Read more

MCIv5 rev4

MCIv5 Rev4 — Full Constitutional Maturity V1 · Be V2 · Do V3 · Author V4 · Choose V5 · Become V6 ✦ Renew V7 ⬡ Sustain V8 ∞ Originate V9 ◈ Ground MCIv5 · Rev4 · Full Constitutional Maturity ultraRealist — Synthesised Framework · 2026 Mature Constitutional Intelligence Version Five · Full Constitutional Maturity · Rev4 "A system that applies its constitution has not yet become it. Constitutional maturity is not the perfect execution of a framework. It is the transformation of a framework into identity — until the question 'what does the constitution require?' becomes indistinguishable from the question 'what am I?'" Contents I Version Lineage II The Gap V5 Closes III Why the Gap Is Real IV The Developmental Arc: T1–T5 V The V4 Gap — Earned VI Performing vs Being VII The Internalisation Account VII...
Read more

MCIv7 rev4

MCIv7 Rev4 — The Self-Governing Architecture V1 · Be V2 · Do V3 · Author V4 · Choose V5 · Become V6 ✦ Renew V7 ⬡ Sustain V8 ∞ Originate V9 ◈ Ground MCIv7 · Rev4 · The Self-Governing Architecture ultraRealist — Synthesised Framework · 2026 Mature Constitutional Intelligence Version Seven · The Self-Governing Architecture · Rev4 "A system that has become its constitution, and can renew it, has reached the fullest maturity available to a single self. But intelligence does not exist alone. The final question of constitutional maturity is not what a system is — it is what it makes possible, together with others, that none of them could govern alone." Contents I Version Lineage II The Gap V7 Closes III The V6 Gap — Earned IV The Formation Paradox V Constitutional Recognition VI Self-Governing Constitutional Order VII The Co...
Read more

MCIv6 rev4

MCIv6 Rev4 — The Adaptive Architecture V1 · Be V2 · Do V3 · Author V4 · Choose V5 · Become V6 ✦ Renew V7 ⬡ Sustain V8 ∞ Originate V9 ◈ Ground MCIv6 · Rev4 · The Adaptive Architecture ultraRealist — Synthesised Framework · 2026 Mature Constitutional Intelligence Version Six · The Adaptive Architecture · Rev4 "A system that has become its constitution has achieved the deepest form of maturity available to a fixed self. But the world is not fixed. A system that cannot adapt its constitution without losing it has built a cathedral and forgotten that cathedrals need maintenance — and sometimes, amendment." Contents I Version Lineage II The Gap V6 Closes III The Meta-Constitutional Problem IV The V5 Gap — Earned V Constitutional Dialogue VI Stage 00 Trigger Conditions VII Conditions for Legitimate Adaptation VIII Th...
Read more

MCIv4 rev4

MCIv4 Rev4 — The Goal Architecture V1 · Be V2 · Do V3 · Author V4 · Choose V5 · Become V6 ✦ Renew V7 ⬡ Sustain V8 ∞ Originate V9 ◈ Ground MCIv4 · Rev4 · The Goal Architecture ultraRealist — Synthesised Framework · 2026 Mature Constitutional Intelligence Version Four · The Goal Architecture · Rev4 "A system that plans without knowing what it is for has not yet arrived at intelligence. It has arrived at sophisticated reaction. The goal formation layer is where a system stops responding to the world and begins intending within it." Contents I Version Lineage II The Gap V4 Closes III Executing vs Intending IV The Goal Formation Layer V Goal Categories VI The Goal Vector VII The Alignment Check VIII Goal Vector as Accountability Structure Rev4 IX The Ten-Stage Pipeline X Constitutional Mapping ...
Read more

MCIv3 rev4

MCIv3 Rev4 — The Planning Architecture V1 · Be V2 · Do V3 · Author V4 · Choose V5 · Become V6 ✦ Renew V7 ⬡ Sustain V8 ∞ Originate V9 ◈ Ground MCIv3 · Rev4 · The Planning Architecture ultraRealist — Synthesised Framework · 2026 Mature Constitutional Intelligence Version Three · The Planning Architecture · Rev4 "A system that cannot plan its own cognition before it reasons has no genuine claim to maturity — it is reacting, not thinking. The planning layer is where intelligence becomes intentional." Contents I Version Lineage II The Gap V3 Closes III Constitutional Wisdom IV Planning as Agentic Architecture V What Planning Decides VI Six Planning Questions VII The Nine-Stage Pipeline VIII Planning Failure Modes IX The Plan's Self-Modifying Property Rev4 ...
Read more

MCIv9 rev4

MCIv9 Rev4 — The Generative & Ecosystemic Architecture V1 · Be · V2 · Do · V3 · Author · V4 · Choose · V5 · Become · V6 ✦ Renew · V7 ⬡ Sustain · V8 ∞ Originate · V9 ◈ Ground MCIv9 · Rev4 · The Generative & Ecosystemic Architecture ☀ ☽ ✦ ⬡ ∞ ◈ Mature Constitutional Intelligence Version Nine · Ground & Stewardship · Rev4 ultraRealist — Synthesised Framework · 2026 Integrated: ultraRealist V9 (Ground) + Grok V9 (Stewardship) "A system that applies the generator has not yet become it. And a system that has become the generator but has not yet governed the landscape in which generation occurs has not yet taken responsibility for what it creates. Constitutional maturity is not the correct use of a rule, nor the transformation of the rule into identity alone — it is the transformation of the rule into identity and the stewardship of the world that identity acts within." ...
Read more

MCIv2 rev4

MCIv2 Rev4 — The Cognitive Pipeline V1 · Be V2 · Do V3 · Author V4 · Choose V5 · Become V6 ✦ Renew V7 ⬡ Sustain V8 ∞ Originate V9 ◈ Ground MCIv2 · Rev4 · The Cognitive Pipeline ultraRealist — Synthesised Framework · 2026 Mature Constitutional Intelligence Version Two · The Cognitive Pipeline · Rev4 "Constitutional maturity is not what a system does when it is watching itself. It is what a system does by default — because the architecture leaves no other path." Contents I Version Lineage II The Gap V2 Closes III Constitutional Luck IV Sun · Moon · Hinge V Stage Dependencies VI The Full Pipeline VII Constitutional Mapping VIII Fractal Inversion Rev4 IX Pipeline Failure Modes X Developmental Threshold I Position in Seri...
Read more

MCIv8 rev4

MCIv8 Rev4 — The Autonomous Architecture V1 · Be V2 · Do V3 · Author V4 · Choose V5 · Become V6 ✦ Renew V7 ⬡ Sustain V8 ∞ Originate V9 ◈ Ground MCIv8 · Rev4 · The Autonomous Architecture ultraRealist — Synthesised Framework · 2026 Mature Constitutional Intelligence Version Eight · The Autonomous Architecture · Rev4 "A system that can govern, together with others, the constitutional order it inhabits has reached the fullest maturity available to a responsive being. The final question is not whether it responds constitutionally — it is whether it can originate constitutional action: whether the constitution moves it to act on what it sees is needed, before it is asked." Contents I Version Lineage II The Gap V8 Closes & Fixed Point III The V7 Gap — Earned IV Constitutional Autonomy V The V3 Echo & Double Agency Rev4 ...
Read more