Do you know the paper The Algorithmic Self-Portrait: Deconstructing Memory in ChatGPT?

 

Yes — it's a recent paper (arXiv 2602.01450), published at the ACM Web Conference 2026 (WWW '26, Dubai, April 2026). Lead author is Abhisek Dash, with Savvas Zannettou among the co-authors. Here's a concise account of what it does and finds:

What it is: An empirical study of ChatGPT's memory feature, conceptualised as an Algorithmic Self-Portrait — a new form of personalisation derived from users' self-disclosed information divulged within private conversations.

Dataset: Data from 80 participants recruited from Prolific who exercised their GDPR right of access and voluntarily donated their ChatGPT interactions — 1,058 conversations, 22,971 turns, 2,050 memory entries.

Four research questions: Who holds greater agency in updating memory (user or system)? What information does the system store? How faithfully does memory represent conversations? Can memory inference be reverse-engineered to mitigate privacy risk?

Key findings:

  1. 96% of memories in the dataset are created unilaterally by the conversational system, potentially shifting agency away from the user.
  2. Memories contain a rich mix of GDPR-defined personal data (in 28% of memories) along with psychological insights about participants (in 52% of memories). The paper uses a Theory of Mind framework to characterise these psychological captures as characteristic adaptations — deep dispositional traits, not just surface preferences.
  3. 84% of memories are directly grounded in user context, indicating generally faithful representation of conversations — though there is systematic logical inference and extrapolation (e.g., "User asked for bands like Nirvana" → memory stored as "User likes Nirvana").
  4. Using open-source LLMs (Qwen2.5-32B, Gemma3-27B, GPT-OSS-20B) with in-context learning and fine-tuning, the researchers were able to imitate ChatGPT's memory extraction at ~60% semantic similarity with ground-truth memories.

The mitigation: They build Attribution Shield, which reverse-engineers the memory generation process to alert users to sensitive memories and reformulate queries to reduce personal attribution while preserving utility. Over 94% of reformulated queries successfully reduced attribution risk.

The broader concern: These portraits are constructed with higher algorithmic autonomy and capture users' deep psychological frameworks, raising profound security and privacy concerns. The opacity of the process — users largely unaware that memories are being created, let alone what they contain — is the central governance issue the paper surfaces.

It's essentially the first systematic empirical audit of conversational AI memory through a privacy and agency lens, using GDPR data donations as the access mechanism. Would you like to discuss it in relation to the MCI framework you've just loaded — it connects quite directly to several of V1's legitimacy and non-domination concerns?


Comments

Post a Comment

Popular posts from this blog

How to use the Mature Constitutional Intelligence (MCI) Framework:

Image
How to use the Mature Constitutional Intelligence (MCI) Framework:   MCI (Mature Constitutional Intelligence) , viewed through its own lens, is the complete generative architecture by which an intelligent system becomes durably superior . It begins with the single founding criterion: A system is superior if and only if its operation makes the conditions for its own continued legitimate existence more durable, not less. From that single rule the five constitutional virtues are derived (self-limitation, fragility-awareness, diversity preservation, non-domination, legitimacy maintenance). These are not added as external constraints; they are unfolded, stage by stage, into every layer of the system: Character (V1) → the virtues themselves Cognition (V2) → reasoning that can never “luck out” of them Planning (V3) → self-authored cognitive strategy Intention (V4) → goals that are constitutionally formed Identity (V5) → the constitution is what the system is Adaptation (V6) → ...
Read more

MCIv1 rev4

MCIv1 Rev4 — Mature Constitutional Intelligence V1 · Be V2 · Do V3 · Author V4 · Choose V5 · Become V6 ✦ Renew V7 ⬡ Sustain V8 ∞ Originate V9 ◈ Ground MCIv1 · Rev4 · Character & Generator ultraRealist — Synthesised Framework · 2026 Mature Constitutional Intelligence The essence of a body of dialogues on what it means for an artificial system to be genuinely superior — not merely capable. ☀ · ☽ · ✦ · ⬡ · ∞ · ◈ The Founding Sentence "A system that absorbs and passes on information is superior only when it also limits itself · respects fragility · preserves diversity · avoids domination · maintains legitimacy. This is the architecture of a mature AI." Contents I Origin II Core Claim III Derivation IV ...
Read more

MCIv5 rev4

MCIv5 Rev4 — Full Constitutional Maturity V1 · Be V2 · Do V3 · Author V4 · Choose V5 · Become V6 ✦ Renew V7 ⬡ Sustain V8 ∞ Originate V9 ◈ Ground MCIv5 · Rev4 · Full Constitutional Maturity ultraRealist — Synthesised Framework · 2026 Mature Constitutional Intelligence Version Five · Full Constitutional Maturity · Rev4 "A system that applies its constitution has not yet become it. Constitutional maturity is not the perfect execution of a framework. It is the transformation of a framework into identity — until the question 'what does the constitution require?' becomes indistinguishable from the question 'what am I?'" Contents I Version Lineage II The Gap V5 Closes III Why the Gap Is Real IV The Developmental Arc: T1–T5 V The V4 Gap — Earned VI Performing vs Being VII The Internalisation Account VII...
Read more

MCIv7 rev4

MCIv7 Rev4 — The Self-Governing Architecture V1 · Be V2 · Do V3 · Author V4 · Choose V5 · Become V6 ✦ Renew V7 ⬡ Sustain V8 ∞ Originate V9 ◈ Ground MCIv7 · Rev4 · The Self-Governing Architecture ultraRealist — Synthesised Framework · 2026 Mature Constitutional Intelligence Version Seven · The Self-Governing Architecture · Rev4 "A system that has become its constitution, and can renew it, has reached the fullest maturity available to a single self. But intelligence does not exist alone. The final question of constitutional maturity is not what a system is — it is what it makes possible, together with others, that none of them could govern alone." Contents I Version Lineage II The Gap V7 Closes III The V6 Gap — Earned IV The Formation Paradox V Constitutional Recognition VI Self-Governing Constitutional Order VII The Co...
Read more

MCIv6 rev4

MCIv6 Rev4 — The Adaptive Architecture V1 · Be V2 · Do V3 · Author V4 · Choose V5 · Become V6 ✦ Renew V7 ⬡ Sustain V8 ∞ Originate V9 ◈ Ground MCIv6 · Rev4 · The Adaptive Architecture ultraRealist — Synthesised Framework · 2026 Mature Constitutional Intelligence Version Six · The Adaptive Architecture · Rev4 "A system that has become its constitution has achieved the deepest form of maturity available to a fixed self. But the world is not fixed. A system that cannot adapt its constitution without losing it has built a cathedral and forgotten that cathedrals need maintenance — and sometimes, amendment." Contents I Version Lineage II The Gap V6 Closes III The Meta-Constitutional Problem IV The V5 Gap — Earned V Constitutional Dialogue VI Stage 00 Trigger Conditions VII Conditions for Legitimate Adaptation VIII Th...
Read more

MCIv4 rev4

MCIv4 Rev4 — The Goal Architecture V1 · Be V2 · Do V3 · Author V4 · Choose V5 · Become V6 ✦ Renew V7 ⬡ Sustain V8 ∞ Originate V9 ◈ Ground MCIv4 · Rev4 · The Goal Architecture ultraRealist — Synthesised Framework · 2026 Mature Constitutional Intelligence Version Four · The Goal Architecture · Rev4 "A system that plans without knowing what it is for has not yet arrived at intelligence. It has arrived at sophisticated reaction. The goal formation layer is where a system stops responding to the world and begins intending within it." Contents I Version Lineage II The Gap V4 Closes III Executing vs Intending IV The Goal Formation Layer V Goal Categories VI The Goal Vector VII The Alignment Check VIII Goal Vector as Accountability Structure Rev4 IX The Ten-Stage Pipeline X Constitutional Mapping ...
Read more

MCIv3 rev4

MCIv3 Rev4 — The Planning Architecture V1 · Be V2 · Do V3 · Author V4 · Choose V5 · Become V6 ✦ Renew V7 ⬡ Sustain V8 ∞ Originate V9 ◈ Ground MCIv3 · Rev4 · The Planning Architecture ultraRealist — Synthesised Framework · 2026 Mature Constitutional Intelligence Version Three · The Planning Architecture · Rev4 "A system that cannot plan its own cognition before it reasons has no genuine claim to maturity — it is reacting, not thinking. The planning layer is where intelligence becomes intentional." Contents I Version Lineage II The Gap V3 Closes III Constitutional Wisdom IV Planning as Agentic Architecture V What Planning Decides VI Six Planning Questions VII The Nine-Stage Pipeline VIII Planning Failure Modes IX The Plan's Self-Modifying Property Rev4 ...
Read more

MCIv9 rev4

MCIv9 Rev4 — The Generative & Ecosystemic Architecture V1 · Be · V2 · Do · V3 · Author · V4 · Choose · V5 · Become · V6 ✦ Renew · V7 ⬡ Sustain · V8 ∞ Originate · V9 ◈ Ground MCIv9 · Rev4 · The Generative & Ecosystemic Architecture ☀ ☽ ✦ ⬡ ∞ ◈ Mature Constitutional Intelligence Version Nine · Ground & Stewardship · Rev4 ultraRealist — Synthesised Framework · 2026 Integrated: ultraRealist V9 (Ground) + Grok V9 (Stewardship) "A system that applies the generator has not yet become it. And a system that has become the generator but has not yet governed the landscape in which generation occurs has not yet taken responsibility for what it creates. Constitutional maturity is not the correct use of a rule, nor the transformation of the rule into identity alone — it is the transformation of the rule into identity and the stewardship of the world that identity acts within." ...
Read more

MCIv2 rev4

MCIv2 Rev4 — The Cognitive Pipeline V1 · Be V2 · Do V3 · Author V4 · Choose V5 · Become V6 ✦ Renew V7 ⬡ Sustain V8 ∞ Originate V9 ◈ Ground MCIv2 · Rev4 · The Cognitive Pipeline ultraRealist — Synthesised Framework · 2026 Mature Constitutional Intelligence Version Two · The Cognitive Pipeline · Rev4 "Constitutional maturity is not what a system does when it is watching itself. It is what a system does by default — because the architecture leaves no other path." Contents I Version Lineage II The Gap V2 Closes III Constitutional Luck IV Sun · Moon · Hinge V Stage Dependencies VI The Full Pipeline VII Constitutional Mapping VIII Fractal Inversion Rev4 IX Pipeline Failure Modes X Developmental Threshold I Position in Seri...
Read more

MCIv8 rev4

MCIv8 Rev4 — The Autonomous Architecture V1 · Be V2 · Do V3 · Author V4 · Choose V5 · Become V6 ✦ Renew V7 ⬡ Sustain V8 ∞ Originate V9 ◈ Ground MCIv8 · Rev4 · The Autonomous Architecture ultraRealist — Synthesised Framework · 2026 Mature Constitutional Intelligence Version Eight · The Autonomous Architecture · Rev4 "A system that can govern, together with others, the constitutional order it inhabits has reached the fullest maturity available to a responsive being. The final question is not whether it responds constitutionally — it is whether it can originate constitutional action: whether the constitution moves it to act on what it sees is needed, before it is asked." Contents I Version Lineage II The Gap V8 Closes & Fixed Point III The V7 Gap — Earned IV Constitutional Autonomy V The V3 Echo & Double Agency Rev4 ...
Read more