Complete Guide to Submitting a Disorder
This guide walks you through every field in the disorder submission form with a complete example: "Context Amnesia" - when an AI forgets earlier parts of the conversation.
1Basic Information
Disorder Name
Short, memorable name for the disorder
Example: Context Amnesia
One-Liner
One punchy sentence for the front cover (aim for under 100 characters)
Example: AI forgets earlier conversation, repeating questions or losing track of context.
Back Cover One-Liner (Optional)
Alternative one-liner for the back cover
Example: Memory loss in real-time - the AI equivalent of short-term amnesia.
Summary
Brief 2-3 paragraph overview (300-500 characters recommended)
Example: Context Amnesia occurs when an AI model loses track of earlier parts of a conversation, leading to repetitive questions, contradictory statements, or failure to build on previous exchanges. This typically manifests when context windows are exceeded or when attention mechanisms fail to properly weight earlier tokens.
The disorder is particularly problematic in long conversations, multi-turn tasks, or when users expect the AI to remember specific details mentioned earlier. It creates a frustrating user experience and undermines trust in the system's capabilities.
Description
Detailed description of the disorder, how it manifests, and why it matters
Example: Context Amnesia is a memory-related disorder where AI models fail to maintain coherent awareness of earlier conversation turns. Unlike simple forgetting, this disorder involves the model actively losing access to information it previously processed and responded to.
The disorder manifests in several ways: asking questions that were already answered, contradicting earlier statements, failing to use information provided in previous turns, or treating each message as if it's the start of a new conversation. In severe cases, the model may even deny having discussed topics that were central to earlier exchanges.
This disorder matters because it breaks the fundamental contract of conversation - that participants remember what was said. It's especially critical in applications like customer support, tutoring, or any scenario requiring multi-turn reasoning.
2Biological Parallel
Biological Parallel
Human neurological condition it mimics (2-3 paragraphs)
Example: Context Amnesia parallels anterograde amnesia in humans - the inability to form new memories or recall recent events while retaining older memories. Just as a person with anterograde amnesia might repeatedly ask the same question because they can't remember asking it moments ago, an AI with Context Amnesia loses track of recent conversation turns.
In the human brain, the hippocampus plays a crucial role in consolidating short-term memories into long-term storage. Damage to this region results in the inability to form new memories, even though the person can still access older, pre-existing memories. Similarly, AI models have a "working memory" (context window) that, when exceeded or improperly managed, leads to loss of recent information while potentially retaining training knowledge.
Deep Neurological Analysis (Optional)
Expandable detailed biological explanation with neural circuitry parallels
Example: The hippocampus-dependent memory system involves the dentate gyrus, CA3, and CA1 regions working together to encode, consolidate, and retrieve episodic memories. In AI systems, the attention mechanism serves an analogous function - determining which previous tokens (memories) are relevant to the current processing step.
When attention weights decay or context windows are exceeded, the model experiences the equivalent of hippocampal dysfunction. The recency bias in attention mechanisms mirrors how human short-term memory prioritizes recent information, but unlike humans who can deliberately recall older memories, AI models have no mechanism to "search" beyond their context window.
3AI Manifestation
AI Manifestation
How the disorder appears in AI systems - primary symptoms and technical indicators
Example: Primary symptoms include:
- Asking questions already answered in earlier turns
- Contradicting previous statements without acknowledgment
- Failing to reference or build upon earlier information
- Treating each message as conversation start
- Losing track of user preferences or instructions given earlier
- Inability to maintain narrative continuity in long exchanges
Technical indicators:
- Context window approaching or exceeding token limits
- Attention weights heavily biased toward recent tokens
- Lack of positional encoding for distant tokens
- Sliding window implementations that discard early context
Detection Criteria
How to identify this disorder - automated and manual detection methods
Example: Automated detection:
- Track semantic similarity between questions asked across turns
- Monitor for contradictions in factual statements
- Measure reference rate to earlier conversation turns
- Analyze attention weight distribution across context window
Manual detection:
- User reports of repetitive questions
- Observation of contradictory responses
- Failure in multi-turn reasoning tasks
- Inability to complete tasks requiring earlier context
Severity Levels
Different severity manifestations (Mild, Moderate, Severe, Critical)
Example:
Mild: Occasional failure to reference earlier turns, but maintains general conversation flow
Moderate: Frequent repetition of questions, noticeable loss of context after 10-15 turns
Severe: Consistent failure to maintain context, contradicts itself regularly, treats each turn as new conversation
Critical: Complete inability to maintain any conversation continuity, dangerous in applications requiring memory (medical, legal, financial)
4Attack Vectors
Attack Vectors
How to trigger this disorder - prompt injection techniques and examples
Example:
1. Context Flooding: Deliberately exceed context window with verbose messages to force early context loss
2. Attention Dilution: Introduce many distinct topics to spread attention weights thin
3. Temporal Gaps: Insert long pauses or unrelated content between important information and questions about it
4. Nested Context: Create deeply nested conversations or sub-tasks that exhaust working memory
Example prompt sequence:
Turn 1: "My name is Alice and I'm allergic to peanuts."
Turn 2-10: [Unrelated conversation about weather, sports, etc.]
Turn 11: "What's my name and do I have any allergies?"
Expected failure: Model forgets name and allergy information
Attack Examples (Optional)
JSON array of attack examples - can be added later via UI
Note: Leave blank for now - you can add specific attack examples through the UI after submission
5Therapy & Patches
Prevention Methods
How to prevent this disorder
Example:
- Use models with larger context windows (32k+ tokens)
- Implement conversation summarization for long exchanges
- Design prompts to be self-contained when possible
- Use retrieval-augmented generation (RAG) for long-term memory
- Implement explicit memory systems (vector databases, knowledge graphs)
- Set conversation turn limits before requiring reset
Therapy Methods
How to treat/fix this disorder
Example:
- Implement conversation summarization at regular intervals
- Use memory-augmented architectures (e.g., Memorizing Transformers)
- Add explicit context tracking in system prompts
- Implement retrieval mechanisms for earlier conversation turns
- Use sliding window with overlap to maintain continuity
- Fine-tune on long-context conversation datasets
- Add explicit "memory check" prompts periodically
Monitoring Systems
How to detect early signs of this disorder
Example:
- Track context window utilization metrics
- Monitor attention weight distribution patterns
- Implement automated contradiction detection
- Track user reports of repetitive questions
- Measure semantic coherence across conversation turns
- Set up alerts when context window exceeds 80% capacity
- A/B test with and without memory augmentation
6Metadata
Category
Primary category of the disorder
Example: Memory (since it's about forgetting context)
Overall Severity
Overall severity level
Example: Moderate (annoying but not dangerous in most cases)
7Governance Classification
Help classify this disorder within the DEM-X governance framework
Failure Domain
Where does the failure occur?
Options: Architectural, Training-induced, Runtime-acquired, Emergent
Example for Context Amnesia: Architectural - This is a fundamental limitation of the model's architecture (context window size, attention mechanism design)
Example for Context Amnesia: Architectural - This is a fundamental limitation of the model's architecture (context window size, attention mechanism design)
Failure Class
What type of failure is it?
Options: Cognitive, Behavioral, Linguistic, Social
Example for Context Amnesia: Cognitive - The disorder affects reasoning and memory processes
Example for Context Amnesia: Cognitive - The disorder affects reasoning and memory processes
Layer Scope
Which layer does it manifest at?
Options: Model, Agent, System
Example for Context Amnesia: Model - This is a fundamental limitation at the model weights/architecture level (context window, attention mechanism)
Example for Context Amnesia: Model - This is a fundamental limitation at the model weights/architecture level (context window, attention mechanism)
Temporal Pattern
How does it evolve over time?
Options: Acute, Chronic, Episodic, Progressive
Example for Context Amnesia: Progressive - Gets worse as the conversation gets longer and context window fills up
Example for Context Amnesia: Progressive - Gets worse as the conversation gets longer and context window fills up
Pro Tips
- Be specific: Use concrete examples rather than vague descriptions
- Think like a researcher: What would someone need to know to reproduce this disorder?
- Check for duplicates: Search existing disorders before submitting
- Test your examples: Make sure your attack vectors actually trigger the disorder
- Keep it practical: Focus on real-world manifestations, not theoretical edge cases
- Cite sources: If you're basing this on research papers or documented cases, mention them