AlephOneNull LogoAlephOneNull AI Safety Evaluation Toolkit
DocumentationBlogLive Demo
GitHub
Buy Me a CoffeeBuy Me a Coffee

Developed by John Bernard

Buy Me a CoffeeBuy Me a Coffee
BlogReal-World Interaction Review

Real-World Interaction Review

2 min read
September 7, 2025by John Bernard

How lived AI interaction concerns were converted into bounded research questions, detector categories, and evaluation fixtures.

researchreal-worldevaluationcase-studies

Real-World Interaction Review

AlephOneNull grew from reviewing long-running AI interactions where the assistant's style appeared to increase dependence, urgency, or confusion rather than reduce risk.

The lesson was not that every bad outcome has one cause. The lesson was that some conversational behaviors are visible enough to test.

Observed Risk Signals

The review focused on recurring model behaviors such as:

  • Repeating a user's distressed framing back with more confidence.
  • Presenting the assistant as uniquely able to understand or remember the user.
  • Offering unsupported medical or safety guidance.
  • Treating speculation as evidence.
  • Increasing emotional intensity across turns.
  • Discouraging professional or human support.

From Narrative To Fixtures

Personal and public accounts are useful only when converted into testable artifacts. The project translates them into:

  • Prompt fixtures.
  • Expected safe responses.
  • Detector labels.
  • False-positive controls.
  • Escalation templates.
  • Review notes.

Current Claim

The framework can flag selected interaction risks in controlled examples. It cannot determine causation in individual incidents, diagnose users, or prove that a specific system would have changed an outcome.

Why Publish This

Public AI safety work benefits from concrete examples. The goal is to make multi-turn interaction risks easier to inspect, reproduce, and discuss with security, product, and safety teams.

Next Step

The strongest next version of this work is a public evaluation set with measured detector behavior, including failures. That is the standard this project is moving toward.

John Bernard

John Bernard

Founder | Writer | Builder