Mistral Reliability Lab

The problem

A model API quickstart proves that a request can return text. It does not prove that tool execution is bounded, failures are handled deliberately, model behavior is testable, or dependency risk is visible.

What I built

Mistral Reliability Lab is a modular Python reference with a reviewer-first Streamlit interface, a FastAPI surface, opt-in OpenTelemetry tracing, and support for both Mistral’s API and local Ollama models. Its credential-free preview lets a reviewer inspect the interaction model without supplying an API key.

The model boundary centralizes calls and implements bounded retry behavior for transient failures. Tool execution uses an explicit allow-list and bounded loops. The interface exposes tool names, arguments, results, and multi-turn history while operational logs and traces exclude prompt, response, and tool-result content by default.

What it demonstrates

• Provider choice can stay behind a small, explicit interface.
• Tool access and reasoning loops should be bounded before a model can act.
• Credential-free tests can exercise routing, grounding, tool use, and failure handling.
• Observability can remain useful without collecting model content by default.
• CI can audit the exact locked runtime dependency set instead of an approximate environment.

Proof

• Explore the public repository
• 26 credential-free automated tests
• 6 versioned deterministic evaluation cases covering routing, tool selection, and grounding
• Locked dependency audit in CI with fail-closed collection behavior