Changelog

Major update introducing cross-provider Structured Outputs, a comprehensive visual analytics dashboard, and a robust artifact export system.

• Structured Outputs (JSON Schema): Enforce strict JSON responses across OpenAI, Anthropic, Mistral, and Gemini using their native provider-level implementations. Includes a new split-context editor with real-time schema validation.
• Visual Analytics Dashboard: Added a responsive chart suite for evaluation tasks to visualize Latency vs. Pass Rate, Accuracy Breakdown, Token Usage (Prompt vs. Completion), and raw engine Throughput (Tokens/Second).
• Artifact Export System: Export visualizations as presentation-ready PNGs and leaderboards as comprehensive CSVs. Features collision-resistant timestamped filenames and embedded metadata title cards.
• High-Density Leaderboard: Redesigned the evaluation data table to aggregate metrics more efficiently, now including a dedicated column for total token consumption per model.
• Robust CSV Parsing: Replaced the regex dataset parser with a character-state machine, safely allowing complex data like newlines and commas inside quoted JSON dataset strings.
• Model Capability Badges: Added visual tag indicators to the LLM selector UI, allowing immediate identification of supported features (e.g., Reasoning, Structured Outputs) per model.
• Editor Architecture: Rebuilt the checkpoint editor using a modular block-based text architecture to eliminate cursor tracking bugs and support optimistic state saving.

Initial public release. A local-first desktop workbench for testing and benchmarking LLMs.

• Prompt Explorer: Version control for prompts with support for dynamic {{variables}}.
• Providers Vault: Connect OpenAI, Anthropic, Mistral, Gemini, and xAI securely using local OS keyring encryption.
• LLM Task Explorer: Send prompts to multiple models concurrently and compare raw API outputs side-by-side.
• Eval Task Explorer: Upload CSV datasets to run bulk evaluations, utilize deterministic manual grading with diagnostic tags (e.g., Hallucination, Format Error), and generate performance leaderboards.
• Local Storage: All prompt history, configuration, and evaluation data is stored entirely in a local SQLite database.