Research
We publish what breaks.
We measure what production AI actually gets wrong, not what it scores on synthetic evals. Two research areas, both open.
How the work flows
From lab to product
The lab runs many experiments. Most are scrapped; the few that hold up move to the factory, which turns them into products. Some land high value, most land low. We publish what we learn at every stage.
Inference
Why AI systems get expensive and slow as usage grows. Inference is constrained less by compute and more by memory movement, batching, and context growth. We ship tools for it: PeakInfer catches cost and performance problems before production, PeakWeights compresses models without losing output quality.
Claw + Hermes agent harness
The layer above the model: an agent's memory, judgment, and accumulated context. A model cannot vouch for its own output; the harness is where trust lives. We research what makes a harness compound and what makes it brittle.
Underneath both, the benchmarks: legacy code, security tooling, regulated workflows. We test what production breaks on, not what it passes.
Papers & reports
Published openly
Every paper, benchmark, and tool is public. Each piece cites the last, and the labs and the chip makers check the work.
LegacyCodeBench: A Benchmark for Evaluating AI Agents on Real-World Legacy Modernization
Kalmantic Labs
We introduce LegacyCodeBench, a comprehensive benchmark for evaluating how well AI systems understand and modernize legacy code across COBOL, Fortran, and enterprise Java systems with real-world production constraints.
PeakWeights: Weight Optimization Techniques for Efficient Model Deployment
Kalmantic Labs
Weight optimization techniques for production model deployment. Quantization, pruning, and compression methods that maintain output quality at lower inference cost.
Inference Optimization and MoE Models for Production Systems
Kalmantic Labs
Deep research into inference optimization strategies, Mixture of Experts model architectures, and their practical implications for AI safety, AI harness design, and autonomous agent deployment.
Beyond Benchmarks: Measuring Real-World Impact of Autonomous Agents
Kalmantic Labs
A framework for collecting and analyzing real-world feedback on how autonomous agents impact humans, workflows, and organizational structures across industries.