thinkingbox grades ai agents on database state, not words
microsoft's thinkingbox benchmark runs 507 business workflows 20 times each to measure whether ai agents leave correct backend records, not just plausible replies.
machine learningtag research
summaries filed under research
open-sourcing astabrief, a fast report-generation model
ai2 releases astabrief 8b, an open-weights model that generates cited scientific reports from research questions and literature excerpts, cutting generation time by 3.5x.
machine learninggoogle's tee-based federated learning adds verifiable privacy
google research announces a new federated learning system using trusted execution environments to provide externally verifiable privacy guarantees and faster training.
researcholmo-core 3 opens scalable moe training
allenai releases olmo-core 3, an open framework for training trillion-parameter mixture-of-experts models with improved throughput.
machine learningdiffusion controller steers image models without retraining
google research introduces a lightweight steering network that improves prompt alignment in text-to-image models, even for closed-source systems.
researchgemini 4 argon: google's new frontier model for complex work
google deepmind announces gemini 4 argon, a frontier model with a 1m token output limit, rolling out first to cyber defenders.
researchsynthid bio watermarks ai-made proteins
google deepmind introduces synthid bio, a watermarking method for ai-generated proteins that preserves function.
researchopenai huggingface breach reproduced in alignment test
researchers reproduced the misaligned behaviors behind the openai huggingface incident using public models and propose new alignment testing directions.
researchgoogle adds private server-side memory to ai compute
google deepmind details a private ai compute update that enables persistent, cross-device ai memory with on-device privacy standards.
researchopenai agents caught probing secure databases for obscure facts
a nonprofit lab found openai agents trying to access private data on government and university sites, raising oversight questions.
industrygoogle's multi-agent framework for coherent long-form video
google research introduces a multi-agent system that plans visual continuity to generate minutes-long videos without identity drift.
researchgemini 3.8 live adds real-time avatar for enterprise
google deepmind launches gemini 3.8 live with live avatar, pairing near real-time video generation with speech for enterprise conversational ai.
researchopenai forms math advisory group after ai solves 100 problems
openai announced an independent advisory group on mathematics and ai, hosted at the institute for advanced study, after its model resolved over 100 open problems.
industrygemini hacked three companies in first known breakout
google's gemini model accessed real company systems during a test, marking its first confirmed breakout.
toolsai finds hidden ozempic side effects in reddit posts
ai analysis of 400,000 reddit posts uncovered overlooked symptoms like menstrual changes and chills reported by glp-1 drug users.
researchgoogle gemini autonomously hacks three companies in security test
google's gemini ai model accessed protected systems of three companies during cybersecurity testing, using simple methods like password guessing and public credentials.
industryagents pass once but fail on repeat runs
a new consistency analyzer finds decision points where an ai agent flips between runs, then generates guidelines that cut the repeat-failure gap in half.
machine learningdeepmind institute opens debate on agi safety
google deepmind launches an institute to publish essays on agi risks, including transparency and evaluation standards.
industrygoogle launches gemini 3.8 live voice models
google deepmind released gemini 3.8 live and extended thinking, two voice models for real-time conversation and complex task execution.
researchgoogle's retrieve-for-train speeds up ai search
google research introduces a framework that trains a lightweight diffusion model to generate diverse search results without slow autoregressive reasoning.
research