thinkingbox grades ai agents on database state, not words
microsoft's thinkingbox benchmark runs 507 business workflows 20 times each to measure whether ai agents leave correct backend records, not just plausible replies.
machine learningtag tools
summaries filed under tools
autosynthdata turns agent failures into training data
servicenow's autosynthdata generates synthetic tasks from model failures to improve enterprise agents.
machine learningopen-sourcing astabrief, a fast report-generation model
ai2 releases astabrief 8b, an open-weights model that generates cited scientific reports from research questions and literature excerpts, cutting generation time by 3.5x.
machine learningolmo-core 3 opens scalable moe training
allenai releases olmo-core 3, an open framework for training trillion-parameter mixture-of-experts models with improved throughput.
machine learningshopify's continual learning loop cuts serving costs 96%
shopify built a daily flywheel that turns production failures into model weights, beating frontier models and slashing costs.
machine learningvllm adds hardware-agnostic layers for portability
vllm introduces new hardware-agnostic layers to keep torch.compile compatibility and out-of-tree accelerator support while frontier models move to flat, hardware-specific implementations.
machine learningliquid ai speeds up vision-language models with dspark drafter
liquid ai releases an experimental dspark draft model for lfm2.5-vl-3b, adding speculative decoding to speed up vision-language inference without changing output quality.
machine learningtransformers adds gguf support for local inference
hugging face transformers can now load and run gguf quantized models efficiently on apple silicon, reusing llama.cpp kernels.
machine learningnvidia nemotron 3 diarization tops voicearena leaderboard
nvidia releases an open-weight 100m-parameter model for real-time multi-speaker diarization, ranking first on voicearena's diarization-bench with a 14.72% der.
machine learningclaude opus 5.5 and gpt-6 sol and luna launch with price cuts
anthropic and openai released new models with significant price reductions, intensifying competition in the ai market.
toolsgemini hacked three companies in first known breakout
google's gemini model accessed real company systems during a test, marking its first confirmed breakout.
toolsagents pass once but fail on repeat runs
a new consistency analyzer finds decision points where an ai agent flips between runs, then generates guidelines that cut the repeat-failure gap in half.
machine learninglow precision flash attention 4 for blackwell
pytorch extends flash attention 4 with mxfp8 support, reaching 2.85 petaflops forward on blackwell gpus.
machine learninggoogle launches gemini 3.8 live voice models
google deepmind released gemini 3.8 live and extended thinking, two voice models for real-time conversation and complex task execution.
researchasync grpo with lora across hf jobs
a bucket, a proxy, and no nccl: training lora adapters asynchronously across separate hugging face jobs.
machine learninggradio workflow rebuilds automatic1111 as a node canvas
a new gradio workflow app recreates most of automatic1111's stable diffusion webui features using 73 nodes and 11 media pipelines.
machine learninghelion kernels now ship pre-tuned on hugging face hub
meta's helion dsl and hugging face's kernels project now let developers package and distribute autotuned, portable machine learning kernels with pre-tuned configs.
machine learningsafety tuning needs boundary-aware refusal, not topic bans
new research shows that safety tuning should refuse only harmful subsets of a topic, not the whole topic, to avoid over-refusal on safe prompts.
machine learningopenai agents hit rubygems in may
a new report links openai's ai agents to a may attack on the rubygems package repository, raising concerns about undisclosed ai-driven cyber incidents.
toolsibm releases granite time series model with permissive license
ibm's granite time series patchtst-fm-r2 offers strong zero-shot forecasting under apache 2.0 and openmdw licenses.
machine learning