smarter json alternative for llm pipelines
toon reduces token waste in llm prompts by compacting repeated json structure into a tabular format.
aitag machine learning
summaries filed under machine learning
aws foundation model building blocks
aws infrastructure for foundation model training and inference integrates accelerated compute, high-bandwidth networking, and distributed storage with open-source orchestration and ml frameworks.
aibuild vector search from scratch in python
a step-by-step guide to creating a vector search engine using only numpy, covering embeddings, normalization, cosine similarity, and visualization.
aiin-kernel broadcast optimization for recsys inference
meta's ikbo eliminates redundant user embedding replication in recommendation models by fusing broadcast logic directly into gpu kernels, cutting latency by up to two-thirds.
aiadding benchmaxxer repellant to the open asr leaderboard
hugging face adds private speech datasets to its asr leaderboard to reduce benchmark gaming and provide a more realistic view of model performance across accents and speaking styles.
ai newsvllm v0 to v1: correctness before corrections in rl
migrating from vllm v0 to v1 for online reinforcement learning required fixing logprob semantics, runtime defaults, weight updates, and fp32 head precision to match training dynamics.
ai newslkv trims llm memory with learned cache budgets
a new method learns optimal key-value cache compression directly from task objectives, improving long-context llm efficiency without heuristic rules.
ai newsratequant tunes kv cache precision for llms
a new method allocates different bit-widths to attention heads in kv cache quantization, avoiding distortion model mismatch to improve large language model serving efficiency.
ai newsemo: pretraining mixture of experts for emergent modularity
emo is a mixture-of-experts model trained so that experts self-organize into task-specific groups, allowing strong performance with only a small subset of experts.
ai news