asynchronous batching speeds up llm inference
separating cpu and gpu work with cuda streams and events reduces idle time, boosting throughput by up to 24%.
aisummaries filed under tools
separating cpu and gpu work with cuda streams and events reduces idle time, boosting throughput by up to 24%.
aifive compact open-weight language models that support structured tool calling for agentic ai workflows.
aia curated list of github repositories that teach self-hosting skills from discovery to deployment, monitoring, and secure access.
aipytorch 2.12 brings up to 100x faster batched eigendecomposition, a device-agnostic graph api, and support for exporting microscaling quantized models.
aifive reusable python scripts handle common time series tasks like resampling, anomaly detection, decomposition, forecasting, and multi-series comparison.
aiarm releases hands-on jupyter labs to deploy pytorch models on edge cpus and npus using executorch.
aia comparison of polars and pandas on three data tasks shows polars is faster and uses less memory due to lazy evaluation and parallelism.
aisimon willison explores patterns for running his llm cli tool directly from a shebang line, enabling scripts that generate content or use tools via natural language prompts.
aia curated list of ten github repositories that help developers learn fastapi through templates, examples, auth tools, microservices, and ml projects.
aijames shore warns that ai coding tools must reduce maintenance costs proportionally to their speed gains, or they risk multiplying long-term work.
aishopify's internal coding agent river operates in public slack channels, letting employees learn from each other's work through visible conversations.
aia step-by-step guide to creating a learning management system that adapts to each learner using local ai models.
aigitlab announces workforce reduction, flatter org, and new values as it bets on agentic engineering expanding software demand.
aijason koebler describes the zombie internet where ai-generated content blends with human writing, making it exhausting to filter and distorting communication.
aiaws infrastructure for foundation model training and inference integrates accelerated compute, high-bandwidth networking, and distributed storage with open-source orchestration and ml frameworks.
aimeta's ikbo eliminates redundant user embedding replication in recommendation models by fusing broadcast logic directly into gpu kernels, cutting latency by up to two-thirds.
ai