source: simon willison: siri ai at wwdc 2026
level: technical
apple revealed new siri ai features at wwdc 2026, taking a cautious approach after past overpromises. the system uses a custom gemini-derived model licensed from google, running on apple's private cloud compute. it will leverage vision llms to extract information from the user's screen, avoiding the need for app developers to add custom integration code. this approach is more feasible now because vision llms have matured since 2024.
a new core ai library was introduced to help developers run their own models on apple hardware. it integrates with meta's pytorch ecosystem through core ai pytorch extensions, a python package that converts pytorch models into core ai programs. this allows models to run efficiently on apple devices by mapping operations to core ai. developers can test these features in the ios 27 developer beta, but access to siri ai requires joining a waiting list.
the gemini models for private cloud compute run on google cloud with nvidia gpus, extending apple's secure infrastructure. apple's security research blog explains that the system maintains privacy protections by using dedicated processes, short-lived software, and attested keys in isolated virtual machines. all binaries will be published for public inspection, similar to apple's existing private cloud compute on apple silicon. early reports from testers who got off the waitlist are expected soon.
why it matters: vision llms reduce integration work for developers, and the core ai library makes it easier to deploy custom models on apple devices.