source: arxiv artificial intelligence: syll: open-source personal automation with cross-surface execution
level: technical
syll is an open-source, self-hosted multimodal agent designed to work across apis, command lines, web surfaces, and desktop guis. unlike many systems that focus on a single interface, syll provides a modular runtime that coordinates computer use across these different surfaces. it combines mcp/api tools, cli execution, and visual gui control, making it easier for agents to handle tasks that span multiple environments.
a key feature is its bidirectional interaction layer. users can teach syll new procedures by simply demonstrating them, and the agent compiles these demonstrations into reusable skills. when the agent acts, it produces multimodal evidence such as logs, keyframes, and approval checkpoints. this allows users to inspect and control what the agent does, supporting auditability and trust. memory, skills, routines, and governance rules are stored as editable local artifacts, so users can directly review and modify them.
syll aims to make personal automation more transparent and user-controlled. by externalizing agent knowledge and decision-making into plain files, it avoids black-box behavior. the system is self-hosted, which means users keep their data and automation logic on their own machines. this design supports both technical and non-technical users who want to build, share, and refine automation routines without relying on cloud services or opaque platforms.
why it matters: it gives ai and data science practitioners a transparent, self-hosted way to automate multi-surface tasks while keeping full audit and control over agent actions.
source: arxiv artificial intelligence: syll: open-source personal automation with cross-surface execution