source: hugging face blog: what building shippy taught us about building agents
level: technical
building an ai agent for maritime domain awareness is a reliability problem. a wrong answer could send a patrol vessel off course, wasting resources and risking safety. the skylight team built shippy to be correct, stay within limits, and handle diverse tasks using live data. the agent has three parts: a soul (system prompt), skills (markdown files for specific requests), and config (runtime settings). skills cover querying vessel events, looking up maritime boundaries, interpreting track data, and generating map links. the soul sets clear boundaries, like never making legal determinations or speculating beyond data.
agents are nondeterministic, but tools can be predictable. shippy uses a purpose-built cli to call the skylight api, avoiding bugs from raw api calls. the cli handles authentication, pagination, and structured output, writing results to a json file to avoid pipe limits. underneath, a standardized api with typed schemas allows independent testing of each layer. for hosting, mothership provisions isolated kubernetes pods per user session, injecting user credentials so data stays private. the sandbox restricts network access and lets the agent run code and install dependencies safely.
evaluation focuses on the whole agent, not just the model. subject-matter experts create scenarios and rubrics, weighting criteria like data accuracy and source attribution. an llm judge scores responses, and tasks must pass a fixed threshold. the eval suite runs against live data on each versioned build, catching regressions. recent runs showed issues like overstepping into tactical advice, geometry errors, and invented commands. future plans include agent-driven map control, model routing for efficiency, and cross-thread memory so shippy remembers user context across conversations.
why it matters: this approach shows how to build reliable ai agents for critical domains by combining deterministic tools, sandboxed isolation, and task-specific evaluation.
source: hugging face blog: what building shippy taught us about building agents