source: simon willison: shot-scraper 1.11
level: technical
shot-scraper 1.11 is a command-line tool for taking website screenshots, recording video demos, and scraping sites using javascript. this release focuses on minor improvements, mainly around command option consistency and fixing how the server: mechanism works for video and multi commands. previously, these commands used a fixed one-second delay before connecting to a server, which could fail if the server took longer to start.
the server: processes used by shot-scraper multi and shot-scraper video now wait up to 30 seconds for the target url to accept connections. they poll for port availability instead of relying on a fixed delay. this change makes the tool more reliable when working with servers that need extra time to start serving traffic. the update addresses issue #197.
several commands now support a --js-file option for loading javascript from a local file, standard input, or a github gist using the gh:username/script syntax. this is an alternative to the existing --javascript option, which accepts javascript code directly as a string. the affected commands are shot-scraper, pdf, html, accessibility, and har. the shot-scraper multi command also supports a js_file: yaml key. additionally, the javascript and html commands now have a --timeout option for consistency with other commands.
why it matters: the improved server startup handling makes automated web scraping and screenshot capture more reliable, especially in data pipelines where servers may have variable startup times.