Stoat – build previews and metrics in pull request comments

Hacker News

Share card

Stoat – build previews and metrics in pull request comments

Hi HN! We're Jared and Liren. We are building Stoat, which allows you to turn pull request comments into developer dashboards: https://stoat.dev/ Builds produce a bunch of useful information that's difficult to access on CI/CD platforms such as GitHub. Even accessing test results for your latest commit can take a minute or two. Code coverage reports may be available in logs (which are hard to search through) or not at all. Generated docs and other reports are usually omitted from builds. Stoat takes any type of file (static sites, code coverage reports, images, etc) at the end of each build, hosts them, and links to them in a single PR comment. Builds also can produce a lot of metrics such as build runtimes, benchmarks, and some business KPIs. These metrics are usually ignored or pushed to something like Datadog and checked reactively after the changes are merged or deployed. For example, there usually isn't an easy way to compare the current PR's impact on build times before merging. Stoat makes it easy to generate and embed charts directly in PR comments so it's easy to compare metrics generated by builds from different branches. If you're increasing the build time 30%, it should be noticeable! To solve these problems Stoat runs in three steps: 1. We consume GitHub build outputs (using an action) 2. We process/aggregate/host these outputs 3. We render this data into a single, templatable PR comment (similar to https://github.com/peter-evans/create-or-update-comment but we aggregate data across build runs using state stored on an external server to avoid race conditions) Here are some example PRs using Stoat: - Tracking build runtimes: https://github.com/stoat-dev/stoat-action/pull/101 - Previewing a variety of build outputs: https://github.com/stoat-dev/example-java/pull/1 - Previewing gifs and videos: https://github.com/stoat-dev/example-remotion/pull/2 Here’s a 4-minute demo video: https://youtu.be/oikep47_7ro In the near future we're planning on adding more tools so users can compare versions of images and files generated by builds (but not checked in), track custom metrics, track integration tests, and more! Longer term, we're aiming to be a platform for DevTools/DevEx teams building out these types of tools internally. We'll have paid plans for private repo support and some advanced features, but our basic features will always be free for public repos. Give it a spin on one of your public repos at https://github.com/stoat-dev/stoat-action#quick-start or check out our docs at https://docs.stoat.dev/ Let us know what you think! Thanks!