Gitlink – Switching Out Git LFS for Cloud Storage

Hacker News

Share card

Gitlink – Switching Out Git LFS for Cloud Storage

I wanted to share an open-source package I created to help deploy a Git LFS server in a Docker container that can point to AWS or GCP with a single configuration. The project is still in its early stages, so I only have a few of the plugins I want, but it's already beneficial to switch out the Git LFS black blob for S3/Cloud Storage. I drew inspiration from two excellent projects - giftless and truss - for the project's architecture. Both taught me a lot, and I wanted to build something similar to giftless but contained in an image so that I could more easily move around, test and deploy to different cloud backends without worrying about managing dependencies. I split the codebase into a plugin architecture that allows you to specify which backends and manipulations to add to a configuration file. Additionally, I had to figure out how to handle secrets better. I opted for a source-drain technique where the config specifies the secret's source, such as Vault or an environment variable, which is then funneled into a "drain" (e.g., a hidden folder mounted on the Docker container). Could this approach be safer than using env vars? I also worked on some deployers and got the ECR upload working, which sends the Docker image to the registry for later deployment on App Runner. While working on this project, I realized I had inadvertently built a Docker wrapper that could work for any Python API. This wrapper abstracts away Docker, secret management, plugins, and deployment, among others. I could run the build-context command and switch out the API file for whatever I wanted. I'm considering spinning that part out to make it a more API-agnostic tool. Let me know if you have any questions or feedback. I'd be happy to discuss this further.