Done
Pinned fields
Click on the next to a field label to start pinning.
Details
Details
Assignee
David Irving
David IrvingReporter
David Irving
David IrvingLabels
Reviewers
Tim Jenness
Story Points
4
RubinTeam
Ops Middleware
Components
Checklist
Checklist
Created November 2, 2023 at 9:47 PM
Updated January 24, 2024 at 4:39 PM
Resolved January 24, 2024 at 4:39 PM
Before going to production with the Butler server, we need to ensure that we can reproduce a past build of its Docker image to allow for emergency fixes and backports to the production version of the service.
Currently the Docker build pulls in the latest version of its dependencies from PyPI at build time. That means external changes could cause breakage in the old code at any time if we attempt to rebuild.
The easiest and most obvious thing to do would be to pin exact versions of the dependencies in the requirements.txt. SQuaRE's safir FastAPI template has a bit of code for doing this in its Makefile (https://github.com/lsst/templates/blob/main/project_templates/fastapi_safir_app/example/Makefile).
However, that gets a bit weird because the integration testing on Jenkins installs its dependencies from conda-forge, ignoring requirements.txt. I'm not sure if it's possible or makes sense to install dependencies as subset of the rubin-env conda environment, but that would make sure that the code that is integration tested is the code that is deployed.
The current GitHub actions for running unit tests also use a different set of dependencies (some are from conda-forge and some are from PyPI). At a minimum these unit tests and the Docker build should probably use the same versions of dependencies.
Currently the Docker build is using a python/Debian base image just because that's what the safir FastAPI template is using. It may make sense to use micromamba instead to make it easier to get common package versions between Jenkins, Docker, and Github Actions (https://hub.docker.com/r/mambaorg/micromamba)