Investigate pydantic for JSON serialization of Butler
Description
is triggered by
Checklist
Lucidchart Diagrams
Issue Matrix
hideActivity
It seems like one of the features of fastapi (the python server recommended by square) is that it uses pydantic models to build the API. It seems like I will be wanting to make this change anyhow as part of the client/server butler implementation.
On GitHub @Jim Bosch made this comment:
Overall this looks fine. Given that we're still constructing dicts a lot internally just to make the pydantic instances, I think it's only better than the previous version without pydantic if:
we do want schema validation (something I'm not really qualified to answer; web APIs are not an area of expertise for me);
we can in fact simplify some of the to_simple/from_simple implementations further, maybe by using different pydantic classes for the minimal versions and/or eliminating minimal forms we're not sure we'll use.
If we do want schema validation, I have absolutely no problem with this; it's only a little bit more complicated than the version without pydantic.
@Kian-Tat Lim I would you mind taking a look and giving a second opinion, especially regarding whether you think we should move forward with this and request pydantic be added to the conda env/
This is an experimental PR that I'm not expecting to merge immediately but it shows where I ended up. I use a trampoline class for now and the user still sees a to_json/from_json as method as before. The difference is that the JSON values coming from an external source now have up front schema validation with type checking. Ideally the thing being serialized would be a pydantic model but this intermediate approach seems to work.
I'm not sure how we can tackle DataCoordinate or DimensionRecord because those have dynamic content and are therefore impossible to model. DimensionRecord would have to be changed to be specific classes associated with specific records that could then be validated.
cc/ @Kian-Tat Lim since this would require pydantic be added to rubin-env (it's a single package that doesn't pull in any dependencies).
This is a short experimental ticket to prototype JSON serialization using pydantic models.
The intent for now is to create simple variants of complex Butler objects as pydantic models and use the pydantic infrastructure for JSON serialization.
If this approach looks promising an RFC would be made to add pydantic to the conda environment.