Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Move utility code out of pipe_base and daf_butler into utils package

Description

Move all the general purpose utility code out of pipe_base and daf_butler and into utils. The license for utils will be changed to BSD 3 clause and a setup.py will be created to allow it to be built and tested with a GitHub Action.

daf_butler has no public APIs for this code and will be migrated to use utils with no deprecation period.

pipe_base will include deprecation notices for code that existed in v22. The logging code that is new (and only used by pipe_base) will not include deprecation messages.

Checklist

Issue Matrix

hide

Lucidchart Diagrams

Activity

Tim Jenness October 8, 2021 at 3:39 PM

All branches merged. I did rename iterable to ensure_iterable.

Jim Bosch October 6, 2021 at 3:51 PM

Only minor comments, many of them preexisting.  Only even slightly big question is whether to rename iterable.

Tim Jenness October 5, 2021 at 7:55 PM

there are a lot of pull requests here but all but utils are just making changes to keep up with the changes to utils, daf_butler, and pipe_base. The timer code has been made more general.

utils now uses mypy (except for a couple of the functions in test.py and wrappers.py) so daf_butler is now able to use those when using mypy. Doing that forced me to fix doImport usage to do type validation and I also added doImportType to ensure that a type and not a module is retrieved. Note that you will need to set MYPYPATH to $UTILS_DIR/python otherwise things will not pass in daf_butler.

I am deliberately not exporting these new lsst.utils functions into the default namespace.

There are now only three routines left in butler utils.py: globToRegex, stripIfNotNone and transaction. Nothing uses those outside of daf_butler and the transaction one in particular seems quite specific. I could move the other two over if you think it's worth it, or else wait until something else needs them. I wasn't sure if you wanted the named.py stuff to move over or any of the time functions in daf_butler.

Done
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Labels

Reviewers

Jim Bosch

Story Points

RubinTeam

Architecture

Components

Checklist

Created September 10, 2021 at 6:26 PM
Updated March 25, 2022 at 8:31 PM
Resolved October 8, 2021 at 3:38 PM