Convert/ingest PS1 and Gaia refcats into gen3 repo for DECam use

Description

Turns out you need reference catalogs for single frame processing!

DECam uses PS1 for photometry and Gaia for astrometry, so I need both. I think the process presently involves using `butler convert`, somehow. I'm pretty sure it's not as simple as adding a couple symlinks to my repo.

Attachments

4

Checklist

Lucidchart Diagrams

Issue Matrix

hide

Activity

Show:

John Parejko 
January 13, 2021 at 7:27 PM

A few docs clarifications, otherwise this looks good.

Meredith Rawls 
January 13, 2021 at 7:07 PM

One followup regarding skymaps. I mentioned that I ran `butler register-skymap REPO -c name='name'` in both of my gen3 repos. Turns out this resulted in two identical skymaps in all but name, which really confused things when I went to export the deepCoadd templates I made in the 2014 repo to the 2015 repo. With help from Jim, this is how I resolved the situation.

(1) do export with elements=() in the call to {{}}saveDatasets

(2) manually edit data_id mentions of origin repo skymap name to be destination repo skymap name in the yaml created during export (do not edit any file paths!)

(3) skip dimensions skymap, tract, and patch when calling import_.

I will attach all the export/import scripts I used to this ticket for future reference.

Meredith Rawls 
January 8, 2021 at 2:05 AM

, would you review this? The only code is a functional `APTemplate.yaml` now that I have managed to get refcats for the HiTS fields into my gen3 repo.

Meredith Rawls 
January 8, 2021 at 1:54 AM

The final step I had to do to get my pipeline to run (besides set various configs correctly) was `butler define-visits REPO DECam`.

To use the refcats in calibration (lsst.pipe.tasks.calibrate.CalibrateTask), these configs are necessary:

Meredith Rawls 
January 7, 2021 at 4:03 AM

I've managed to export refcats from one repo and import them to another.

The other necessary step (for a pipeline which includes building coadds, like the one I'm running to build templates) is registering a skymap. With all the raws in place, I ran `butler register-skycap . -c name="decam_hits2014_v1"`.

Both the hits2014-3 and hits2015-3 repos now have panstarrs and gaia refcats in them, as well as registered skymaps, for use with decam processing.

Done
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Reviewers

John Parejko

Story Points

RubinTeam

Sprint

Checklist

Created December 18, 2020 at 7:52 AM
Updated January 13, 2021 at 8:49 PM
Resolved January 13, 2021 at 8:49 PM