Skip to content

DM-55972: Switch test suite from direct vcrpy use to pytest-recording - #195

Open
mfisherlevine wants to merge 2 commits into
mainfrom
tickets/DM-55972
Open

DM-55972: Switch test suite from direct vcrpy use to pytest-recording#195
mfisherlevine wants to merge 2 commits into
mainfrom
tickets/DM-55972

Conversation

@mfisherlevine

@mfisherlevine mfisherlevine commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Test classes are now marked with @pytest.mark.vcr and configured via tests/conftest.py, which keeps the existing flat cassette directory and bare-function-name cassette naming so no recordings need regenerating. vcrpy (which pytest-recording wraps) is retained only for setUpClass, which runs outside the per-test fixture the plugin uses.

mfisherlevine and others added 2 commits August 27, 2026 22:14
@mfisherlevine @claude
Switch test suite from direct vcrpy use to pytest-recording
ebfeb8e 
Test classes are now marked with @pytest.mark.vcr and configured via
tests/conftest.py, which keeps the existing flat cassette directory and
bare-function-name cassette naming so no recordings need regenerating.
vcrpy (which pytest-recording wraps) is retained only for setUpClass,
which runs outside the per-test fixture the plugin uses, and for the
setUp in test_ics, which is recorded separately from its tests and so
nests its own cassette inside the plugin's.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mfisherlevine @claude
Additionally support pytest-vcr in the test suite
bc29ae0 
pytest-vcr and pytest-recording are mutually exclusive, and the stack
environment will move from the former to the latter at some point.
Rather than having to land this branch at the exact moment that
happens, conftest.py now detects which plugin is loaded and adapts to
it: the record mode is read from --vcr-record or --record-mode as
appropriate, and the cassette name is overridden via both
vcr_cassette_name and default_cassette_name.

This commit is purely additive and should be reverted once the
environment has moved to pytest-recording.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 27, 2026

Copy link
Copy Markdown

mypy-coverage report

❌ 188 unannotated, 25 partial definition(s).

  • Root: /home/runner/work/summit_utils/summit_utils
  • Config: mypy.ini
  • Files scanned: 61
  • Files excluded: 0

Summary

metric value
❌ body-checked by mypy 76.4%
❌ fully annotated 73.3%
annotated 585
partial 25
unannotated 188

Files with gaps

file fully typed % body checked % unannotated partial
python/lsst/summit/utils/astrometry/anet.py 93.8% 100.0% 0 1
python/lsst/summit/utils/bestEffort.py 83.3% 100.0% 0 1
python/lsst/summit/utils/consdbClient.py 78.3% 100.0% 0 5
python/lsst/summit/utils/guiders/plotting.py 96.3% 96.3% 1 0
python/lsst/summit/utils/guiders/reading.py 84.6% 88.5% 6 2
python/lsst/summit/utils/guiders/transformation.py 68.0% 80.0% 5 3
python/lsst/summit/utils/imageExaminer.py 95.8% 100.0% 0 1
python/lsst/summit/utils/nightReport.py 96.0% 100.0% 0 1
python/lsst/summit/utils/plotRadialAnalysis.py 72.7% 100.0% 0 3
python/lsst/summit/utils/quickLook.py 66.7% 100.0% 0 2
python/lsst/summit/utils/simonyi/mountAnalysis.py 75.0% 87.5% 1 1
python/lsst/summit/utils/simonyi/mountData.py 80.0% 80.0% 1 0
python/lsst/summit/utils/spectrumExaminer.py 93.3% 100.0% 0 1
python/lsst/summit/utils/tmaUtils.py 93.8% 98.5% 1 3
python/lsst/summit/utils/utils.py 97.9% 100.0% 0 1
tests/conftest.py 0.0% 0.0% 7 0
tests/test_bestEffortIsr.py 25.0% 25.0% 6 0
tests/test_blockUtils.py 40.0% 40.0% 6 0
tests/test_butlerUtils.py 8.1% 8.1% 34 0
tests/test_consdbClient.py 0.0% 0.0% 24 0
tests/test_dateTime.py 33.3% 33.3% 4 0
tests/test_efdUtils.py 15.4% 15.4% 11 0
tests/test_guider.py 94.4% 94.4% 1 0
tests/test_ics.py 33.3% 33.3% 4 0
tests/test_imageExaminer.py 40.0% 40.0% 3 0
tests/test_nightReport.py 10.5% 10.5% 17 0
tests/test_plotting.py 25.0% 25.0% 3 0
tests/test_quickLook.py 30.8% 30.8% 9 0
tests/test_tmaUtils.py 10.3% 10.3% 26 0
tests/test_utils.py 32.0% 32.0% 17 0
tests/utils.py 0.0% 0.0% 1 0

Unannotated definitions (188)

python/lsst/summit/utils/guiders/plotting.py

  • L355 GuiderPlotter.stripPlot._zero (function)

python/lsst/summit/utils/guiders/reading.py

  • L62 make_subplot (function)
  • L536 GuiderData.__iter__ (method)
  • L540 GuiderData.items (method)
  • L545 GuiderData.keys (method)
  • L549 GuiderData.values (method)
  • L554 GuiderData.__getitem__ (method)

python/lsst/summit/utils/guiders/transformation.py

  • L473 stampToCcd (function)
  • L872 DriftResult.__str__ (method)
  • L902 DriftResult.summary (method)
  • L914 getObsAltAz (function)
  • L953 DeltaAltAz (function)

python/lsst/summit/utils/simonyi/mountAnalysis.py

  • L283 plotMountErrors.offset_time_aware (function)

python/lsst/summit/utils/simonyi/mountData.py

  • L122 getAzElRotHexDataForPeriod.calcDeltaT (function)

python/lsst/summit/utils/tmaUtils.py

  • L190 getAzimuthElevationDataForEvent.calcDeltaT (function)

tests/conftest.py

  • L57 _getVcrPlugin (function)
  • L71 pytest_configure (function)
  • L85 vcr_cassette_dir (function)
  • L91 vcr_config (function)
  • L95 _getCassetteName (function)
  • L102 default_cassette_name (function)
  • L108 vcr_cassette_name (function)

tests/test_bestEffortIsr.py

  • L33 BestEffortIsrTestCase.setUpClass (method)
  • L43 BestEffortIsrTestCase.test_getExposure (method)
  • L52 BestEffortIsrTestCase.test_getExposureFromExpRecord (method)
  • L70 BestEffortIsrTestCase.test_raises (method)
  • L77 BestEffortIsrTestCase.test_quicklook_connections (method)
  • L91 setup_module (function)

tests/test_blockUtils.py

  • L113 BlockParserTestCase.setUpClass (method)
  • L128 BlockParserTestCase.tearDown (method)
  • L133 BlockParserTestCase.test_parsing (method)
  • L155 BlockParserTestCase.test_notFoundBehavior (method)
  • L182 BlockParserTestCase.test_actualValues (method)
  • L205 setup_module (function)

tests/test_butlerUtils.py

  • L70 ButlerUtilsTestCase.setUp (method)
  • L117 ButlerUtilsTestCase.test_getLatissDefaultCollections (method)
  • L123 ButlerUtilsTestCase.test_RECENT_DAY (method)
  • L142 ButlerUtilsTestCase.test_sanitizeDayObs (method)
  • L152 ButlerUtilsTestCase.test_getMostRecentDayObs (method)
  • L162 ButlerUtilsTestCase.test_getSeqNumsForDayObs (method)
  • L173 ButlerUtilsTestCase.test_getMostRecentDataId (method)
  • L182 ButlerUtilsTestCase.test_getDatasetRefForDataId (method)
  • L191 ButlerUtilsTestCase.test__dayobs_present (method)
  • L199 ButlerUtilsTestCase.test__seqnum_present (method)
  • L207 ButlerUtilsTestCase.test__expid_present (method)
  • L215 ButlerUtilsTestCase.test_getDayObs (method)
  • L224 ButlerUtilsTestCase.test_getSeqNum (method)
  • L233 ButlerUtilsTestCase.test_getExpId (method)
  • L242 ButlerUtilsTestCase.test_datasetExists (method)
  • L248 ButlerUtilsTestCase.test_sortRecordsByDayObsThenSeqNum (method)
  • L271 ButlerUtilsTestCase.test_getDaysWithData (method)
  • L277 ButlerUtilsTestCase.test_getExpIdFromDayObsSeqNum (method)
  • L282 ButlerUtilsTestCase.test_updateDataIdOrDataCord (method)
  • L292 ButlerUtilsTestCase.test_fillDataId (method)
  • L308 ButlerUtilsTestCase.test_getExpRecordFromDataId (method)
  • L313 ButlerUtilsTestCase.test_getDayObsSeqNumFromExposureId (method)
  • L319 ButlerUtilsTestCase.test_removeDataProduct (method)
  • L323 ButlerUtilsTestCase.test_getLatissOnSkyDataIds (method)
  • L348 ButlerUtilsTestCase.test__assureDict (method)
  • L361 ButlerUtilsTestCase.test__get_dayobs_key (method)
  • L370 ButlerUtilsTestCase.test__get_seqnum_key (method)
  • L379 ButlerUtilsTestCase.test__get_expid_key (method)
  • L388 ButlerUtilsTestCase.test_updateDataId (method)
  • L405 ButlerUtilsTestCase.test_getExpRecord (method)
  • L434 ButlerInitTestCase.test_dafButlerRaiseTypes (method)
  • L456 ButlerInitTestCase.test_makeDefaultLatissButlerRaiseTypes (method)
  • L477 ButlerInitTestCase.test_DAF_BUTLER_REPOSITORY_INDEX_value (method)
  • L489 setup_module (function)

tests/test_consdbClient.py

  • L38 client (function)
  • L45 test_table_name (function)
  • L55 test_add_flexible_metadata_key (function)
  • L113 test_get_flexible_metadata_keys (function)
  • L128 test_get_flexible_metadata (function)
  • L155 test_insert_flexible_metadata (function)
  • L164 test_schema (function)
  • L185 test_clean_token_url_response (function)
  • L208 test_client (function)
  • L216 test_timeout_override (function)
  • L224 test_get_passes_timeout (function)
  • L228 test_get_passes_timeout.fake_get (function)
  • L240 test_post_passes_timeout (function)
  • L244 test_post_passes_timeout.fake_post (function)
  • L256 test_timeout_propagates (function)
  • L259 test_timeout_propagates.fake_get (function)
  • L268 test_insert_obs_id (function)
  • L283 test_insert_by_seq_num (function)
  • L298 test_insert_by_seq_num_detector (function)
  • L315 test_insert_allow_update (function)
  • L330 test_insert_bad_obs_id_tuple (function)
  • L338 test_insert_no_values (function)
  • L345 test_getCcdVisitTableForDay_dedupes_overlapping_columns (function)
  • L390 test_getCcdVisitTableForDay_keeps_columns_when_no_overlap (function)

tests/test_dateTime.py

  • L41 DateTimeTestCase.test_getDayObsAsTimes (method)
  • L64 DateTimeTestCase.test_calcDayOffset (method)
  • L75 DateTimeTestCase.test_getDayObsForTime (method)
  • L92 setup_module (function)

tests/test_efdUtils.py

  • L64 EfdUtilsTestCase.setUpClass (method)
  • L87 EfdUtilsTestCase.tearDown (method)
  • L92 EfdUtilsTestCase.test_makeEfdClient (method)
  • L95 EfdUtilsTestCase.test_getTopics (method)
  • L114 EfdUtilsTestCase.test_getEfdData (method)
  • L172 EfdUtilsTestCase.test_raiseIfTopicNotInSchema (method)
  • L184 EfdUtilsTestCase.test_getMostRecentRowWithDataBefore (method)
  • L194 EfdUtilsTestCase.test_efdTimestampToAstropy (method)
  • L199 EfdUtilsTestCase.test_astropyToEfdTimestamp (method)
  • L205 EfdUtilsTestCase.test_clipDataToEvent (method)
  • L258 setup_module (function)

tests/test_guider.py

  • L293 setup_module (function)

tests/test_ics.py

  • L45 M1M3ICSTestCase.setUp (method)
  • L60 M1M3ICSTestCase.tearDown (method)
  • L65 M1M3ICSTestCase.test_analysis (method)
  • L96 setup_module (function)

tests/test_imageExaminer.py

  • L35 ImageExaminerTestCase.setUpClass (method)
  • L49 ImageExaminerTestCase.test_imageExaminer (method)
  • L67 setup_module (function)

tests/test_nightReport.py

  • L44 NightReportTestCase.setUpClass (method)
  • L64 NightReportTestCase.test_saveAndLoad (method)
  • L78 NightReportTestCase.test_getSortedData (method)
  • L89 NightReportTestCase.test_getExpRecordDictForDayObs (method)
  • L106 NightReportTestCase.test_getObsInfoAndMetadataForSeqNum (method)
  • L116 NightReportTestCase.test_rebuild (method)
  • L127 NightReportTestCase.test_getExposureMidpoint (method)
  • L148 NightReportTestCase.test_getTimeDeltas (method)
  • L154 NightReportTestCase.test_makeStarColorAndMarkerMap (method)
  • L165 NightReportTestCase.test_printObsTable (method)
  • L175 NightReportTestCase.test_plotPerObjectAirMass (method)
  • L191 NightReportTestCase.test_makeAltAzCoveragePlot (method)
  • L205 NightReportTestCase.test_calcShutterTimes (method)
  • L213 NightReportTestCase.test_getDatesForSeqNums (method)
  • L219 NightReportTestCase.test_doesNotRaise (method)
  • L230 NightReportTestCase.test_internals (method)
  • L250 setup_module (function)

tests/test_plotting.py

  • L45 PlottingTestCase.setUpClass (method)
  • L49 PlottingTestCase.test_plot (method)
  • L96 setup_module (function)

tests/test_quickLook.py

  • L40 QuickLookIsrTaskTestCase.setUp (method)
  • L60 QuickLookIsrTaskTestCase.test_runQuickLook (method)
  • L82 QuickLookIsrTaskTestCase.test_runQuickLookMissingData (method)
  • L87 QuickLookIsrTaskTestCase.test_runQuickLookBadDark (method)
  • L109 QuickLookIsrTaskRunQuantumTests.setUp (method)
  • L235 QuickLookIsrTaskRunQuantumTests.tearDown (method)
  • L238 QuickLookIsrTaskRunQuantumTests.test_runQuantum (method)
  • L292 raiseExitMockError (function)
  • L307 setup_module (function)

tests/test_tmaUtils.py

  • L63 getTmaEventTestTruthValues (function)
  • L89 writeNewTmaEventTestTruthValues (function)
  • L117 makeValid (function)
  • L124 _turnOn (function)
  • L140 TmaUtilsTestCase.test_tmaInit (method)
  • L156 TmaUtilsTestCase.test_tmaReferences (method)
  • L170 TmaUtilsTestCase.test_getAxisAndType (method)
  • L182 TmaUtilsTestCase.test_initStateLogic (method)
  • L223 TMAEventMakerTestCase.setUpClass (method)
  • L236 TMAEventMakerTestCase.tearDown (method)
  • L241 TMAEventMakerTestCase.test_events (method)
  • L246 TMAEventMakerTestCase.test_rowDataForValues (method)
  • L262 TMAEventMakerTestCase.test_monotonicTimeInDataframe (method)
  • L267 TMAEventMakerTestCase.test_monotonicTimeApplicationOfRows (method)
  • L283 TMAEventMakerTestCase.test_fullDaySequence (method)
  • L294 TMAEventMakerTestCase.test_endToEnd (method)
  • L318 TMAEventMakerTestCase.test_noDataBehaviour (method)
  • L329 TMAEventMakerTestCase.test_helperFunctions (method)
  • L340 TMAEventMakerTestCase.test_filterBadValues (method)
  • L436 TMAEventMakerTestCase.test_getEvent (method)
  • L456 TMAEventMakerTestCase.test_printing (method)
  • L479 TMAEventMakerTestCase.test_getAxisData (method)
  • L497 TMAEventMakerTestCase.test_plottingAndCommands (method)
  • L518 TMAEventMakerTestCase.test_findEvent (method)
  • L568 TMAEventMakerTestCase.test_eventAssociatedWith (method)
  • L619 setup_module (function)

tests/test_utils.py

  • L66 ExpSkyPositionOffsetTestCase.setUp (method)
  • L82 ExpSkyPositionOffsetTestCase.test_getExpPositionOffset (method)
  • L146 MiscUtilsTestCase.test_getFieldNameAndTileNumber (method)
  • L179 MiscUtilsTestCase.test_getAirmassSeeingCorrection (method)
  • L191 MiscUtilsTestCase.test_getFilterSeeingCorrection (method)
  • L198 MiscUtilsTestCase.test_getBandpassSeeingCorrection (method)
  • L214 MiscUtilsTestCase.test_quickSmooth (method)
  • L222 MiscUtilsTestCase.test_getCurrentDayObsDatetime (method)
  • L233 MiscUtilsTestCase.test_getCurrentDayObsInt (method)
  • L240 MiscUtilsTestCase.test_getCurrentDayObsHumanStr (method)
  • L247 MiscUtilsTestCase.test_getSunAngle (method)
  • L265 QuantileTestCase.test_quantiles (method)
  • L286 ImageBasedTestCase.test_fluxFromFootprint (method)
  • L340 IdTestCase.test_exposure_id (method)
  • L347 IdTestCase.test_ccdexposure_id (method)
  • L360 IdTestCase.test_calcEclipticCoords (method)
  • L376 setup_module (function)

tests/utils.py

  • L40 getVcr (function)

@codecov

codecov Bot commented Aug 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 23.38%. Comparing base (6e79a37) to head (bc29ae0).

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #195   +/-   ##
=======================================
  Coverage   23.38%   23.38%           
=======================================
  Files          38       38           
  Lines        7070     7070           
=======================================
  Hits         1653     1653           
  Misses       5417     5417
Files with missing lines Coverage Δ
python/lsst/summit/utils/efdUtils.py 25.47% <ø> (ø)
python/lsst/summit/utils/tmaUtils.py 25.06% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@mfisherlevine