ci(conda): add a publish input to Conda Update's manual run (#2060)

docs/source/contributing/releasing.rst

@@ -402,7 +402,8 @@ this run, it *skipped*.
 ``tag``, ``pypi`` and ``conda`` all guard against this the same way the
 other workflows gated on a test run already do for the identical problem --
 ``cron-vendor.yml``, ``deploy-pages.yml`` and ``cron-conda.yml`` each open
-with ``!cancelled() && needs.head.result != 'failure'``, ``head`` being their
+with ``!cancelled()`` and require ``needs.head.result != 'failure'`` (and, in
+``cron-conda.yml``, its ``mode`` job's success, :issue:`2060`), ``head`` being their
 call to ``cron-head.yml``, so that a skipped gate does not cascade-skip them,
 while an outright failure or a genuine run cancellation still does. ``tag``, ``pypi`` and ``conda`` are
 stricter about their *publishing* predecessors -- ``github``, and for

docs/source/contributing/workflows.rst

@@ -393,14 +393,40 @@ exists`` step finds the release rather than skipping the GitHub upload
 
 To debug Vendor Update, Conda Update or GitHub Pages without waiting for
 Saturday, start a manual run on a branch, as in ``gh workflow run cron-conda.yml
---ref <branch>``. A manual run is build-only: Cron Head's test gate and every
-build and test step the schedule runs still run -- each ``conda-dist`` leg,
+--ref <branch>``. A manual run is build-only by default: Cron Head's test gate
+and every build and test step the schedule runs still run -- each ``conda-dist`` leg,
 Windows included, builds the dispatched commit with ``conda-update``'s changes
 applied -- while every step that pushes, tags, uploads, deploys or posts a
 report skips, no job that runs names an environment, and the Create Release run
 it raises releases nothing. :file:`tests/project/test_build_only_dispatch.py`
 pins this.
 
+Conda Update's manual run also takes a ``publish`` input, off by default, for
+putting a version's missing platform files on Anaconda (:issue:`2060`). A
+scheduled run uploads only a build it bumped, and it bumps only when
+``Verify Changed files`` sees a change, which :file:`util/conda_vendor.py` does
+not make when no dependency moved. ``gh workflow run cron-conda.yml --ref main
+-f publish=true`` therefore runs as the schedule does, through the same test
+gate, environments and ``repository-maintenance`` group, but with the build
+bumped regardless: ``Decide whether to bump`` says so although nothing changed,
+so ``Bump Version``, ``Commit Changes`` and ``Push Changes`` run, ``conda-tag``
+tags the pushed commit, every ``conda-dist`` leg uploads it, the GitHub
+Release steps run, and ``discussion`` posts the report as ``scheduled``. The
+``mode`` job is the one place the input is read: its ``publishes`` output is
+``'true'`` on the schedule, on a ``workflow_run`` and on such a run, and
+``'false'`` on any other manual run, and every step or job that publishes or
+names an environment requires it to be ``'true'``. On any ref but ``main``,
+``publish`` fails ``mode`` with an ``::error``, so the run fails and
+``conda-update`` never starts. The workflow-level concurrency group stays keyed
+on the run, so a manual run never displaces a pending scheduled one.
+No later run repeats a forced bump, so where ``Push Changes`` does not push it
+because ``main`` moved past the checkout or during the push, the run stays
+green having published nothing, and says so in a warning and in its report
+(:issue:`2060`); the remedy is to dispatch Conda Update again with ``publish``
+on ``main``. Where another run already pushed the same build, that run tags and
+uploads it, and a second dispatch is needed only if it did not.
+:file:`tests/project/test_conda_publish_dispatch_2060.py` pins this.
+
 ``gate-only: true`` selects not only ``unit-tests.yml``'s ``gate`` (the full
 suite, on ``latest``) but also ``changelog`` (checks ``CHANGELOG.md`` against its
 source entry) and ``python-versions``, which reads ``gate``'s version (see
@@ -1818,12 +1844,15 @@ it, so the scheduled jobs reading them name it too
 * Conda Update's ``conda-dist`` -- ``environment: anaconda``
 * Vendor Update's ``vendor-update`` -- ``environment: conda-tag``
 
+A manual run names neither, except Conda Update's with ``publish``, whose jobs
+name them as the schedule's do (:issue:`2060`).
+
 Both ``conda-dist`` and the release ``conda`` job have a ``windows-latest`` leg,
 which builds, smoke-tests and publishes the ``win-64`` package (:issue:`2060`).
 It blocks as the other legs do: no step continues on error. Its one step that
 reads ``ANACONDA_TOKEN`` is ``Upload conda packages to Anaconda (Windows)``, the
 Intel leg's anaconda-client call (:issue:`2044`) under ``bash -el``, which skips a
-manual run as every upload does.
+build-only manual run as every upload does.
 :file:`tests/project/test_conda_windows_leg_2060.py` pins the leg.
 
 A required reviewer on ``conda-tag`` or ``anaconda`` would therefore hold every

tests/project/test_build_only_dispatch.py

@@ -11,9 +11,12 @@
   step that pushes, tags, uploads, deploys, posts or reads a secret is
   unreachable on ``workflow_dispatch``. Read conservatively: the step's own
   ``if:``, or its job's, has to hold ``github.event_name != 'workflow_dispatch'``
-  as a top-level ``&&`` conjunct, which makes it false whatever else is true.
-  A job that still runs names no environment, and no run joins a concurrency
-  group a real run waits in.
+  as a top-level ``&&`` conjunct, which makes it false whatever else is true --
+  or, in Conda Update, ``needs.mode.outputs.publishes == 'true'``, which its
+  ``mode`` job writes 'false' on a manual run without ``publish`` (#2060). A
+  job that still runs names no environment, and no run joins a concurrency
+  group a real run waits in. :mod:`tests.project.test_conda_publish_dispatch_2060`
+  covers a manual Conda Update run with ``publish``.
 * :class:`TestAManualRunStillBuilds`: the jobs and steps that build and test
   run on a manual run as they do on the schedule.
 * :class:`TestCreateReleaseIgnoresAManualVendorUpdate`: Create Release, which
@@ -49,6 +52,12 @@
 DISPATCH = 'workflow_dispatch'
 #: The guard each publishing step or job carries, whitespace removed.
 GUARD = "github.event_name!='workflow_dispatch'"
+#: Conda Update's form of it (#2060): its `mode` job's verdict, which is 'false'
+#: on a manual run without `publish`, as test_conda_publish_dispatch_2060.py runs it to show.
+PUBLISH_GUARD = "needs.mode.outputs.publishes=='true'"
+#: Each guard's own text, for the scan that strips them.
+_GUARDS = re.compile(r"(github\.event_name != 'workflow_dispatch'|needs\.mode\.outputs\.publishes == 'true')"
+                     r"(\s*&&\s*)?")
 #: Scheduled workflows with no manual run, and why none is needed.
 EXEMPT = {
     'codeql-analysis.yml': 'runs on every push to main and every pull request',
@@ -103,7 +112,7 @@ def conjuncts(condition: 'object') -> 'Optional[list[str]]':
 
 def guarded(entry: 'dict[str, Any]') -> 'bool':
     """Whether ``entry``'s ``if:`` is false on a manual run, whatever else holds."""
-    return GUARD in (conjuncts(entry['if']) or []) if 'if' in entry else False
+    return bool({GUARD, PUBLISH_GUARD} & set(conjuncts(entry['if']) or [])) if 'if' in entry else False
 
 
 def publishes(entry: 'dict[str, Any]') -> 'bool':
@@ -125,11 +134,32 @@ def exposed(workflow: 'dict[str, Any]') -> 'list[str]':
     return found
 
 
+def mode_context(event_name: 'str') -> 'dict[str, Any]':
+    """``github`` and Conda Update's ``mode`` output on ``event_name``, ``publish`` unset."""
+    publishes = 'false' if event_name == DISPATCH else 'true'
+    return {'github': {'event_name': event_name}, 'needs': {'mode': {'outputs': {'publishes': publishes}}}}
+
+
+def job_group(group: 'str', event_name: 'str', run_id: 'str') -> 'str':
+    """A job's ``concurrency:`` group on ``event_name``, each ``${{ }}`` in it evaluated.
+
+    ``format('<prefix>{0}', github.run_id)`` is read as the string it makes, as
+    the evaluator models no ``format``.
+
+    """
+    context = mode_context(event_name)
+    context['github'] = {**context['github'], 'run_id': run_id, 'ref': 'refs/heads/main'}
+    group = re.sub(r"format\('([^'{]*)\{0\}',\s*github\.run_id\)", lambda match: f"'{match.group(1)}{run_id}'",
+                   group)
+    return re.sub(r'\$\{\{(.*?)\}\}', lambda match: str(evaluate_expression('${{' + match.group(1) + '}}', context)),
+                  group)
+
+
 def dispatch_environment(job: 'dict[str, Any]', event_name: 'str' = DISPATCH) -> 'object':
     value = job.get('environment', '')
     if isinstance(value, dict):
         value = value.get('name', '')
-    return evaluate_expression(value, {'github': {'event_name': event_name}}) if '${{' in value else value
+    return evaluate_expression(value, mode_context(event_name)) if '${{' in value else value
 
 
 class TestEveryScheduledWorkflowCanBeStarted(unittest.TestCase):
@@ -171,7 +201,7 @@ def test_the_scan_sees_each_publisher(self) -> None:
         }
         for name in BUILD_ONLY:
             text = (WORKFLOWS / name).read_text(encoding='utf-8')
-            stripped = re.sub(r"github\.event_name != 'workflow_dispatch'(\s*&&\s*)?", 'true && ', text)
+            stripped = _GUARDS.sub('true && ', text)
             found = exposed(_workflows.loads(stripped))
             with self.subTest(workflow=name):
                 for publisher in expected[name]:
@@ -185,6 +215,9 @@ def test_the_conjunct_reading_is_conservative(self) -> None:
         self.assertFalse(guarded({'if': "github.event_name != 'workflow_dispatch' || a"}))
         self.assertFalse(guarded({'if': "(github.event_name != 'workflow_dispatch' && a)"}))
         self.assertTrue(guarded({'if': "a && github.event_name != 'workflow_dispatch'"}))
+        self.assertTrue(guarded({'if': "a && needs.mode.outputs.publishes == 'true'"}))
+        self.assertFalse(guarded({'if': "needs.mode.outputs.publishes != 'true' && a"}))
+        self.assertFalse(guarded({'if': "needs.mode.outputs.publishes == 'true' || a"}))
         self.assertFalse(guarded({}))
 
     def test_a_running_job_names_no_environment(self) -> None:
@@ -219,8 +252,9 @@ def group(event_name: 'str', run_id: 'str') -> 'str':
                     if guarded(job) or not shared:
                         continue
                     with self.subTest(job=job_id):
-                        self.assertIn(f"github.event_name == '{DISPATCH}' &&", shared)
-                        self.assertIn('github.run_id', shared)
+                        self.assertNotEqual(job_group(shared, DISPATCH, '1'), job_group(shared, DISPATCH, '2'))
+                        self.assertEqual(job_group(shared, 'schedule', '1'), job_group(shared, 'schedule', '2'))
+                        self.assertNotEqual(job_group(shared, DISPATCH, '1'), job_group(shared, 'schedule', '1'))
 
     def test_no_report_is_posted(self) -> None:
         for name in BUILD_ONLY:
@@ -259,8 +293,9 @@ def test_conda_update_hands_conda_dist_a_version(self) -> None:
         version = next(step for step in _workflows.steps('cron-conda.yml', 'conda-update')
                        if step.get('name') == 'Get Version')
         for event_name, expected in ((DISPATCH, True), ('schedule', False)):
-            context = {'github': {'event_name': event_name},
-                       'steps': {'verify-changed-files': {'outputs': {'files_changed': 'false'}}}}
+            context = {**mode_context(event_name),
+                       'steps': {'verify-changed-files': {'outputs': {'files_changed': 'false'}},
+                                 'bump': {'outputs': {'bump': 'false'}}}}
             with self.subTest(event=event_name):
                 self.assertIs(evaluate_condition(version['if'], context), expected)
 
@@ -274,7 +309,7 @@ def test_conda_dist_builds_the_dispatched_commit_with_the_changes(self) -> None:
             name = step.get('id') or step.get('name')
             for os_label in _workflows.job('cron-conda.yml', 'conda-dist')['strategy']['matrix']['os']:
                 for event_name in (DISPATCH, 'schedule'):
-                    context = {'github': {'event_name': event_name}, 'matrix': {'os': os_label},
+                    context = {**mode_context(event_name), 'matrix': {'os': os_label},
                                'steps': {'conda_build': {'outcome': 'success'},
                                          'release': {'outputs': {'published': 'true'}}},
                                STATUS: {'cancelled': False}}

tests/project/test_conda_intel_cli_upload_2044.py

@@ -87,11 +87,13 @@ def legs(name: 'str', job_id: 'str') -> 'list[str]':
 def runs(entry: 'dict[str, Any]', os: 'str', published: 'str' = 'false') -> 'bool':
     """Whether ``entry`` runs on leg ``os``, with ``check_leg`` having found ``published``.
 
-    On the schedule: a manual run of cron-conda.yml uploads nothing on any leg,
-    which tests/project/test_build_only_dispatch.py pins.
+    On the schedule, where cron-conda.yml's `mode` says the run publishes: a
+    build-only manual run uploads nothing on any leg, which
+    tests/project/test_build_only_dispatch.py pins.
 
     """
     context = {'matrix': {'os': os}, 'github': {'event_name': 'schedule'},
+               'needs': {'mode': {'outputs': {'publishes': 'true'}}},
                'steps': {'check_leg': {'outputs': {'PCAPKIT_LEG_EXISTS': published}}}}  # type: dict[str, object]
     return evaluate_condition(str(entry.get('if', 'true')), context)