ci(publish): fix Soda PyPI --skip-existing + single approval gate#2759
Merged
Conversation
Two fixes to publish.yaml from the first 4.14.0 run: 1. Drop the Soda PyPI (pypi.cloud.soda.io) upload. It errored on `--skip-existing` (unsupported by that index), and it shouldn't be a target: that index serves the soda-library 1.x line that scan-launcher builds from, and soda-library's connectors share package names with soda-core's (soda-postgres, soda-snowflake, ...). Publishing soda-core's 4.x there collides with soda-library's identically-named 1.x packages. Nothing consumes soda-core clean releases from cloud (launchers pull soda-core dev builds from the dev platform index). soda-core clean releases belong on public PyPI only. Removes the now-unused AWS secrets-manager fetch too. 2. Collapse the 14 environment approvals into one. The matrix publish job referenced `production-release` directly, so each of the 14 legs raised its own approval (14 emails). A tiny `approve` gate job now carries the environment; the matrix depends on it and drops the environment. Safe because PYPI_API_TOKEN is a repo-level secret. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Niels-b
approved these changes
Jun 16, 2026
Reverses the "drop Soda PyPI" half of this branch — soda-core clean releases are still needed on pypi.cloud.soda.io. The original failure was not that the registry is wrong, but that #2758 passed --skip-existing to every leg, and pypi.cloud.soda.io (devpi) rejects that flag: UnsupportedConfiguration: 'https://pypi.cloud.soda.io' does not have support for the following features: --skip-existing Fix per-registry instead of removing the leg: - public PyPI -> keeps `twine upload --skip-existing` (supported) - pypi.cloud.soda.io -> upload plain; capture output and treat an "already exists" / 409 rejection as success, so re-runs and backfills stay idempotent Keeps this branch's single-approval gate (one approval email per run, not one per matrix leg). Restores the AWS secrets fetch the Soda leg needs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
paulteehan
changed the title
|
Niels-b
approved these changes
Jun 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #2758 — two fixes surfaced by the first v4.14.0 publish run.
1. Fix the Soda PyPI (
pypi.cloud.soda.io) upload — don't drop itThe run failed on the Soda leg:
The registry wasn't the problem — the flag was. #2758 added
--skip-existingto every upload leg, but
pypi.cloud.soda.iois a devpi index that rejects it.The old
release.yamlalways uploaded to Soda PyPI plain (no--skip-existing),which is why it worked.
soda-core clean releases are still needed on Soda PyPI, so we keep the leg and fix
idempotency per-registry:
twine upload --skip-existing(supported)."already exists" / 409 rejection as success, so re-runs and backfills stay safe.
2. Collapse 14 approval emails into one
The matrix
publishjob referencedenvironment: production-releasedirectly, soeach of the 14 matrix legs raised its own deployment approval → 14 emails per
release. A tiny
approvegate job now carries the environment; the matrixneeds:it and no longer references the environment itself. Safe becausePYPI_API_TOKENis a repo-level secret, so the matrix keeps secret access.Verification
resolve→approve(env) →publish(matrix, no env).--skip-existing(devpi-compatible) and toleratesduplicate-version rejections.
--skip-existingon public PyPI + the duplicate-tolerant Soda leg make re-runningon an already-published tag (e.g.
v4.14.0) a safe no-op.🤖 Generated with Claude Code