Automate npm publishing for WordPress packages #37820
Replies: 12 comments 21 replies
-
|
Draft PR: #37751 |
Beta Was this translation helpful? Give feedback.
-
|
The current schedule for npm publishing looks as follows:
It would be great if we could develop a revised strategy where the "feature-freeze" mode in WordPress Core doesn't have any impact on when and how WordPress packages are published to npm. It might require a few changes in how we handle the dist-tags, but maybe we should exercise extreme cases to find something in the middle that works in most cases. The other issue discussed in #35630 is the lack of identifying npm releases tied to a specific WordPress major version or the Gutenberg plugin version. It would be beneficial for plugin authors to be able to install npm packages pinned to those configurations when willing to support ranges of plugins/WordPress core versions. At the moment, we don't even publish npm packages for every Gutenberg plugin release so we might want to address that as well. |
Beta Was this translation helpful? Give feedback.
-
For context, the existing script currently prompts the user to do so manually after it finishes: gutenberg/bin/plugin/commands/packages.js Lines 418 to 424 in 29ec9e0 I think that this part is the easiest to automate; we already have precedent in the build-plugin workflow: gutenberg/.github/workflows/build-plugin-zip.yml Lines 141 to 151 in 29ec9e0 The main task here will be to figure out the correct SHA(s) produced by |
Beta Was this translation helpful? Give feedback.
-
|
Yes, the only challenge is that one of the commits is optional, but they always have the same title so we can use that for matching. |
Beta Was this translation helpful? Give feedback.
-
|
I sorted out cherry-picking all necessary commits to the |
Beta Was this translation helpful? Give feedback.
-
As for the ones that are part of the |
Beta Was this translation helpful? Give feedback.
-
|
Right, it's actually one function Still, there are some steps that are obsolete on CI – you don't need to clone the repository into a temp directory and take care of the cleanup. |
Beta Was this translation helpful? Give feedback.
-
|
I added |
Beta Was this translation helpful? Give feedback.
-
I'm leaning towards changing the workflow trigger from manual (which I've currently put for testing purposes in #37751) to maybe upon release (which is also the trigger for the "Update Changelog and upload Gutenberg plugin to WordPress.org plugin repo" workflow) -- and limit it to publishing RC1, to replicate our currently documented recommendation. |
Beta Was this translation helpful? Give feedback.
-
|
I like that. Having a way to trigger the release on-demand and tied to the Gutenberg plugin RC1 seems perfect. I'd also like to explain why we switched to RC1 as the source for npm publishing. If the publishing happens just after triggering the release process for the plugin, we remove the risk that the changelog files will have merge conflicts when backporting them back to We could also rethink the way CHANGELOGs are maintained but let's keep it out of the scope for now. |
Beta Was this translation helpful? Give feedback.
-
|
I'm experimenting with triggering npm publishing at the time when RC1 for the Gutenberg plugin is released. See #39259. |
Beta Was this translation helpful? Give feedback.
-
This might be the trickiest one, especially in a lerna context. I'm hoping that we'll be able to find a solution that works by "just" storing some kind of token in GH Secrets. |
Beta Was this translation helpful? Give feedback.
-
It looks like it bypasses 2FA when using the token from CI. I also found an example of how it can be configured on CI in semantic-release/npm#277 (comment): - name: Publish
run: |
echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" >> ~/.npmrc
npm publish --tag ${{ steps.deployment.outputs.tag }}
env:
NPM_TOKEN: ${{ secrets.NPM_DEPLOY_KEY }} |
Beta Was this translation helpful? Give feedback.
-
|
I found this excellent article https://dev.to/astagi/publish-to-npm-using-github-actions-23fn which contains all the steps required to set up the token and use it in a GitHub action. I'm experimenting with it in #39259. |
Beta Was this translation helpful? Give feedback.
-
Yeah, we can try to figure out something like that. OTOH, as a first step for automation, I like to replicate the exact behavior of the current script in CI, since there's always a bit of a risk of feature creep (especially if some changes need to be agreed upon -- easier to tackle those in follow-up PRs). Maybe there's some way to automatically determine (through some REST API maybe?) if WP Core is currently in feature-freeze mode. |
Beta Was this translation helpful? Give feedback.
-
|
For the sake of initial implementation, we can completely ignore the struggle with detecting the feature freeze or when a minor version bump has to happen to account for the next major WordPress version. We can focus on the development releases with the |
Beta Was this translation helpful? Give feedback.
-
This makes sense but could be a bit tricky to do, especially if we want our npm releases to follow semver. I think this is best discussed separately from the goal of automating the npm release process :) |
Beta Was this translation helpful? Give feedback.
-
|
What if we get a bit creative and use It should be straightforward in the case of Gutenberg plugin releases. After publishing packages to npm, you can read all production packages and run the following command using the extracted package name and version: npm dist-tag add <pkg>@<version> [<tag>] |
Beta Was this translation helpful? Give feedback.
-
I'm proposing a new strategy in #39259 where the "feature-freeze" mode in WordPress Core isn't taken into account when publishing npm packages with the |
Beta Was this translation helpful? Give feedback.
-
|
I'm fine with changes, just want to mention two things:
|
Beta Was this translation helpful? Give feedback.
-
Every plugin release (RC1) will trigger at least a
As long as those changes are applied as a |
Beta Was this translation helpful? Give feedback.
-
|
I was thinking about the changes to the publishing process based on the feedback in #39259 from @tellthemachines and this is my current thinking. Proposed revised strategy for npm publishingThe Gutenberg repository follows the WordPress SVN repository in the branching strategy for every major WordPress release. In addition to that, it also contains two other special branches that control npm publishing workflows:
Release types and their schedule:
|
Beta Was this translation helpful? Give feedback.
-
In the automated workflow,
It's very important to describe it properly because it's very confusing. My current understanding is that we use |
Beta Was this translation helpful? Give feedback.
-
Thanks for explaining, this works. It creates a difference between WP branches and Gutenberg wp/* branches (not fully synced) but that's fine I think. |
Beta Was this translation helpful? Give feedback.
-
Yes, it's a bit unfortunate, but at the same time it better aligns with the process we have been doing for such a long time. @tellthemachines correctly pointed out that |
Beta Was this translation helpful? Give feedback.
-
We have very different workflows and release cadence between Core and Gutenberg, so I think it's fine to have that difference as long as it's clearly documented. And hopefully at some point Core will evolve to creating the release branch straight after Beta 😄 |
Beta Was this translation helpful? Give feedback.
-
|
I updated #39389 with all the changes discussed above. I hope I was able to cover all the discussion points and it's easier to follow now. |
Beta Was this translation helpful? Give feedback.
-
|
I opened #40927 which automates the process of publishing npm packages targeting the given major WordPress release. |
Beta Was this translation helpful? Give feedback.
-
|
The remaining tasks I can think of at the moment:
|
Beta Was this translation helpful? Give feedback.
-
|
I used the following function to apply async function addDistTagToPackages( { gitWorkingDirectoryPath } ) {
const { dependencies, devDependencies } = require( path.resolve(
gitWorkingDirectoryPath,
'package.json'
) );
const packageJsonFiles = await glob(
path.resolve( gitWorkingDirectoryPath, 'packages/*/package.json' )
);
const operations = packageJsonFiles
.map( ( packageJsonFile ) => require( packageJsonFile ) )
.filter(
( packageJson ) =>
packageJson.private !== true &&
( Object.keys( dependencies ).includes( packageJson.name ) ||
Object.keys( devDependencies ).includes(
packageJson.name
) )
)
.map(
( packageJson ) =>
`npm dist-tag add ${ packageJson.name }@${ packageJson.version } wp-5.7 --otp 123456;`
);
let skip = true;
for ( const operation of operations ) {
if ( operation.includes( 'jest-preset-default' ) ) {
skip = false;
}
if ( skip ) continue;
try {
//console.log( operation );
await command( operation, {
cwd: gitWorkingDirectoryPath,
stdio: 'inherit',
} );
} catch ( error ) {}
}
}We already had WordPress 6.0 covered. With all that it's possible to easily switch between releases WP 5.7-6.0 using |
Beta Was this translation helpful? Give feedback.
-
We have GitHub automation in place for handling the Gutenberg plugin releases.
Prior work:
It would be great to bring the same experience when publishing WordPress packages to npm. It's still a partially manual process that has its quirks when the WordPress trunk branch can be closed or in "feature-freeze" mode as documented in Synchronizing WordPress Trunk.
Let's discuss:
trunkor change the flow in a way where we could auto-approve all CLI prompts?Beta Was this translation helpful? Give feedback.
All reactions