Have to destroy and re-deploy pipeline stack after replacing Github token - amazon-web-services

Question

I uses CDK to create and deploy pipeline(with AWS CodePipeline https://aws.amazon.com/codepipeline/), and today Source stage stopped working, complaining about insufficient permission, which turns out to be that the Github token is expired. Error:
"Could not access the GitHub repository: "pandaWebsite". The access token might be invalid or has been revoked. Edit the pipeline to reconnect with GitHub."
So I re-genereate the Github token, and updated it in AWS Secrets Manager. And click "Retry" button in pipeline, and it still failed. Eventually I have to run cdk destroy to destroy the pipeline, and run cdk deploy to re-deploy the pipeline, and then it works.
My question is, why I have to destroy and re-deploy the pipeline? I was expecting that once I updated the token in Secrets Manager, it should just work.
More context
AWS Secrets Manager is where I stored the Github token, and my CDK code fetch from it. See code here:
// Add Source stage to fetch code from GitHub repository.
private addSourceStage(
pipeline: codepipeline.Pipeline,
sourceCode: codepipeline.Artifact
) {
pipeline.addStage({
stageName: "Source",
actions: [
new codepipeline_actions.GitHubSourceAction({
actionName: "Checkout",
owner: "yangliu",
repo: "pandaWebsite",
branch: "main",
// read the value from Secrets Manager
oauthToken: CDK.SecretValue.secretsManager(
"github-token"
),
output: sourceCode,
trigger: codepipeline_actions.GitHubTrigger.WEBHOOK,
}),
],
});
}