-
Notifications
You must be signed in to change notification settings - Fork 763
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] maxSurge behaves differently during updates #1741
Comments
It's a bit strange. |
one possible explanation is that after step 2, one new pod get ready faster than the other two, so kruise will start step 3 but only creating one replicas since maxSurge is 3 |
to facilitate the process display, I reduced replicas to 5
NAME READY STATUS RESTARTS AGE NAME READY STATUS RESTARTS AGE NAME READY STATUS RESTARTS AGE NAME READY STATUS RESTARTS AGE NAME READY STATUS RESTARTS AGE NAME READY STATUS RESTARTS AGE NAME READY STATUS RESTARTS AGE |
the last 2 pods were creating and ready one by one, and it's easy to reappear when initialDelaySeconds is a little bit longger ( for example, 10s) |
I0913 09:25:21.115721 1 cloneset_sync_utils.go:118] Calculate diffs for CloneSet project-123/nginx-clone-lossy-test-123, replicas=5, partition=0, maxSurge=3, maxUnavailable=0, allPods=6, newRevisionPods=3, newRevisionActivePods=3, oldRevisionPods=3, oldRevisionActivePods=3, unavailableNewRevisionCount=1, unavailableOldRevisionCount=0, preDeletingNewRevisionCount=0, preDeletingOldRevisionCount=0, toDeleteNewRevisionCount=0, toDeleteOldRevisionCount=0. Result: {scaleUpNum:1 scaleUpNumOldRevision:0 scaleDownNum:0 scaleDownNumOldRevision:0 scaleUpLimit:1 deleteReadyLimit:0 useSurge:2 useSurgeOldRevision:0 updateNum:2 updateMaxUnavailable:1} I0913 09:25:21.135176 1 event.go:282] Event(v1.ObjectReference{Kind:"CloneSet", Namespace:"project-123", Name:"nginx-clone-lossy-test-123", UID:"6c530a5f-3f29-4716-ab95-a7a9d5806e09", APIVersion:"apps.kruise.io/v1alpha1", ResourceVersion:"30661475", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' succeed to create pod nginx-clone-lossy-test-123-5n2c9 I0913 09:25:21.157225 1 cloneset_sync_utils.go:118] Calculate diffs for CloneSet project-123/nginx-clone-lossy-test-123, replicas=5, partition=0, maxSurge=3, maxUnavailable=0, allPods=7, newRevisionPods=4, newRevisionActivePods=4, oldRevisionPods=3, oldRevisionActivePods=3, unavailableNewRevisionCount=2, unavailableOldRevisionCount=0, preDeletingNewRevisionCount=0, preDeletingOldRevisionCount=0, toDeleteNewRevisionCount=0, toDeleteOldRevisionCount=0. Result: {scaleUpNum:0 scaleUpNumOldRevision:0 scaleDownNum:1 scaleDownNumOldRevision:3 scaleUpLimit:0 deleteReadyLimit:0 useSurge:1 useSurgeOldRevision:0 updateNum:1 updateMaxUnavailable:2} |
after nginx-clone-lossy-test-123-5n2c9 created, useSurge was set to 1, then pod was creating and ready one by one |
@ABNER-1 We use maxSurge as update speed control, and our users have questions about this. Can you try to reproduce and analyze this problem ? |
What happened:
I tested cloneset with 8 replicas and updateStrategy below:
updateStrategy:
maxSurge: 3
maxUnavailable: 0
partition: 0
type: ReCreate
I thought the update should be divided into three batches(3, 3, 2),which should update as below:
But actually, update was divided into four batches(3, 3, 1, 1),the last two old pods was updated one by one
What you expected to happen:
Each batch of updates should be as many as possible without exceeding maxSurge.
How to reproduce it (as minimally and precisely as possible):
updateStrategy:
maxSurge: 3
maxUnavailable: 0
partition: 0
type: ReCreate
Anything else we need to know?:
Environment:
kubectl version
): 1.23.5The text was updated successfully, but these errors were encountered: