Skip to content

Fix bug when deploying new routing table#6249

Open
nadav-govari wants to merge 2 commits intomainfrom
nadav/fix
Open

Fix bug when deploying new routing table#6249
nadav-govari wants to merge 2 commits intomainfrom
nadav/fix

Conversation

@nadav-govari
Copy link
Copy Markdown
Collaborator

Description

When deploying the new IngesterStatus broadcast, old ingesters dont broadcast their status, which causes the control plane running the new code to only see the ingesters running the new code as Ready. This causes all of the shards on the cluster to be placed on only new ingesters, which breaks the cluster. This self-resolves once the deployment is complete.

This change results in the same behavior as today - if a node doesn't broadcast an ingester status, we assume it's Ready, which is what we did before. This should fix deploys.

How was this PR tested?

Tested that ingester status unwraps correctly and on a test cluster.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants