Skip to main content

Fire Arrow Server 2.6.7

· One min read
Till Gerken
Till Gerken
CTO

Fire Arrow Server 2.6.7 has been released.

  • (bugfix) Subscription delivery no longer stops across a cluster when leadership changes hands

Bug Fixes​

Subscription delivery stopped across a cluster​

In cluster mode, the leader delivers subscription notifications. When leadership moved while another instance had not finished joining the cluster, for example one just added by autoscaling, no instance delivered notifications anymore. Delivery stopped for every subscription until an instance was restarted, and nothing was logged. Only deployments running more than one instance were affected.

Starting with 2.6.7, if notifications wait longer than 30 seconds and the leader is not delivering them, another instance takes over. Around such a takeover, a notification can occasionally arrive twice. None are lost, so consumers should tolerate an occasional duplicate. No configuration change is required.

The new metric fire_arrow_outbox_fallback_drains increases whenever a takeover happens. Alert on any increase, because it means the leader stopped delivering.