Skip to main content

Fire Arrow Server 1.16.2

· 3 min read
Till Gerken
Till Gerken
CTO

Fire Arrow Server 1.16.2 has been released.

  • (bugfix) Polling-only CarePlan-event subscriptions no longer stall server startup when Azure Queue is not enabled
  • (feature) New fire-arrow.search.reindex-on-startup-when-index-missing setting rebuilds the search index on cold start for deployments using ephemeral storage

Polling-Only Subscriptions Could Stall Server Startup​

CarePlan-event subscriptions that use the polling-only channel (channel://nop) are internal bookkeeping — they let the server track which CarePlans are opted in to task materialization without sending messages to an external endpoint. These subscriptions intentionally carry no payload.

Before 1.16.2, the server's subscription delivery pipeline expected every message-channel subscription to carry a JSON payload. When a channel://nop subscription fired, the deliverer rejected the empty payload and the application's retry mechanism repeated the attempt indefinitely. In deployments that had Azure Queue enabled, a separate interceptor short-circuited nop subscriptions before the deliverer ran, so the problem never surfaced. In deployments without Azure Queue — the more common configuration — nothing intercepted the nop delivery, and the retry storm saturated the delivery thread pool.

The visible effect depended on how long startup took under the added load. On smaller instances (two cores or fewer), the retry overhead could push startup past the container health-check deadline, causing the platform to kill the process before the full-text search index was created. On the next cold start the Lucene index directory was missing entirely, and full-text search returned no results until an external reindex was triggered.

Starting with 1.16.2, a dedicated interceptor drains nop subscriptions on all deployments, regardless of whether Azure Queue is enabled. The interceptor suppresses delivery for any subscription whose endpoint is channel://nop or whose message payload is absent, so the deliverer is never invoked and no retries occur. Subscription matching and CarePlan task materialization are unaffected — the server still records the nop match internally and materializes Tasks as before. No configuration change is required.

Automatic Search Reindex on Cold Start​

Deployments that keep the Lucene search index on ephemeral storage — for example Azure App Service's /tmp volume — lose the index on every cold start. Until the index is rebuilt, full-text search and search parameters backed by Lucene return empty results. Rebuilding previously required an operator to trigger a manual $reindex after noticing the gap.

1.16.2 adds an opt-in setting that automates this:

fire-arrow:
  search:
    reindex-on-startup-when-index-missing: true   # default: false

When enabled, the server inspects the Lucene index directory during startup. If the directory is missing or contains no indexed data (a freshly committed but empty index), the server marks all resources for reindexing. The reindex runs asynchronously in the background — it does not block startup or delay readiness probes — and HAPI's scheduled reindex job picks up the marked resources within seconds.

The check is conservative: an index that already contains segment data is left alone, so a normal restart with a populated index does not trigger a redundant reindex. The setting defaults to false; enable it only on deployments where the index storage is known to be ephemeral.