[XTA-15079] Add SPOG ?o= routing support and fix final-flush auth on close

[samikshya-db]

Summary

Ports the SPOG (Single Panel of Glass) routing fix from
databricks/databricks-jdbc#1316
to the Node.js driver, and fixes a related close-ordering bug uncovered
while validating the port against a real SPOG host.

Change 1: SPOG header injection

SPOG replaces workspace-specific hostnames with account-level vanity URLs.
When httpPath carries ?o=<workspaceId>, endpoints that don't include
the workspace in their URL path need the workspace conveyed via the
x-databricks-org-id header instead.

• Parse ?o=<digits> out of httpPath in DBSQLClient.connect() and
  stash the org-id as x-databricks-org-id on a new
  ClientConfig.customHeaders field. A user-supplied customHeaders
  entry (case-insensitively keyed) takes precedence.
• DatabricksTelemetryExporter spreads config.customHeaders into the
  outgoing POST headers; auth headers still win on collision.
• FeatureFlagCache does the same for the feature-flag GET.
• ConnectionOptions.customHeaders exposed so callers can also inject
  their own out-of-band headers.

OAuth / OIDC token requests use openid-client's private HTTP transport
and never see customHeaders, matching JDBC's exemption.

Not needed vs JDBC: this driver passes options.path through
unmodified (no split-on-= parser), uses Thrift only (no SEA JSON body
warehouse-ID extraction), and exposes no Volume API surface.

Change 2: Close-ordering fix

While verifying Change 1 against a real SPOG warehouse, the final
client.close() flush emitted:

[warn] Telemetry: Authorization header missing — metrics will be dropped
[debug] Telemetry export auth failure: Telemetry export: missing Authorization header

Root cause: TelemetryClientProvider.releaseClient called
unregisterContext before client.close(). On the last refcount, that
emptied the FIFO of (context, authProvider) pairs before the final
flush, so the exporter's getAuthProvider() walk returned undefined and
the batch was dropped.

Fix: skip the unregister on the last release so the FIFO snapshot
survives through close()'s final flush. Multi-refcount path is
unchanged. No leak — the TelemetryClient is unreachable as soon as
releaseClient returns, and the retained (context, authProvider)
pair dies with it.

Live verification

Ran against dogfood-spog.staging.azuredatabricks.net warehouse
ad6f9b54e6593746?o=7064161269814046:

customHeaders={"x-databricks-org-id":"7064161269814046"}     ← SPOG header injected
Feature flag enableTelemetryForNodeJs: true                  ← header propagated via SPOG
Result: [{"v":1}]                                            ← SELECT 1 worked
Successfully exported 3 telemetry metrics                    ← final flush no longer drops

Test plan

Unit tests added (all passing locally):

• buildCustomHeaders parses ?o= into x-databricks-org-id
• No header when ?o= is absent and no user-supplied customHeaders
• User-supplied customHeaders is preserved alongside parsed org-id
• User-supplied x-databricks-org-id (any case) wins over ?o=
• Non-numeric o=<value> does not inject a header
• End-to-end DBSQLClient.connect() populates
  config.customHeaders['x-databricks-org-id'] from the path
• DatabricksTelemetryExporter attaches config.customHeaders to
  the POST headers
• Auth headers still override customHeaders on key collision
• No header is attached when config.customHeaders is empty
• FeatureFlagCache.fetchFeatureFlag attaches customHeaders to
  the GET headers
• releaseClient keeps the context registered through close() on
  last refcount
• releaseClient drops the context immediately when other
  refcounts remain

This pull request and its description were written by Isaac.

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

[msrathore-db]

Have we tested this for auth and telemetry?

[samikshya-db]

yes, added test details in description

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

[msrathore-db]

[M1] buildCustomHeaders(options) reads this.httpPath, not options.path — misleading signature

The method takes options: ConnectionOptions but only reads options.customHeaders. The workspace ID is sourced from this.httpPath via this.extractWorkspaceId(). The unit test at tests/unit/DBSQLClient.test.ts:803-805 makes this concrete — it sets (client as any).httpPath separately from the path in the options literal.

This creates a hidden ordering dependency at the call site (line 588): this.httpPath = options.path must run before this.config.customHeaders = this.buildCustomHeaders(options). A future refactor that swaps those two lines silently breaks SPOG with no test failure — extractWorkspaceId() returns undefined, no header is injected.

Suggested — make it pure / static so the dependency is explicit:

private static buildCustomHeaders(
  httpPath: string | undefined,
  userHeaders: Record<string, string> | undefined,
): Record<string, string> | undefined {
  const merged: Record<string, string> = { ...(userHeaders ?? {}) };
  const hasOrgIdAlready = Object.keys(merged).some((k) => k.toLowerCase() === 'x-databricks-org-id');
  if (!hasOrgIdAlready) {
    const orgId = DBSQLClient.extractWorkspaceId(httpPath); // also static
    if (orgId) merged['x-databricks-org-id'] = orgId;
  }
  return Object.keys(merged).length > 0 ? merged : undefined;
}

Call site at line 588: this.config.customHeaders = DBSQLClient.buildCustomHeaders(options.path, options.customHeaders);

Bonus: the (client as any).httpPath = ... test workaround disappears.

This pull request and its description were written by Isaac.

[msrathore-db]

Please verify all purpose cluster url is working and also M2M is working

[msrathore-db]

[H2] No log signal when SPOG header is injected, dropped, or overridden — JDBC parity gap

buildCustomHeaders and extractWorkspaceId emit zero log statements at any level. When SPOG routing misbehaves in prod, an oncall engineer has nothing to grep for. JDBC at DatabricksConnectionContext.java:1349-1364 logs all three branches: success injection, user-supplied-wins, and non-numeric/malformed path. The PR description's "live verification" confirms one path worked once — but a future regression in extraction or a non-numeric ?o= on a customer host would be invisible.

Suggested:

private buildCustomHeaders(options: ConnectionOptions): Record<string, string> | undefined {
  const merged: Record<string, string> = { ...(options.customHeaders ?? {}) };
  const hasOrgIdAlready = Object.keys(merged).some((k) => k.toLowerCase() === 'x-databricks-org-id');
  if (hasOrgIdAlready) {
    this.logger.log(LogLevel.debug, 'SPOG: x-databricks-org-id already set by caller, not extracting from httpPath');
  } else {
    const orgId = this.extractWorkspaceId();
    if (orgId) {
      merged['x-databricks-org-id'] = orgId;
      this.logger.log(LogLevel.debug, `SPOG: injecting x-databricks-org-id=${orgId} (extracted from ?o= in httpPath)`);
    }
  }
  return Object.keys(merged).length > 0 ? merged : undefined;
}

Also: at extractWorkspaceId (line 329), a non-numeric ?o=tenant_xyz is silently dropped — add a LogLevel.warn so customers with malformed paths get a signal.

This pull request and its description were written by Isaac.

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).