[improve][client] PIP-234: Support sharing the memory limit controller across multiple isolated Pulsar client instances

[oneby-wang]

Fixes #25212

Motivation

#19074 enables sharing of Pulsar client resources across multiple isolated Pulsar client instances, but this solution lacks limiting the total amount of memory across all of the Pulsar client instances. This PR introduces a shared MemoryLimitController solution for PIP-234.

Modifications

1. Modify MemoryLimitController to support registering multiple trigger callbacks and deregistering the callback when the client instance gets closed.
2. Add shared MemoryLimitController in PulsarClientSharedResources and PulsarClientSharedResourcesBuilder, then apply the shared MemoryLimitController to PulsarClientImplBuilder.
3. Add tests to verify the code change.

Verifying this change

• Make sure that the change passes the CI checks.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

PR in forked repository: oneby-wang#29

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR adds support for sharing a single MemoryLimitController across multiple isolated Pulsar client instances (PIP-234), including multi-trigger registration/deregistration and tests that validate shared-memory behavior for producers and consumers.

Changes:

• Extend MemoryLimitController to support multiple triggers and allow deregistration on client shutdown.
• Add shared-resource support for a shared MemoryLimitController (and associated OpenTelemetry wiring/metrics scaffolding) via PulsarClientSharedResources.
• Add integration tests validating shared memory limiting across multiple clients.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
pulsar-client/src/main/java/org/apache/pulsar/client/impl/PulsarClientSharedResourcesImpl.java	Instantiates and applies shared memory limit controller; optionally creates shared memory buffer metrics
pulsar-client/src/main/java/org/apache/pulsar/client/impl/PulsarClientSharedResourcesBuilderImpl.java	Adds builder config objects and builder methods for shared memory limit and OpenTelemetry
pulsar-client/src/main/java/org/apache/pulsar/client/impl/PulsarClientResourcesConfigurer.java	Adds factory methods to create shared MemoryLimitController and InstrumentProvider
pulsar-client/src/main/java/org/apache/pulsar/client/impl/PulsarClientImpl.java	Accepts injected shared MemoryLimitController, registers trigger, and deregisters on shutdown
pulsar-client/src/main/java/org/apache/pulsar/client/impl/MemoryLimitController.java	Supports multiple trigger callbacks via CopyOnWriteArraySet and exposes register/deregister
pulsar-client-api/src/main/java/org/apache/pulsar/client/api/PulsarClientSharedResourcesBuilder.java	Adds public builder methods for shared memory limit controller and OpenTelemetry
pulsar-client-api/src/main/java/org/apache/pulsar/client/api/PulsarClientSharedResources.java	Introduces new shared resource types: MemoryLimitController, OpenTelemetry
pulsar-client-api/src/main/java/org/apache/pulsar/client/api/OpenTelemetryConfig.java	New public config interface for OpenTelemetry configuration
pulsar-client-api/src/main/java/org/apache/pulsar/client/api/MemoryLimitConfig.java	New public config interface for shared memory limit configuration
pulsar-broker/src/test/java/org/apache/pulsar/client/impl/PulsarTestClient.java	Updates constructor call for new PulsarClientImpl builder param
pulsar-broker/src/test/java/org/apache/pulsar/client/impl/ProducerMemoryLimitTest.java	Adds test validating producers share a memory limit controller across clients
pulsar-broker/src/test/java/org/apache/pulsar/client/impl/ConsumerMemoryLimitTest.java	Adds test validating consumers share a memory limit controller across clients