[Feature Request] Support Kafka DescribeCluster API (API Key 60, KIP-700)

[DCjanus]

Kafka introduced the DescribeCluster API (API Key 60) in KIP-700, starting from Kafka 2.8.0.
The protocol definition is documented in the official spec

This API exposes cluster-level metadata directly and avoids the overhead of issuing a full MetadataRequest.
The Java client (AdminClient) already uses this API when supported by the broker, as seen in the related PR:
apache/kafka#9905

Current Situation in Sarama

ClusterAdmin.DescribeCluster() currently:

• Sends a full MetadataRequest (including all topics)
• Extracts only limited cluster-level fields from the response

This approach is heavier than necessary and scales poorly on clusters with many topics.

Proposal

• Add protocol support for API Key 60 (DescribeCluster)
• Make DescribeCluster() prefer this API when broker support is advertised
• Fallback to MetadataRequest automatically based on ApiVersions negotiation

Compatibility

This change is naturally backward-compatible.
Older brokers simply do not advertise API Key 60, so Sarama can continue using existing logic without behavior changes.

---

Supporting DescribeCluster allows Sarama to align with Kafka’s protocol evolution, match the Java client’s behavior, and reduce unnecessary metadata overhead.

If this feature request is acceptable, I’m willing to help with the implementation when time permits.
Anyone interested is also welcome to take this issue directly.

[DCjanus]

Highlighting DescribeCluster (API key 60) version differences to streamline compatibility planning:

• v0 (KIP-700, Kafka 2.8.0): initial DescribeCluster definition, only IncludeClusterAuthorizedOperations plus the minimal response fields.
• v1 (KIP-919, Kafka 3.7.0): request adds EndpointType (default 1=brokers, 2=controllers); response echoes EndpointType and allows MISMATCHED_ENDPOINT_TYPE / UNSUPPORTED_ENDPOINT_TYPE errors.
• v2 (KIP-1073, Kafka 4.0.0): request adds IncludeFencedBrokers; each broker entry in the response gains IsFenced to mark fenced nodes in KRaft clusters.

Implementation-wise, we can reuse the existing Kafka version gating and prefer v2 → v1 → v0 before falling back to MetadataRequest, so every broker generation gets the richest data it understands.

Ref:

• DescribeClusterRequest
• DescribeClusterResponse

[DCjanus]

Current status:

1. PR perf(admin): modernize DescribeCluster RPC handling #3390 already calls DescribeCluster when the cluster supports it.
2. For compatibility we still return the legacy ([]*Broker, controllerID, error) tuple, so DescribeCluster-only fields (e.g. IsFenced, ClusterAuthorizedOperations) are discarded; see the TODO in convertDescribeClusterBrokers.

Next step:

• Extend Broker with optional fields (plus accessors) so callers can read the richer DescribeCluster metadata without changing the public signature. This keeps downstream breakage minimal.

Request for feedback:

• I am not yet deeply familiar with the broker/connection code, so I cannot promise this is the optimal long-term shape. If you see a better way to surface these fields, please chime in and I can adjust the follow-up patch accordingly.