kafka: Add topic partition size metric

[piochelepiotr]

Summary

• Adds kafka.log.partition.size metric from kafka.log:type=Log,name=Size,topic=<topic>,partition=<partition> JMX bean
• Reports the on-disk size in bytes of each topic partition log, tagged with topic, partition, and kafka_cluster_id
• Collected per broker — includes both leader and follower replicas

Motivation

Topic/partition disk usage is useful for capacity planning and identifying hot partitions. This data is already exposed via JMX but wasn't collected by the integration.

Test plan

• Deploy against a Kafka cluster with topics and data
• Verify kafka.log.partition.size is emitted with correct topic and partition tags
• Confirm no impact on existing metrics

🤖 Generated with Claude Code

[datadog-datadog-prod-us1]

⚠️ Tests

✨ Fix all issues with BitsAI or with Cursor

⚠️ Warnings

🧪 1 Test failed

❄️ Known flaky: test from test_check.py     (Fix with Cursor)

Needed at least 1 candidates for 'kafka.log.partition.size', got 0
Expected:
        MetricStub(name='kafka.log.partition.size', type=None, value=None, tags=None, hostname=None, device=None, flush_first_value=None)
Difference to closest:
        Expected name: kafka.log.partition.size
        Found kafka.replication.partition_count

Similar submitted:
Score   Most similar
0.63    MetricStub(name='kafka.replication.partition_count', type=0, value=0.0, tags=['dd.internal.jmx_check_name:kafka', 'instance:kafka-localhost-9999', 'jmx_domain:kafka.server', 'kafka_cluster_id:va-p7g4MSKuzXctH37kXmA', 'name:PartitionCount', 'type:ReplicaManager'], hostname='default', device=None, flush_first_value=False)
...

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: eae54a3 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.03%. Comparing base (3a624a5) to head (eae54a3).

Additional details and impacted files

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.