Repository navigation
Conversation
The instrumentation emitted the pre-1.17 messaging attributes unconditionally and passed a hardcoded 1.11.0 schema URL, so OTEL_SEMCONV_STABILITY_OPT_IN had no effect on it. Read the MESSAGING opt-in mode and pass it to the producer and consumer wrappers. The default mode keeps emitting messaging.destination, messaging.kafka.partition and messaging.url unchanged. The messaging mode emits messaging.destination.name, messaging.destination.partition.id, server.address, server.port, messaging.operation.name and messaging.operation.type instead, and messaging/dup emits both generations. messaging.system is emitted in every mode. messaging.url carries the whole bootstrap_servers value JSON-encoded. The stable conventions define server.address as one host and server.port as one port, so the host and the port of the first entry are parsed out of it rather than the JSON being copied across, the way opentelemetry-instrumentation-confluent-kafka already does it. An entry naming no port, or a port that is not a number, leaves server.port unset instead of recording a wrong one. The schema URL now follows the mode in force rather than being a literal, so a consumer is told which generation of attributes it received. Refs open-telemetry#3495
Instrument a producer and a consumer under each of the three modes and assert the exact attribute names, the value types and the schema URL the span carries. messaging.destination.partition.id is a string while messaging.kafka.partition was an integer, so both are checked.
Describe the messaging opt-in values the instrumentation now reads, and give the mapping from each old attribute to the stable one that replaces it.
Pull request dashboard statusWaiting on reviewers · refreshed 2026-11-10 19:27 UTC Review the latest changes. Status above doesn't look right?
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
opentelemetry-instrumentation-kafka-pythonemits the pre-1.17 messaging attributes unconditionally and passeshttps://opentelemetry.io/schemas/1.11.0toget_traceras a literal, soOTEL_SEMCONV_STABILITY_OPT_INhas no effect on it. TheMESSAGINGsignal type and themessaging/messaging/dupmodes already exist inopentelemetry-instrumentation/src/opentelemetry/instrumentation/_semconv.py; this wires the package up to them.KafkaInstrumentor._instrumentresolves theMESSAGINGmode once and passes it to_wrap_sendand_wrap_next,_get_attributesbuilds the attribute dict for the mode in force, and the schema URL comes from_get_schema_url_for_signal_typesinstead of a literal.https://opentelemetry.io/schemas/1.11.0messaginghttps://opentelemetry.io/schemas/1.44.0messaging/duphttps://opentelemetry.io/schemas/1.44.0The old and stable attributes correspond as follows:
messaging.destinationmessaging.destination.namemessaging.kafka.partition(int)messaging.destination.partition.id(str)messaging.urlserver.addressandserver.portmessaging.urlcarries the wholebootstrap_serversvalue JSON-encoded. The stable conventions defineserver.addressas one host andserver.portas one port, so_bootstrap_servers_attributesparses the host and the port out of the first entry rather than copying the JSON across. It handles the spellings kafka-python accepts (onehost:portentry, a comma-separated string of them, a list of them), strips a leading security protocol as inSASL_SSL://broker:9093, and keeps a bracketed or bare IPv6 literal whole rather than splitting it on its last colon. This follows_set_bootstrap_servers_attributesinopentelemetry-instrumentation-confluent-kafka.messaging.systemis emitted in every mode, since the key did not change. The stable modes also addmessaging.operation.nameandmessaging.operation.type, which the old conventions had no equivalent of.Default behaviour is unchanged. The four
TestKafkaSamplingtests added in #5301 assert the exact attribute dict handed to the sampler and pass without modification to their expectations, which is the check that the default path did not move.Refs #3495
Decisions worth overturning
Two choices in
_bootstrap_servers_attributesfollow whatopentelemetry-instrumentation-confluent-kafkaalready does. Both are easy to reverse.bootstrap_serversentry is reported.server.addressis singular, and a bootstrap list names candidates rather than the broker actually used.server.portis left out when the first entry names no port, as with kafka-python's own default of the bare stringlocalhost. kafka-python does substitute 9092 in that case, so emittingserver.port = 9092would describe the connection accurately. It is left out because that value comes from a library default rather than from anything the user configured.Gaps
{topic} sendand{topic} receivein every mode, matchingopentelemetry-instrumentation-aiokafkaandopentelemetry-instrumentation-confluent-kafka. The stable conventions want{operation} {destination}. That is a separate behaviour change.messaging.client.idandmessaging.consumer.group.nameare not emitted. Adding them is new data rather than a migration of existing data.aio-pika,pika,confluent-kafka,boto3sqs,celery,botocore) need the same treatment, one PR each.opentelemetry-instrumentation-aiokafkais the inverse case: it emits the stable attributes unconditionally with no way to ask for the old ones. Not addressed here.Type of change
Checklist:
tox -e precommittox -e typecheckNotes on the checklist:
ruff0.16.1 (check and format), the version pinned in.pre-commit-config.yaml, andpylint4.0.5 against.pylintrcwere run directly oninstrumentation/opentelemetry-instrumentation-kafka-pythonrather than throughtox -e precommit. Both are clean; pylint scores 10.00/10.tox -e typecheckwas not run.instrumentation/opentelemetry-instrumentation-kafka-pythonis not in theincludelist of[tool.pyright]inpyproject.toml, so the environment does not cover this package.test-requirements.oldest.txt(kafka-python 2.3.2, Python 3.10) andtest-requirements.latest.txt(kafka-python 3.0.8, Python 3.12), with the core packages installed fromopentelemetry-python@mainthe waytox.inidoes. 32 passed in both.