How can I send AVRO messages to a Kafka topic hosted in Confluent Cloud while utilizing OAuth for both authorization and authentication within Neoload?

Hi all,

I'm attempting to produce AVRO messages to my Kafka topic in Confluent Cloud using the Neoload Kafka send and receive message element, but I encountered an issue where I was unable to produce.

I have two topics in Confluent Cloud, one accepting messages as strings and the other as AVRO. I've configured my Kafka send message element with all the relevant details (provided below), and for authorization and authentication, I'm using OAuth. I can successfully produce string messages to my string topic, but I'm unable to do the same for the AVRO topic. However, I also have an API key that can act as a substitute for OAuth, and with the API key, I can produce AVRO messages to the topic without any issue.

In order to produce AVRO messages, we need to specify the schema in the configuration, which I did under the parameter value.schema. Despite this, I'm still unable to produce using OAuth. The error message I'm receiving is “Sending failed: User is denied operation Read on Subject: com.xyxyxyx.domain.boiler_plate.model.UserInfo; error code: 40301”.

According to my developer, a crucial aspect regarding our Kafka topic is when sending AVRO messages to Kafka using OAuth authentication and utilizing the "io.confluent.kafka.serializers.KafkaAvroSerializer" for serialization, it's essential to ensure compatibility with version "7.4.0" of the serializer. Earlier versions may have limitations that could affect serialization. To guarantee seamless integration and proper serialization when sending messages to Kafka topics, we must explicitly specify version "7.4.0" of the serializer in our configuration. This ensures that our system operates smoothly and remains compatible with the required dependencies.

If this is the case, how can we pass this version along with our configuration in Neoload? If not, what could be the issue? I have all the necessary read and write access to the topic, and using IntelliJ with the same configuration details, I could successfully send AVRO messages to the topic without any issue. Only in Neoload am I facing this issue.



value : { "usernames": "Neoload_AVRO", "age": 18, "phone": "1498376382", "street": "AAAAAA", "city": "Tamilnadu", "country": "India" }
security.protocol: SASL_SSL
auto.register.schemas: false
use.latest.version: true
sasl.mechanism: OAUTHBEARER
sasl.jaas.config: required clientId=' ' scope='produce' clientSecret=' ' extension_logicalCluster=' ' extension_identityPoolId=' ';
bearer.auth.credentials.source: SASL_OAUTHBEARER_INHERIT
key.serializer: org.apache.kafka.common.serialization.StringSerializer
value.serializer: io.confluent.kafka.serializers.KafkaAvroSerializer
specific.avro.reader: true


Hope I have given you all the necessary information. Thanks in advance

0 replies

Be the first to reply!