Serde Issue with Custom Schema Registry Naming

[PabloLec]

Hello,

I have a problem with Serde, most likely related to the names of my Schema Registry.
My schema registry is well configured in the Kafka UI, it shows up correctly on the left side bar, and it's also visible under "Value Serde" on the topic screen.

However, it doesn't seem to be working; I get a tooltip saying "Fallback serde was used", but there's no error in the console.

Unfortunately, my topics aren't using the TopicRecordNameStrategy, and it's not feasible to modify them.

In practice, for instance, I have the topic "contract" and its corresponding subject "avrocontractevent". The naming format is thus "avro[TOPIC]event". For your information, my schemas are correct and are being used by applications.

My configuration is as follows:

  kafka:
    clusters:
      -
        name: local
        bootstrapServers: kafka-broker:9092
        schemaRegistry: http://kafka-schema-registry:8081
        schemaNameTemplate: "avro%sevent"

I tried using schemaNameTemplate, but to no avail. Do you see any issues with this configuration, or have any leads to explore?

Thank you for your assistance.

PS: I'm using docker image provectuslabs/kafka-ui:latest

[Haarolean]

Hi, you can specify schemaNameTemplate either on cluster level or serde level:
serde:

            kafka.clusters.0.serde.4.properties.keySchemaNameTemplate: "%s-key"
            kafka.clusters.0.serde.4.properties.schemaNameTemplate: "%s-value"

cluster:

kafka:
  clusters:
    -
      name: ...
      schemaRegistryKeySchemaNameTemplate: xxx
      schemaRegistrySchemaNameTemplate: yyy

In your case you've tried to use serde property on cluster property level.

[Haarolean]

@PabloLec should be fixed within #4116

[PabloLec]

@Haarolean
Hi, thank you for your help!

I've tested it and I'm still encountering my issue, perhaps it's different from the one that was fixed or maybe my configuration isn't right.

I'm using the master docker image, the last commit displayed on the UI is indeed the most recent (b0583a3 30/08/2023 10:47:27).

To recap my problem:

• The naming of my subjects is as follows: "avro[TOPIC]event"
• My schemas are correct and Schema Registry seems to be properly loaded and used in the UI.
• However, Serde fails with the message Fallback serde was used.
• So, the problem could be that my naming convention isn't being recognized. It might possibly be my configuration that's incorrect.
• I don't have any specific logs when I use the UI and encounter Serde errors.

My current configuration likely has unnecessary fields but I'm trying to cover the possibilities that I know for debugging:

kafka:
  clusters:
    -
      name: local
      bootstrapServers: kafkabroker:9092
      schemaRegistry: http://kafka-schema-registry:8081
      schemaRegistryKeySchemaNameTemplate: "avro%sevent"
      schemaRegistrySchemaNameTemplate: "avro%sevent"
      serde:
        - 
          name: SchemaRegistry
          keySchemaNameTemplate: "avro%sevent"
          schemaNameTemplate: "avro%sevent"

[Haarolean]

@PabloLec thanks, we'll take a look

[PabloLec]

@Haarolean Any update on the subject?
Perhaps you'd prefer that I open an Issue to more easily track the potential problem?

[Haarolean]

@PabloLec #4255
Feel free to raise one. I don't have r/w permissions at the moment