Custom Classification model behavior

[galvangoh]

• Package Name: azure-ai-documentintelligence
• Package Version: 1.0.0b2
• Operating System: Windows
• Python Version: 3.10.9

Describe the bug
I trained the custom model via studio on 2 different titled document which share very similar pattern in its template. My classified label from the custom model is used to decide the logic flow of my application later on. Sometimes, document A gets classified as document B and vice versa.

The documentation mentioned "Custom classification models are deep-learning-model types that combine layout and language features to accurately detect and identify documents...". I don't think the "layout" here points to the layout model because blocks of text is extracted by the layout model which the custom classification does not do that. Unless there is a way to composed prebuilt and custom models, how can I classify my documents more properly? I'm happy to remain at the current version of the API and will only upgrade if there are improvement to the classification capability of the base neural model itself.

In the screenshot below, I show 2 document which I want to classify into its own label (see top right).

Questions:

1. As the document are titled differently, does the custom model picks up the title as a feature to distinguish them properly?
2. Handwritings and stamps can appear in random locations in the document. Does the custom model picks these as features during training?
3. What sort of training data is sufficient so that both document can be correctly classified?

Expected behavior
Accurate classification of different titled document despite sharing the same template.

Screenshots

[xiangyan99]

Thanks for the feedback, we’ll investigate asap.