created data marix does not contain the proper information

[soar-high]

Hi, I am having a shopping card for Coop. It has a data matrix to be read by the scanners.
When scanning the card with Catima there is created a data matrix which looks similar, but not identical to the one on the card. And it does not work in the shop.
Is there any way to investigate and optimize?

Kind regards

[TheLastProject]

Most of the time an issue like that is caused by a bad lighting condition causing an incorrect value to be scanned, so step 1 would be to make sure the barcode is readable well and the location it's scanned in isn't too dark or have light reflecting on the barcode.

It is also important to not manually edit the Card ID, as that by default changes the barcode value (see #1453).

If neither of those are the culprit, it might be something else. I don't know what kind of data coop puts in their card, but there have been some reports about issues with handling of binary data (but so far nothing that I ran into for any card I own).

It may also be a good idea to check zxing's online decoder to see if it's not a bug there because Catima uses zxing under the hood to actually do the barcode detection and generation.

[soar-high]

Hi, Thank you for the advices. Pictures / scans are clear, no reflections or anything else.  #1453 I couldn't reproduce or find any changes in the resulting data matrix.  Running the data matrix Catima generates through zxing (screenshot) and compare the output with the output from the card's data matrix photo there are differences in the raw bytes only. All "e8" on the cards matrix which are followed by a "d.." are changed to "1e" in the Catima matrix raw bytes. The "e8" followed by a "4.." stayed as "e8". Raw text and Parsed result are the same to my eyes. But most code scanners in the shops do not accept the Catima data matrix. Using QR-code scanner of my S22 the resulting text is equivalent to the parsed result except of the "�" which are missing in the S22's string. And it seems to me that the "�" are exactly at the positions of the "e8" followed by "d.." raw bytes.  Does that make any sense for an IT specialist? Kind regards

…

Message ID: ***@***.***>

[nicopap]

Hi,

I get the same issue scanning the "Coop" card (the Swiss retail chain).

Taking a more careful look at it, it seems the data matrix is interpreted as an ASCII text string containing a series of three numbers separated by the "Group Separator" ASCII character (0x1d, 29, or "GS"), which is a non-printable ASCII character (they are displayed as "�" in the zxing.org tool you linked).

The ZXing library outputs text with those group separator ASCII characters. The "Card ID" field contains the characters in question even though they are invisible, but when generating the Data Matrix barcode, Catima seems to ignore them or mix them up in a way that alters the value encoded.

I'm not an android developer, and I'm not quite familiar with barcode standards, but I'll take the time to investigate and see if I can open a PR to fix the issue.

[nicopap]

Taking another look, it seems more complex.

The "source barcode" is the barcode printed on the Coop shopping card.

The issue is that the barcode shown in Catima is different from the source barcode, and the Catima barcode doesn't work at the retail chain's points of sale.

When using https://zxing.org/w/decode.jspx, both the source barcode and the Catima barcode produce the same "Raw text", but a different "Raw bytes". As soar-high pointed out, the "e8" bytes are converted into "1e" bytes.

I've a copy of the two datamatrix barcodes, but I will only share it privately as I am worried that it contains sensitive information.

My hunch is that the "Group Separator" character is not part of the original text, but is rather an artifact of the output of the ZXing library, converting in ASCII the original encoding of groups. You'll notice that in the wikipedia page for "Data Matrix" there is a "codeword" table, and e8 has the FNC1 "Interpretation", FNC1 is group separator. See https://en.wikipedia.org/wiki/Data_Matrix#Encoding.

No idea how to fix this. So don't expect a PR from me :/

I've taken a glance at the code, and the roundtrip from source barcode to Catima barcode is:

1. Utils.getBarcodesFromBitmapReal uses zxing.Result.getText
2. It is stored somewhere (it's what is displayed in the "Card ID" field)
3. MultiFormatWriter.encode is used in BarcodeImageWriterTask.generate

[c22]

I'm also experiencing issues with Data Matrix support in Catima. It may not be directly related to what is happening here, so apologies if this is derailing a bit.

I have a 26x26 Data Matrix ticket that Catima was not able to scan. I scanned it with BinaryEye with no issues, but it scans in as a 14x14 Data Matrix with seemingly no way to bump up the error correction or represent it in the original format. I suspect what is happening is ZXing is reading out/decoding the final value, rather than each individual bit and then encoding it again from the decoded value with no way to bump up error correction or compare it to the original (encoded) bits from the scanned image.

It's not clear to me from reading ZXing documentation if it's possible to use getRawBytes on the Result of DataMatrixReader.decode() then directly construct a new DataMatrix from those bytes, and potentially preserve the original encoded value exactly as read (which may include encoding errors, codewords, etc.) but maybe it's something worth exploring?

It's also curious that Catima can't read this particular DataMatrix but BinaryEye can, as my understanding is they both use ZXing under the hood.