API for getting layer specific data.

[therahedwig]

Right now, libsai can decrypt the file system, and look at each entry. To get the information about the layers, and then to get the pixeldata(if applicable) requires using VirtualFileEntry::Read(). There is a struct for the LayerHeader in sai.cpp, but if I want to access that from an outside class, I have to duplicate these structs in that outside class.

Ideally, we'd implement the fiddly bits into libsai, and then use a bit of api to figure out if a VirtualFileEntry is a layer we can read, and then have a layer object that gives us simpler access to layer properties and contents(pixeldata, vectordata).

Do you have any idea how you'd like to see this architecture wise?

[Wunkolo]

Looking back at this old library, there are some decisions that I think I'd like to try and do to facilitate something like this without having to make redundant copies of raster data. Stuff like having it use memory-mapped-IO for read-only access since the code skips around pretty randomly around the file and the encryption allows for "pretty-random" access as well. And with a setup like that, there can be an API like std::optional<sai::RasterLayer> ReadLayer(sai::VirtualFileEntry& ...) of some kind that simply points to the already-mapped data and provides utility functions like flattening out the specially formatted data into the usual RGBA formats and so forth. The less copies the better!

[therahedwig]

So... a way to do this would be to have...

class Layer

• layerclass (necessary?)
• ID
• bounds
• opacity
• visible
• preserveopacity
• clipping
• blending
• name?
• parentid?

Then

RasterLayer : public Layer

• PixelData

Mask : public Layer -- Are masks also 4 channel encoded, or are they single channel/gray?

• pixeldata

LineWorkLayer : public Layer

• VectorData

FolderLayer : public Layer

• isOpen

Then...

• std::optionalsai::Layer ReadLayer(sai::VirtualFileEntry& ...)
  Then we can use dynamic casting to make the layer one of it's appropriate subclasses.

Where would you want said ReadLayer? In Document? Or VirtualFileSystem?

EDIT:
Also, it seems that I would need to put the LayerHeader and LayerBounds structs into the header anyway so I can access them for such a new class.

At the least, the only time I had to read from a stream, the data had to be accessed by reading through the stream which by itself caused the offset. I have the feeling we want to read through the stream, parse the data into the LayerHeader struct(like is being done for Document::Thumbnail()), and then access that. Or would you rather I used VirtualFileSystem::seek(offset) for this?

[Wunkolo]

Sounds like a good pattern that is similar to how the structures themselves are laid out in memory. We can use the serialized layer headers or just encapsulate it into a more higher level structure describing the layer too, abstracting away all the grit of the layer system to have a more convenient API. Either is fine ultimately.

I like the idea that a "raster layer" or "document" is just a view of a particular virtual file system and its entries. And by using memory mapped IO, I can do away with the concept of having stateful seek pointer or having a redundant amount of IO related syscalls. the VFS comes first before its interpretation as a layer.

It would involve another dependency, but I've used mio before for rapidly traversing large and complex file formats, and beyond that we would just need a small ring of caches for the table blocks and the data blocks, similar to the one in-place now, to expedite redundant decryption.

Do you know of any libraries that read from virtual file systems or archives with patterns that can be referenced from?
I'd love to refactor this library using some more modern C++11/17 concepts. Like using std::path and std::visit and such.

I can't believe I let myself use strtok.

[therahedwig]

So, I discussed this with Boudewijn, going over all the options.

Right now, I think my concern is to get stuff working in Krita, so I am going to implement the layer class to parse through the layers. I will just parse through the entry and fill it into the layer headers and see how much I can get working. (That is, right now they'll be separate objects rather than alternate views)

After that we can worry about performance and using memory mapping. I am far more worried about getting the import correct at this point :D

[Wunkolo]

I've added a bit of an API related to iterating all of the layers and sublayers in a document.

Check out the new Document.cpp sample to see the new pattern.
In general, you would iterate the layers first, and then iterate the sublayers(masks) to apply the masks to any layers you have iterated.
All sublayers are guaranteed to be dependent upon a layer object. In the context of making a loader for Krita, iterate the layers and create the Krita-equivalent, and then iterate the sublayers and apply these masks to your Krita-layers by mapping their parent-layer identifier to the equivalent Krita layer.

I've also added the new _Tag user-literal so that we can use plaintext tag names and four-letter constants without any weird warnings. There is no obvious way to statically guarantee that "..."_Tag is only passed a char Foo[5] but this isn't exactly code that is trying to be exposed to the end-user.