Update leading to different base64 encoding in picture srcset

[globalexport]

Hi,
It seems like there is a base64 string missing after updating this plugin (and other parts).

Has such a result been observed by somebody else?

Before (Production)

Statamic 3.2.32 Pro
Laravel 8.81.0
PHP 8.0.20
doublethreedigital/duplicator 2.1.0
optimoapps/statamic-bard-text-align 1.0.2
spatie/statamic-responsive-images 2.8.5
statamic/ssg 0.9.0

After (Dev)

Statamic 3.3.17 Pro
Laravel 8.83.18
PHP 8.0.16
doublethreedigital/duplicator 2.3.1
optimoapps/statamic-bard-text-align 1.0.2
spatie/statamic-responsive-images 2.12.4
statamic/ssg 1.0.1

[globalexport]

php please glide:clear fixes this problem temporarily.

[ncla]

Could you please provide steps to reproduce this? Is this happening on SSG generated pages or without SSG?

[globalexport]

Hi @ncla,

Sorry for the late reply. It is happening on SSG generated pages.
Sadly, I do not have a way to provide easy reproduction steps, as the project has become quite complex.

As long as nobody runs into my issue, I would need to create a fresh project.

[globalexport]

I can confirm now that the default SSG rendering process seems to be working correctly. There has to be something broken on my side.

[ncla]

It is possible that this could occur again at random. The placeholder fetching can fail silently, however only if it's a FileNotFoundException exception. See code here: https://github.com/spatie/statamic-responsive-images/blob/main/src/Breakpoint.php#L262-L272

It is also possible that it is a bug in ssg package. I am experiencing image issues as well in one of my projects. statamic/ssg#91

When you ran this test, did you try running ssg:generate multiple times?

[globalexport]

I ran ssg:generate five times in a row after you mentioned it. It worked every time.

My project is quite different from the test setup. I customised the ssg process (different commands and workflow). I did not touch the image handling though. Now I am thinking about carefully moving the project's code into the test setup.

[globalexport]

Side note:

I found out that the error shown in the first post is not restricted to the ssg output.
It can happen in the dynamic part as well.

• php please glide:clear => dynamic site looking correct
• After running SSG => ssg site images looking correct but images of the dynamic site broken.
• php please glide:clear => dynamic site looking correct again

[globalexport]

As you mentioned, I am touching the exception block for the images in question, if I do not clear the glide cache beforehand.

statamic-responsive-images/src/Breakpoint.php

Line 270 in 6aa7591

} catch (FileNotFoundException $e) {

[ncla]

In the SSG issue statamic/ssg#91 I wrote this:

This exception will only go away if I do php please glide:clear && php artisan cache:clear, but then I stumble upon this exception elsewhere when I decide to ssg:generate again. It seems that if Glide cache retrieval/generation fails once, then it is stuck forever?

Which sounds like your issue: ssg:generate fails on some image and then it is stuck in this fail state forever until you do glide:clear. Except in my case, in the steps you described here

• php please glide:clear => dynamic site looking correct
• After running SSG => ssg site images looking correct but images of the dynamic site broken.
• php please glide:clear => dynamic site looking correct again

In the 2nd step, my ssg:generate actually has failed on regular Glide manipulation too (that are not using responsive tag, but overall in project the use of responsive tag exists).

[ncla]

CMS 3.3.8 version introduced some new Glide cache stuff (statamic/cms#5725) which could make this code section outdated in Breakpoint.php. https://github.com/spatie/statamic-responsive-images/blob/main/src/Breakpoint.php#L262-L272

It has not been updated since 3.3.8. Maybe there is more correct, up-to-date way of fetching an image from cache.

            try {
                $source = base64_encode($server->getCache()->read($path));
                $cache = $server->getCache();
                $mimetype = method_exists($cache, 'getMimetype')
                    ? $cache->getMimetype($path)
                    : $cache->mimeType($path);

                $base64Placeholder = "data:{$mimetype};base64,{$source}";
            } catch (FileNotFoundException $e) {
                return '';
            }

It is possible that this section actually is buggy since 3.3.8.

[globalexport]

Yes, you are right. Nothing is done to mitigate the outcome this exception is leading to (invalid base64 string).
I am re-opening this issue.

[globalexport]

I have used one of my addons to overwrite (alias) the Breakpoint.class.

Condition && $this->placeholderSrc() != '':

    public function getSrcSet(bool $includePlaceholder = true, string $format = null): string
    {
        return $this->getWidths()
            ->map(function (int $width) use ($format) {
                return "{$this->buildImageJob($width, $format, $this->ratio)->handle()} {$width}w";
            })
            ->when($includePlaceholder && $this->placeholderSrc() != '', function (Collection $widths) {
                return $widths->prepend($this->placeholderSrc());
            })
            ->implode(', ');
    }

Return empty strings here:

    public function placeholder(): string
    {
        $base64Svg = base64_encode($this->placeholderSvg());
        return $base64Svg !== '' ? "data:image/svg+xml;base64,{$base64Svg}" : '';
    }
[...]
    private function placeholderSrc(): string
    {
        return $this->placeholder() !== '' ? $this->placeholder() . ' 32w' : '';
    }

Now the faulty 32w is missing in the srcset and the other images are rendered.

I also found out that
php please responsive:generate is not creating the same amount of images than php please responsive:regenerate.
I do not know why there is a difference at the moment. This also was problematic in my case.

There is a lot of testing necessary now.

[ncla]

So if I understood correctly, getting rid of placeholder yielded consistent error free output?

I also found out that
php please responsive:generate is not creating the same amount of images than php please responsive:regenerate.
I do not know why there is a difference at the moment. This also was problematic in my case.

It would be great if you could open separate issue for this. One issue at a time. :)

[globalexport]

Yes, your are correct.
No errors up to this moment, but I am still testing a small subset of all pages (approx. 30 of 500+).

[ncla]

Unfortunately the issue I linked previously in this conversation to statamic/ssg#91 might not be relevant to this issue, which I thought it could be. My issue stemmed from Glide tag pairs instead, see issue I opened here: statamic/ssg#110

So we are back to where we were before: could you try and make a reproducible repository with this bug? Then from there I can take a look. You said you had some custom SSG generation stuff, which obviously makes things complicated.

[ncla]

Hey, is there any updates on this?

[globalexport]

Hi @ncla,

Sorry that I did not comment on your last post.

I managed to temporarily fix my problem with the above solution:
#146 (comment)

There was no more time left for a detailed investigation.

[globalexport]

I am going to close the issue, as it probably has something to do with my setup.

Thank you very much for looking into this!

[ncla]

Hey @globalexport, I think the latest update / pull request should address the issue you had and you may remove your work around.

You were hitting that exception because you are on older Laravel framework version, while I was on newer - that's why I couldn't reproduce this issue. 🙈 I failed to notice this. Newer Laravel versions has some changes to Flysystem which changed some Glide stuff.

You can read the #171 issue and and #174 PR, but essentially the issue was that placeholder generation was reading cached manipulated image that didn't exist in the cache storage. This storage changes location depending on the assets.image_manipulations.cache setting. In SSG context, that is probably set to true. But when you are checking on your website, you are using "dynamic" cache or false setting. Glide Path Cache Store currently is only aware of the relative path from the storage to the manipulated images. Once you change cache setting, Path Cache Store still thinks you have already generated the image, and is looking in a different cache storage directory.

That's why glide:clear helped intermediately as it clears Glide Path Cache Store too.

[globalexport]

Hi @ncla,

Thanks a lot for the explanation!
I am going to update some statamic instances this week.
It would be quite nice being able to remove the local fixes again. :)

[globalexport]

Hi @ncla,

It is looking very well on one of the smaller projects.
I updated everything (statamic, laravel, reponsive images, ssg) and removed my local fixes.
The missing base 64 string is back in both the dynamic and static frontends.

Thanks a lot!