fix(certificate): unable to refresh certificate entity with vault ref…

…erence when initial with an invalid string (#10984)

When set the cert to an invalid string in the vault, and then correct the
string, kong will not start to use the correct certificate even though `kong
vault` command shows the correct certiticate being returned.

This is because the initial invalid string has already been cached in the L2
cache by mlcache, and it will not be updated via the L3 cache callback
afterward.

When mlcache fails during the execution of the `l1_serializer`, it caches the
result retrieved from the L3 cache callback the first time in the L2 cache
permanently. This means that subsequent calls to `cache.get()` will never fetch
data from the L3 cache but will directly retrieve it from the L2 cache.

To avoid this situation, when the `l1_serializer` fails, we no longer allow
mlcache to store the data retrieved from L3 into L2.

changelog/unreleased/kong-ee/fix-certificate-reference.yml

@@ -0,0 +1,3 @@
+message: Fixed an issue that certificate entity configured with vault reference may not get refreshed on time when initial with an invalid string.
+type: bugfix
+scope: Core

kong/resty/mlcache/init.lua

@@ -412,6 +412,28 @@ local function set_shm(self, shm_key, value, ttl, neg_ttl, flags, shm_set_tries,
 end
 
 
+local function del_shm(self, shm_key, value)
+    local shm = self.shm
+    local dict = self.dict
+
+    if value == nil then
+        if self.dict_miss then
+            shm = self.shm_miss
+            dict = self.dict_miss
+        end
+    end
+
+    local ok, err = dict:delete(shm_key)
+
+    if not ok then
+        ngx_log(WARN, "could not delete from lua_shared_dict '" .. shm
+                      .. "': " .. err)
+        return
+    end
+
+    return true
+end
+
 local function set_shm_set_lru(self, key, shm_key, value, ttl, neg_ttl, flags,
                                shm_set_tries, l1_serializer, throw_no_mem)
 
@@ -421,7 +443,13 @@ local function set_shm_set_lru(self, key, shm_key, value, ttl, neg_ttl, flags,
         return nil, err
     end
 
-    return set_lru(self, key, value, ttl, neg_ttl, l1_serializer)
+    ok, err = set_lru(self, key, value, ttl, neg_ttl, l1_serializer)
+    if not ok and err then
+        -- l1_serializer returned nil + err, do not store the cached vaule in L2
+        del_shm(self, shm_key, value)
+    end
+
+    return ok, err
 end