Log a fail/warn when config metadata default values are unable to be resolved #28062

onobc · 2021-09-18T15:55:44Z

Example

I created a simple app w/ the following config props:

@ConfigurationProperties(prefix = "foo")
public class TestConfigProps {

    /**
     * Paths of things
     */
    private List<File> paths = new ArrayList<>(Collections.singletonList(new File(".")));

    public List<File> getPaths() {
        return paths;
    }

    public void setPaths(final List<File> paths) {
        this.paths = paths;
    }
}

Warn Mode

I ran the config processor which output the following:

I then added the following hint:

{
  "properties": [
    {
      "name": "foo.paths",
      "type": "java.util.List<java.io.File>",
      "defaultValue": [
        "."
      ]
    }
  ]
}

I ran the config processor again - problem fixed:

Fail Mode

When using "fail mode" its the same as above w/ the exception of the problem output looking like the following:

TODOS

add ability to disable error mode??
add test for configprocessor validation checks
docs?

Fixes gh-28039

...va/org/springframework/boot/configurationprocessor/fieldvalues/UnresolvableDefaultValue.java

...main/java/org/springframework/boot/configurationprocessor/fieldvalues/FieldValuesParser.java

snicoll

Thanks for the initial work on this. I've added a couple comments to hopefully give you directions.

snicoll · 2021-09-21T06:52:40Z

...ngframework/boot/configurationprocessor/fieldvalues/javac/JavaCompilerFieldValuesParser.java

@@ -165,7 +167,11 @@ private Object getValue(VariableTree variable) throws Exception {
 			Class<?> wrapperType = WRAPPER_TYPES.get(variable.getType());
 			Object defaultValue = DEFAULT_TYPE_VALUES.get(wrapperType);
 			if (initializer != null) {
-				return getValue(initializer, defaultValue);
+			    Object initializerResolvedValue = getValue(initializer, defaultValue);


It needs to be a bit more precise than that. I think the idea is that we want to fail if we have a FieldValuesParser available and ignore if we don't. We we could do is changing the contract to return a ValueWrapper and just return that here. Our fallback implementation when we couldn't find an implementation should then return null.

Note that we probably want to differentiate between null as "detected as null" and null as "couldn't figure out". This could be an attribute on ValueWrapper or something.

Thx for the direction @snicoll . It makes sense. I will get back to this in the next couple days.

@snicoll

It needs to be a bit more precise than that. I think the idea is that we want to fail if we have a FieldValuesParser available and ignore if we don't.

So, when no FVP is available (ie the env is not an com.sun.tools.javac.processing.JavacProcessingEnvironment) the default FVP is NONE which simply returns an empty map. Because this code is in the FVP it will only apply when the FVP (itself) is available.

We we could do is changing the contract to return a ValueWrapper and just return that here. Our fallback implementation when we couldn't find an implementation should then return null.

Again, the fallback impl currently is an empty map.

I may be missing something here - please clarify.

Again, the fallback impl currently is an empty map.

I am aware of that and I am asking you to change the contract. You can't know what fields are available if you don't have an implementation, but you could change the contract so that, if you're being requested a field with name "foo" you can then return null or a ValueWrapper that holds additional information as I've suggested. So instead of a map a POJO that can tell you if the result contains a field with a given name and another method that gives you a ValueWrapper based on a field name.

I wanted to submit what I have so far before calling it a day as I know you are 7hr ahead of me @snicoll .

I have not moved from map to POJO and changed the contract yet as the map (w/ its ValueWrapper values) pretty much fulfills the above mentioned contract. I don't mind moving to POJO but am curious if you still think it will add value to the current proposal.

I am now marching through the JCFPV tests and updating it. I will continue that tomorrow.

snicoll · 2021-09-21T06:53:20Z

...rg/springframework/boot/configurationprocessor/ConfigurationMetadataAnnotationProcessor.java

@@ -353,6 +356,20 @@ private ConfigurationMetadata mergeAdditionalMetadata(ConfigurationMetadata meta
 		return metadata;
 	}

+	private void validateDefaultValuesResolved(ConfigurationMetadata metadata) {
+        // TODO do we want this? How do we configure annotation processor w/ 'levers'


Ideally fail when we know that we're in an environment where default values can be discovered. Ignore otherwise.

snicoll · 2021-09-30T08:11:27Z

@Bono007 do you have an update for us? If you don't, no worries, we can pick things up as they are as RC1 is our last chance to include this in 2.6.

onobc · 2021-09-30T11:45:40Z

Hi @snicoll - things got busy on my end but I have carved out some time this morning to work on this.

To help me gauge if this needs to be done in "a day" or "the next few days", is 10/21 still accurate?

I will give an update (code or no code) by today @ "12:00:00 -5:00" (~5hrs from time of writing)

snicoll · 2021-09-30T12:46:18Z

Thanks for the follow-up Chris. Yeah that's the RC1 release date so we still have plenty of time. Please do not rush it and let us know if you have any question.

onobc · 2021-09-30T19:44:55Z

Thanks for the follow-up Chris. Yeah that's the RC1 release date so we still have plenty of time. Please do not rush it and let us know if you have any question.

@snicoll my plans to carve out time this morning were foiled by a Kafka Streams production outage that I am putting out. It sounds though it is ok to have something in the next 1-2 days is ok though (correct me if that is not the case). I should have time later today to get back to it.

Thanks

onobc · 2021-10-03T03:15:54Z

...rg/springframework/boot/configurationprocessor/ConfigurationMetadataAnnotationProcessor.java

@@ -353,6 +356,22 @@ private ConfigurationMetadata mergeAdditionalMetadata(ConfigurationMetadata meta
 		return metadata;
 	}

+	private void validateDefaultValuesResolved(ConfigurationMetadata metadata) {
+        metadata.getItems().forEach((itemMetadata -> {
+			if (itemMetadata.getDefaultValue() instanceof ValueWrapper) {


I have not propagated the change from Object -> ValueWrapper up from FVP to ItemMetadata yet but will do that shortly. Unless you can think of a reason to keep it Object.

One reason to keep it Object is to prevent the ripple around all areas that are accessing ItemMetadata#defaultValue (JsonConverter, MetadataStore, etc..).

Yeah. I didn't even think of changing ItemMetadata to be honest. I was hoping we could trigger the warning/error locally to field resolution by passing the necessary bits to it.

Unfortunately we have to wait until the additional metadata is merged to see if the user added a manual hint.

onobc · 2021-10-06T03:05:32Z

@snicoll I was looking through the build failures and our update to the processor is suggesting that we have quite a few "hints" to add ourselves...

> Task :spring-boot-project:spring-boot-autoconfigure:compileJava FAILED
error: Could not resolve default value for property 'server.tomcat.additional-tld-skip-patterns' using expression 'new ArrayList<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'server.tomcat.relaxed-path-chars' using expression 'new ArrayList<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'server.tomcat.relaxed-query-chars' using expression 'new ArrayList<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'server.undertow.accesslog.dir' using expression 'new File("logs")'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'server.undertow.max-headers' using expression 'UndertowOptions.DEFAULT_MAX_HEADERS'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'server.undertow.max-parameters' using expression 'UndertowOptions.DEFAULT_MAX_PARAMETERS'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'server.undertow.options.server' using expression 'new LinkedHashMap<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'server.undertow.options.socket' using expression 'new LinkedHashMap<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.activemq.packages.trusted' using expression 'new ArrayList<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.artemis.embedded.cluster-password' using expression 'UUID.randomUUID().toString()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.artemis.embedded.server-id' using expression 'serverIdCounter.getAndIncrement()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.cache.cache-names' using expression 'new ArrayList<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.data.elasticsearch.client.reactive.endpoints' using expression 'new ArrayList<>(Collections.singletonList("localhost:9200"))'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.datasource.xa.properties' using expression 'new LinkedHashMap<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.flyway.init-sqls' using expression 'new ArrayList<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.flyway.jdbc-properties' using expression 'new HashMap<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.flyway.placeholders' using expression 'new HashMap<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.flyway.schemas' using expression 'new ArrayList<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.freemarker.settings' using expression 'new HashMap<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.integration.channel.max-broadcast-subscribers' using expression 'Integer.MAX_VALUE'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.integration.channel.max-unicast-subscribers' using expression 'Integer.MAX_VALUE'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.integration.endpoint.no-auto-startup' using expression 'new ArrayList<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.integration.endpoint.read-only-headers' using expression 'new ArrayList<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.jersey.init' using expression 'new HashMap<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.jpa.properties' using expression 'new HashMap<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.kafka.bootstrap-servers' using expression 'new ArrayList<>(Collections.singletonList("localhost:9092"))'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.kafka.consumer.key-deserializer' using expression 'StringDeserializer.class'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.kafka.consumer.value-deserializer' using expression 'StringDeserializer.class'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.kafka.producer.key-serializer' using expression 'StringSerializer.class'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.kafka.producer.value-serializer' using expression 'StringSerializer.class'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.ldap.embedded.base-dn' using expression 'new ArrayList<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.mail.properties' using expression 'new HashMap<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.mvc.contentnegotiation.media-types' using expression 'new LinkedHashMap<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.rabbitmq.stream.port' using expression 'DEFAULT_STREAM_PORT'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.security.user.password' using expression 'UUID.randomUUID().toString()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.security.user.roles' using expression 'new ArrayList<>()'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.session.servlet.filter-order' using expression 'SessionRepositoryFilter.DEFAULT_ORDER'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.task.execution.pool.max-size' using expression 'Integer.MAX_VALUE'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.task.execution.pool.queue-capacity' using expression 'Integer.MAX_VALUE'. Add hint in additional-spring-configuration.metadata.json
error: Could not resolve default value for property 'spring.webservices.servlet.init' using expression 'new HashMap<>()'. Add hint in additional-spring-configuration.metadata.json
40 errors

I will start marching through the above list and will add these to the additional metadata.

onobc · 2021-10-06T03:17:35Z

When I encountered the above list I was a bit surprised - I did not expect to see so many entries in violation. I am thinking we should leave the default behavior as Kind.ERROR but provide the ability to switch to Kind.WARN. In most cases I bet the list of updates needed is small but I am sure there are some cases where this will be a pain to absorb at once. Thoughts?

snicoll · 2021-10-06T06:00:02Z

Thanks for the follow-up.

I did not expect to see so many entries in violation.

There are a lot of false positive in this list. Creating an empty list or map shouldn't lead to a violation. UUID.randomUUID().toString() on the default security password is interesting. We'd probably need a way to escape that check.

onobc · 2021-10-06T13:28:07Z

Thanks for the follow-up.

Np.

There are a lot of false positive in this list. Creating an empty list or map shouldn't lead to a violation. UUID.randomUUID().toString() on the default security password is interesting. We'd probably need a way to escape that check.

I will add these exemptions and then add "hints" for the remaining entries.

...boot-autoconfigure/src/main/resources/META-INF/additional-spring-configuration-metadata.json

onobc · 2021-10-07T04:56:54Z

...n/java/org/springframework/boot/configurationprocessor/fieldvalues/javac/ExpressionTree.java

@@ -56,6 +62,16 @@ Object getLiteralValue() throws Exception {
 		return hasLiteralValue() ? this.literalValueMethod.invoke(getInstance()) : null;
 	}

+	Object getNoArgConstructor() throws Exception {


This takes expressions of type "NEW_CLASS" and make sure they have no-args and returns the identifier.
An expression of "new ArrayList<>()" results in "ArrayList<>".

onobc · 2021-10-07T05:01:24Z

...ngframework/boot/configurationprocessor/fieldvalues/javac/JavaCompilerFieldValuesParser.java

@@ -39,6 +40,10 @@
 */
 public class JavaCompilerFieldValuesParser implements FieldValuesParser {

+	private static final Pattern COLLECTION_PATTERN = Pattern.compile("^.*(List|Set)<.*>$");


This is just a simple pragmatic approach and is not perfect and will also detect "com.foo.SomeFooList<>".

onobc · 2021-10-07T05:02:07Z

...ngframework/boot/configurationprocessor/fieldvalues/javac/JavaCompilerFieldValuesParser.java

+			if (noArgConstructor != null) {
+				String expressionStr = noArgConstructor.toString();
+				if (COLLECTION_PATTERN.matcher(expressionStr).matches()) {
+					return ValueWrapper.of("[]", expressionStr);


Should we just skip the field altogether or does it make sense to set the default of "[]" ?

onobc · 2021-10-07T05:03:44Z

...roject/spring-boot/src/main/resources/META-INF/additional-spring-configuration-metadata.json

+    {
+      "name": "spring.jta.atomikos.properties.default-max-wait-time-on-shutdown",
+      "type": "java.lang.Long",
+      "defaultValue": "Long.MAX_VALUE"


Should we always expand these well-known constants in here? As a user I know what Long.MAX_VALUE is and it makes sense to me when I see it in auto-complete in IDE.

Very good question. I was tempted to do that at some point but the problem is that putting this value in a configuration file would not actually bind properly so I think it is semantically wrong to put that representation.

However, I would expect the IDE to be able to resolve that and show the hint rather than the raw, not very helpful, value.

onobc · 2021-10-08T02:28:17Z

...ator-autoconfigure/src/main/resources/META-INF/additional-spring-configuration-metadata.json

@@ -528,6 +536,10 @@
        "level": "error"
      }
    },
+    {
+      "name": "management.metrics.export.wavefront.sender.message-size",
+      "defaultValue": "2147483647B"


Another interesting one DataSize.ofBytes(Integer.MAX_VALUE).

onobc · 2021-10-08T02:31:25Z

...boot-autoconfigure/src/main/resources/META-INF/additional-spring-configuration-metadata.json

@@ -356,6 +368,14 @@
      "name": "spring.artemis.broker-url",
      "defaultValue": "tcp://localhost:61616"
    },
+    {
+      "name": "spring.artemis.embedded.cluster-password",
+      "defaultValue": "UUID.randomUUID().toString()"


Yep, like you said, this is an interesting one. It is a random string value. We could instead do something like <random-uuid-string> but "UUID.randomUUID().toString()" says that as well and also gives a hint to how its random generated.

onobc · 2021-10-08T02:33:31Z

...boot-autoconfigure/src/main/resources/META-INF/additional-spring-configuration-metadata.json

@@ -1984,6 +2049,10 @@
        "request"
      ]
    },
+    {
+      "name": "spring.session.servlet.filter-order",
+      "defaultValue": -2147483598


Does not convey the same hint that SessionRepositoryFilter.DEFAULT_ORDER does (which in turn is = Integer.MIN_VALUE + 50)

onobc · 2021-10-08T02:38:34Z

...ngframework/boot/configurationprocessor/fieldvalues/javac/JavaCompilerFieldValuesParser.java

 			}
+
+			Object identifierForNewClass = expression.getIdentifierForNewClassNoArgConstructor();


The newly added ExpressionTree. getIdentifierForNewClassNoArgConstructor simplifies the regex inspection needed. I could have skipped adding the methods to ExpressionTree and instead done a

if (expression.getKind().equals("NEW_CLASS")) {

but that would have complicated the regex needed to handle the prefix of new and then the suffix of ( ) which would get ugly quickly as the random spacing that can be added to the expression etc...

… resolved. Fixes gh-28039

…em.defaultValue

…favor of direct isEquals

wilkinsona · 2023-07-20T16:05:17Z

Thanks, @onobc. Unfortunately, having seen what it takes to implement it we've realised that my initial idea wasn't a particularly good one. I'm afraid we're going to decline this and try some alternatives idea. Sorry for the wasted effort. We'll repurpose #28039 to do that.

onobc · 2023-07-20T16:19:30Z

@wilkinsona no worries - we try some approaches and some work and some don't. :)

spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Sep 18, 2021

onobc commented Sep 18, 2021

View reviewed changes

...va/org/springframework/boot/configurationprocessor/fieldvalues/UnresolvableDefaultValue.java Outdated Show resolved Hide resolved

onobc commented Sep 18, 2021

View reviewed changes

...main/java/org/springframework/boot/configurationprocessor/fieldvalues/FieldValuesParser.java Outdated Show resolved Hide resolved

snicoll reviewed Sep 21, 2021

View reviewed changes

snicoll marked this pull request as draft September 21, 2021 06:54

snicoll changed the title ~~WIP: Logs a fail/warn when config metadata default values are unable to be resolved~~ Log a fail/warn when config metadata default values are unable to be resolved Sep 21, 2021

snicoll added the status: waiting-for-feedback We need additional information before we can continue label Sep 30, 2021

spring-projects-issues added status: feedback-provided Feedback has been provided and removed status: waiting-for-feedback We need additional information before we can continue labels Sep 30, 2021

onobc commented Oct 3, 2021

View reviewed changes

onobc commented Oct 7, 2021

View reviewed changes

...boot-autoconfigure/src/main/resources/META-INF/additional-spring-configuration-metadata.json Show resolved Hide resolved

onobc commented Oct 7, 2021

View reviewed changes

onobc commented Oct 8, 2021

View reviewed changes

onobc marked this pull request as ready for review October 19, 2021 13:42

philwebb force-pushed the main branch 2 times, most recently from 7aa1646 to 5b11628 Compare November 19, 2021 18:42

philwebb force-pushed the main branch from 1ca278f to 902dd0b Compare November 19, 2021 20:17

onobc added 11 commits January 8, 2022 10:50

Logs a fail/warn when config metadata default values are unable to be…

b501f1f

… resolved. Fixes gh-28039

WIP: Added ValueWrapper

ab95c45

Updated AbstractFieldValuesProcessorTests to verify ValueWrapper

19e4701

Update property descriptor tests to verify ValueWrapper on MetadataIt…

b54a4e1

…em.defaultValue

Remove "convenience" methods in AbstractFieldValuesProcessorTests in …

0cf66c9

…favor of direct isEquals

Fix format / checkstyle

97dbe4c

Simple exclusion of no arg constructor well-known collection/map types

1934985

Add test for compile fail/succeed w/ bad default values

11df4cc

Add option to disable error on unresolved default values.

91c3828

Fixed checkstyle on test

6eb73c8

Add missing default for spring.integration.poller.max-messages-per-poll

2c0466d

snicoll added the for: team-meeting An issue we'd like to discuss as a team to make progress label Jul 20, 2023

wilkinsona closed this Jul 20, 2023

wilkinsona mentioned this pull request Jul 20, 2023

Improve ways in which configuration property metadata default values can be provided #28039

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Log a fail/warn when config metadata default values are unable to be resolved #28062

Log a fail/warn when config metadata default values are unable to be resolved #28062

onobc commented Sep 18, 2021 •

edited

Loading

snicoll left a comment

snicoll Sep 21, 2021

onobc Sep 21, 2021

onobc Oct 1, 2021

snicoll Oct 2, 2021

onobc Oct 3, 2021 •

edited

Loading

snicoll Sep 21, 2021

snicoll commented Sep 30, 2021

onobc commented Sep 30, 2021

snicoll commented Sep 30, 2021

onobc commented Sep 30, 2021

onobc Oct 3, 2021

onobc Oct 4, 2021

snicoll Oct 4, 2021

onobc Oct 4, 2021

onobc commented Oct 6, 2021

onobc commented Oct 6, 2021

snicoll commented Oct 6, 2021

onobc commented Oct 6, 2021 •

edited

Loading

onobc Oct 7, 2021

onobc Oct 7, 2021

onobc Oct 7, 2021

onobc Oct 7, 2021

snicoll Oct 7, 2021

onobc Oct 8, 2021

onobc Oct 8, 2021 •

edited

Loading

onobc Oct 8, 2021

onobc Oct 8, 2021 •

edited

Loading

wilkinsona commented Jul 20, 2023

onobc commented Jul 20, 2023

		}

		Object identifierForNewClass = expression.getIdentifierForNewClassNoArgConstructor();

Log a fail/warn when config metadata default values are unable to be resolved #28062

Log a fail/warn when config metadata default values are unable to be resolved #28062

Conversation

onobc commented Sep 18, 2021 • edited Loading

Example

Warn Mode

Fail Mode

TODOS

snicoll left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

onobc Oct 3, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

snicoll commented Sep 30, 2021

onobc commented Sep 30, 2021

snicoll commented Sep 30, 2021

onobc commented Sep 30, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

onobc commented Oct 6, 2021

onobc commented Oct 6, 2021

snicoll commented Oct 6, 2021

onobc commented Oct 6, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

onobc Oct 8, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

onobc Oct 8, 2021 • edited Loading

Choose a reason for hiding this comment

wilkinsona commented Jul 20, 2023

onobc commented Jul 20, 2023

onobc commented Sep 18, 2021 •

edited

Loading

onobc Oct 3, 2021 •

edited

Loading

onobc commented Oct 6, 2021 •

edited

Loading

onobc Oct 8, 2021 •

edited

Loading

onobc Oct 8, 2021 •

edited

Loading