Propose indexing metadata section in Forc manifest

text/rfcs/0006-metadata-in-forc-manifest.md

@@ -0,0 +1,92 @@
+- Feature Name: Indexing Metadata Section in Forc Manifest
+- Start Date: 2024-01-25
+- Author(s): @deekerno
+- RFC PR: [FuelLabs/rfcs#21](https://github.com/FuelLabs/rfcs/pull/21)
+
+## Summary
+
+An optional section describing indexing-specific metadata should be added to the Forc manifest.
+
+## Motivation
+
+The objective of this change is to allow a Sway contract developer to describe the necessary information for indexing the contract in the Forc manifest, if desired. The expected outcome is that developers will be able to define indexing-specific _metadata_ in the same place and manner that they define other important information, e.g. authoring information, dependencies, etc. 
+
+## Guide-Level Explanation
+
+_Indexing-specific metadata_ refers to the set of information necessary to allow for an indexing service to begin processing resultant data from contract execution. The service should use this information to create distinct and separate tables for contract data as well as route creation on a publicly-available querying API.
+
+Consider the following example of a Forc project manifest: 
+```
+[project]
+authors = ["User"]
+entry = "main.sw"
+license = "Apache-2.0"
+name = "my-fuel-project"
+
+[project.metadata.indexing]
+namespace = "examaple-namespace"
+identifier = "example-identifier"
+schema_path = "/path/to/autogenerated/schema"
+```
+
+An indexing service should use the data shown above in the `[project.metadata.indexing]` section to generate unique identifiers for database tables (e.g. `example-namespace_example-identifer`) as well as create a public API route through which dapps can query the service for information (e.g. `https://indexer.fuel.network/api/example-namespace/example-identifer`).
+
+Sway developers should find this section to be a convenient way to specify information necessary for indexing their data as it would be defined in the same location and format as  project metadata. It can added manually by the user or via a `forc` plugin (e.g. a yet-to-be-developed `forc index create`):
+
+As mentioned earlier, this section should be optional as some developers may not have a desire to index their Sway contract. However, for those that do want to index their Sway contract, they should use the `forc index deploy` command after ensuring that the `[project.metadata.indexing]` section has been correctly defined and their contract has been successfully built and deployed. The command should leverage the required project metadata as well as the contract ABI to create an asset package that will be uploaded to an indexing service; the service should process the asset package and begin indexing the Sway contract. 
+
+## Reference-Level Explanation
+
+To be able to support an indexing subsection of a `[project.metadata]` section, an equivalent mapping for metadata should be added to [the `PackageManifest` type](https://github.com/FuelLabs/sway/blob/d06f3b4f6ed88f1ed9a3f8e601870ce5615b17c0/forc-pkg/src/manifest.rs#L141-L153):
+
+```rust
+/// A direct mapping to a `Forc.toml`.
+#[derive(Serialize, Deserialize, Clone, Debug, PartialEq, Eq)]
+#[serde(rename_all = "kebab-case")]
+pub struct PackageManifest {
+    pub project: Project,
+    pub network: Option<Network>,
+    pub dependencies: Option<BTreeMap<String, Dependency>>,
+    pub patch: Option<BTreeMap<String, PatchMap>>,
+    /// A list of [configuration-time constants](https://github.com/FuelLabs/sway/issues/1498).
+    pub build_target: Option<BTreeMap<String, BuildTarget>>,
+    build_profile: Option<BTreeMap<String, BuildProfile>>,
+    pub contract_dependencies: Option<BTreeMap<String, ContractDependency>>,
+}
+```
+
+This could be done in the way that `cargo` does it in which there is a `metadata` field with a value of type `Option<toml::Value>`, which would satisfy the optional property and ensure that the value is correctly-constructed TOML syntax. Alternatively, because we control exactly what type of data can be successfully processed by Forc, we could define a type for each type of metadata that we want to support and use it to ensure the information is properly constructed. In this case, an `IndexingMetadata` type could be created and it could be a part of a larger `PackageMetadata` struct for type safety.
+
+A Forc plugin dedicated to indexing (e.g. `forc index`) should then parse the manifest. If the necessary indexing metadata section is present, the plugin should leverage the adjusted `PackageManifest` type and other information from Forc in order to build an asset bundle and upload it to an indexing service. If it is not present, then the plugin should return an error. At no point should a well-defined metadata section lead to an error in `forc` or the Sway compiler.
+
+## Drawbacks
+
+In my opinion, there are no foreseeable drawbacks. This optional section is intended to be leveraged by a tool separate from the Sway compiler and should be able to be ignored with no side effects.
+
+## Rationale and Alternatives
+
+The original version of the indexer required users to create a separate project directory and write/edit three separate files in order to be able to index their Sway smart contract. While it allowed for powerful customization, it increased the mental load on smart contract developers; furthermore, this is in addition to the mental load placed upon them by other user constraints arising from the nature of the original version's architecture. Thus, as the indexing team considers other ways to reduce the set of responsibilities on the user, I think this would be a beneficial addition that (due to its proposed optional status) should not have any detrimental effects to the rest of the developer experience.
+
+An alternative would be to specify this information in a separate file and keep the information in `Forc.toml` purely related to building the contract. However, I do believe that having this optional indexing metadata section inline with the other project metadata makes for a more cohesive developer experience.
+
+## Prior Art
+
+### Cargo
+
+The [Cargo manifest format](https://doc.rust-lang.org/cargo/reference/manifest.html) contains a number of sections, many of which are actually optional and not explicitly needed for compiling a Rust project (e.g. `[badges]`); in fact, there is a [`[metadata]` section](https://doc.rust-lang.org/cargo/reference/manifest.html#the-metadata-table) available in the manifest format which mirrors how the indexing metadata section should be used.
+
+### Fuel Indexer
+
+The original version of the Fuel indexer uses [a YAML manifest](https://docs.fuel.network/docs/indexer/project-components/manifest/) that contains the necessary data to start processing and storing blockchain data in a database; it contained the same data specified in the example snippet above as well as other metadata used to change the behavior of contract indexing.
+
+### Smart Contract Languages
+
+As far as I can tell, there does not seem to be equivalent prior art (in regards to indexing) in any of the most popular smart contract languages.
+
+## Unresolved Questions
+
+Should the metadata section allow for a general, well-constructed value? If so, it allows for users to define whatever they want in the manifest even though it won't be processed by the compiler; this could allow for unforeseen third-party/downstream uses.
+
+## Future Possibilities
+
+One possibility is to implement a way for `forc index` to determine the block height at which a contract was deployed and add that field as part of the optional indexing section. This would allow the indexing service to skip directly to that point in the chain and prevent the unnecessary processing of blocks prior to contract deployment. If added, it would not need to be exact as the indexing service has historically been fast enough to process thousands of blocks in a short period of time; however, at millions of blocks, it would be beneficial at scale. As far as I can tell, `forc` does not persist the chain height at time of contract deployment; it may be prudent to log the information somewhere in the project directory.