Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NFT provenance tracker SDK - grant application #1832

Closed
wants to merge 28 commits into from

Conversation

Ekaanth
Copy link

@Ekaanth Ekaanth commented Jul 3, 2023

Project Abstract

One-liner: A on-chain system capable of tracking and displaying the entire lifecycle of NFTs across multiple parachains in the Polkadot ecosystem via SDK.

An NFT Provenance Tracker SDK, an on-chain system capable of tracking and displaying the entire lifecycle of NFTs across multiple parachains in the Polkadot ecosystem. The tracker will collect data like NFT creation, past ownership history, price history, metadata, and locking/unlocking events. The data will be made accessible via SDK.

This is a new funding application and is not in response to an RFP, or a follow-up grant.

Grant level

  • Level 1: Up to $10,000, 2 approvals
  • Level 2: Up to $30,000, 3 approvals
  • Level 3: Unlimited, 5 approvals (for >$100k: Web3 Foundation Council approval)

Application Checklist

  • The application template has been copied and aptly renamed (project_name.md).
  • I have read the application guidelines.
  • Payment details have been provided (bank details via email or BTC, Ethereum (USDC/DAI), or Polkadot/Kusama (USDT) address in the application).
  • The software delivered for this grant will be released under an open-source license specified in the application.
  • The initial PR contains only one commit (squash and force-push if needed).
  • The grant will only be announced once the first milestone has been accepted (see the announcement guidelines).
  • I prefer the discussion of this application to take place in a private Element/Matrix channel. My username is: @abhichat85:matrix.org @humblefool94:matrix..org

@CLAassistant
Copy link

CLAassistant commented Jul 3, 2023

CLA assistant check
All committers have signed the CLA.

Copy link
Member

@semuelle semuelle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the application. A couple of questions:

  • When you talk about NFTs, do you mean the NFTs pallet, PSP34s, uniques pallet, or EVM-compatible ones?
  • Your tech stack mentions Elasticsearch, AWS and nginx, but none of your deliverables touches those. How will I be able to replicate your results?
  • Indexing NFT-related events seems to be a common indexer task. See, for example, https://github.com/emarai/squid-polkadot-psp34-nft-indexing. Why should we fund another, NFT-only indexer?
  • Please expand the Ecosystem Fit section, for example to explain who your target audience is and how this project compares to similar, existing ones.

@semuelle semuelle self-assigned this Jul 4, 2023
@diposs diposs mentioned this pull request Jul 5, 2023
10 tasks
@Ekaanth
Copy link
Author

Ekaanth commented Jul 5, 2023

Hi @semuelle, thanks for raising this absolutely valid questions. Here are our answers which will hopefully give you more clarity on this grant application.

When you talk about NFTs, do you mean the NFTs pallet, PSP34s, uniques pallet, or EVM-compatible ones?

We will not be focusing currently on the EVM-compatible ones but stick to more Polkadot ecosystem NFT's which are powered mostly by the Uniques Pallet and going forward the NFT's pallet will be used. So this will be a tool for aggregation for NFTs created through both pallets.

Your tech stack mentions Elasticsearch, AWS, and nginx, but none of your deliverables touches those. How will I be able to replicate your results?

I think there's some confusion related to this point. The deliverables are geared towards building and delivering the individual modules which will eventually be released as an SDK which other projects can integrate. This project will be a proof of concept (PoC) to demonstrate this particular use case and how this can be leveraged by other projects and parachains in the Polkadot ecosystem. Since this will be an SDK, anyone will be able to integrate this within their application. This acts as an enabler for this particular use case for provenance tracking of NFTs and fractional fungible tokens associated with those NFTs. This can extend to not only digital assets but also real-world assets (which is what is the primary focus of Metaquity Network). This grant application is in relation to taking this project to the proof of concept stage. The ancillary tech stack mentioned such as Elasticsearch, AWS, and others shall be implemented when we take this project to the production stage.

Indexing NFT-related events seem to be a common indexer task. See, for example, https://github.com/emarai/squid-polkadot-psp34-nft-indexing. Why should we fund another, NFT-only indexer?

We have envisioned this project as a more generic use case for our larger project and other projects in the ecosystem. PSP34 and especially the ones mentioned in the link is focused on the Astar network but this SDK can be implemented by any parachain using NFTs in some form or the other. More importantly, this is not going to be just an NFT-only indexer module, we can extend this to re-fungible tokens built on top of these NFTs and to track provenance of the entire asset lifecycle, which includes extending this use case to track fractional ownership of assets in the future.

Please expand the Ecosystem Fit section to explain who your target audience is and how this project compares to similar, existing ones.

Our target audience is basically the parachains and projects in the Polkadot ecosystem who are working with NFTs and possibly refungibility of NFT's to fungible tokens and make their lives simpler by providing an open source tool to track the entire lifecycle of their assets. Also upcoming parachains like Metaquity Network (ours) and others can use this to build these provenance use cases which will eventually give rise to more effective collaborations between the DeX's and the parachains.

@Ekaanth Ekaanth requested a review from semuelle July 5, 2023 18:52
Copy link
Member

@semuelle semuelle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates, @Ekaanth. At this point, I think this functionality makes most sense as part of an NFT marketplace, rather than standalone. But since your deliverables are the SDK only, that would work. Could you

  • change the specification of deliverables 0a to 0e? They are either the template text or empty.
  • Add the contents of your previous comment to the application document? It's easier to read everything in one place.

Also please expand on the comparison to existing solutions, even if they don't have the same scope as your project, e.g. general purpose indexers.

I will hand over the review to one of my colleagues, as I will be out of office for a bit. Best of luck.

@semuelle semuelle removed their assignment Jul 12, 2023
@takahser takahser self-assigned this Jul 12, 2023
@takahser takahser self-requested a review July 12, 2023 20:27
@Ekaanth
Copy link
Author

Ekaanth commented Jul 13, 2023

Hi @semuelle and @takahser ,

Thanks for your comments, we have added all the relevant information to the application proposal.

In regard to your question

Also please expand on the comparison to existing solutions, even if they don't have the same scope as your project, e.g. general-purpose indexers.

Existing general purpose indexers are as you call it "generic". Their aim is to index all kinds of data points. However, this project specifically aims at tracking the provenance lifecycle of NFT's using uniques pallet and NFT pallet. The further goal of this project is to extend this functionality for further tracking of fractional or refungible tokens of those NFTs. This is especially useful for applications working towards this functionality and projects working with Real-world assets.

Further, this project can also be expanded to NFT 2.0 or future NFT standards.

@semuelle Thanks for delegating to @takahser , hope you have happy holidays.

@takahser
Copy link
Collaborator

@Ekaanth I haven't found the time to review your proposal and above conversation in detail yet, but I'm going to provide you feedback soon.

@metaquity
Copy link

@takahser - Looking forward for your feedback soon :)

Copy link
Collaborator

@takahser takahser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Ekaanth

I've added a couple of inline-comments. On top of that I'd like to note that:

  • some of the mandatory deliverables are missing; feel free to add these
  • it seems to me as Milestone 1, 0b. & 1. represent research that is targeted towards this project which is why I personally wouldn't support them - I'd rather expect them to be part of the proposal so we can get a better understanding, what you're planning to build
  • in general, it'd be good if you could explain some concrete examples of where your SDK would be useful
  • in particular I'd be curious to learn if/how you're planning to use this SDK for your own project


Architecture:

![](https://i.ibb.co/TkWNWWK/Screenshot-2023-07-03-at-14-13-35.png)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • what's a crown job?
  • could you upload a better-quality version, it's a bit hard to read.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically, we will analyze the new and old blocks produced and read block information only if there is information related to NFT.
Our team has made improvements to the image to enhance its clarity.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the role of the block analyzer? Is it something you're going to implement, like a pallet or a smart contract? Or is it an existing component?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will be using subsquid to read the old and new blocks that are generated to index the blocks

applications/metaquity_nft_provenance_tracker.md Outdated Show resolved Hide resolved
applications/metaquity_nft_provenance_tracker.md Outdated Show resolved Hide resolved
applications/metaquity_nft_provenance_tracker.md Outdated Show resolved Hide resolved
| **0a.** | License | Apache 2.0 |
| **0b.** | Documentation | Architectural documentation of NFT-scanner |
| 1. | Data model definition | Designing and defining the architectural overview of the data model for this application |
| 2. | NFT scanner module | This module will scan the blockchain for any NFT-related data from the genesis block and invoke the indexer module |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In which layer of the stack will this module be located? What tech is going to be used here?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quote from your proposal:

This module will be in middle layer to scan the blockchain for the blocks containing NFT related data from the genesis block and invoke indexer module (We will be indexing only blocks that has NFT related data.)

By "middle layer" you mean the "middleware layer" as depicted in your architecture diagram? Could you distinguish which technologies are used in which layer and how (protocols or other means of communication) they interact with each other?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The middleware layer is written mostly in typescript; they communicate or interact with the data using JSON.

applications/metaquity_nft_provenance_tracker.md Outdated Show resolved Hide resolved
applications/metaquity_nft_provenance_tracker.md Outdated Show resolved Hide resolved
@takahser takahser added the changes requested The team needs to clarify a few things first. label Jul 19, 2023
@Ekaanth
Copy link
Author

Ekaanth commented Aug 7, 2023

Hi @takahser any update on the proposal?

Copy link
Collaborator

@Noc2 Noc2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the application and your updates so far. One quick comment from me: Could you add more technical details to the specification of the milestone tables? For example the programming language and the functionality like what kind of NFTs, events you will support, etc. Also we usually don't fund the initial design phase, regarding "Designing and defining the architectural overview of the data model for this application" since this isn't necessarily useful for others.

@semuelle
Copy link
Member

I only just noticed that you requested a private discussion, @Ekaanth. Sorry about that. Would you like to switch over to Matrix, or are you okay with continuing the conversation here?

Copy link
Collaborator

@takahser takahser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Ekaanth Sorry for the delay here. I've updated the inline comments that I mentioned earlier. In general, the architecture, scope of the grant and deliverables are not clear to me yet.

@semuelle
Copy link
Member

pinging @Ekaanth

@github-actions

This comment was marked as duplicate.

@Ekaanth
Copy link
Author

Ekaanth commented Aug 25, 2023

Hello @semuelle, apologies for my delayed response. I wanted to inform you that we have made some updates to the proposal's architecture and overview.

@semuelle semuelle requested a review from takahser August 29, 2023 13:07
Copy link
Collaborator

@takahser takahser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Ekaanth thanks for the update. However, some of the inline comments that I posted earlier still aren't addressed, in particular this and this one. Please address both of them. Also, feel free to have a look at the new ones. To optimize time-efficiency here, please ping us once you've addressed all inline comments.

| **0a.** | License | Apache 2.0 |
| **0b.** | Documentation | We will provide both readme.md and official docs documentation |
| **0c.** | Testing and Testing Guide | Testing guide will be mentioned in official docs & core unit tests will be provided |
| 1. | Data model definition | Designing and defining the architectural overview of the data model for this application |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is another research-leaning deliverable that I wouldn't support. Please focus on the deliverables, rather on activities - While I agree that you should have a well-defined model (rather than having non-typed js) I'd expect it the related code do be self-documenting by being clear and expressive and supplement it with inline-code where deemed necessary. Architectural diagrams or model diagrams can make sense here, too, however, I'd expect it to be part of the Documentation (0b.). To conclude, I suggest you remove this deliverable and incorporate it into the "Documentation" deliverable instead.

Comment on lines 108 to 109
| **0c.** | Testing and Testing Guide | Testing guide will be mentioned in official docs & core unit tests will be provided |
| 1. | Data model definition | Designing and defining the architectural overview of the data model for this application |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mandatory Docker deliverable is still missing here.

Comment on lines 122 to 124
| **0b.** | Documentation | We will provide both readme.md and official architectural documentation of the NFT scanner module |
| **0c.** | Testing and Testing Guide | Core functions will be fully covered by comprehensive unit tests to ensure functionality and robustness. In the guide, we will describe how to run these tests. |
| **0e.** | Article | Add an article covering how to integrate NFT-SDK in a project. |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mandatory Docker deliverable is still missing here.

| **0b.** | Documentation | We will provide both readme.md and official docs documentation |
| **0c.** | Testing and Testing Guide | Testing guide will be mentioned in official docs & core unit tests will be provided |
| 1. | Data model definition | Designing and defining the architectural overview of the data model for this application |
| 2. | NFT scanner module | This module will be in middle layer to scan the blockchain for the blocks containing NFT related data from the genesis block and invoke indexer module (We will be indexing only blocks that has NFT related data.)|
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous, related inline comment still hasn't been addressed.

@semuelle semuelle removed their request for review September 5, 2023 08:43
@semuelle
Copy link
Member

pinging @Ekaanth

@Ekaanth
Copy link
Author

Ekaanth commented Sep 19, 2023

Hi @semuelle, sorry for the late response, we have made the relevant changes.

Copy link
Collaborator

@takahser takahser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Ekaanth thanks for the update, it looks much better indeed and I can kind of grasp what you're planning to implement, at least from a high level. I still have some critics, especially when it comes to the deliverables - those are still quite vague, though it's already better than before. Try to be as specific as possible here and keep in mind that at the time you deliver the milestone the evaluator will have to check the specifications verify that the code has been implemented accordingly. For example, for this deliverable:

| 9. | Implementation of SDK | Develop SDK to facilitate the user's call to action |

You could basically just implement const helloWorld = () => "hello world";" and the spec would be fulfilled, if you argue that calling the helloWorld` function is the "call to action". This is why we're laying such a high emphasis on the deliverables being as detailed as possible.

That being said, I'm going to mark the proposal as ready for review since I think it's improved enough already to have other team members have a look at it. Personally, I won't approve it yet and I'm also not sure if $20k are justified for querying and storing blockchain data using subscan/subsquid.


- API: This layer helps in fetching details of the blockchain from the DB to the SDK, This is a transportation layer where it is fetching information from the DB based on the user commands. This module will also be self-hosted. The API endpoints will be private and accessible through SDK, the API endpoints will be written using node(typescript)

- SDK: This package will have all the necessary functions to fetch details about the NFT asset. Users will be able to download this package and query for the information of the asset on blockchain by providing various SDK commands. This module will be essentially integrated on the client side. Essentially the SDK will written in typescript and deployed on the NPM marketplace.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- SDK: This package will have all the necessary functions to fetch details about the NFT asset. Users will be able to download this package and query for the information of the asset on blockchain by providing various SDK commands. This module will be essentially integrated on the client side. Essentially the SDK will written in typescript and deployed on the NPM marketplace.
- SDK: This package will have all the necessary functions to fetch details about the NFT asset. Users will be able to download this package and query for the information of the asset on blockchain by providing various SDK commands. This module will be essentially integrated on the client side. Essentially, the SDK will written in typescript and published on the NPM marketplace.


- Blockchain layer: Access to the blockchain data source and crawl through the blocks from the genesis block, we will use using existing solution for crawling through the blockchain, we are exploring subscan for this solution and these are self-hosted or we will connect to external RPC nodes for fetching the data. We will be looking at subsquid for this solution.

- Middleware layer: Contains the crawlers/plugins which are written in typescript through which we can crawl the blockchain and collect information about NFTs write into the database and index it using elastic search. These indexers can index data based on the common NFR events and platform-specific events for better data availability. This module can be a self-hosted or SaaS this module can read and index any newer blockchain that adopts `pallet-nft` for the NFTs.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Middleware layer: Contains the crawlers/plugins which are written in typescript through which we can crawl the blockchain and collect information about NFTs write into the database and index it using elastic search. These indexers can index data based on the common NFR events and platform-specific events for better data availability. This module can be a self-hosted or SaaS this module can read and index any newer blockchain that adopts `pallet-nft` for the NFTs.
- Middleware layer: Contains the crawlers/plugins which are written in typescript through which we can crawl the blockchain and collect information about NFTs write into the database and index it using elastic search. These indexers can index data based on the common NFT events and platform-specific events for better data availability. This module can be a self-hosted or SaaS this module can read and index any newer blockchain that adopts `pallet-nft` for the NFTs.

| **0d.** | Docker | Dockerfiles and docker-compose.yml for running a development environment that locally spins up, which will be able to interact with the database to fetch the indexed information locally. |
| **0e.** | Article | Add an article covering how to integrate NFT-SDK in a project. |
| 1. | NFT Schema Development | Development of data models for the indexing of the NFT-related extrinsic so that users can get compressive information of the NFT lifecycle |
| 2. | Collection schema development | Development of NFT-collection schema models |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Development of NFT-collection schema models

By schema models do you mean data schemas? So basically, which attributes an object would have and which relations objects have between each other?

Suggested change
| 2. | Collection schema development | Development of NFT-collection schema models |
| 2. | Collection schema development | Development of NFT-collection data schemas |

| **0e.** | Article | Add an article covering how to integrate NFT-SDK in a project. |
| 1. | NFT Schema Development | Development of data models for the indexing of the NFT-related extrinsic so that users can get compressive information of the NFT lifecycle |
| 2. | Collection schema development | Development of NFT-collection schema models |
| 3. | NFT fractionalization schema development | Development of fractionalization schema models |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the difference to the previous deliverable 2. | Collection schema development here?

| 9. | Implementation of SDK | Develop SDK to facilitate the user's call to action |
| 10. | NFT indexer module | Index all the data that is scanned by the scanner module and pass it to the database layer. |
| 11. | Design and development of the NFT provenance SDK | A typescript-based SDK that consists of API calls to the database to fetch the NFT lifecycle. |
| 12. | Technical Documentation for the SDK | We will provide a compressive technical document outlining the process of integrating the SDK, available functionality, and data model types of the data that is fetched. |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably be part of 0b. Documentation

Comment on lines +112 to +123
| 1. | NFT Schema Development | Development of data models for the indexing of the NFT-related extrinsic so that users can get compressive information of the NFT lifecycle |
| 2. | Collection schema development | Development of NFT-collection schema models |
| 3. | NFT fractionalization schema development | Development of fractionalization schema models |
| 4. | NFT lifecycle schema development | Developing schema models for the NFT lifecycle, which includes the ownership, transfer, locking, or fractionalization of NFT |
| 5. | NFT pallet event indexer | Development of the module to monitor the latest block generation via subsquid |
| 6. | Asset pallet event and fractionalization indexer | Development of the module to monitor the pallet NFT actions and NFT fractionalization pallet actions |
| 7. | Integration of subsquid | Using subsquid to get the latest block information |
| 8. | Implementation of APIs | Development of endpoints that can ge used for SDKs request to fetch more details about the NFT lifecycle |
| 9. | Implementation of SDK | Develop SDK to facilitate the user's call to action |
| 10. | NFT indexer module | Index all the data that is scanned by the scanner module and pass it to the database layer. |
| 11. | Design and development of the NFT provenance SDK | A typescript-based SDK that consists of API calls to the database to fetch the NFT lifecycle. |
| 12. | Technical Documentation for the SDK | We will provide a compressive technical document outlining the process of integrating the SDK, available functionality, and data model types of the data that is fetched. |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, these deliverables are very vague:

  • I'm not sure we need multiple deliverables that are concerned with implementing "schema models". I'd expect that to be part of a module/library/SDK etc. Don't get me wrong - I think it's good that you're doing that (rather than using vanillajs with no custom data structures in the form of types/interfaces/classes, etc.) but I don't think there's a need to break them into deliverables. It's sufficient if you mention it in the spec of a deliverable (e.g. something like the SDK is going to be implemented using TypeScript and we're going to use custom interfaces for X, Y and Z).
  • Not sure what's the difference between 9. | Implementation of SDK and 11. | Design and development of the NFT provenance SDK
  • Not sure what is meant by "facilitate the user's call to action"
  • It'd be helpful to know which technologies and languages are being used in each deliverable.

@takahser takahser added ready for review The project is ready to be reviewed by the committee members. and removed changes requested The team needs to clarify a few things first. labels Sep 21, 2023
@semuelle
Copy link
Member

pinging @Ekaanth & @metaquity

@semuelle
Copy link
Member

semuelle commented Oct 4, 2023

I am closing this application due to inactivity. Please let me know if you want to continue working on it, @Ekaanth & @metaquity.

@semuelle semuelle closed this Oct 4, 2023
@Ekaanth
Copy link
Author

Ekaanth commented Oct 14, 2023

Thanks @semuelle @takahser for following through with the grant application, We did not proceed much since Kodvadot has played a similar grant and they have been approved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready for review The project is ready to be reviewed by the committee members.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants