Developer Guide

Welcome to the Netman Developer Guide!

Here you will find a breakdown of the following items

• TLDR
• The Netman Buffer Object Life Cycle
  • How the heck does the api work?
  • What is a provider?
• How to create a provider!
  • Initial Considerations
  • Integration with api
  • Help my provider is broke!

TLDR

The Netman Buffer Object Life Cycle

The "Netman Buffer Object" is the object that api creates to help keep track of data associated with a neovim buffer. This is automatically created by the api when neovim creates a buffer (via the FileReadCmd, or BufReadCmd). This object is only created for buffers where the file uri being opened is associated with a provider.

But how does api associate a uri with a provider?

Netman is a clever little program, it only pays attention to buffer open events based on the file name (as neovim considers it). Netman selectively creates event listeners specifically for protocols as registered by providers. This means that if you have a provider loaded for ssh

spoiler alert

you do if you are using Netman

and no other providers, Netman will only listen to files opened that start with ssh related protocols (as defined by the ssh provider). There is no limit to the number of providers that Netman can have loaded at one time, Netman will only listen for events related to the protocols that the loaded providers specify they can handle. These events listeners (called autocommands in vim speak) are all cleanly housed in the Netman command group (called augroup in vim speak).

Netman will use this object to track internal metadata about the uri (its provider, if it has a local file stored somewhere, what buffer its on, a provider cache, etc). This is to help prevent Netman from having to redo logic when seeing a uri that is has already processed. Additionally, this buffer object contains a provider specific cache that is passed to the provider on most function calls so the provider can safely store "relevant" information to the uri.

Ok thats cool but what about when the user is done with the file?

Netman has its greedy little hooks into a handful of places in neovim's event system, one of those places being the BufUnload event. When neovim fires this event on a buffer that Netman is watching, Netman will receive the event and proceed to clear out the object from its memory. Additionally, if the provider associated with the buffer has implemented the close_connection function, Netman will call out to it to inform it that the buffer for the uri was closed.

Sounds pretty cool, but

How the heck does the api work?

The api is the main "guts" of Netman. It sits between neovim (and therefore the end user) and the provider. Both of them communicate with it via a standard set of functions, and this allows the api to communicate "between" them in an abstract way (so the end user doesn't have to care how to interface with a protocol and the provider doesn't have to care about how to interface with a user).

Sounds cool but why would a user talk to Netman instead of neovim?

The best part of all of this is that Netman's api cleanly integrates itself into neovim and thus the user (and neovim) don't have to care about how to talk to it. The user will simply utilize neovim as they would regularly do, except Netman provides additional functionality to interface with remote data via the provider structure. When a user opens a remote location (via uri), Netman will take over and ensure a clean experience between the user and the provider.

How does Netman do that?

Netman has the following events set for providers that are registered with it

• FileReadCmd
  • This autocommand is used to capture when neovim is opening a file with a protocol that Netman supports.
• BufReadCmd
  • This autocommand is used to capture when neovim is opening a buffer with a protocol that Netman supports.
• FileWriteCmd
  • This autocommand is used to capture when the user is writing out to a file (before the write occurs) for a buffer that Netman is watching.
• BufWriteCmd
  • This autocommand is used to capture when the user is writing out their buffer (before the write occurs), when the buffer is one that Netman is watching
• BufUnload
  • This autocommand is used to capture when a relevant buffer is being closed by the user. Netman will reach out to the associated provider to inform it that the buffer is being closed

When a ReadCmd event is fired, Netman forwards the associated uri to its read command, where the api establishes which provider should handle the read, and then provides the results from the provider to the user. This is laid out more in the api documentation

Additionally, Netman does expose a vim command :Nmread which directs to the read api. This can be used by the user but is more meant for you the developer.

When a WriteCmd event is fired, Netman forwards the associated uri to its write command, where the api grabs the cached provider and informs it that the user wishes to write out their buffer to this uri. More details on how write works is explained in the api documentation

Additionally, Netman does expose a vim command :Nmwrite which directs to the write api. This can be used by the user but is more meant for you the developer.

There is alot of talk about providers

What is a provider?

A provider is a program that sits between Netman and an external data source that is not reachable in "traditional" means. An example of a provider is the builtin ssh provider, netman.providers.ssh. In this case, the ssh provider sits between Netman and ssh related programs (ssh, sftp, scp), and since it has implemented the required provider interface, api is able to safely assume that it can be communicated with to gather information from the various ssh related programs when a user requests to do so (with a uri, such as sftp://myhost/my/super/secret/file).

A provider should return consistent data (as declared in the api documentation), though it does not have to store anything within the local filesystem.

That sounds pretty cool but Netman doesn't support X protocol.

How to create a provider!

So you want to create a provider for protocol X? You've come to the right place! We are going to be creating a provider for docker, follow along!

Initial Considerations

Before we can begin creating our shiny new provider, we need to take a few things into consideration.

First! Have we searched for providers that might do what we are looking for on github? It could be that a provider exists to handle X protocol. If not (or you feel like making your own anyway), onto the second question

Second! What is the target program(s) of our provider? For docker, we care about the docker program. Thus, we will need to ensure that docker exists when the provider is initialized, and if it doesn't exist, we should not intialize (along with log) that we were not able to find all our dependencies. This is critical for users when they are expecting to be able to use a provider and it isn't present. The inevitable "It didn't work" can be prevented with proper error handling and communication with the user on when your dependencies aren't met.

Third! What edge cases might we run into while communicating with our program (docker in this case), and how will we handle them? It is important to know what might go wrong beforehand and ensure that we account for those scenarios, or at the very least call them out so the user has some point of reference upon errors. In docker's case, the following considerations need to be accounted for

• Docker isn't installed
  • In this case, we should simply not initialize (this is covered more below)
• The target container doesn't exist
  • In this case, we should simply reject (return nil) any requests to this container, as well as notify the user that their request is nonsensical.
• The target container isn't running
  • Here, we can handle this in one of 3 ways
    • We can die and error out that the container isn't running
    • We can prompt the user to see if we should attempt to start the container
    • We can attempt to autostart the container
• The target container doesn't have the appropriate programs installed for introspection
  • Our preferred method of traversing the file system will be to execute find within the container (much like we do in the ssh provider). This should be available on most containers but it might not be. If this is the case, we can do one of the following options
    • We can die and complain that we can't properly introspect the container
    • We can try a fallback program for introspection (such as ls)
    • We can try utilizing docker tools to externally interface with the container contents

With these considerations in mind (and valid research done), lets dive into creating our new provider!

First Steps

The first thing to do is create a new repo for our provider (docker will be included in the Netman core but for the sake of this guide, we will create a new repository)

$ git init docker-provider-netman
Initialized empty Git repository in /home/miversen/git/docker-provider-netman/.git/

There is no specific naming convention for providers, name your provider whatever you would like!

Once we have our provider repo started, lets enter the directory and create the following file structure

$ cd docker-provider-netman
$ mkdir -p lua/docker-provider-netman
$ cd lua/docker-provider-netman
$ touch init.lua

Here we are creating the basic lua file structure that neovim will be looking for when a user attempts to import our plugin. At this point, your project should look something like this

❯ ls -R docker-provider-netman 
.:
lua

./lua:
docker-provider-netman

./lua/docker-provider-netman:
init.lua

Or more clearly

> lua
    > docker-provider-netman
        > init.lua

Lets add the init.lua (in its current blank form) to the repo

$ git add init.lua

We can verify that the file has been added via

$ git status
On branch master

No commits yet

Changes to be committed:
  (use "git rm --cached <file>..." to unstage)
        new file:   init.lua

Lets stage and commit it so we have a safe fallback when we are working!

$ git stage init.lua
$ git commit -m "Initial Filestructure"
[master (root-commit) 64aacb6] Initial Filestructure
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 lua/docker-provider-netman/init.lua

You should get something like the above. If you are having issues with working through git, atlassian provides a very helpful walkthrough of how git works and how to use it. Moving forward, we will focus on the development of the plugin and less on the stuff going on around it (meaning, there won't be more shell commands or explainations about git/shell)

Creating our basic provider structure

The Netman API calls the following attributes that must be defined on every provider

• read
• write
• delete
• name
• get_metadata
• protocol_patterns
• name
• version

It also calls out the following optional functions

• init
• close_connection

Integration with api

How to troubleshoot your shiny new provider