Scaling of compute nodes

[judofyr]

Hi folks! Super excited about the work you're doing here, and I'm trying to understand more how it works (or is intended to work in the long-term). One quick questions:

You mention on the front page that Neon can "automatically scale based on demand". I assume that this refers to the compute nodes, but I don't see any mention how the system works when there's multiple of them.

• According to https://github.com/neondatabase/neon/blob/main/safekeeper/README.md there is only one primary compute node. Is this correct? Writes are only accepted by a single compute node?
• Is it possible to spin up more read-only compute nodes? If so, how do they choose an LSN?

[hlinnaka]

Hi folks! Super excited about the work you're doing here, and I'm trying to understand more how it works (or is intended to work in the long-term).

Thanks!

One quick questions:

You mention on the front page that Neon can "automatically scale based on demand". I assume that this refers to the compute nodes, but I don't see any mention how the system works when there's multiple of them.

It's largely not implemented yet; we are working on it. The compute node is shut down automatically if it's idle for 5 minutes, and started up again on connection. That works. The storage system is multi-tenant and runs on large instances, so that provides scalability by the feat of having a large instance that you can "burst" to. Similarly the compute nodes run in docker containers in a larger host, so we could scale individual containers up and down on-the-fly pretty easily, but there's no code to actually do it yet.

• According to https://github.com/neondatabase/neon/blob/main/safekeeper/README.md there is only one primary compute node. Is this correct? Writes are only accepted by a single compute node?

That's right.

• Is it possible to spin up more read-only compute nodes? If so, how do they choose an LSN?

Yes. You can choose an LSN yourself, or you can give a timestamp that is converted to an LSN, by finding the commit record closest to the timestamp. That's implemented in the storage already, but we haven't exposed it in the cloud service yet, we haven't figured out the right user interface for it yet.

[docteurklein]

   Writes are only accepted by a single compute node?

That's right.

Does that mean writes are (and will always be) vertically capped by the capacities of this single node?
Or is it planned to scale writes?