Falcon fails under websocket-bench

[jpcamara]

Running falcon using

bundle exec bento --falcon

If you run websocket bench, it misses almost every broadcast:

./websocket-bench broadcast ws://localhost:8080/cable \
                --concurrent 8 \
                --sample-size 100 \
                --step-size 200 \
                --payload-padding 200 \
                --total-steps 10 \
                --origin http://0.0.0.0 \
                --server-type=actioncable
200 / 200 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
100 / 100 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
[2024-10-01T22:51:35-04:00] clients:   200    95per-rtt:   7ms    min-rtt:   1ms    median-rtt:   1ms    max-rtt:   8ms
200 / 200 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
100 / 100 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
[2024-10-01T22:51:35-04:00] clients:   400    95per-rtt:   7ms    min-rtt:   1ms    median-rtt:   1ms    max-rtt:   7ms
200 / 200 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
100 / 100 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
[2024-10-01T22:51:35-04:00] clients:   600    95per-rtt:   7ms    min-rtt:   1ms    median-rtt:   1ms    max-rtt:   7ms
200 / 200 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
100 / 100 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
[2024-10-01T22:51:35-04:00] clients:   800    95per-rtt:   7ms    min-rtt:   1ms    median-rtt:   1ms    max-rtt:   8ms
200 / 200 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
100 / 100 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
[2024-10-01T22:51:36-04:00] clients:  1000    95per-rtt:   8ms    min-rtt:   1ms    median-rtt:   1ms    max-rtt:   8ms
200 / 200 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
100 / 100 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
[2024-10-01T22:51:36-04:00] clients:  1200    95per-rtt:   1ms    min-rtt:   1ms    median-rtt:   1ms    max-rtt:   1ms
200 / 200 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
100 / 100 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
[2024-10-01T22:51:36-04:00] clients:  1400    95per-rtt:  13ms    min-rtt:   1ms    median-rtt:   1ms    max-rtt:  13ms
200 / 200 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
100 / 100 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
[2024-10-01T22:51:36-04:00] clients:  1600    95per-rtt:  17ms    min-rtt:   1ms    median-rtt:   1ms    max-rtt:  17ms
200 / 200 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
100 / 100 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
[2024-10-01T22:51:36-04:00] clients:  1800    95per-rtt:   1ms    min-rtt:   1ms    median-rtt:   1ms    max-rtt:   1ms
200 / 200 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
100 / 100 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
Missing received broadcasts: expected 1100000, got 22982
[2024-10-01T22:51:38-04:00] clients:  2000    95per-rtt:   1ms    min-rtt:   1ms    median-rtt:   1ms    max-rtt:   1ms

If you modify Async::Executor to run an async reactor in a separate thread, things get a little better:

module Async
    class Executor
      class Timer < Data.define(:task)
        def shutdown = task.stop
      end

      private attr_reader :semaphore

      def initialize(max_size: 10)
        # @semaphore = ::Async::Semaphore.new(max_size)
        @queue = Queue.new
        Thread.new do
          Async do
            loop do
              block = @queue.pop
              Async do
                block.call
              end
            end
          end
        end
      end

      def post(task = nil, &block)
        task ||= block
        # semaphore.async(&task)
        @queue.push(task)
      end

It takes longer to run (like a minute instead of the seconds everything else takes), but the numbers are better:

200 / 200 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
100 / 100 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
[2024-10-01T22:52:47-04:00] clients:   200    95per-rtt:  82ms    min-rtt:   1ms    median-rtt:   1ms    max-rtt: 108ms
200 / 200 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
100 / 100 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
[2024-10-01T22:52:48-04:00] clients:   400    95per-rtt:   6ms    min-rtt:   1ms    median-rtt:   2ms    max-rtt:   6ms
200 / 200 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
100 / 100 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
[2024-10-01T22:52:51-04:00] clients:   600    95per-rtt:  10ms    min-rtt:   1ms    median-rtt:   2ms    max-rtt:  10ms
200 / 200 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
100 / 100 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
[2024-10-01T22:52:56-04:00] clients:   800    95per-rtt:  27ms    min-rtt:   1ms    median-rtt:   2ms    max-rtt:  27ms
200 / 200 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
100 / 100 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
[2024-10-01T22:53:02-04:00] clients:  1000    95per-rtt:  30ms    min-rtt:   1ms    median-rtt:   3ms    max-rtt:  31ms
200 / 200 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
100 / 100 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
[2024-10-01T22:53:10-04:00] clients:  1200    95per-rtt:  37ms    min-rtt:   1ms    median-rtt:   3ms    max-rtt:  37ms
200 / 200 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
100 / 100 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
[2024-10-01T22:53:20-04:00] clients:  1400    95per-rtt:  45ms    min-rtt:   1ms    median-rtt:   3ms    max-rtt:  65ms
200 / 200 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
100 / 100 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
[2024-10-01T22:53:35-04:00] clients:  1600    95per-rtt:  39ms    min-rtt:   1ms    median-rtt:   4ms    max-rtt:  39ms
200 / 200 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
100 / 100 [-----------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 2302 p/s
[2024-10-01T22:53:48-04:00] clients:  1800    95per-rtt: 129ms    min-rtt:   1ms    median-rtt:   4ms    max-rtt: 129ms
200 / 200 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
100 / 100 [-----------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 1382 p/s
Missing received broadcasts: expected 1100000, got 920988
[2024-10-01T22:54:07-04:00] clients:  2000    95per-rtt: 201ms    min-rtt:   1ms    median-rtt:   4ms    max-rtt: 201ms

Not saying this is correct per se, but definitely seem to be some issues with how it's currently setup. I'm still looking at it a bit as is @ioquatix but I wanted to post it here in the meantime.