Document about the ways to write unit tests for Bolt apps

[offbyone]

I am looking to write unit tests for my Bolt API, which consists of an app with several handlers for events / messages. The handlers are both decorated using the app.event decorator, and make use of the app object to access things like the db connection that has been put on it. For example:

# in main.py
app = App(
    token=APP_TOKEN,
    signing_secret=SIGNING_SECRET,
    process_before_response=True,
    token_verification_enabled=token_verification_enabled,
)
app.db = db

# in api.py:
from .main import app

@app.command("/slashcommand")
def slash_command(ack, respond):
    ack()
    results = app.db.do_query()
    respond(...)

The thing is, I cannot find any framework pointers, or documentation, on how I would write reasonable unit tests for this. Presumably ones where the kwargs like ack and respond are replaced by test doubles. How do I do this?

The page URLs

• https://slack.dev/bolt-python/

Requirements

Please read the Contributing guidelines and Code of Conduct before creating this issue or pull request. By submitting, you are agreeing to those rules.

[seratch]

Hi @offbyone, thanks for taking the time to write in!

As we have been receiving feedback about testing, it's on our radar. I cannot tell when we can release built-in test support but I myself will be working on it in the latter half of this year.

At this moment, the only thing I can suggest is to do something similar with the tests this project has. You can find so many tests under this package: https://github.com/slackapi/bolt-python/tree/v1.6.1/tests/scenario_tests

The key points are:

• Have your own mock server like this, which responds to Web API calls / response_url calls from test code. You don't need to use the standard HTTP server module for it.
• App instances in tests need to have client argument in its constructor. The client's base_url needs to point the mock server's URL this way
• Prepare request payload example data like this

In addition to above,

I cannot find any framework pointers

You can use any testing frameworks (my recommendation is pytest). The forthcoming testing support won't require your tests to rely on a specific test framework. We may provde some additional utility for pytest etc. for convenience, though.

Presumably ones where the kwargs like ack and respond are replaced by test doubles. How do I do this?

For respond, you can have a local mock HTTP server for handling the request. Regarding the ack call, it will be eventually converted to either an HTTP response or WebSocket message (Socket Mode) but before that, you can verify the value by checking response: Response returned by App#dispatch method as this test does.

I know that this approach is not so easy but it should be reasonable enough. Please consider going with it for now.

I hope this was helpful to you. If everything is clear for now, would you mind closing this issue? For the testing support task, I will come up with a bit more detailed issue soon. Also, for better documentation, we will add the section for testing once we release built-in testing support.

[offbyone]

Yeah, it ... helps. I guess I'd consider, if I were you, leaving this issue open to track that there is a gap, and resolving it when that documentation exists. If you don't track issues that way, please feel free to close it, but as far as I'm concerned this is still a bug until I can go to your documentation site and see instructions on how to test my bot :D

[seratch]

if I were you, leaving this issue open to track that there is a gap, and resolving it when that documentation exists

Yeah, I understand this way. It's also fine to keep this open for now. I have a link to this in the new issue that organizes the tasks/discussions around testing support. Thanks for the nice suggestion.

[offbyone]

I have a further question: Since this kind of testing requires constructing the app object differently, but the object is created at the module level, does someone there have a working pattern for how to do this in the bot code itself? Your example tests all create the app in the tests, but in a real Bolt application by the time you've imported the code, the App already exists and has been configured. So it's really tricky trying to sort out the right method here.

[carlos-alberto]

Hey @seratch is there any update on when this will be released?

[seratch]

@carlos-alberto I did a quick prototyping late last year. After that, we haven't had good progress due to other priorities. But we're aiming to provide a better solution sometime this year!

[I-Dont-Remember]

If it helps those who stumble on this in the future, this is an open-source Bolt app with a moderately sized test suite as an example to dig through.

We've taken the approach of keeping our core functionality in a separate module that doesn't depend on slack_bolt, letting us easily run tests without dealing with bolt trying to start a server or anything like that.

Then in app.py you can just import the module and use the Slack @decorated functions essentially as wrappers over your unit-testable business logic.

⚠ This only helps for unit tests, unfortunately haven't delved into integration tests, so unfortunately no review/feedback on seratch's suggestion.

[jimenezj8]

Any progress on this since the last update?

[seratch]

Hey @jimenezj8, this is still on my radar and actually I did quick prototyping last year. However, I cannot tell when we can deliver a stable solution due to our bandwidth and priorities. We will update you when we come up with something in the future.

[jimenezj8]

Thanks for the update @seratch 🙂 will keep my eyes peeled for another

[fr12k]

Hi i was looking into how to write unit tests for our slack bot and found this wonderful library.
https://github.com/gabrielfalcao/HTTPretty
That make's it easy to mock http requests.

Just be aware that the 1.1 version is broken if you want to use it use 1.0.5.
gabrielfalcao/HTTPretty#425

[ChadDa3mon]

Came across this as well and figured I'd try to contribute a bit. Here's some modified code from my current app, hopefully it can help someone else. Normally I'd have all of the payload stuff in my conftest.py file to keep things clean, but for the purposes of this I figured I'd put it all in one place.

import pytest
from unittest.mock import AsyncMock
from my.production.code import easter_egg

# This is the shortcut we expect to receive from Slack (well, a heavily truncated one)
mock_shortcut = {
        "message": {
            "text": "Hello, World!",
            "ts": "1234567890.123456"
        },
        "user": {
            "id": "U123456",
            "name": "John Doe"
        },
        "channel": {
            "name": "general"
        },
        "trigger_id": "trigger_12345"
    }

# Upon receiving this shortcut, we will send a 'view' to our user
# This is what we expect it to look like
expected_modal = {
    "type": "modal",
    "callback_id": "hello-modal",
    "title": {
        "type": "plain_text",
        "text": "Greetings!"
    },
    "submit": {
        "type": "plain_text",
        "text": "Good Bye"
    },
    "blocks":[
        {
          "type": "section",
          "text": {
            "type": "plain_text",
            "text": "Frosty the Snowman",
            "emoji": True
          }
        },
        {
          "type": "video",
          "title": {
            "type": "plain_text",
            "text": "Frosty the Snowman",
            "emoji": True
          },
          "title_url": "https://www.youtube.com/watch?v=RRxQQxiM7AA",
          "description": {
            "type": "plain_text",
            "text": "Frosty the Snowman",
            "emoji": True
          },
          "video_url": "https://www.youtube.com/embed/vjscH2WBWjw?feature=oembed&autoplay=1",
          "alt_text": "Frosty the Snowman",
          "thumbnail_url": "https://i.ytimg.com/vi/bSzBBK8gC6c/hqdefault.jpg",
          "author_name": "",
          "provider_name": "YouTube",
          "provider_icon_url": "https://a.slack-edge.com/80588/img/unfurl_icons/youtube.png"
        }
    ]
}

@pytest.mark.asyncio
async def test_easter_egg():
    ack = AsyncMock()
    client = AsyncMock()
    client.views_open = AsyncMock()

    # Test my actual production function 'easter_egg'
    # This should ack the shortcut and then open a view to the user
    await easter_egg(ack, mock_shortcut, client)
    ack.assert_awaited_once # Make sure it was acknowledged
    client.views_open.assert_awaited_once_with( # Make sure the view we get back is correct
        trigger_id=mock_shortcut["trigger_id"],
        view=expected_modal
    )

[leon-adarcewicz]

Any progress on this since the last update?

[zimeg]

@leon-adarcewicz the bolt-python-jira-functions sample app can be good reference with a few of the unit tests for listeners - similar to what @ChadDa3mon shared above! 🧪

The tests for Workflow Buddy from @I-Dont-Remember are also solid at testing the overall listener logic while mocking HTTP requests is another great option from @fr12k 🙏

I'm not aware of updates to built-in test helpers for Bolt Python, but these all seem like good testing approaches to me. I hope this is helpful for now, and in some neartime I'll explore this a bit more to include on slack.dev/bolt-python 📚

[leon-adarcewicz]

Thank you @zimeg!

[dsoprea]

@seratch Any feedback on the current state of unit-testing support?

[seratch]

No updates from me lately. To align with our platform roadmap and priorities, I am not actively working on the prototype right now.

[NathanSmeltzer]

This may help with testing, but I haven't tried it yet:
https://github.com/ygalblum/slack-server-mock