Support for Examples

[ursenzler]

Yesterday, I had an idea about how to add support for examples in MSpec.

I'd like to write this:

[Subject("Examples")]
public class WhenIDoThis
{
    Establish context = () => SetUpEverything();

    Examples<Example> examples = () =>
        {
             return new[] { new Example("hello"), new Example("world") };
        }

    Because<Example> of = example => DoSomething(example);

    It should_do_that = () => CheckThat_HoldingForAllExamples();

    It<Example> should_do_that_too = example => CheckSomethingExampleSpecific(example);
}

with

public class Example
{
public Example(string someData) { this.SomeData = someData; }
public string SomeData { get; private set; }
}

This would allow to add specific examples to a specification.

Of course, the examples should be shown in the reports (e.g. show all Properties of Example, or call ToString on Example).

A goody would be, that an individual example could be run from the R# runner (like TestData of NUnit or InlineData of xUnit)

What do you think about this?

[agross]

This looks interesting. What would be a good example of a spec that would require examples?

[brase]

Reminds me of the concept of row tests in classic test frameworks.

When you run specs based on selenium and mspec you need to write a lot of duplicated code. Think of validators for form fields. Behaviours can help you there to write less code but it does not feel correct and there is still a lot of duplicated code.

[ursenzler]

An example from my project:
We support that a user can use functions to evaluate values in documents. There are about a dozen of different functions, which can be used in over 20 different places. No we have 12x20 specs for this.
With examples we could melt the specs down to 20 specs each with 12 examples (the different functions).

Typical cases from the literature are:

• examples for calculating parking lot pricing (5min -> x, 15min->x, 20min -> y, ....)
• example for discount calculations

Examples help communication with non-technical people, too.

[Slesa]

You can badly read It as a sentence. Wouldn't it better to call it
Processing should_do_something =

[ursenzler]

It and Processing are both okay for me. Processing reads better, but it's a new keyword.

[danielmarbach]

Hy folks

I looked into this a bit this weekend, here is my current idea which allows existing users to grasp the feature without learning something new:

public class ContextWithExample : IFakeContext
{
    public static bool ItInvoked = false;
    public static bool ContextInvoked = false;
    public static bool CleanupInvoked = false;

    public static List<Example> becauseExamples = new List<Example>();
    public static List<Example> itExamples = new List<Example>();

    Establish<Example> foo = () =>
    {
        ContextInvoked = true;

        return new[] { new Example("foo"), new Example("bar")};
    };

    Because<Example> of = example =>
    {
        becauseExamples.Add(example);
    };

    It<Example> is_a_specification_with_example = example =>
    {
        itExamples.Add(example);
    };

    It is_a_specification = () =>
    {
        ItInvoked = true;
    };

    public void Reset()
    {
        ItInvoked = false;
        ContextInvoked = false;

        becauseExamples.Clear();
        itExamples.Clear();
    }

    public class Example
    {
        public Example(string content)
        {
            Content = content;
        }

        public string Content { get; private set; }

        public override string ToString()
        {
            return Content;
        }
    }
}

I see the argument that the new generic It doesn't read so nicely but nonetheless I think introducing a new assertion clause makes no sense. There is one caveat:

When you are using resharper the generic type is always selected first. But I think almost all mspec users have live templates which they rely on. Or this could be used by not putting the new generic delegates onto the Framework.cs but one namespace deeper which would users force to add an explicit using. My gut feeling is more towards relying that users have their templates setup so I wouldn't bury the example feature in deeper namespaces.

The current design / architecture of mspec model only uses a Context per Context and several specifications. If we design it properly the examples API above could become a first class citizen but this would require significant refactoring internally. OR we push the examples feature into the context and specification model and use the Result model to extend the result with additional stuff from the examples. Any opinions on this @agross ?

[danielmarbach]

I looked again over it and also tried to define custom delegate types. I came to the conclusion that we have to seperate the establish from the example phase. Here the adaption:

public class ContextWithExample : IFakeContext
{
    public static bool ItInvoked = false;
    public static bool ContextInvoked = false;
    public static bool CleanupInvoked = false;

    public static List<Example> becauseExamples = new List<Example>();
    public static List<Example> itExamples = new List<Example>();

    Establish<Example> foo = example =>
    {
        ContextInvoked = true;
    };

    Examples<Example> examples = () =>
    {
        return new[] { new Example("foo"), new Example("bar") };
    };

    Because<Example> of = example =>
    {
        becauseExamples.Add(example);
    };

    It<Example> is_a_specification_with_example = example =>
    {
        itExamples.Add(example);
    };

    It is_a_specification = () =>
    {
        ItInvoked = true;
    };

    public void Reset()
    {
        ItInvoked = false;
        ContextInvoked = false;

        becauseExamples.Clear();
        itExamples.Clear();
    }

    public class Example
    {
        public Example(string content)
        {
            Content = content;
        }

        public string Content { get; private set; }

        public override string ToString()
        {
            return Content;
        }
    }
}

and here with custom delegates:

 [Subject(typeof(Account), "Funds transfer example")]
 public class when_transferring_between_two_accounts_with_examples
 {
  static Account fromAccount;
  static Account toAccount;

  Given<Transfer> accounts = transfer =>
  {
      fromAccount = new Account { Balance = transfer.FromAccountBalanceBeforeTransfer };
      toAccount = new Account { Balance = transfer.ToAccountBalanceBeforeTransfer };
  };

  Examples<Transfer> transfers = () =>
  {
      return new[] {new Transfer { Amount = 1m, FromAccountBalanceBeforeTransfer = 1m, ToAccountBalanceBeforeTransfer = 1m, FromAccountBalanceAfterTransfer = 0m, ToAccountBalanceAfterTransfer = 2m }, };
  };

  When<Transfer> transfer_is_made =
     transfer => fromAccount.Transfer(transfer.Amount, toAccount);

  Then<Transfer> should_debit_the_from_account_by_the_amount_transferred =
    transfer => fromAccount.Balance.ShouldEqual(transfer.FromAccountBalanceAfterTransfer);

  Then<Transfer> should_credit_the_to_account_by_the_amount_transferred =
    transfer => toAccount.Balance.ShouldEqual(transfer.ToAccountBalanceAfterTransfer);

  public class Transfer
  {
      public decimal FromAccountBalanceBeforeTransfer { get; set; }
      public decimal ToAccountBalanceBeforeTransfer { get; set; }
      public decimal FromAccountBalanceAfterTransfer { get; set; }
      public decimal ToAccountBalanceAfterTransfer { get; set; }

      public decimal Amount { get; set; }

      public override string ToString()
      {
          return
              string.Format(
                            "Transfering {0} from account with initial balance {1} to account with initial balance {2}", Amount, FromAccountBalanceBeforeTransfer, ToAccountBalanceBeforeTransfer);
      }
  }

}

[danielmarbach]

With the new examples delegate it would be possible to implement the support in the context and providing a new IContextRunner for that purpose. The support would be built like the current SetupForEachSpecification is done. The context behaves differently when examples are provided. The context runner could call EstablishContext for each example which then internally make executes the establish and because with the current sample. The ugly thing is that introduces more state into the context but would be straight forward to implement. Thoughts?

[danielmarbach]

First extremely rough draft can be found here:

https://github.com/danielmarbach/machine.specifications/tree/Examples

[danielmarbach]

Also see here http://www.planetgeek.ch/2013/05/26/rowtest-theory-testdata-support-for-machine-specifications/

[philippdolder]

I like how it looks like. Though I personally would prefer having the feature in the Examples sub-namespace to prevent distraction in IntelliSense when you're not using the examples feature in your current spec.

@danielmarbach how does that feature work when I have a base spec. Can I have a non-generic Establish block in the base and an Establish<Example> block in my spec where I have the Because<Example> and It<Example>?

[ursenzler]

I'd prefer to keep the example stuff in the main namespace. Hiding it in a namespace will not help people finding and using it.

[philippdolder]

My point is that ReSharper automatically suggests the generic types first. So if not using templates you usually will have to do more keyboard presses to achieve the same as now. Otherwise I would also suggest having the feature in the main namespace. But that's just my 50 cent

[danielmarbach]

There is definitely the drawback when you hide it in a namespace that users have difficulties to find it. On the other hand when you have it in the same namespace you pollute intelli-sense with the generic one which is always chosen first with resharper installed. That can also be a PITA. For example xunit has the theories also in a seperate namespace. NUnit has it in the same namespace. Just saying :D

[danielmarbach]

https://github.com/danielmarbad ch/machine.specifications/tree/Examples has updated implementation. Can anyone chim in and do the resharper support? My time to work on that stuff is currently extremely limited.

[citizenmatt]

Hi guys. I'm afraid I can't offer to implement this for you (I'm off on holiday! Hooray!) but here are a few pointers. I've updated the plugin dev guide to describe how to add support for row tests. (Updated 18/07/2013 with fixed link)

The dev guide is written with nunit as the example, as it is a simple structure that the majority of people will be familiar with - test class -> test method -> row test.

I'm not entirely sure how it maps to the mspec model, but the concept is the same - in the runner, when you encounter a test that you don't have a RemoteTask for, you create it locally and call IRemoteTaskServer.CreateDynamicElement. In your IUnitTestProvider you also implement IDynamicUnitTestProvider and create a child element that represents the row test.

[marcofranssen]

This is great feature. It really simplifies testing different code paths on different values used in the examples.

[denisivan0v]

Will this feature be included in future releases? I really need it in my project.
Is there a roadmap for the releases?

[danielmarbach]

My biggest hurdle was the resharper support. I have it running roughly for the consolerunner that is why I hesitaed to include it

Am 08.10.2013 um 08:07 schrieb denisivan0v [email protected]:

Will this feature be included in future releases? I really need it in my project.
Is there a roadmap for the releases?

—
Reply to this email directly or view it on GitHub.

[dotnetprofessional]

What's the progress on this feature? Its been over a year since it was first described as a useful feature (which it would be). I find the lack of this feature to be the biggest issue using the framework.

[ursenzler]

The MSpec teams needs to solve some infrastructure problems first, before
adding new functionality (e.g. version independent runner, multi version
targeting of R#).
But believe me, I ask them every week about it :-)
And I'm currently looking for a sponsor that would pay for this feature.

On Fri, Apr 25, 2014 at 4:51 AM, dotnetprofessional <
[email protected]> wrote:

What's the progress on this feature? Its been over a year since it was
first described as a useful feature (which it would be). I find the lack of
this feature to be the biggest issue using the framework.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/141#issuecomment-41354482
.

[danielmarbach]

@ursenzler Thanks Urs for jumping in.

We have a working spike on @fabiantrottman ‘s clone. You can use it at your own risk if you want to rush things. I’m currently working on my freetime on this, Fabian can from time to time work on it half freetime half education in bbv software services. We do our best to progress but I cannot give any due date at the moment.

[FabianTrottmann]

I got it working based on a redesigned unstable mspec version which is quite similar to the current devlope branch (actually the develope branch is based on this redesign). Currently i'm working on the integration of the example feature into this develope branch (we had to fix some bugs first to get it ready for the examples). I'll send a pull request in the next days. Sorry for the delay

[danielmarbach]

No need to say sorry. 0.9.0 will be about version independance anyway. I started pulling out the runner utility. Resharper comes next

[dotnetprofessional]

@danielmarbach, @FabianTrottmann

Wasn't trying to give you guys a hard time over the delay, more to get an update. So I appreciate the quick response on that. Though if I sparked more interest in @FabianTrottmann to complete it, well, that can't be a bad thing :)

Look forward to the release...

[RedwoodForest]

@dotnetprofessional @danielmarbach, @FabianTrottmann

We're looking forward to this one too!

Thanks for all your hard work on this project.

[dgioulakis]

Any updates on this feature? Thanks guys!

[erlandsona]

Any updates on this feature? Thanks guys!

[octaviobffernandes]

Hey guys, any updates on this?
This is a very interesting feature, specially for those used to Theory/TestData features of xUnit and NUnit. It would add a big value to MSpec, IMO.

[ivan-prodanov]

Guys, it's been 4 years. Any updates?