Skip to content

Releases: ponylang/ponyc

0.52.5

30 Dec 20:53
Compare
Choose a tag to compare

Fix compiler crash when attempting to mutate a val field

In Pony 0.52.4 we fixed an unsoundness bug that had been introduced over the summer. While fixing that bug, an error case was missed and as a result, a compiler crash was created where an error message should have been generated.

We've fixed the crash and it now displays a "left side is immutable" error message.

[0.52.5] - 2022-12-30

Fixed

0.52.4

29 Dec 11:07
Compare
Choose a tag to compare

Fix crash when handling parameters with invalid types

Fixes a compiler assert when checking a parameter for autorecovery that has an invalid type.

Fix some infinite loops during typechecking

Some recursive constraints could cause the typechecker to go into
an infinite loop while trying to check subtyping. These infinite
loops have now been fixed. An example program that demonstrates
the issue is below:

primitive Foo[T: (Unsigned & UnsignedInteger[T])]
  fun boom() =>
    iftype T <: U8 then
      None
    end

actor Main
  new create(env: Env) =>
    Foo[U8].boom()

The use of T: UnsignedInteger[T] is valid and expected, but wasn't
being handled correctly in this situation by the typechecker in this context.

Fix a major bug which allowed mutating fields of a val object

A bug was introduced in 0.51.2 which caused some checks to allow mutating fields of a val object due to changes to some operations on capabilities. This bug is fixed: immutability is now checked separately from other properties.

If you are using Pony 0.51.2 to 0.52.3, you should upgrade to the 0.52.4 release that contains this fix as soon as possible.

[0.52.4] - 2022-12-29

Fixed

  • Fix an assert in call.c when checking an invalid argument for autorecover (PR #4278)
  • Fix an issue with infinite loops while typechecking some expressions (PR #4274)
  • Fix soundness bug introduced in Pony 0.51.2 (PR #4283)

0.52.3

16 Dec 13:01
Compare
Choose a tag to compare

Deprecate macOS on Intel support

We have moved macOS on Intel from being a fully supported platform to a best effort platform. We are no longer providing prebuilt ponyc binaries for macOS on Intel. We also no longer do any CI testing using macOS on Intel. We are not planning on intentionally breaking macOS on Intel but supporting it is no longer a priority.

Fix segfault caused by unsafe garbage collection optimization

Prior to this change, there was an unsafe optimization in the Pony runtime. The optimization is detailed in the ORCA paper on the garbage collection protocol and is usually safe, but sadly not always.

The optimization cuts down on the amount of tracing that is done when an object is sent from one actor to another. It is based on the observation that for the sake of reference counting, we don't need to count every object in a graph that is sent from actor A to actor B so long as the root of the graph being sent is immutable. This optimization provides a large performance boost over tracing all objects sent from one actor to another. It also will from time to time, introduce a segfault that takes down the runtime.

Issue #1118 is the most obvious instance of the bug caused by the optimization. The core of the problem is that when an actor's reference count hits 0, it should be able to be reaped. However, if a reference to an actor is sent to another actor inside an immutable object, the actor will not be traced on send and might get reaped while references to it exist. Once that happens, a segfault is guaranteed.

We have fixed the safety problem by tracing every object sent between actors. In not very rigorous testing using a modified version of message-ubench, we saw a 1/3 drop in performance compared to running with the safety problem/optimization enabled. It should be noted that the 1/3 drop in performance is probably the high-end in terms of performance hit and many Pony programs will see little to no performance change.

Our plan moving forward is to start adding smarts to the compiler in an incremental fashion so that we can turn the "problematic optimization" back on when it isn't problematic. We will never recover "all lost performance" because there are times when the optimization is unsafe and no amount of compiler smarts will allow us to turn the optimization on in those situations. We can however, over time, turn the optimization back on for more and more types. We expect this process to take a while and welcome community involvement in the project.

Fix "Don't include 'home file' in documentation search"

PR #4243 incorrectly implemented the "don't include 'home file' in documentation search". It has been fixed.

The original release notes are now accurate (before it wasn't working!):

We've updated the documentation generation to not include the "home file" that lists the packages in the search index. The "exclude from search engine" functionality is only available in the mkdocs-material-insiders theme.

The home file is extra noise in the index that provides no value. For anyone using the insiders theme, this will be an improvement in the search experience.

[0.52.3] - 2022-12-16

Fixed

  • Fix segfault caused by unsafe garbage collection optimization (PR #4256)
  • Fix incorrectly implemented PR #4243 (PR #4276)

Changed

  • Remove macOS on Intel as a supported platform (PR #4270)

0.52.2

01 Dec 13:53
Compare
Choose a tag to compare

Don't include generated documentation "home file" in search index

We've updated the documentation generation to not include the "home file" that lists the packages in the search index. The "exclude from search engine" functionality is only available in the mkdocs-material-insiders theme.

The home file is extra noise in the index that provides no value. For anyone using the insiders theme, this will be an improvement in the search experience.

Add support for erase left and erase line in term.ANSI

We've implemented RFC #75 that set out a means of updating ANSI in the term package to support not just ANSI erase right, but also erase left, and erase line.

Existing functionality continues to work as is. All existing erase right functionality will continue to work without breakage. The erase method has been augmented to take an option parameter for the direction to erase. EraseRight is the default, thus keeping existing behavior. EraseLeft and EraseLine options have been added as well.

To erase left, one would do:

ANSI.erase(EraseLeft)

To erase right, you could do either of the following:

ANSI.erase()
ANSI.erase(EraseRight)

And finally, to erase an entire line:

ANSI.erase(EraseLine)

Improve TCP Backpressure on Windows

Our prior setting of backpressure for TCP writes on Windows was naive. It was based purely on the number of buffers currently outstanding on an IOCP socket. The amount of data didn't matter at all. Whether more data could be accepted or not, also wasn't taken into consideration. We've enhanced the backpressure support at both the Pony level in TCPConnection and in the runtime API.

Two runtime API methods have been updated on Windows.

pony_os_writev

The Windows version of pony_os_writev will now return the number of buffers accepted or zero if backpressure is encountered. All other errors still cause an error that must be handled on the Pony side of the API via a try block.

pony_os_send

The Windows version of pony_os_send will now return the number of bytes accepted or zero if backpressure is encountered. All other errors still cause an error that must be handled on the Pony side of the API via a try block.

The changes are considered non-breaking as previously, the return values from both functions had no meaning.

Fix multiple races within actor/cycle detector interactions

The Pony runtime includes an optional cycle detector that is on by default. The cycle detector runs and looks for groups of blocked actors that will have reference counts above 0 but are unable to do any more work as all members are blocked and don't have additional work to do.

Over time, we have made a number of changes to the cycle detector to improve it's performance and mitigate its impact on running Pony programs. In the process of improving the cycle detectors performance, it has become more and more complicated. That complication led to several race conditions that existed in the interaction between actors and the cycle detector. Each of these race conditions could lead to an actor getting freed more than once, causing an application crash or an attempt to access an actor after it had been deleted.

We've identified and fixed what we believe are all the existing race conditions in the current design.

Update supported FreeBSD to 13.1

Prebuilt FreeBSD ponyc binaries are now built for FreeBSD 13.1. All CI is also done using FreeBSD 13.1. We will make a "best effort" attempt to not break FreeBSD 13.0, but the only supported version of FreeBSD going forward is 13.1.

Previously built ponyc versions targeting FreeBSD 13.0 will continue to be available.

Drop Rocky 8 support

In July of 2021, we added prebuilt binaries for Rocky 8. We did this at the request of a ponyc user. However, after attempting an update to get Rocky 8 building again, we hit a problem where in the new CI environment we built, we are no longer able to build ponyc. We do not have time to investigate beyond the few hours we have already put in. If anyone in the community would like to get the Rocky 8 environment working again, we would be happy to add support back in.

The current issue involves libatomic not being found during linking. If you would like to assist, hop into the CI stream on the ponylang Zulip.

Fix compiler crash when calling (this.)create()

Fixed a crash when calling .create() (with an implicit this).

[0.52.2] - 2022-12-01

Fixed

  • Fix multiple races within actor/cycle detector interactions (PR #4251)
  • Fix crash when calling (this.)create() (PR #4263)

Added

  • Don't include "home file" in documentation search (PR #4243)
  • Add support for erase left and erase line in term.ANSI (PR #4246)

Changed

  • Improve TCP backpressure handling on Windows (PR #4252)
  • Update supported FreeBSD to 13.1 (PR #4185)
  • Drop Rocky 8 support (PR #4262)

0.52.1

14 Nov 18:05
Compare
Choose a tag to compare

"Exclude source files" from documentation source

In the previous ponyc release, we updated the default theme used by our documentation generation from a custom theme based on mkdocs-material to using the latest mkdocs-material. With the change came the possibility of using all the latests mkdocs-material features. Mkdocs-material is a heavily developed theme that has a ton of great features. Some of mkdocs-material's features are only available to sponsors of the project.

One of the many excellent insider's features is the ability to exclude some files from the mkdocs search index. We've added support for this feature in the generated documentation so that, iff you are a mkdocs-material sponsor, the source code files that we link to from generated documentation will no longer be included in the source index.

If you've used our generated documentation extensively, you know that having the source implementation linked is a great boon, but it is also a pain as it clutters up the search index and you often end up on a source implementation page from a search.

If you are using the built in documentation generation that comes with ponyc, we suggest that you take a look into becoming a sponsor of mkdocs-material as we will be adding support for additional insiders features if folks find them to be useful as part of the generated documentation.

[0.52.1] - 2022-11-14

Added

  • Update docgen to generate source files that are ignored (PR #4239)

0.52.0

10 Nov 20:24
Compare
Choose a tag to compare

Fixed an overflow in Time.nanos() values on Windows

Based on how were were calculating nanos based of the QueryPerformanceCounter and QueryPerformanceFrequency on Windows, it was quite likely that we would eventually overflow the U64 we used to represent nanos and "go backwards in time".

Fix incorrect interaction between String/Array reserve and Pointer realloc

This was an interesting one. So, there's was an interesting interaction between String and Array's reserve and Pointer's realloc. Generally, it went like this:

reserve when it called realloc would give realloc a single value, the size of the newly reserved chunk of memory. This would result in realloc copying not just the in-use items from old memory to newly allocated memory, but in fact, the entire old memory chunk.

This could lead to very surprising results if reserve had been called by undefined. And in general, it could lead to suboptimal performance as we could end up copying more than was needed.

realloc has been updated to take to parameters, now, the amount of memory to alloc and the number of bytes to copy from old memory to new.

Sort package types in documentation

In order to make documentation on a class, actor, primitive or struct easier to find, they are now sorted alphabetically inside their package.

Adapt documentation generation to only depend the mkdocs material theme

We used to have our own mkdocs theme for ponylang documentation, it was based off of the mkdocs-material theme. This change puts the adaptations we need into the mkdocs.yml file itself, so we don't depend on our own theme anymore, but only on the upstream mkdocs-material. This eases the maintenance burden by not having to catch up with the material theme codebase.

This is a Breaking Change in so far that users that want to generate documentation with ponyc now have to install the python package mkdocs-material instead of mkdocs-ponylang to generate their documentation in HTML form with mkdocs. The library documentation github action will have the correct dependencies installed, so no changes should be needed when using it.

Fix broken documentation generation on Windows

Generating documentation has been broken on Windows: ponyc was not able to write the documentation files, due to path handling issues on Windows. This is now fixed and documentation generation is working again on Windows.

[0.52.0] - 2022-11-10

Fixed

  • Avoid fairly easy to trigger overflow in Windows Time.nanos code (PR #4227)
  • Fix incorrect interaction between String/Array reserve and Pointer realloc (PR #4223)
  • Fix broken documentation generation on Windows (PR #4226)

Changed

  • Sort package types in documentation (PR #4228)
  • Adapt documentation generation to use the mkdocs material theme (PR #4226

0.51.4

29 Oct 10:14
Compare
Choose a tag to compare

Reverts Ansi.erase to erase to the right, not the left

With the release of 0.49.0, we changed Ansi.erase to be erasing left rather than right. This made sense based on the comment on the Ansi.erase method. However, the actual usage in the standard library of Ansi.erase was expecting it to erase to the right. Because of this change, since 0.49.0, the readline support in the term package has been broken.

We've reverted the changes from the original PR and updated the comment on Ansi.erase to note that it erases to right, not the left.

We would welcome an RFC to discuss supporting both left and right erasure as part of the Ansi primitive.

[0.51.4] - 2022-10-29

Fixed

  • Fix broken readline package (PR #4199)

0.51.3

02 Oct 12:07
Compare
Choose a tag to compare

Fix bug in StdStream.print

When printing via StdStream.print strings containing null bytes, the standard library was printing the string until the first null byte and then padding the printed string with space characters until the string size was reached.

This bug was introduced in Pony 0.12.0 while fixing a different printing bug.

We've fixed the bug so that printing strings with null bytes works as expected.

Enhance checking for identity comparison with new objects

Consider the following program:

class C

actor Main
  new create(env: Env) =>
    env.out.print(if C is C then "C is C" else "C is NOT C" end)

This will fail to compile with the message identity comparison with a new object will always be false.

Nevertheless, the checking wasn't exhaustive and some cases weren't covered, like the one in the following example:

class C

actor Main
  new create(env: Env) =>
    env.out.print(if C.create() is C.create() then "C is C" else "C is NOT C" end)

We've made the check exhaustive and we believe it now covers all possible cases.

[0.51.3] - 2022-10-02

Fixed

  • Fix bug in StdStream.print (PR #4180)
  • Fix identity comparison check with desugared creations (PR #4182)

0.51.2

26 Aug 19:17
Compare
Choose a tag to compare

Allow constructor expressions to be auto-recovered in assignments or arguments

Previously for the following code:

actor Main
  new create(env: Env) =>
    Bar.take_foo(Foo(88))
    let bar: Foo iso = Foo(88)

class Foo
  new create(v: U8) =>
    None

primitive Bar
  fun take_foo(foo: Foo iso) =>
    None

You'd get compilation errors "argument not assignable to parameter" and "right side must be a subtype of left side" for the two lines in Main.create().

We've added checks to see if the constructor expressions can be implicitly auto-recovered, and if they can, no compilation error is generated.

This only applies to cases where the type of the parameter (or the let binding) is a simple type, i.e. not a union, intersection or tuple type.

Support for RISC-V

Pony now supports compiling for RISC-V linux glibc systems. 64 bit rv64gc/lp64d is tested via CI. In theory 32 bit rv32gc/ilp32d might work but is untested currently.

Enhance runtime stats tracking

The current pony runtime stats tracking that was previously
implemented under the USE_MEMTRACK and USED_MEMTRACK_MESSAGES
defines has been enhanced. The new defines are called
USE_RUNTIMESTATS and USE_RUNTIMESTATS_MESSAGES.

Runtime stats tracking tracks the following actor info:

  • heap memory allocated
  • heap memory used
  • heap num allocated
  • heap realloc counter
  • heap alloc counter
  • heap free counter
  • heap gc counter
  • system message processing cpu usage
  • app message processing cpu usage
  • garbage collection cpu usage
  • messages sent counter
  • system messages processed counter
  • app messages processed counter

Runtime tracking tracks the following scheduler info:

  • mutemap memory used
  • mutemap memory allocated
  • memory used for gc acquire/release actormaps and actors created
  • memory allocated for gc acquire/release actormaps and actors created
  • created actors counter
  • destroyed actors counter
  • actor system message processing cpu for all actor runs on the scheduler
  • actor app message processing cpu for all actor runs on the scheduler
  • actor garbage collection cpu for all actor runs on the scheduler
  • scheduler message processing cpu usage
  • scheduler misc cpu usage while waiting to do work
  • memory used by inflight messages
  • memory allocated by inflight messages
  • number of inflight messages

There is also a new command line argument available to Pony programs
called --ponyprintstatsinterval that will print out scheduler statistics
every --ponyprintstatsinterval seconds along with printing out actor
statistics whenever an actor is destroyed. The output looks something like:

$ ./helloworld --ponyprintstatsinterval 1
Hello, world.
Actor stats for actor: 140661474674688, heap memory allocated: 0, heap memory used: 0, heap num allocated: 0, heap realloc counter: 0, heap alloc counter: 0, heap free counter: 0, heap gc counter: 0, system cpu: 2628, app cpu: 964, garbage collection marking cpu: 0, garbage collection sweeping cpu: 0, messages sent counter: 3, system messages processed counter: 1, app messages processed counter: 1
Actor stats for actor: 140661474675200, heap memory allocated: 0, heap memory used: 0, heap num allocated: 0, heap realloc counter: 0, heap alloc counter: 0, heap free counter: 0, heap gc counter: 0, system cpu: 1451, app cpu: 170026, garbage collection marking cpu: 0, garbage collection sweeping cpu: 0, messages sent counter: 0, system messages processed counter: 1, app messages processed counter: 2
Actor stats for actor: 140661474848256, heap memory allocated: 9664, heap memory used: 7408, heap num allocated: 63, heap realloc counter: 0, heap alloc counter: 63, heap free counter: 0, heap gc counter: 0, system cpu: 0, app cpu: 5561, garbage collection marking cpu: 0, garbage collection sweeping cpu: 0, messages sent counter: 8, system messages processed counter: 0, app messages processed counter: 1
Actor stats for actor: 140661474672640, heap memory allocated: 0, heap memory used: 0, heap num allocated: 0, heap realloc counter: 0, heap alloc counter: 0, heap free counter: 0, heap gc counter: 0, system cpu: 1820, app cpu: 103, garbage collection marking cpu: 0, garbage collection sweeping cpu: 0, messages sent counter: 3, system messages processed counter: 1, app messages processed counter: 1
Scheduler stats for index: 1, total memory allocated: -232, total memory used: -320, created actors counter: 0, destroyed actors counter: 1, actors app cpu: 0, actors gc marking cpu: 0, actors gc sweeping cpu: 0, actors system cpu: 2668, scheduler msgs cpu: 203532, scheduler misc cpu: 1871418, memory used inflight messages: 0, memory allocated inflight messages: 0, number of inflight messages: 0
Scheduler stats for index: 0, total memory allocated: -464, total memory used: -576, created actors counter: 0, destroyed actors counter: 1, actors app cpu: 176654, actors gc marking cpu: 0, actors gc sweeping cpu: 0, actors system cpu: 10118, scheduler msgs cpu: 172241, scheduler misc cpu: 1839211, memory used inflight messages: 0, memory allocated inflight messages: 0, number of inflight messages: 0
Scheduler stats for index: 3, total memory allocated: -56, total memory used: -64, created actors counter: 0, destroyed actors counter: 0, actors app cpu: 0, actors gc marking cpu: 0, actors gc sweeping cpu: 0, actors system cpu: 1820, scheduler msgs cpu: 150303, scheduler misc cpu: 1774791, memory used inflight messages: 0, memory allocated inflight messages: 0, number of inflight messages: 0
Scheduler stats for index: 2, total memory allocated: -896, total memory used: -1088, created actors counter: 0, destroyed actors counter: 2, actors app cpu: 0, actors gc marking cpu: 0, actors gc sweeping cpu: 0, actors system cpu: 583713, scheduler msgs cpu: 142592, scheduler misc cpu: 1286404, memory used inflight messages: 0, memory allocated inflight messages: 0, number of inflight messages: 0
Actor stats for actor: 140661474841600, heap memory allocated: 0, heap memory used: 0, heap num allocated: 0, heap realloc counter: 0, heap alloc counter: 0, heap free counter: 0, heap gc counter: 0, system cpu: 592420, app cpu: 0, garbage collection marking cpu: 0, garbage collection sweeping cpu: 0, messages sent counter: 0, system messages processed counter: 7, app messages processed counter: 0

This runtime stats tracking info has been exposed to pony programs as
part of the runtime_info package and an example runtime_info program
has been added to the examples directory.

The runtime stats tracking in a pony program can be used for some
useful validations for those folks concerned about
heap allocations in the critical path (i.e. if they
rely on the compiler's HeapToStack optimization pass
to convert heap allocations to stack allocations and
want to validate it is working correctly).

Example of possible use to validate number of heap allocations:

use "collections"
use "runtime_info"

actor Main
  new create(env: Env) =>
    let num_allocs_before = ActorStats.heap_alloc_counter(ActorStatsAuth(env.root))

    let ret = critical()

    let num_allocs_after = ActorStats.heap_alloc_counter(ActorStatsAuth(env.root))

    env.out.print("Allocations before: " + num_allocs_before.string())
    env.out.print("Allocations after: " + num_allocs_after.string())

    env.out.print("Critical section allocated " + (num_allocs_after - num_allocs_before).string() + " heap objects")

    env.out.print("Allocations at end: " + ActorStats.heap_alloc_counter(ActorStatsAuth(env.root)).string())

  fun critical(): U32 =>
    var x: U32 = 1
    let y: U32 = 1000

    for i in Range[U32](1, y) do
      x = x * y
    end

    x

Fix compiler crash related to using private types as default arguments

The compiler crash described in #4130 happened with code that attempted to call a method in a different package when this remote method used a package-private type as a default parameter.

// In the "lib" package

primitive _Private

primitive Public
  fun apply[T](v: (T | _Private) = _Private): None => None

// In main
use lib = "lib"

actor Main
  new create(env: Env) =>
    let p = lib.Public.apply[U8]()
    env.out.print(p.string())

It was decided that this code is valid Pony code, and this PR fixes the crash so that code like the above compiles.

Fix "incorrect" atomics usage

When reviewing other code, we noticed that an old PR that was porting code from one atomics system to another changed the semantics of some atomics. The changes would have no impact on X86, but could have an impact on CPUs with weaker memory models like Arm.

The changes would be tricky to easily evaluate if the changes (all weaken the semantics) are ok. Given that we do not have the time to evaluate the changes fully at the moment, we are switched to having stronger semantics.

This change might fix incorrect atomics usage and memory issues on Arm and RISC-V. It will definitely cause some amount of slowdown on those platforms.

If anyone wants to do a thorough review of various atomics and prove that we can
weaken any safely, we will happily accept PRs that contain proof of the safety.

[0.51.2] - 2022-08-26

Fixed

  • Fix for crash when methods with default private type params in remote packages are called (PR #4167)
  • Fix incorrect atomics usage (PR #4159)

Added

  • Auto-recover constructor expressions (PR #4124)
  • Support for RISC-V (PR #3435)
  • Enhance runtime stats tracking (PR #4144)

0.51.1

29 Jun 23:53
Compare
Choose a tag to compare

Fix String.f32 and String.f64 errors with non null terminated strings

Until now, the String.f32 and String.f64 methods required null-terminated strings in order to work properly. This wasn't documented, and wasn't the intended behaviour. We've now fixed these cases and these functions should work as expected for non-null-terminated strings.

Fix for infinite Ranges

Even though all Range objects that contain NaN parameters or a zero step parameter are considered infinite, some did not iterate at all or produced an error after the first iteration. For example, the code from the Range documentation:

  // this Range will produce 0 at first, then infinitely NaN
  let nan: F64 = F64(0) / F64(0)
  for what_am_i in Range[F64](0, 1000, nan) do
    wild_guess(what_am_i)
  end 

did not run as expected, but rather produced an error on the first iteration. This is now fixed, and .next() calls on the above Range[F64](0, 1000, nan) now first yields 0 and subsequently indefinetely NaN values. Likewise, Range(10, 10, 0) will now indefinitely yield 10.

Update to basing musl images off of Alpine 3.12

We supply nightly and release Docker images for ponyc based on Alpine Linux. We've updated the version of Alpine we use from 3.12 which recently reached it's end of life to Alpine 3.16 which is supported until 2024.

More efficient actor heap garbage collection

We have improved the actor heap garbage collection process
to make it more efficient by deferring some initialization
work and handling as part of the normal garbage collection
mark and sweep passes.

Fix compiler crash related to explicit FFI return types

After we re-added the ability to override the return type for FFI function calls in a previous release, we forgot to reintroduce some checks in the compiler that ensured that the specified return types would be known to the code generation pass. This caused some programs that introduced a new type (for example, a bare lambda) in the context of a FFI return type to crash the compiler. This is now fixed.

Support void* (Pointer[None]) parameters in bare lambdas and functions

Unlike conventional FFI functions, bare lambdas @{(...)} and bare functions fun @bare() => ... did not support Pointer[None] parameters, Pony's equivalent of void*. This is despite the fact that like FFI functions, bare lambdas and functions are strictly intended for use with FFI calls and callbacks.

This commit allows bare lambdas and bare functions with Pointer[None] parameters to accept arguments of any Pointer[A] type for these parameters. Therefore, code like the following now works:

use @printf[I32](fmt: Pointer[None] tag, ...)

actor Main
  new create(env: Env) =>
    let printer = @{(fmt: Pointer[None] tag): I32 => @printf(fmt)}
    let cb = this~print()
    printer("Hello".cstring())
    cb(" world!\n".cstring())

  fun @print(fmt: Pointer[None]): I32 =>
    @printf(fmt)

Systematic testing for the runtime

Pony is a concurrent and parallel language. Different actors can be run
at the same time on multiple CPUs. The Pony runtime coordinates all of
this interleaving of actors and contains a fair amount of complexity.
Runtime functionality such as the message queues and the backpressure
system rely on atomic operations which can be tricky to get right across
multiple platforms.

Systematic testing will allow us to coordinate the interleaving of the
multiple runtime scheduler threads in a deterministic and reproducible
manner. This ability to reproduce a particular runtime behavior will be
invaluable for debugging runtime issues.

The overall idea and some details of the implementation for systematic
testing has been shamelessly stolen from the Verona runtime (see:
https://github.com/microsoft/verona/blob/master/docs/explore.md#systematic-testing
for details). This implementation doesn't include replayable runtime
unit tests like Verona, but it sets a foundation for allowing replayable
runs of programs (and probably tests) for debugging runtime issues such
as backpressure/etc. Additionally, while all development and testing was
done on Linux, in theory this systematic testing functionality should
work on other operating systems (Windows, MacOS, Freebsd, etc) barring
issues related to lack of atomics for tracking the active thread and
whether a thread has stopped executing or not (unlikely to be an issue
on MacOS/Freebsd/other pthread based threading implementations).

Instructions for using systematic testing...

The following instructions were tested on Linux but should in theory
function correctly on other *nix environments that use the Pony
make/cmake based build system.

Start with a clean environment:

make clean config=debug

Configure with systematic testing and pthreads enabled:

make configure config=debug use=scheduler_scaling_pthreads,systematic_testing

Build:

make build config=debug

Compile helloworld:

./build/debug-systematic_testing/ponyc examples/helloworld/

Run helloworld:

./helloworld

Output will look something like:

me@home:~/ponyc$ ./helloworld
Systematic testing using seed: 360200870782547...
(rerun with `<app> --ponysystematictestingseed 360200870782547` to reproduce)
<SNIPPED LOTS OF OUTPUT>
thread 139871784978176: yielding to thread: 139871776585472.. next_index: 3
Hello, world.
thread 139871776585472: yielding to thread: 139871768192768.. next_index: 4
<SNIPPED LOTS OF OUTPUT>
Systematic testing successfully finished!
me@home:~/ponyc$ 

[0.51.1] - 2022-06-29

Fixed

  • Fix String.f32 and String.f64 errors with non null terminated strings (PR #4132)
  • Fix for infinite Ranges (PR #4127)
  • Ensure reachability of types returned by FFI calls (PR #4149)
  • Support void* (Pointer[None]) parameters in bare lambdas and functions (PR #4152)

Added

  • Avoid clearing chunks at start of GC (PR #4143)
  • Systematic testing for the runtime (PR #4140)

Changed

  • Update to basing musl images off of Alpine 3.16 (PR #4139)