Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce PerformanceOptimizedDelegateCalculator #747

Conversation

bcremer
Copy link
Contributor

@bcremer bcremer commented Apr 6, 2023

Reference: #634 (comment)

Benchmark with added \Money\Money::registerCalculator(\Money\Calculator\PerformanceOptimizedDelegateCalculator::class); in vendor/autoload.php:

$ composer benchmark -- --ref=without_optimized_calculator
> vendor/bin/phpbench run --retry-threshold=3 --iterations=15 --revs=1000  --warmup=2 '--ref=without_optimized_calculator'
PHPBench (1.2.10) running benchmarks... #standwithukraine
with configuration file: /home/pushapidev/current/moneyphp/phpbench.json
with PHP version 8.1.17, xdebug ✔, opcache ❌
comparing [actual vs. without_optimized_calculator]

\Benchmark\Money\NumberInstantiationBench

    benchConstructorWithZeroIntegerAmount...R1 I3 - [Mo0.130μs vs. Mo0.129μs] +0.31% (±0.92%)
    benchConstructorWithPositiveIntegerAmou.R1 I9 - [Mo0.180μs vs. Mo0.180μs] +0.41% (±1.20%)
    benchConstructorWithNegativeIntegerAmou.R1 I14 - [Mo0.180μs vs. Mo0.181μs] -0.07% (±1.62%)
    benchConstructorWithZeroAndFractionalAm.R1 I10 - [Mo0.192μs vs. Mo0.190μs] +0.67% (±0.86%)
    benchConstructorWithFractionalAmount....R2 I7 - [Mo0.237μs vs. Mo0.236μs] +0.27% (±0.95%)
    benchConstructorWithNegativeFractionalA.R1 I12 - [Mo0.238μs vs. Mo0.241μs] -1.20% (±1.19%)
    benchFromStringWithZeroIntegerAmount....R2 I9 - [Mo0.200μs vs. Mo0.200μs] -0.02% (±0.53%)
    benchFromStringWithPositiveIntegerAmoun.R2 I14 - [Mo0.260μs vs. Mo0.257μs] +1.21% (±0.81%)
    benchFromStringWithNegativeIntegerAmoun.R1 I11 - [Mo0.258μs vs. Mo0.262μs] -1.43% (±0.67%)
    benchFromStringWithZeroAndFractionalAmo.R1 I10 - [Mo0.288μs vs. Mo0.291μs] -0.87% (±1.33%)
    benchFromStringWithFractionalAmount.....R1 I6 - [Mo0.343μs vs. Mo0.355μs] -3.30% (±0.68%)
    benchFromStringWithNegativeFractionalAm.R1 I8 - [Mo0.347μs vs. Mo0.358μs] -3.13% (±1.30%)

\Benchmark\Money\MoneyOperationBench

    benchAdd................................R1 I14 - [Mo0.432μs vs. Mo0.649μs] -33.49% (±0.75%)
    benchSubtract...........................R2 I7 - [Mo0.429μs vs. Mo0.650μs] -33.97% (±0.97%)
    benchMultiply...........................R1 I13 - [Mo0.746μs vs. Mo0.682μs] +9.41% (±1.88%)
    benchDivide.............................R1 I9 - [Mo0.824μs vs. Mo0.822μs] +0.32% (±1.47%)
    benchSum................................R2 I9 - [Mo1.102μs vs. Mo1.117μs] -1.32% (±1.07%)
    benchMin................................R1 I13 - [Mo0.573μs vs. Mo0.545μs] +5.24% (±1.70%)
    benchMax................................R1 I7 - [Mo0.576μs vs. Mo0.516μs] +11.56% (±1.15%)
    benchAvg................................R2 I12 - [Mo0.575μs vs. Mo0.516μs] +11.43% (±0.54%)
    benchRatioOf............................R1 I1 - [Mo0.396μs vs. Mo0.376μs] +5.46% (±0.99%)
    benchMod................................R1 I2 - [Mo0.416μs vs. Mo0.578μs] -28.02% (±1.10%)
    benchIsSameCurrency.....................R3 I13 - [Mo0.061μs vs. Mo0.077μs] -20.80% (±1.02%)
    benchIsZero.............................R1 I7 - [Mo0.128μs vs. Mo0.138μs] -6.96% (±0.90%)
    benchAbsolute...........................R1 I14 - [Mo0.180μs vs. Mo0.212μs] -15.13% (±0.53%)
    benchNegative...........................R2 I12 - [Mo0.526μs vs. Mo0.984μs] -46.55% (±1.23%)
    benchIsPositive.........................R1 I7 - [Mo0.123μs vs. Mo0.140μs] -11.75% (±1.10%)
    benchCompare............................R1 I10 - [Mo0.154μs vs. Mo0.167μs] -7.64% (±0.91%)
    benchLessThan...........................R2 I8 - [Mo0.173μs vs. Mo0.191μs] -9.06% (±0.70%)
    benchLessThanOrEqual....................R8 I14 - [Mo0.174μs vs. Mo0.196μs] -11.36% (±1.03%)
    benchEquals.............................R6 I14 - [Mo0.193μs vs. Mo0.215μs] -10.15% (±1.78%)
    benchGreaterThan........................R2 I10 - [Mo0.173μs vs. Mo0.188μs] -8.12% (±1.00%)
    benchGreaterThanOrEqual.................R4 I9 - [Mo0.174μs vs. Mo0.190μs] -8.54% (±0.87%)

\Benchmark\Money\MoneyInstantiationBench

    benchConstructorWithZeroIntegerAmount...R2 I7 - [Mo0.102μs vs. Mo0.119μs] -14.18% (±0.91%)
    benchConstructorWithPositiveIntegerAmou.R1 I10 - [Mo0.132μs vs. Mo0.159μs] -16.57% (±1.23%)
    benchConstructorWithNegativeIntegerAmou.R1 I2 - [Mo0.131μs vs. Mo0.157μs] -16.79% (±0.86%)
    benchConstructorWithZeroStringAmount....R1 I10 - [Mo0.095μs vs. Mo0.118μs] -19.56% (±0.90%)
    benchConstructorWithPositiveStringAmoun.R1 I10 - [Mo0.102μs vs. Mo0.123μs] -17.03% (±0.86%)
    benchConstructorWithNegativeStringAmoun.R1 I3 - [Mo0.102μs vs. Mo0.119μs] -13.66% (±1.33%)

Subjects: 39, Assertions: 0, Failures: 0, Errors: 0

This is a weird outlier as the method is not affected by the calculator at all.

benchIsSameCurrency.....................R3 I13 - [Mo0.061μs vs. Mo0.077μs] -20.80% (±1.02%)

*
* @psalm-immutable
*/
final class PerformanceOptimizedDelegateCalculator implements Calculator
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably need to expand https://github.com/moneyphp/money/tree/8953b873725c465bc55eee12a1c91b923e0d63c7/benchmark to include the raw calculators (as 3 almost-exactly-same benchmark classes)

@bcremer bcremer force-pushed the introduce-PerformanceOptimizedDelegateCalculator branch from cd72c9a to fdfd57c Compare April 6, 2023 09:54
@bcremer
Copy link
Contributor Author

bcremer commented Apr 6, 2023

@Ocramius @frederikbosch Added calculator benchmarks.

Unfortunately I fail to replicate the improved add/substract performance on calculator level. I have to recheck my setup.

$ composer benchmark -- benchmark/Calculator/
> vendor/bin/phpbench run --retry-threshold=3 --iterations=15 --revs=1000  --warmup=2 'benchmark/Calculator/'
PHPBench (1.2.10) running benchmarks... #standwithukraine
with configuration file: /home/pushapidev/current/moneyphp/phpbench.json
with PHP version 8.1.17, xdebug ❌, opcache ✔

\Benchmark\Money\Calculator\BcMathCalculatorBench

    benchCompare............................R1 I3 - Mo0.463μs (±0.75%)
    benchAdd................................R2 I10 - Mo0.137μs (±1.43%)
    benchSubtract...........................R8 I11 - Mo0.142μs (±0.96%)
    benchMultiply...........................R1 I0 - Mo0.267μs (±0.70%)
    benchDivide.............................R1 I8 - Mo0.256μs (±0.74%)
    benchCeil...............................R1 I14 - Mo0.481μs (±1.09%)
    benchFloor..............................R1 I5 - Mo0.468μs (±1.24%)
    benchAbsolute...........................R1 I7 - Mo0.102μs (±1.30%)
    benchRound..............................R1 I8 - Mo0.575μs (±0.99%)
    benchShare..............................R1 I2 - Mo0.646μs (±1.23%)
    benchMod................................R1 I2 - Mo0.247μs (±1.24%)


\Benchmark\Money\Calculator\GmpCalculatorBench

    benchCompare............................R3 I10 - Mo3.459μs (±1.21%)
    benchAdd................................R3 I9 - Mo0.221μs (±0.57%)
    benchSubtract...........................R2 I12 - Mo0.224μs (±0.78%)
    benchMultiply...........................R1 I11 - Mo1.285μs (±1.25%)
    benchDivide.............................R2 I14 - Mo1.775μs (±1.08%)
    benchCeil...............................R1 I8 - Mo0.617μs (±1.00%)
    benchFloor..............................R1 I7 - Mo0.595μs (±1.52%)
    benchAbsolute...........................R1 I0 - Mo0.102μs (±1.82%)
    benchRound..............................R1 I4 - Mo0.687μs (±1.95%)
    benchShare..............................R2 I9 - Mo2.336μs (±1.42%)
    benchMod................................R1 I4 - Mo1.240μs (±0.98%)


\Benchmark\Money\Calculator\PerformanceOptimizedDelegateCalculatorBench

    benchCompare............................R2 I3 - Mo0.548μs (±0.59%)
    benchAdd................................R1 I13 - Mo0.241μs (±1.01%)
    benchSubtract...........................R2 I14 - Mo0.244μs (±1.01%)
    benchMultiply...........................R3 I11 - Mo0.303μs (±0.55%)
    benchDivide.............................R1 I5 - Mo0.272μs (±0.46%)
    benchCeil...............................R2 I11 - Mo0.503μs (±0.94%)
    benchFloor..............................R1 I12 - Mo0.490μs (±1.21%)
    benchAbsolute...........................R3 I14 - Mo0.132μs (±1.54%)
    benchRound..............................R3 I14 - Mo0.599μs (±0.94%)
    benchShare..............................R3 I14 - Mo0.662μs (±0.98%)
    benchMod................................R2 I12 - Mo0.271μs (±0.72%)


Subjects: 33, Assertions: 0, Failures: 0, Errors: 0

@Ocramius
Copy link
Contributor

Ocramius commented Apr 6, 2023

That's quite an improvement! Some performance overhead due to delegated method calls, but overall OK 👍

@bcremer
Copy link
Contributor Author

bcremer commented Apr 6, 2023

I really can't make sense of the following.

When calling Money::add() the GmpCalculator it's faster.
But when I compare the calculators directly BcMathCalculator::add is faster than GmpCalculator::add.

final class CompareCalculatorsBench
{
    public function benchAddBcMath(): void
    {
        Money::registerCalculator(BcMathCalculator::class);
        $currency = new Currency('EUR');
        $a  = new Money('100', $currency);
        $b  = new Money('50', $currency);
        $a->add($b);
    }

    public function benchAddGmp(): void
    {
        Money::registerCalculator(GmpCalculator::class);
        $currency = new Currency('EUR');
        $a  = new Money('100', $currency);
        $b  = new Money('50', $currency);
        $a->add($b);
    }

    public function benchAddCalculatorBcMatch(): void
    {
        BcMathCalculator::add('100', '50');
    }

    public function benchAddCalculatorGmp(): void
    {
        GmpCalculator::add('100', '50');
    }
}
$ composer benchmark -- benchmark/CompareCalculatorsBench.php
> vendor/bin/phpbench run --retry-threshold=3 --iterations=15 --revs=1000  --warmup=2 'benchmark/CompareCalculatorsBench.php'
PHPBench (1.2.10) running benchmarks... #standwithukraine
with configuration file: /home/pushapidev/current/moneyphp/phpbench.json
with PHP version 8.1.17, xdebug ❌, opcache ✔

\Benchmark\Money\CompareCalculatorsBench

    benchAddBcMath..........................R2 I4 - Mo0.872μs (±0.90%)
    benchAddGmp.............................R1 I3 - Mo0.664μs (±1.00%)
    benchAddCalculatorBcMatch...............R1 I11 - Mo0.125μs (±1.27%)
    benchAddCalculatorGmp...................R1 I10 - Mo0.208μs (±1.17%)

Subjects: 4, Assertions: 0, Failures: 0, Errors: 0

@bcremer
Copy link
Contributor Author

bcremer commented Apr 6, 2023

OK, turns out Gmp is in fact slower than BcMath for add operations.

The increased performance of \Money\Money::add() is a side effect of the different result representations of both calculators:

// \Money\Calculator\BcMathCalculator::add
bcadd('50', '100', 14); 
// Return: string(18) "150.00000000000000"

// \Money\Calculator\GmpCalculator::add
gmp_strval(gmp_add('50', '100')); 
// Return: string(3) "150"

The result is passed into the \Money\Money constructor.

The result representation is critical for the code path in \Money\Money::__construct

    public function __construct(int|string $amount, Currency $currency)
    {
        $this->currency = $currency;

        if (filter_var($amount, FILTER_VALIDATE_INT) === false) {
            $numberFromString = Number::fromString((string) $amount);
            if (! $numberFromString->isInteger()) {
                throw new InvalidArgumentException('Amount must be an integer(ish) value');
            }

            $this->amount = $numberFromString->getIntegerPart();

            return;
        }

        $this->amount = (string) $amount;
     }

The result of BcMathCalculator::add() (150.00000000000000) does not pass the FILTER_VALIDATE_INT and thus additional (slow) number functions are executed.

The results of GmpCalculator::add() 150 does pass for FILTER_VALIDATE_INT and is directly used to populate the amount.

This is very good news. We can easily improve the default BcMathCalculator by reducing the $scale to 0 for all operations that will not return a decimal.

> vendor/bin/phpbench run --retry-threshold=3 --iterations=15 --revs=1000  --warmup=2 'benchmark/MoneyOperationBench.php' '--ref=before_bcmath_scale'
PHPBench (1.2.10) running benchmarks... #standwithukraine
with configuration file: /home/pushapidev/current/moneyphp/phpbench.json
with PHP version 8.1.17, xdebug ✔, opcache ✔
comparing [actual vs. before_bcmatch_scale]

\Benchmark\Money\MoneyOperationBench

    benchAdd................................R4 I9 - [Mo0.298μs vs. Mo0.610μs] -51.12% (±0.63%)
    benchSubtract...........................R1 I5 - [Mo0.298μs vs. Mo0.611μs] -51.34% (±1.04%)
    benchMultiply...........................R1 I12 - [Mo0.622μs vs. Mo0.629μs] -1.16% (±1.30%)
    benchDivide.............................R1 I13 - [Mo0.731μs vs. Mo0.726μs] +0.67% (±1.08%)
    benchSum................................R1 I12 - [Mo0.646μs vs. Mo1.063μs] -39.23% (±0.65%)
    benchMin................................R1 I14 - [Mo0.494μs vs. Mo0.493μs] +0.11% (±1.01%)
    benchMax................................R1 I6 - [Mo0.509μs vs. Mo0.512μs] -0.48% (±0.74%)
    benchAvg................................R1 I3 - [Mo1.462μs vs. Mo1.822μs] -19.79% (±1.40%)
    benchRatioOf............................R1 I0 - [Mo0.365μs vs. Mo0.349μs] +4.68% (±1.99%)
    benchMod................................R6 I13 - [Mo0.382μs vs. Mo0.383μs] -0.48% (±1.06%)
    benchIsSameCurrency.....................R1 I6 - [Mo0.059μs vs. Mo0.060μs] -1.88% (±1.31%)
    benchIsZero.............................R1 I11 - [Mo0.106μs vs. Mo0.105μs] +0.25% (±1.22%)
    benchAbsolute...........................R3 I14 - [Mo0.157μs vs. Mo0.157μs] -0.27% (±1.18%)
    benchNegative...........................R1 I9 - [Mo0.392μs vs. Mo0.710μs] -44.84% (±0.90%)
    benchIsPositive.........................R2 I6 - [Mo0.102μs vs. Mo0.102μs] -0.03% (±0.86%)
    benchCompare............................R3 I14 - [Mo0.133μs vs. Mo0.133μs] +0.06% (±0.98%)
    benchLessThan...........................R2 I3 - [Mo0.149μs vs. Mo0.149μs] -0.07% (±0.75%)
    benchLessThanOrEqual....................R2 I8 - [Mo0.149μs vs. Mo0.149μs] -0.01% (±1.25%)
    benchEquals.............................R1 I4 - [Mo0.169μs vs. Mo0.169μs] +0.13% (±0.93%)
    benchGreaterThan........................R1 I7 - [Mo0.149μs vs. Mo0.149μs] +0.11% (±1.60%)
    benchGreaterThanOrEqual.................R1 I14 - [Mo0.149μs vs. Mo0.149μs] +0.28% (±1.28%)

Subjects: 21, Assertions: 0, Failures: 0, Errors: 0

@bcremer
Copy link
Contributor Author

bcremer commented Apr 6, 2023

It's maybe better to close this PR and open a fresh one with only the improved BCMatchCalculator.

Edit:
Replaced this PR by: #748

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants