Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SteaneReedMuller Code #251

Draft
wants to merge 17 commits into
base: master
Choose a base branch
from
Draft

SteaneReedMuller Code #251

wants to merge 17 commits into from

Conversation

Fe-r-oz
Copy link
Contributor

@Fe-r-oz Fe-r-oz commented Mar 30, 2024

No description provided.

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Mar 30, 2024

improved SRM(1,r)

Outputs:

julia> parity_checks(SteaneReedMuller(1,3))
+ XXXXXXXX
+ ZZZZZZZZ
+ __ZZXXYY
+ _ZXY_ZXY
+ _XZYZY_X

julia> parity_checks(SteaneReedMuller(1,4))
+ XXXXXXXXXXXXXXXX
+ ZZZZZZZZZZZZZZZZ
+ ____ZZZZXXXXYYYY
+ __ZZXXYY__ZZXXYY
+ _ZXY_ZXY_ZXY_ZXY
+ _X_XZYZYZYZY_X_X

julia> parity_checks(SteaneReedMuller(1,5))
+ XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
+ ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ
+ ________ZZZZZZZZXXXXXXXXYYYYYYYY
+ ____ZZZZXXXXYYYY____ZZZZXXXXYYYY
+ __ZZXXYY__ZZXXYY__ZZXXYY__ZZXXYY
+ _ZXY_ZXY_ZXY_ZXY_ZXY_ZXY_ZXY_ZXY
+ _X_X_X_XZYZYZYZYZYZYZYZY_X_X_X_X
canonicalize:

julia> canonicalize!(parity_checks(SteaneReedMuller(1,3)))
+ XZ_YZXY_
+ _XZYZY_X
+ _ZXY_ZXY
+ __ZZXXYY
+ ZZZZZZZZ

julia> canonicalize!(parity_checks(Gottesman(3)))
+ XZ_Y_YXZ
+ _XZY_XZY
- _ZXYZ_YX
- __ZZYYXX
+ ZZZZZZZZ

Had to close #249 because of the merge conflicts introduced by #250.

Gottesman subset is being tests with official Gottesman test.

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Mar 30, 2024

It was told that the Stabilizer can accept two bool arguments, no need to pass ;UINt8 argument, but the following happens when testing for large stabilizer circuits;

Stabilizer(extended_Hx, extended_Hz)
TypeError: in typeassert, expected Matrix{UInt64}, got a value of type Vector{Any}

SteaneReedMuller(1,3) to (1,7) don't have this error.

@Krastanov
Copy link
Member

It was told that the Stabilizer can accept two bool arguments, no need to pass ;UINt8 argument, but the following happens when testing for large stabilizer circuits;

Stabilizer(extended_Hx, extended_Hz) TypeError: in typeassert, expected Matrix{UInt64}, got a value of type Vector{Any}

SteaneReedMuller(1,3) to (1,7) don't have this error.

An easy way to check what constructors are available is:

julia> methods(Stabilizer)
# 10 methods for type constructor:
  [1] Stabilizer(s::Stabilizer)
  [2] Stabilizer(xzs::AbstractMatrix{Bool})
  [3] Stabilizer(phases::Tzv, nqubits::Int64, xzs::Tm) where {Tzv<:AbstractVector{UInt8}, Tm<:(AbstractMatrix{<:Unsigned})}
  [4] Stabilizer(paulis::AbstractArray{PauliOperator{Tz, Tv}, 1}) where {Tz, Tv}
  [5] Stabilizer(phases::AbstractVector{UInt8}, xs::AbstractMatrix{Bool}, zs::AbstractMatrix{Bool})
  [6] Stabilizer(phases::AbstractVector{UInt8}, xzs::AbstractMatrix{Bool})
  [7] Stabilizer(xs::AbstractMatrix{Bool}, zs::AbstractMatrix{Bool})
  [8] Stabilizer(tab::T) where T<:QuantumClifford.Tableau
  [9] Stabilizer(g::Graphs.SimpleGraphs.SimpleGraph)
 [10] Stabilizer(c::QuantumClifford.ECC.AbstractECC)

I think you have been using method 5, but method 7 should be more convenient. Given your error message, it seems the problem is that somewhere a Vector{Any} is being fed instead of a Matrix{Bool}. More information is necessary to pinpoint the exact issue.

Also, it is worrisome that this happens only for some sizes. It implies there might be some size-dependent type-instability, which is usually a bad sign.

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Mar 31, 2024

The Hx and Hz are both Boolean Matrices before they are passed down to the Stabilizer.

I will check out how the vector{any} comes into play. Also, I will introspect on the tip: size-dependent type-instability,

@Krastanov
Copy link
Member

On the master branch tests pass. You can disregard nightly/performance/downgrade/buildkite, but the rest should pass here. Let's keep this in draft form until the tests pass.

@Krastanov Krastanov marked this pull request as draft March 31, 2024 01:28
@Krastanov
Copy link
Member

To run the tests locally, you can do ] test QuantumClifford. Avoid pushing here until things work locally.

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Mar 31, 2024

julia> naive_encoding_circuit(parity_checks(SteaneReedMuller(3,6)))
690-element Vector{QuantumClifford.AbstractOperation}:
 QuantumClifford.sHadamard(1)
 QuantumClifford.sZCX(1,43)
 QuantumClifford.sZCX(1,44)
 QuantumClifford.sZCX(1,45)
 QuantumClifford.sZCX(1,46)
 QuantumClifford.sZCX(1,47)
 QuantumClifford.sZCX(1,49)
 QuantumClifford.sZCX(1,50)
 QuantumClifford.sZCX(1,51)
 QuantumClifford.sZCX(1,52)
 QuantumClifford.sZCX(1,54)
 QuantumClifford.sZCX(1,55)
 QuantumClifford.sZCX(1,56)
 QuantumClifford.sZCX(1,58)
 QuantumClifford.sZCX(1,59)
 QuantumClifford.sZCX(1,61)
 QuantumClifford.sHadamard(2)
 ⋮
 QuantumClifford.sSWAP(33,27)
 QuantumClifford.sSWAP(32,28)
 QuantumClifford.sSWAP(31,29)
 QuantumClifford.sSWAP(30,28)
 QuantumClifford.sSWAP(29,26)
 QuantumClifford.sSWAP(28,26)
 QuantumClifford.sSWAP(27,25)
 QuantumClifford.sSWAP(26,24)
 QuantumClifford.sSWAP(25,23)
 QuantumClifford.sSWAP(24,23)
 QuantumClifford.sSWAP(23,22)
 QuantumClifford.sSWAP(22,21)
 QuantumClifford.sSWAP(21,20)
 QuantumClifford.sSWAP(20,19)
 QuantumClifford.sSWAP(19,18)
 QuantumClifford.sSWAP(18,17)
 QuantumClifford.sSWAP(17,16)

julia> naive_encoding_circuit(parity_checks(SteaneReedMuller(3,4)))
127-element Vector{QuantumClifford.AbstractOperation}:
 QuantumClifford.sHadamard(1)
 QuantumClifford.sPhase(1)
 QuantumClifford.sZCX(1,16)
 QuantumClifford.sHadamard(2)
 QuantumClifford.sZCX(2,16)
 QuantumClifford.sHadamard(3)
 QuantumClifford.sZCZ(3,2)
 QuantumClifford.sZCX(3,16)
 QuantumClifford.sHadamard(4)
 QuantumClifford.sZCX(4,16)
 QuantumClifford.sHadamard(5)
 QuantumClifford.sPhase(5)
 QuantumClifford.sZCZ(5,3)
 QuantumClifford.sZCX(5,16)
 QuantumClifford.sHadamard(6)
 QuantumClifford.sPhase(6)
 QuantumClifford.sZCZ(6,4)
 ⋮
 QuantumClifford.sZCZ(15,8)
 QuantumClifford.sZCZ(15,9)
 QuantumClifford.sZCZ(15,10)
 QuantumClifford.sZCZ(15,11)
 QuantumClifford.sZCZ(15,12)
 QuantumClifford.sZCZ(15,13)
 QuantumClifford.sZCX(15,16)
 QuantumClifford.sZ(1)
 QuantumClifford.sZ(2)
 QuantumClifford.sZ(3)
 QuantumClifford.sZ(4)
 QuantumClifford.sZ(5)
 QuantumClifford.sZ(6)
 QuantumClifford.sZ(7)
 QuantumClifford.sZ(9)
 QuantumClifford.sZ(11)
 QuantumClifford.sZ(13)

julia> naive_encoding_circuit(parity_checks(SteaneReedMuller(5,6)))
1732-element Vector{QuantumClifford.AbstractOperation}:
 QuantumClifford.sHadamard(1)
 QuantumClifford.sPhase(1)
 QuantumClifford.sZCX(1,64)
 QuantumClifford.sHadamard(2)
 QuantumClifford.sZCX(2,64)
 QuantumClifford.sHadamard(3)
 QuantumClifford.sZCZ(3,2)
 QuantumClifford.sZCX(3,64)
 QuantumClifford.sHadamard(4)
 QuantumClifford.sZCX(4,64)
 QuantumClifford.sHadamard(5)
 QuantumClifford.sZCZ(5,3)
 QuantumClifford.sZCX(5,64)
 QuantumClifford.sHadamard(6)
 QuantumClifford.sZCZ(6,4)
 QuantumClifford.sZCX(6,64)
 QuantumClifford.sHadamard(7)
 ⋮
 QuantumClifford.sZ(36)
 QuantumClifford.sZ(37)
 QuantumClifford.sZ(38)
 QuantumClifford.sZ(39)
 QuantumClifford.sZ(41)
 QuantumClifford.sZ(42)
 QuantumClifford.sZ(44)
 QuantumClifford.sZ(46)
 QuantumClifford.sZ(47)
 QuantumClifford.sZ(49)
 QuantumClifford.sZ(50)
 QuantumClifford.sZ(51)
 QuantumClifford.sZ(54)
 QuantumClifford.sZ(55)
 QuantumClifford.sZ(57)
 QuantumClifford.sZ(59)
 QuantumClifford.sZ(61)

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Apr 1, 2024

code_k(c::....) = ....will give 0 at (4, 2) based on Eq 2. Steane calls it ''code size falls to 0.'' Not sure how the base test perceives k =0 for frame or other calculations that require k.

I've come to appreciate that even my mistakes uncover loopholes in the tests. The base test is more comprehensive now as a result.

code_k(c::....) = ....will give 0 at (4, 2) based on Eq 2. Steane calls it ''code size falls to 0.'' Not sure how the base test perceives k =0 for frame or other calculations that require k.

As per Steane's paper, I have made sure that when the k falls to 0, it should be a throw as Steane point that the method does not work in these cases. Kindly please confirm this as well from the paper.

n + k

elseif _nplusk_qrm(t, r) == 0
             throw(ArgumentError("Invalid parameters when attempting to construct a Steane-Reed-Muller code. The method to construct Steane-Reed-Muller code fails when k is 0."))  

This helps to identify the following cases:

julia> _nplusk_qrm(3, 2)
0

julia> _nplusk_qrm(4, 2)
0

julia> _nplusk_qrm(4, 1)
0

julia> _nplusk_qrm(4, 3)
0


julia> _nplusk_qrm(5, 1)
0

julia> _nplusk_qrm(5, 2)
0

julia> _nplusk_qrm(5, 3)
0

julia> _nplusk_qrm(5, 4)
0

julia> _nplusk_qrm(6, 1)
0

julia> _nplusk_qrm(6, 2)
0

julia> _nplusk_qrm(6, 3)
0

julia> _nplusk_qrm(6, 4)
0

julia> _nplusk_qrm(6, 5)
0

Edit:
Actually, I wanted to quote this as a new comment by >, but ended up editing the same comment instead. In the edit, I have added the cases where k falls to zero and that should be thrown when the SteaneReedMuller runs.

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Apr 1, 2024

I don't know precisely the definition of code_s(c::...) so I didn't included it. Maybe it's total number of syndromes bits?

I have included code_k, code_n. Distance is 2^c.t + 2^(c.t-1) as the improved base requires them for calculations of frames

If there are improvements that can be made, if further attention is required in tests, etc. please let me know.

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Apr 1, 2024

Maybe it's better to include a sentence in docstring why Generator matrics are actually stabilizers meaning same as parity check tableaus for some values, this could really prevent confusion if people see parity check Tableau and ask "Why is formed in the structure of Generator Matrixs which is G = H as follows

 [ G1| 0 ]  
 [ 0| G2 ]
 [ Dx|Dz ]

G = H (in some cases)
After reading the paper, the reason is that

"The method works for k > 0, and the generators for k < 0 are stabilisers of k > 0 codes."

It would be helpful if permission is granted to add this line in the docstring.

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Apr 1, 2024

I don't know precisely the definition of code_s(c::...) so I didn't included it. Maybe it's total number of syndromes bits?

As per def. of code_s in CSS:
code_s(c::CSS) = size(c.Hx, 1) + size(c.Hz, 1)

I guess code_s is total rows of stabilizer since in CSS, we have different rows for Hz and Hx. This can be seen in different examples of CSS.

In Steane's case, I have something like

code_s(c::...) = _nplusk(c.t, c.r)

since that's tells us total number of rows in this case.

Edit:

I don't think code_s(c::...) = _nplusk(c.t, c.r) is correct guess.

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Apr 1, 2024

More information is necessary to pinpoint the exact issue.
[5] Stabilizer(phases::AbstractVector{UInt8}, xs::AbstractMatrix{Bool}, zs::AbstractMatrix{Bool})
[7] Stabilizer(xs::AbstractMatrix{Bool}, zs::AbstractMatrix{Bool})

Very sorry for not providing complete details. My bad.

Problem was in the whennaive_encoding_circuit(which starts from line 18 to line 53), Maybe because in [5], we provide a parameter phases, (what is phased btw?) and in [7], we don't, so maybe it expects a unit8 phase parameter?

Complete appropriate trace:
I have added (please see here in the trace). This is from where we see the source innaive_encoding_circuit

 BoundsError: attempt to access 16-element view(::Vector{UInt8}, 17:32) with eltype UInt8 at index [17]
  Stacktrace:
    [1] throw_boundserror(A::SubArray{UInt8, 1, Vector{UInt8}, Tuple{UnitRange{Int64}}, true}, I::Tuple{Int64})
      @ Base ./abstractarray.jl:737
    [2] checkbounds
      @ ./abstractarray.jl:702 [inlined]
    [3] getindex
      @ ./subarray.jl:322 [inlined]
    [4] naive_encoding_circuit(code::QuantumClifford.ECC.SteaneReedMuller; undoperm::Bool)
      @ QuantumClifford.ECC ~/Desktop/New/b10/QuantumClifford.jl/src/ecc/circuits.jl:53 (please see here)
    [5] naive_encoding_circuit
      @ ~/Desktop/New/b10/QuantumClifford.jl/src/ecc/circuits.jl:18 [inlined] (please see here)
    [6] test_with_pframes(code::QuantumClifford.ECC.SteaneReedMuller)
      @ Main.var"##ecc#256" ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:47

@Krastanov
Copy link
Member

It might help to run the other ECC tests as well, to see what fails. The one about encoding&syndrome circuits is probably too involved, with too many pieces that work together, so it is difficult to debug.

What other tests fail? In particular, what are the test_ecc_codeproperties results.

It is indeed quite probably that the issue is the k=0 codes. If it happens only for them, we should: (1) understand what specifically about the assumptions in the tests is wrong and (2) filter out these codes only for these specific tests.

@Krastanov
Copy link
Member

"The method works for k > 0, and the generators for k < 0 are stabilisers of k > 0 codes."

It would be helpful if permission is granted to add this line in the docstring.

I would need to understand better what that means, but yeah, go ahead and add it. I will ask more questions when I get to reviewing this.

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Apr 2, 2024

What other tests fail? In particular, what are the test_ecc_codeproperties results.

Every other test seem to work well. It's seems to me that it's the k = 0, and the way the code_base is not to be setup does provides error when k = 0.

Please have a look at CI tests. They provide the same error that is locally.


 Testing Running tests...
Starting tests with 1 threads out of `Sys.CPU_THREADS = 4`...
skipping gpu tests (set GPU_TESTS=true to test gpu)
Test Summary: | Pass  Total  Time
throws        |  101    101  4.3s
Test Summary: | Pass  Total  Time
paulis        |  451    451  4.9s
Test Summary: | Pass  Total  Time
stabs         |   95     95  8.9s
Test Summary: | Pass  Total  Time
stabcanon     |  226    226  3.7s
Test Summary: | Pass  Total  Time
mul_leftright | 1260   1260  3.0s
Test Summary: | Pass  Total  Time
inner         |    9      9  0.6s
Test Summary: | Pass  Total  Time
embed         |    5      5  0.0s
Test Summary: | Pass  Total  Time
gf2           |   90     90  2.2s
Test Summary: | Pass  Total   Time
projections   |  379    379  10.6s
Test Summary: | Pass  Total  Time
expect        |    7      7  0.1s
Test Summary: | Pass  Total  Time
trace         |  286    286  3.7s
Test Summary: | Pass  Total  Time
cliff         |  197    197  6.6s
Test Summary: | Pass  Total  Time
symcliff      |  329    329  4.3s
Test Summary: | Pass  Total   Time
symcontrolled |  203    203  12.4s
Test Summary: | Pass  Total  Time
classicalreg  |    8      8  1.3s
Test Summary: | Pass  Total   Time
random        |  192    192  16.6s
Test Summary: | Pass  Total  Time
noisycircuits |   91     91  5.6s
Test Summary: | Pass  Total  Time
syndromemeas  |  100    100  0.2s
Test Summary: | Pass  Total  Time
bitpack       |   96     96  7.5s
Test Summary: | Pass  Total  Time
memorylayout  |   40     40  7.4s
Test Summary: | Pass  Total  Time
graphs        |   22     22  0.6s
Test Summary: | Pass  Total  Time
hash          |    4      4  0.2s
Test Summary: | Pass  Total  Time
entanglement  |   57     57  3.6s
Test Summary: | Pass  Total  Time
enumerate     |    5      5  2.1s
Test Summary: | Pass  Total   Time
quantumoptics |  346    346  15.9s
naive syndrome circuits - zero syndrome for logical states: Error During Test at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:37
  Got exception outside of a @test
  ArgumentError: dimension size must be ≥ 0, got -10 for dimension 1
  Stacktrace:
    [1] BitArray
      @ ./bitarray.jl:32 [inlined]
    [2] BitArray
      @ ./bitarray.jl:71 [inlined]
    [3] falses(dims::Tuple{Int64})
      @ Base ./bitarray.jl:405
    [4] falses
      @ ./bitarray.jl:403 [inlined]
    [5] quantum_mallows(rng::Random._GLOBAL_RNG, n::Int64)
      @ QuantumClifford ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:189
    [6] random_destabilizer(rng::Random._GLOBAL_RNG, n::Int64; phases::Bool)
      @ QuantumClifford ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:73
    [7] random_destabilizer
      @ ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:72 [inlined]
    [8] random_stabilizer(rng::Random._GLOBAL_RNG, n::Int64; phases::Bool)
      @ QuantumClifford ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:166
    [9] random_stabilizer
      @ ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:166 [inlined]
   [10] random_stabilizer
      @ ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:167 [inlined]
   [11] test_naive_syndrome(c::QuantumClifford.ECC.SteaneReedMuller, e::Bool)
      @ Main.var"##ecc#256" ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:12
   [12] macro expansion
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:39 [inlined]
   [13] macro expansion
      @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
   [14] top-level scope
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:38
   [15] include(mod::Module, _path::String)
      @ Base ./Base.jl:495
   [16] include(x::String)
      @ Main.var"##ecc#256" ~/.julia/packages/SafeTestsets/raUNr/src/SafeTestsets.jl:28
   [17] macro expansion
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/runtests.jl:20 [inlined]
   [18] macro expansion
      @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
   [19] macro expansion
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/runtests.jl:20 [inlined]
   [20] top-level scope
      @ ~/.julia/packages/SafeTestsets/raUNr/src/SafeTestsets.jl:30
   [21] eval(m::Module, e::Any)
      @ Core ./boot.jl:385
   [22] macro expansion
      @ ~/.julia/packages/SafeTestsets/raUNr/src/SafeTestsets.jl:28 [inlined]
   [23] top-level scope
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/runtests.jl:19
   [24] include(fname::String)
      @ Base.MainInclude ./client.jl:489
   [25] top-level scope
      @ none:6
   [26] eval
      @ ./boot.jl:385 [inlined]
   [27] exec_options(opts::Base.JLOptions)
      @ Base ./client.jl:291
   [28] _start()
      @ Base ./client.jl:552
naive syndrome circuits - zero syndrome for logical states: Error During Test at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:59
  Got exception outside of a @test
  BoundsError: attempt to access 16-element view(::Vector{UInt8}, 17:32) with eltype UInt8 at index [17]
  Stacktrace:
    [1] throw_boundserror(A::SubArray{UInt8, 1, Vector{UInt8}, Tuple{UnitRange{Int64}}, true}, I::Tuple{Int64})
      @ Base ./abstractarray.jl:737
    [2] checkbounds
      @ ./abstractarray.jl:702 [inlined]
    [3] getindex
      @ ./subarray.jl:322 [inlined]
    [4] naive_encoding_circuit(code::QuantumClifford.ECC.SteaneReedMuller; undoperm::Bool)
      @ QuantumClifford.ECC ~/Desktop/New/b10/QuantumClifford.jl/src/ecc/circuits.jl:53
    [5] naive_encoding_circuit
      @ ~/Desktop/New/b10/QuantumClifford.jl/src/ecc/circuits.jl:18 [inlined]
    [6] test_with_pframes(code::QuantumClifford.ECC.SteaneReedMuller)
      @ Main.var"##ecc#256" ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:47
    [7] macro expansion
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:61 [inlined]
    [8] macro expansion
      @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
    [9] top-level scope
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:60
   [10] include(mod::Module, _path::String)
      @ Base ./Base.jl:495
   [11] include(x::String)
      @ Main.var"##ecc#256" ~/.julia/packages/SafeTestsets/raUNr/src/SafeTestsets.jl:28
   [12] macro expansion
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/runtests.jl:20 [inlined]
   [13] macro expansion
      @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
   [14] macro expansion
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/runtests.jl:20 [inlined]
   [15] top-level scope
      @ ~/.julia/packages/SafeTestsets/raUNr/src/SafeTestsets.jl:30
   [16] eval(m::Module, e::Any)
      @ Core ./boot.jl:385
   [17] macro expansion
      @ ~/.julia/packages/SafeTestsets/raUNr/src/SafeTestsets.jl:28 [inlined]
   [18] top-level scope
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/runtests.jl:19
   [19] include(fname::String)
      @ Base.MainInclude ./client.jl:489
   [20] top-level scope
      @ none:6
   [21] eval
      @ ./boot.jl:385 [inlined]
   [22] exec_options(opts::Base.JLOptions)
      @ Base ./client.jl:291
   [23] _start()
      @ Base ./client.jl:552
Test Summary:                                                | Pass  Error  Total  Time
ecc                                                          |  340      2    342  9.4s
  naive syndrome circuits - zero syndrome for logical states |  320      1    321  4.7s
  naive syndrome circuits - zero syndrome for logical states |   16      1     17  2.2s
  is degenerate function - test on popular codes             |    4             4  0.3s
ERROR: LoadError: Some tests did not pass: 340 passed, 0 failed, 2 errored, 0 broken.
in expression starting at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/runtests.jl:61
ERROR: Package QuantumClifford errored during testing

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Apr 2, 2024

Breakdown after running each encoding and decoding file separately

Firstly,
test_ecc_decoder_all_setups pass for the given values of SteaneReedMuller

julia> include("test_ecc_decoder_all_setups.jl")
WARNING: redefinition of constant Main.code_instance_args. This may fail, cause incorrect answers, or produce other errors.
Test Summary:                       |Time
table decoder, good for small codes | None  8.1s
Test Summary:                               |Time
belief prop decoders, good for sparse codes | None  0.1s
Test Summary:                                                          |Time
matching decoder, good as long as column weight of the code is limited | None  9.2s
Test.DefaultTestSet("matching decoder, good as long as column weight of the code is limited", Any[], 0, false, false, true, 1.712044735925158e9, 1.712044745110257e9, false, "/home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_decoder_all_setups.jl")

K<0 error in test_ecc.

julia> include("test_ecc.jl")
WARNING: redefinition of constant Main.code_instance_args. This may fail, cause incorrect answers, or produce other errors.
naive syndrome circuits - zero syndrome for logical states: Error During Test at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:37
  Got exception outside of a @test
  ArgumentError: dimension size must be ≥ 0, got -10 for dimension 1
  Stacktrace:
    [1] BitArray
      @ ./bitarray.jl:32 [inlined]
    [2] BitArray
      @ ./bitarray.jl:71 [inlined]
    [3] falses(dims::Tuple{Int64})
      @ Base ./bitarray.jl:405
    [4] falses
      @ ./bitarray.jl:403 [inlined]
    [5] quantum_mallows(rng::Random._GLOBAL_RNG, n::Int64)
      @ QuantumClifford ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:189
    [6] random_destabilizer(rng::Random._GLOBAL_RNG, n::Int64; phases::Bool)
      @ QuantumClifford ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:73
    [7] random_destabilizer
      @ ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:72 [inlined]
    [8] random_stabilizer(rng::Random._GLOBAL_RNG, n::Int64; phases::Bool)
      @ QuantumClifford ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:166
    [9] random_stabilizer
      @ ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:166 [inlined]
   [10] random_stabilizer
      @ ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:167 [inlined]
   [11] test_naive_syndrome(c::SteaneReedMuller, e::Bool)
      @ Main ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:12
   [12] macro expansion
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:39 [inlined]
   [13] macro expansion
      @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
   [14] top-level scope
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:38
   [15] include(fname::String)
      @ Base.MainInclude ./client.jl:489
   [16] top-level scope
      @ REPL[7]:1
   [17] eval
      @ ./boot.jl:385 [inlined]
   [18] eval_user_input(ast::Any, backend::REPL.REPLBackend, mod::Module)
      @ REPL ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/REPL/src/REPL.jl:150
   [19] repl_backend_loop(backend::REPL.REPLBackend, get_module::Function)
      @ REPL ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/REPL/src/REPL.jl:246
   [20] start_repl_backend(backend::REPL.REPLBackend, consumer::Any; get_module::Function)
      @ REPL ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/REPL/src/REPL.jl:231
   [21] run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool, backend::Any)
      @ REPL ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/REPL/src/REPL.jl:389
   [22] run_repl(repl::REPL.AbstractREPL, consumer::Any)
      @ REPL ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/REPL/src/REPL.jl:375
   [23] (::Base.var"#1013#1015"{Bool, Bool, Bool})(REPL::Module)
      @ Base ./client.jl:432
   [24] #invokelatest#2
      @ ./essentials.jl:892 [inlined]
   [25] invokelatest
      @ ./essentials.jl:889 [inlined]
   [26] run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, history_file::Bool, color_set::Bool)
      @ Base ./client.jl:416
   [27] exec_options(opts::Base.JLOptions)
      @ Base ./client.jl:333
   [28] _start()
      @ Base ./client.jl:552
Test Summary:                                              | Pass  Error  Total  Time
naive syndrome circuits - zero syndrome for logical states |  320      1    321  2.5s
ERROR: LoadError: Some tests did not pass: 320 passed, 0 failed, 1 errored, 0 broken.
in expression starting at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:37


test_ecc_codeproperties: I have not provided the code_s in the file as I don't know it's definition.


julia> include("test_ecc_codeproperties.jl")
WARNING: redefinition of constant Main.code_instance_args. This may fail, cause incorrect answers, or produce other errors.
Test Summary: | Pass  Total  Time
is CSS        |   21     21  0.4s
code tableau consistency: Test Failed at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:34
  Expression: code_s(code) + code_k(code) == code_n(code)
   Evaluated: 6 == 16

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:672 [inlined]
 [2] macro expansion
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:34 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
 [4] top-level scope
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:30
code tableau consistency: Test Failed at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:35
  Expression: size(H, 1) < size(H, 2)
   Evaluated: 16 < 16

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:672 [inlined]
 [2] macro expansion
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:35 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
 [4] top-level scope
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:30
code tableau consistency: Test Failed at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:38
  Expression: QuantumClifford.stab_looks_good(copy(H))

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:672 [inlined]
 [2] macro expansion
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:38 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
 [4] top-level scope
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:30
code tableau consistency: Test Failed at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:35
  Expression: size(H, 1) < size(H, 2)
   Evaluated: 64 < 64

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:672 [inlined]
 [2] macro expansion
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:35 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
 [4] top-level scope
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:30
code tableau consistency: Test Failed at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:38
  Expression: QuantumClifford.stab_looks_good(copy(H))

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:672 [inlined]
 [2] macro expansion
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:38 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
 [4] top-level scope
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:30
code tableau consistency: Test Failed at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:34
  Expression: code_s(code) + code_k(code) == code_n(code)
   Evaluated: 8 == 64

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:672 [inlined]
 [2] macro expansion
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:34 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
 [4] top-level scope
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:30
code tableau consistency: Test Failed at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:35
  Expression: size(H, 1) < size(H, 2)
   Evaluated: 64 < 64

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:672 [inlined]
 [2] macro expansion
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:35 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
 [4] top-level scope
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:30
code tableau consistency: Test Failed at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:38
  Expression: QuantumClifford.stab_looks_good(copy(H))

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:672 [inlined]
 [2] macro expansion
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:38 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
 [4] top-level scope
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:30
Test Summary:            | Pass  Fail  Total  Time
code tableau consistency |  118     8    126  0.7s
ERROR: LoadError: Some tests did not pass: 118 passed, 8 failed, 0 errored, 0 broken.
in expression starting at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:29

syndromes

include("test_ecc_syndromes.jl")
WARNING: redefinition of constant Main.code_instance_args. This may fail, cause incorrect answers, or produce other errors.
naive and shor measurement circuits: Error During Test at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_syndromes.jl:45
  Got exception outside of a @test
  BoundsError: attempt to access 16-element view(::Vector{UInt8}, 17:32) with eltype UInt8 at index [17]
  Stacktrace:
    [1] throw_boundserror(A::SubArray{UInt8, 1, Vector{UInt8}, Tuple{UnitRange{Int64}}, true}, I::Tuple{Int64})
      @ Base ./abstractarray.jl:737
    [2] checkbounds
      @ ./abstractarray.jl:702 [inlined]
    [3] getindex
      @ ./subarray.jl:322 [inlined]
    [4] naive_encoding_circuit(code::SteaneReedMuller; undoperm::Bool)
      @ QuantumClifford.ECC ~/Desktop/New/b10/QuantumClifford.jl/src/ecc/circuits.jl:53
    [5] naive_encoding_circuit
      @ ~/Desktop/New/b10/QuantumClifford.jl/src/ecc/circuits.jl:18 [inlined]
    [6] pframe_naive_vs_shor_syndrome(code::SteaneReedMuller)
      @ Main ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_syndromes.jl:10
    [7] macro expansion
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_syndromes.jl:47 [inlined]
    [8] macro expansion
      @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
    [9] top-level scope
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_syndromes.jl:46
   [10] include(fname::String)
      @ Base.MainInclude ./client.jl:489
   [11] top-level scope
      @ REPL[5]:1
   [12] eval
      @ ./boot.jl:385 [inlined]
   [13] eval_user_input(ast::Any, backend::REPL.REPLBackend, mod::Module)
      @ REPL ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/REPL/src/REPL.jl:150
   [14] repl_backend_loop(backend::REPL.REPLBackend, get_module::Function)
      @ REPL ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/REPL/src/REPL.jl:246
   [15] start_repl_backend(backend::REPL.REPLBackend, consumer::Any; get_module::Function)
      @ REPL ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/REPL/src/REPL.jl:231
   [16] run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool, backend::Any)
      @ REPL ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/REPL/src/REPL.jl:389
   [17] run_repl(repl::REPL.AbstractREPL, consumer::Any)
      @ REPL ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/REPL/src/REPL.jl:375
   [18] (::Base.var"#1013#1015"{Bool, Bool, Bool})(REPL::Module)
      @ Base ./client.jl:432
   [19] #invokelatest#2
      @ ./essentials.jl:892 [inlined]
   [20] invokelatest
      @ ./essentials.jl:889 [inlined]
   [21] run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, history_file::Bool, color_set::Bool)
      @ Base ./client.jl:416
   [22] exec_options(opts::Base.JLOptions)
      @ Base ./client.jl:333
   [23] _start()
      @ Base ./client.jl:552
Test Summary:                       | Pass  Error  Total  Time
naive and shor measurement circuits |   88      1     89  0.4s
ERROR: LoadError: Some tests did not pass: 88 passed, 0 failed, 1 errored, 0 broken.
in expression starting at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_syndromes.jl:45

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Apr 2, 2024

What's does stab_looks_good actually checks?

@Krastanov
Copy link
Member

What's does stab_looks_good actually checks?

This list of properties

function stab_looks_good(s)

@Krastanov
Copy link
Member

K<0

I think I understand what k=0 is. I do not think it makes sense to have situations where k<0.

@Krastanov
Copy link
Member

Every other test seem to work well. It's seems to me that it's the k = 0, and the way the code_base is not to be setup does provides error when k = 0.

Please have a look at CI tests. They provide the same error that is locally.

It is not obvious to me that the other tests work. If I am reading this right, all these tests are not executed because the test before them failed. https://github.com/QuantumSavory/QuantumClifford.jl/blob/master/test/runtests.jl#L62-L74

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Apr 2, 2024

Every other test seem to work well.

I phrased it wrong as a general statement. Testing individually, test_ecc_decoder_all_setups.jl passes.

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Apr 2, 2024

I think I understand what k=0 is. I do not think it makes sense to have situations where k<0.

I have been trying to ask whether we should include k = 0 for sometime ;/ Even prepared an equation image for it!

I will add an condition that does not include k =0 cases to avoid cases which have the following error
ArgumentError: dimension size must be ≥ 0, got -10 for dimension 1

I will make sure that the verify with Steane Paper, so that the algorithm is correct.

Thanks for your comments! I will try to improve the work.

@Krastanov
Copy link
Member

I think I understand what k=0 is. I do not think it makes sense to have situations where k<0.

I have been trying to ask whether we should include k = 0 for sometime ;/ Even prepared an equation image for it!

I will add an condition that does not include k =0 cases to avoid cases which have the following error ArgumentError: dimension size must be ≥ 0, got -10 for dimension 1

I will make sure that the verify with Steane Paper, so that the algorithm is correct.

Thanks for your comments! I will try to improve the work.

To clarify, k=0 can make perfect sense. I do not understand what k<0 is.

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Apr 2, 2024

I do not understand what k<0 is.

This k<0 has been dangerous for me as well ;/ I don't understand how 'generators for k < 0 are stabilizers of k > 0 codes.'

Given my limited knowledge on the subject as I am just starting out, I have not read how generator matrices are considered as stabilizers/parity matrices when k goes negative.

The paper does not elaborate on this either.

Copy link

codecov bot commented Apr 2, 2024

Codecov Report

Attention: Patch coverage is 0% with 64 lines in your changes are missing coverage. Please review.

Project coverage is 0.00%. Comparing base (cae94d8) to head (2197365).
Report is 2 commits behind head on master.

❗ Current head 2197365 differs from pull request most recent head 378b7c9. Consider uploading reports for the commit 378b7c9 to get more accurate results

Files Patch % Lines
src/ecc/codes/steanereedmuller.jl 0.00% 64 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #251       +/-   ##
==========================================
- Coverage   82.63%   0.00%   -82.64%     
==========================================
  Files          58      44       -14     
  Lines        3794    3392      -402     
==========================================
- Hits         3135       0     -3135     
- Misses        659    3392     +2733     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Apr 2, 2024

Docstring added from paper which tries to explain the k< 0.
I wanted to add unicode for subscript z, but I am not sure how to add that.
These codes have peculiar properties, and thus, maybe we need to less the strictness some code properties.

Edit
Added the condition to not consider the codes when k<0. They are not in stabilizer matrices!

@Fe-r-oz Fe-r-oz marked this pull request as ready for review April 2, 2024 18:49
@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Apr 2, 2024

Seeking out review so that I can make it better.

I have not read how generator matrices are considered as stabilizers/parity matrices when k goes negative.
The paper does not elaborate on this either.

Paper has 1 small paragraph about it, so I had carefully added it in docstring.

Edit
Tagged my comment from earlier using >

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Apr 4, 2024

Kindly please review this as well. Looking forward to your comments!

@Krastanov
Copy link
Member

I will look into it sometime late next week. Ping me if I have not by then.

@Krastanov Krastanov self-requested a review April 4, 2024 21:20
Copy link
Contributor

github-actions bot commented Apr 14, 2024

Benchmark Result

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmarks:
  • Target: 14 Apr 2024 - 11:40
  • Baseline: 14 Apr 2024 - 11:48
  • Package commits:
  • Target: e00308
  • Baseline: 64f695
  • Julia commits:
  • Target: b9aeaf
  • Baseline: b9aeaf
  • Julia command flags:
  • Target: None
  • Baseline: None
  • Environment variables:
  • Target: None
  • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID time ratio memory ratio
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 0.94 (5%) ✅ 1.00 (1%)
["circuitsim", "mctrajectories_union", "q1001_r1"] 0.93 (5%) ✅ 1.00 (1%)
["circuitsim", "pftrajectories", "q1001_r1"] 1.07 (5%) ❌ 1.00 (1%)
["circuitsim", "pftrajectories", "q101_r1"] 1.09 (5%) ❌ 1.00 (1%)
["clifford", "dense", "cnot250_on_diag500_stab"] 1.14 (5%) ❌ 1.00 (1%)
["clifford", "dense", "cnot_on_dense500_stab"] 0.95 (5%) ✅ 1.00 (1%)
["clifford", "dense", "dense500_on_diag500_stab"] 1.14 (5%) ❌ 1.00 (1%)
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 0.95 (5%) ✅ 0.99 (1%) ✅
["ecc", "evaluate_decoder", "shor_pybp_comm"] 0.82 (5%) ✅ 1.00 (1%)
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 0.89 (5%) ✅ 1.00 (1%)
["stabilizer", "canon", "diag_rref500"] 1.06 (5%) ❌ 1.00 (1%)
["stabilizer", "project", "stabilizer"] 1.05 (5%) ❌ 1.00 (1%)
["stabilizer", "tensor", "diag_pow5_20"] 0.90 (5%) ✅ 1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Target

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2263 s 0 s 298 s 6834 s 0 s
 #2 0 MHz 1850 s 0 s 265 s 7274 s 0 s
 #3 0 MHz 2832 s 0 s 300 s 6257 s 0 s
 #4 0 MHz 2533 s 0 s 302 s 6546 s 0 s
 Memory: 15.606494903564453 GB (13352.0390625 MB free)
 Uptime: 944.78 sec
 Load Avg: 1.04 1.11 0.8
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 3385 s 0 s 386 s 9880 s 0 s
 #2 0 MHz 3281 s 0 s 351 s 10017 s 0 s
 #3 0 MHz 3869 s 0 s 397 s 9380 s 0 s
 #4 0 MHz 3349 s 0 s 408 s 9880 s 0 s
 Memory: 15.606494903564453 GB (13222.359375 MB free)
 Uptime: 1371.9 sec
 Load Avg: 1.07 1.08 0.92
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmark: 14 Apr 2024 - 11:40
  • Package commit: e00308
  • Julia commit: b9aeaf
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["circuitsim", "mctrajectories", "q1001_r1"] 15.156 ms (5%) 500.77 KiB (1%) 18019
["circuitsim", "mctrajectories", "q101_r1"] 171.779 μs (5%) 50.53 KiB (1%) 1818
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 13.436 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 118.991 μs (5%) 256 bytes (1%) 8
["circuitsim", "mctrajectories_union", "q1001_r1"] 12.931 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_union", "q101_r1"] 117.960 μs (5%) 256 bytes (1%) 8
["circuitsim", "pftrajectories", "q1001_r1"] 63.549 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r100"] 174.865 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r10000"] 1.107 ms (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q101_r1"] 6.492 μs (5%) 9.42 KiB (1%) 201
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 44.293 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] 154.527 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.083 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] 5.859 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 4.478 μs (5%) 9.47 KiB (1%) 202
["circuitsim", "pftrajectories_union", "q1001_r1"] 22.602 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r100"] 132.907 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r10000"] 1.054 ms (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q101_r1"] 2.334 μs (5%) 96 bytes (1%) 2
["clifford", "dense", "cnot250_on_dense500_destab"] 11.636 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_dense500_stab"] 5.733 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_destab"] 1.122 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_stab"] 560.663 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot_on_dense500_destab"] 51.095 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_dense500_stab"] 23.043 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_destab"] 26.490 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_stab"] 13.325 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "dense500_on_dense500_destab"] 11.619 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_dense500_stab"] 5.716 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_destab"] 979.803 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_stab"] 560.663 μs (5%) 512 bytes (1%) 8
["clifford", "symbolic", "cnot250_on_dense500_destab"] 1.446 ms (5%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 686.788 μs (5%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 1.158 ms (5%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 569.710 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_destab"] 4.729 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 2.304 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 4.649 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 2.294 μs (5%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 1.864 ms (5%) 1.77 MiB (1%) 42497
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] 4.154 ms (5%) 3.60 MiB (1%) 81172
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 4.585 ms (5%) 3.71 MiB (1%) 82804
["ecc", "evaluate_decoder", "shor_pybp_comm"] 55.796 ms (5%) 4.76 MiB (1%) 144207
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 126.504 ms (5%) 9.57 MiB (1%) 284018
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 138.460 ms (5%) 9.66 MiB (1%) 285104
["ecc", "evaluate_decoder", "shor_table_comm"] 256.788 μs (5%) 241.42 KiB (1%) 4593
["ecc", "evaluate_decoder", "shor_table_naivesyn"] 873.325 μs (5%) 334.20 KiB (1%) 4790
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.274 ms (5%) 427.73 KiB (1%) 5876
["ecc", "evaluate_decoder", "toric8_bp_comm"] 687.924 ms (5%) 14.117 ms 173.01 MiB (1%) 1037289
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 1.380 s (5%) 28.500 ms 348.16 MiB (1%) 2109359
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 1.431 s (5%) 28.882 ms 359.72 MiB (1%) 2195029
["ecc", "evaluate_decoder", "toric8_pybp_comm"] 458.415 ms (5%) 8.11 MiB (1%) 163696
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] 947.037 ms (5%) 18.04 MiB (1%) 360388
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] 951.169 ms (5%) 20.56 MiB (1%) 392802
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] 4.253 ms (5%) 1.95 MiB (1%) 23176
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] 12.541 ms (5%) 5.58 MiB (1%) 79355
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] 21.109 ms (5%) 8.09 MiB (1%) 111769
["ecc", "evaluate_decoder", "toric8_table_comm"] 2.647 ms (5%) 1.64 MiB (1%) 23082
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] 10.236 ms (5%) 4.93 MiB (1%) 79160
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] 18.772 ms (5%) 7.44 MiB (1%) 111574
["pauli", "mul", "100"] 14.838 ns (5%)
["pauli", "mul", "1000"] 17.930 ns (5%)
["pauli", "mul", "100000"] 723.346 ns (5%)
["pauli", "mul", "20000000"] 180.556 μs (5%)
["stabilizer", "canon", "cano500"] 3.109 ms (5%)
["stabilizer", "canon", "diag_cano500"] 676.749 μs (5%)
["stabilizer", "canon", "diag_gott500"] 4.184 ms (5%) 5.19 MiB (1%) 34521
["stabilizer", "canon", "diag_rref500"] 675.567 μs (5%)
["stabilizer", "canon", "gott500"] 6.551 ms (5%) 5.19 MiB (1%) 34531
["stabilizer", "canon", "md_cano500"] 1.592 ms (5%)
["stabilizer", "canon", "md_rref500"] 1.629 ms (5%)
["stabilizer", "canon", "rref500"] 3.196 ms (5%)
["stabilizer", "project", "destabilizer"] 18.223 μs (5%) 288 bytes (1%) 5
["stabilizer", "project", "stabilizer"] 9.428 μs (5%) 80 bytes (1%) 2
["stabilizer", "tensor", "diag_pow5_20"] 1.139 ms (5%) 23.97 MiB (1%) 34
["stabilizer", "tensor", "pow5_20"] 2.869 μs (5%) 6.44 KiB (1%) 31
["stabilizer", "trace", "destabilizer"] 34.114 μs (5%) 80 bytes (1%) 2
["stabilizer", "trace", "stabilizer"] 24.495 μs (5%) 112 bytes (1%) 3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2263 s 0 s 298 s 6834 s 0 s
 #2 0 MHz 1850 s 0 s 265 s 7274 s 0 s
 #3 0 MHz 2832 s 0 s 300 s 6257 s 0 s
 #4 0 MHz 2533 s 0 s 302 s 6546 s 0 s
 Memory: 15.606494903564453 GB (13352.0390625 MB free)
 Uptime: 944.78 sec
 Load Avg: 1.04 1.11 0.8
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmark: 14 Apr 2024 - 11:48
  • Package commit: 64f695
  • Julia commit: b9aeaf
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["circuitsim", "mctrajectories", "q1001_r1"] 15.134 ms (5%) 500.77 KiB (1%) 18019
["circuitsim", "mctrajectories", "q101_r1"] 171.709 μs (5%) 50.53 KiB (1%) 1818
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 14.271 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 118.701 μs (5%) 256 bytes (1%) 8
["circuitsim", "mctrajectories_union", "q1001_r1"] 13.836 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_union", "q101_r1"] 116.867 μs (5%) 256 bytes (1%) 8
["circuitsim", "pftrajectories", "q1001_r1"] 59.240 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r100"] 173.462 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r10000"] 1.100 ms (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q101_r1"] 5.971 μs (5%) 9.42 KiB (1%) 201
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 44.884 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] 157.844 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.087 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] 5.849 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 4.488 μs (5%) 9.47 KiB (1%) 202
["circuitsim", "pftrajectories_union", "q1001_r1"] 22.612 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r100"] 134.370 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r10000"] 1.056 ms (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q101_r1"] 2.344 μs (5%) 96 bytes (1%) 2
["clifford", "dense", "cnot250_on_dense500_destab"] 11.423 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_dense500_stab"] 5.735 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_destab"] 1.123 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_stab"] 490.432 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot_on_dense500_destab"] 50.644 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_dense500_stab"] 24.376 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_destab"] 26.249 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_stab"] 13.715 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "dense500_on_dense500_destab"] 11.427 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_dense500_stab"] 5.730 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_destab"] 980.164 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_stab"] 490.212 μs (5%) 512 bytes (1%) 8
["clifford", "symbolic", "cnot250_on_dense500_destab"] 1.433 ms (5%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 684.583 μs (5%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 1.161 ms (5%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 568.315 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_destab"] 4.749 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 2.304 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 4.649 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 2.284 μs (5%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 1.822 ms (5%) 1.77 MiB (1%) 42665
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] 4.042 ms (5%) 3.61 MiB (1%) 81606
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 4.828 ms (5%) 3.76 MiB (1%) 84120
["ecc", "evaluate_decoder", "shor_pybp_comm"] 68.329 ms (5%) 4.76 MiB (1%) 144207
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 141.506 ms (5%) 9.57 MiB (1%) 284018
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 142.219 ms (5%) 9.66 MiB (1%) 285104
["ecc", "evaluate_decoder", "shor_table_comm"] 252.741 μs (5%) 241.42 KiB (1%) 4593
["ecc", "evaluate_decoder", "shor_table_naivesyn"] 869.669 μs (5%) 334.20 KiB (1%) 4790
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.266 ms (5%) 427.73 KiB (1%) 5876
["ecc", "evaluate_decoder", "toric8_bp_comm"] 682.276 ms (5%) 11.528 ms 171.80 MiB (1%) 1030191
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 1.388 s (5%) 24.881 ms 350.12 MiB (1%) 2120895
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 1.419 s (5%) 26.709 ms 356.86 MiB (1%) 2178201
["ecc", "evaluate_decoder", "toric8_pybp_comm"] 465.957 ms (5%) 8.11 MiB (1%) 163696
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] 952.794 ms (5%) 18.04 MiB (1%) 360388
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] 950.216 ms (5%) 20.56 MiB (1%) 392802
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] 4.224 ms (5%) 1.95 MiB (1%) 23176
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] 12.500 ms (5%) 5.58 MiB (1%) 79355
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] 20.975 ms (5%) 8.09 MiB (1%) 111769
["ecc", "evaluate_decoder", "toric8_table_comm"] 2.645 ms (5%) 1.64 MiB (1%) 23082
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] 10.174 ms (5%) 4.93 MiB (1%) 79160
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] 18.636 ms (5%) 7.44 MiB (1%) 111574
["pauli", "mul", "100"] 14.626 ns (5%)
["pauli", "mul", "1000"] 18.873 ns (5%)
["pauli", "mul", "100000"] 729.046 ns (5%)
["pauli", "mul", "20000000"] 173.172 μs (5%)
["stabilizer", "canon", "cano500"] 3.081 ms (5%)
["stabilizer", "canon", "diag_cano500"] 708.669 μs (5%)
["stabilizer", "canon", "diag_gott500"] 4.091 ms (5%) 5.19 MiB (1%) 34521
["stabilizer", "canon", "diag_rref500"] 637.937 μs (5%)
["stabilizer", "canon", "gott500"] 6.507 ms (5%) 5.19 MiB (1%) 34531
["stabilizer", "canon", "md_cano500"] 1.591 ms (5%)
["stabilizer", "canon", "md_rref500"] 1.611 ms (5%)
["stabilizer", "canon", "rref500"] 3.138 ms (5%)
["stabilizer", "project", "destabilizer"] 18.484 μs (5%) 288 bytes (1%) 5
["stabilizer", "project", "stabilizer"] 8.967 μs (5%) 80 bytes (1%) 2
["stabilizer", "tensor", "diag_pow5_20"] 1.264 ms (5%) 23.97 MiB (1%) 34
["stabilizer", "tensor", "pow5_20"] 2.851 μs (5%) 6.44 KiB (1%) 31
["stabilizer", "trace", "destabilizer"] 32.781 μs (5%) 80 bytes (1%) 2
["stabilizer", "trace", "stabilizer"] 24.866 μs (5%) 112 bytes (1%) 3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 3385 s 0 s 386 s 9880 s 0 s
 #2 0 MHz 3281 s 0 s 351 s 10017 s 0 s
 #3 0 MHz 3869 s 0 s 397 s 9380 s 0 s
 #4 0 MHz 3349 s 0 s 408 s 9880 s 0 s
 Memory: 15.606494903564453 GB (13222.359375 MB free)
 Uptime: 1371.9 sec
 Load Avg: 1.07 1.08 0.92
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Runtime information

Runtime Info
BLAS #threads 2
BLAS.vendor() lbt
Sys.CPU_THREADS 4

lscpu output:

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 48 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Vendor ID: AuthenticAMD
Model name: AMD EPYC 7763 64-Core Processor
CPU family: 25
Model: 1
Thread(s) per core: 2
Core(s) per socket: 2
Socket(s): 1
Stepping: 1
BogoMIPS: 4890.85
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm
Virtualization: AMD-V
Hypervisor vendor: Microsoft
Virtualization type: full
L1d cache: 64 KiB (2 instances)
L1i cache: 64 KiB (2 instances)
L2 cache: 1 MiB (2 instances)
L3 cache: 32 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected

Cpu Property Value
Brand AMD EPYC 7763 64-Core Processor
Vendor :AMD
Architecture :Unknown
Model Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00
Cores 16 physical cores, 16 logical cores (on executing CPU)
No Hyperthreading hardware capability detected
Clock Frequencies Not supported by CPU
Data Cache Level 1:3 : (32, 512, 32768) kbytes
64 byte cache line size
Address Size 48 bits virtual, 48 bits physical
SIMD 256 bit = 32 byte max. SIMD vector size
Time Stamp Counter TSC is accessible via rdtsc
TSC runs at constant rate (invariant from clock frequency)
Perf. Monitoring Performance Monitoring Counters (PMC) are not supported
Hypervisor Yes, Microsoft
Benchmark Result

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmarks:
  • Target: 14 Apr 2024 - 12:08
  • Baseline: 14 Apr 2024 - 12:15
  • Package commits:
  • Target: 390841
  • Baseline: 64f695
  • Julia commits:
  • Target: b9aeaf
  • Baseline: b9aeaf
  • Julia command flags:
  • Target: None
  • Baseline: None
  • Environment variables:
  • Target: None
  • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID time ratio memory ratio
["circuitsim", "mctrajectories_union", "q1001_r1"] 0.92 (5%) ✅ 1.00 (1%)
["circuitsim", "pftrajectories", "q1001_r1"] 1.12 (5%) ❌ 1.00 (1%)
["circuitsim", "pftrajectories", "q1001_r100"] 1.07 (5%) ❌ 1.00 (1%)
["circuitsim", "pftrajectories", "q1001_r10000"] 1.05 (5%) ❌ 1.00 (1%)
["circuitsim", "pftrajectories", "q101_r1"] 1.12 (5%) ❌ 1.00 (1%)
["clifford", "dense", "cnot_on_dense500_stab"] 0.95 (5%) ✅ 1.00 (1%)
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 0.94 (5%) ✅ 0.97 (1%) ✅
["ecc", "evaluate_decoder", "shor_pybp_comm"] 1.08 (5%) ❌ 1.00 (1%)
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 0.88 (5%) ✅ 1.00 (1%)
["ecc", "evaluate_decoder", "toric8_bp_comm"] 0.97 (5%) 0.99 (1%) ✅
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 0.99 (5%) 0.98 (1%) ✅
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 1.01 (5%) 1.02 (1%) ❌
["pauli", "mul", "20000000"] 1.05 (5%) ❌ 1.00 (1%)
["stabilizer", "tensor", "diag_pow5_20"] 1.14 (5%) ❌ 1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Target

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2360 s 0 s 290 s 7985 s 0 s
 #2 0 MHz 2461 s 0 s 290 s 7886 s 0 s
 #3 0 MHz 2536 s 0 s 311 s 7785 s 0 s
 #4 0 MHz 2244 s 0 s 305 s 8086 s 0 s
 Memory: 15.606494903564453 GB (13355.05859375 MB free)
 Uptime: 1069.87 sec
 Load Avg: 1.01 1.08 0.78
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 3222 s 0 s 378 s 11328 s 0 s
 #2 0 MHz 2942 s 0 s 389 s 11600 s 0 s
 #3 0 MHz 4166 s 0 s 400 s 10358 s 0 s
 #4 0 MHz 3718 s 0 s 405 s 10804 s 0 s
 Memory: 15.606494903564453 GB (13236.34375 MB free)
 Uptime: 1500.63 sec
 Load Avg: 1.0 1.08 0.92
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmark: 14 Apr 2024 - 12:8
  • Package commit: 390841
  • Julia commit: b9aeaf
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["circuitsim", "mctrajectories", "q1001_r1"] 15.294 ms (5%) 500.77 KiB (1%) 18019
["circuitsim", "mctrajectories", "q101_r1"] 171.821 μs (5%) 50.53 KiB (1%) 1818
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 13.320 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 118.832 μs (5%) 256 bytes (1%) 8
["circuitsim", "mctrajectories_union", "q1001_r1"] 13.144 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_union", "q101_r1"] 116.928 μs (5%) 256 bytes (1%) 8
["circuitsim", "pftrajectories", "q1001_r1"] 66.755 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r100"] 181.770 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r10000"] 1.152 ms (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q101_r1"] 6.692 μs (5%) 9.42 KiB (1%) 201
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 43.873 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] 158.295 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.129 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] 5.857 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 4.488 μs (5%) 9.47 KiB (1%) 202
["circuitsim", "pftrajectories_union", "q1001_r1"] 22.311 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r100"] 136.175 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r10000"] 1.101 ms (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q101_r1"] 2.304 μs (5%) 96 bytes (1%) 2
["clifford", "dense", "cnot250_on_dense500_destab"] 11.740 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_dense500_stab"] 5.792 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_destab"] 1.122 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_stab"] 561.700 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot_on_dense500_destab"] 50.554 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_dense500_stab"] 23.243 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_destab"] 26.449 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_stab"] 13.395 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "dense500_on_dense500_destab"] 11.746 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_dense500_stab"] 5.794 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_destab"] 980.844 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_stab"] 561.700 μs (5%) 512 bytes (1%) 8
["clifford", "symbolic", "cnot250_on_dense500_destab"] 1.435 ms (5%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 687.866 μs (5%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 1.161 ms (5%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 570.026 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_destab"] 4.729 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 2.304 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 4.659 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 2.304 μs (5%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 1.841 ms (5%) 1.77 MiB (1%) 42651
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] 4.173 ms (5%) 3.61 MiB (1%) 81578
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 4.573 ms (5%) 3.70 MiB (1%) 82622
["ecc", "evaluate_decoder", "shor_pybp_comm"] 63.400 ms (5%) 4.76 MiB (1%) 144207
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 119.778 ms (5%) 9.57 MiB (1%) 284018
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 140.635 ms (5%) 9.66 MiB (1%) 285104
["ecc", "evaluate_decoder", "shor_table_comm"] 258.123 μs (5%) 241.42 KiB (1%) 4593
["ecc", "evaluate_decoder", "shor_table_naivesyn"] 870.818 μs (5%) 334.20 KiB (1%) 4790
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.263 ms (5%) 427.73 KiB (1%) 5876
["ecc", "evaluate_decoder", "toric8_bp_comm"] 678.360 ms (5%) 15.917 ms 172.26 MiB (1%) 1032858
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 1.398 s (5%) 26.988 ms 347.24 MiB (1%) 2103955
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 1.402 s (5%) 37.628 ms 353.22 MiB (1%) 2156767
["ecc", "evaluate_decoder", "toric8_pybp_comm"] 454.812 ms (5%) 8.11 MiB (1%) 163696
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] 957.254 ms (5%) 18.04 MiB (1%) 360388
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] 989.646 ms (5%) 20.56 MiB (1%) 392802
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] 4.277 ms (5%) 1.95 MiB (1%) 23176
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] 12.431 ms (5%) 5.58 MiB (1%) 79355
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] 20.979 ms (5%) 8.09 MiB (1%) 111769
["ecc", "evaluate_decoder", "toric8_table_comm"] 2.685 ms (5%) 1.64 MiB (1%) 23082
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] 10.147 ms (5%) 4.93 MiB (1%) 79160
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] 18.603 ms (5%) 7.44 MiB (1%) 111574
["pauli", "mul", "100"] 14.205 ns (5%)
["pauli", "mul", "1000"] 17.968 ns (5%)
["pauli", "mul", "100000"] 717.629 ns (5%)
["pauli", "mul", "20000000"] 192.479 μs (5%)
["stabilizer", "canon", "cano500"] 3.049 ms (5%)
["stabilizer", "canon", "diag_cano500"] 709.055 μs (5%)
["stabilizer", "canon", "diag_gott500"] 4.051 ms (5%) 5.19 MiB (1%) 34521
["stabilizer", "canon", "diag_rref500"] 627.062 μs (5%)
["stabilizer", "canon", "gott500"] 6.400 ms (5%) 5.19 MiB (1%) 34531
["stabilizer", "canon", "md_cano500"] 1.548 ms (5%)
["stabilizer", "canon", "md_rref500"] 1.562 ms (5%)
["stabilizer", "canon", "rref500"] 3.118 ms (5%)
["stabilizer", "project", "destabilizer"] 18.335 μs (5%) 288 bytes (1%) 5
["stabilizer", "project", "stabilizer"] 9.598 μs (5%) 80 bytes (1%) 2
["stabilizer", "tensor", "diag_pow5_20"] 1.432 ms (5%) 23.97 MiB (1%) 34
["stabilizer", "tensor", "pow5_20"] 2.792 μs (5%) 6.44 KiB (1%) 31
["stabilizer", "trace", "destabilizer"] 31.298 μs (5%) 80 bytes (1%) 2
["stabilizer", "trace", "stabilizer"] 24.115 μs (5%) 112 bytes (1%) 3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2360 s 0 s 290 s 7985 s 0 s
 #2 0 MHz 2461 s 0 s 290 s 7886 s 0 s
 #3 0 MHz 2536 s 0 s 311 s 7785 s 0 s
 #4 0 MHz 2244 s 0 s 305 s 8086 s 0 s
 Memory: 15.606494903564453 GB (13355.05859375 MB free)
 Uptime: 1069.87 sec
 Load Avg: 1.01 1.08 0.78
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmark: 14 Apr 2024 - 12:15
  • Package commit: 64f695
  • Julia commit: b9aeaf
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["circuitsim", "mctrajectories", "q1001_r1"] 15.503 ms (5%) 500.77 KiB (1%) 18019
["circuitsim", "mctrajectories", "q101_r1"] 169.847 μs (5%) 50.53 KiB (1%) 1818
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 13.717 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 118.171 μs (5%) 256 bytes (1%) 8
["circuitsim", "mctrajectories_union", "q1001_r1"] 14.247 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_union", "q101_r1"] 117.240 μs (5%) 256 bytes (1%) 8
["circuitsim", "pftrajectories", "q1001_r1"] 59.481 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r100"] 169.186 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r10000"] 1.097 ms (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q101_r1"] 6.001 μs (5%) 9.42 KiB (1%) 201
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 44.593 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] 154.789 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.084 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] 5.831 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 4.458 μs (5%) 9.47 KiB (1%) 202
["circuitsim", "pftrajectories_union", "q1001_r1"] 22.603 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r100"] 132.027 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r10000"] 1.052 ms (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q101_r1"] 2.344 μs (5%) 96 bytes (1%) 2
["clifford", "dense", "cnot250_on_dense500_destab"] 11.725 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_dense500_stab"] 5.756 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_destab"] 1.122 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_stab"] 561.591 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot_on_dense500_destab"] 50.644 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_dense500_stab"] 24.485 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_destab"] 26.129 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_stab"] 13.725 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "dense500_on_dense500_destab"] 11.725 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_dense500_stab"] 5.760 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_destab"] 980.134 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_stab"] 561.591 μs (5%) 512 bytes (1%) 8
["clifford", "symbolic", "cnot250_on_dense500_destab"] 1.440 ms (5%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 687.125 μs (5%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 1.161 ms (5%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 569.425 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_destab"] 4.769 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 2.294 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 4.658 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 2.294 μs (5%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 1.850 ms (5%) 1.78 MiB (1%) 42833
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] 4.157 ms (5%) 3.62 MiB (1%) 81830
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 4.881 ms (5%) 3.81 MiB (1%) 85170
["ecc", "evaluate_decoder", "shor_pybp_comm"] 58.758 ms (5%) 4.76 MiB (1%) 144207
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 136.706 ms (5%) 9.57 MiB (1%) 284018
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 140.608 ms (5%) 9.66 MiB (1%) 285104
["ecc", "evaluate_decoder", "shor_table_comm"] 253.654 μs (5%) 241.42 KiB (1%) 4593
["ecc", "evaluate_decoder", "shor_table_naivesyn"] 866.530 μs (5%) 334.20 KiB (1%) 4790
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.265 ms (5%) 427.73 KiB (1%) 5876
["ecc", "evaluate_decoder", "toric8_bp_comm"] 696.963 ms (5%) 12.891 ms 174.39 MiB (1%) 1045374
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 1.409 s (5%) 29.752 ms 354.52 MiB (1%) 2146767
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 1.383 s (5%) 24.310 ms 344.77 MiB (1%) 2107074
["ecc", "evaluate_decoder", "toric8_pybp_comm"] 456.334 ms (5%) 8.11 MiB (1%) 163696
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] 965.308 ms (5%) 18.04 MiB (1%) 360388
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] 983.870 ms (5%) 20.56 MiB (1%) 392802
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] 4.262 ms (5%) 1.95 MiB (1%) 23176
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] 12.409 ms (5%) 5.58 MiB (1%) 79355
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] 20.895 ms (5%) 8.09 MiB (1%) 111769
["ecc", "evaluate_decoder", "toric8_table_comm"] 2.626 ms (5%) 1.64 MiB (1%) 23082
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] 10.054 ms (5%) 4.93 MiB (1%) 79160
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] 18.498 ms (5%) 7.44 MiB (1%) 111574
["pauli", "mul", "100"] 14.526 ns (5%)
["pauli", "mul", "1000"] 18.820 ns (5%)
["pauli", "mul", "100000"] 728.864 ns (5%)
["pauli", "mul", "20000000"] 183.152 μs (5%)
["stabilizer", "canon", "cano500"] 3.078 ms (5%)
["stabilizer", "canon", "diag_cano500"] 709.035 μs (5%)
["stabilizer", "canon", "diag_gott500"] 4.037 ms (5%) 5.19 MiB (1%) 34521
["stabilizer", "canon", "diag_rref500"] 627.144 μs (5%)
["stabilizer", "canon", "gott500"] 6.392 ms (5%) 5.19 MiB (1%) 34531
["stabilizer", "canon", "md_cano500"] 1.537 ms (5%)
["stabilizer", "canon", "md_rref500"] 1.555 ms (5%)
["stabilizer", "canon", "rref500"] 3.119 ms (5%)
["stabilizer", "project", "destabilizer"] 17.944 μs (5%) 288 bytes (1%) 5
["stabilizer", "project", "stabilizer"] 9.808 μs (5%) 80 bytes (1%) 2
["stabilizer", "tensor", "diag_pow5_20"] 1.254 ms (5%) 23.97 MiB (1%) 34
["stabilizer", "tensor", "pow5_20"] 2.849 μs (5%) 6.44 KiB (1%) 31
["stabilizer", "trace", "destabilizer"] 31.048 μs (5%) 80 bytes (1%) 2
["stabilizer", "trace", "stabilizer"] 24.206 μs (5%) 112 bytes (1%) 3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 3222 s 0 s 378 s 11328 s 0 s
 #2 0 MHz 2942 s 0 s 389 s 11600 s 0 s
 #3 0 MHz 4166 s 0 s 400 s 10358 s 0 s
 #4 0 MHz 3718 s 0 s 405 s 10804 s 0 s
 Memory: 15.606494903564453 GB (13236.34375 MB free)
 Uptime: 1500.63 sec
 Load Avg: 1.0 1.08 0.92
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Runtime information

Runtime Info
BLAS #threads 2
BLAS.vendor() lbt
Sys.CPU_THREADS 4

lscpu output:

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 48 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Vendor ID: AuthenticAMD
Model name: AMD EPYC 7763 64-Core Processor
CPU family: 25
Model: 1
Thread(s) per core: 2
Core(s) per socket: 2
Socket(s): 1
Stepping: 1
BogoMIPS: 4890.84
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm
Virtualization: AMD-V
Hypervisor vendor: Microsoft
Virtualization type: full
L1d cache: 64 KiB (2 instances)
L1i cache: 64 KiB (2 instances)
L2 cache: 1 MiB (2 instances)
L3 cache: 32 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected

Cpu Property Value
Brand AMD EPYC 7763 64-Core Processor
Vendor :AMD
Architecture :Unknown
Model Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00
Cores 16 physical cores, 16 logical cores (on executing CPU)
No Hyperthreading hardware capability detected
Clock Frequencies Not supported by CPU
Data Cache Level 1:3 : (32, 512, 32768) kbytes
64 byte cache line size
Address Size 48 bits virtual, 48 bits physical
SIMD 256 bit = 32 byte max. SIMD vector size
Time Stamp Counter TSC is accessible via rdtsc
TSC runs at constant rate (invariant from clock frequency)
Perf. Monitoring Performance Monitoring Counters (PMC) are not supported
Hypervisor Yes, Microsoft
Benchmark Result

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmarks:
  • Target: 14 Apr 2024 - 13:08
  • Baseline: 14 Apr 2024 - 13:15
  • Package commits:
  • Target: 960242
  • Baseline: 64f695
  • Julia commits:
  • Target: b9aeaf
  • Baseline: b9aeaf
  • Julia command flags:
  • Target: None
  • Baseline: None
  • Environment variables:
  • Target: None
  • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID time ratio memory ratio
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 0.92 (5%) ✅ 1.00 (1%)
["clifford", "dense", "cnot_on_dense500_stab"] 1.06 (5%) ❌ 1.00 (1%)
["clifford", "dense", "dense500_on_diag500_destab"] 1.15 (5%) ❌ 1.00 (1%)
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 0.89 (5%) ✅ 0.96 (1%) ✅
["ecc", "evaluate_decoder", "shor_pybp_comm"] 0.93 (5%) ✅ 1.00 (1%)
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 1.27 (5%) ❌ 1.00 (1%)
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 1.36 (5%) ❌ 1.00 (1%)
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 1.01 (5%) 0.99 (1%) ✅
["stabilizer", "canon", "diag_cano500"] 0.94 (5%) ✅ 1.00 (1%)
["stabilizer", "canon", "diag_rref500"] 0.87 (5%) ✅ 1.00 (1%)
["stabilizer", "tensor", "diag_pow5_20"] 0.81 (5%) ✅ 1.00 (1%)
["stabilizer", "trace", "destabilizer"] 0.93 (5%) ✅ 1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Target

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2282 s 0 s 294 s 13346 s 0 s
 #2 0 MHz 2785 s 0 s 306 s 12841 s 0 s
 #3 0 MHz 2109 s 0 s 283 s 13529 s 0 s
 #4 0 MHz 2420 s 0 s 296 s 13217 s 0 s
 Memory: 15.606498718261719 GB (13357.84375 MB free)
 Uptime: 1599.32 sec
 Load Avg: 1.06 1.08 0.77
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 3283 s 0 s 381 s 16590 s 0 s
 #2 0 MHz 3967 s 0 s 414 s 15888 s 0 s
 #3 0 MHz 2882 s 0 s 376 s 16992 s 0 s
 #4 0 MHz 3941 s 0 s 392 s 15930 s 0 s
 Memory: 15.606498718261719 GB (13173.26953125 MB free)
 Uptime: 2034.01 sec
 Load Avg: 1.0 1.06 0.91
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmark: 14 Apr 2024 - 13:8
  • Package commit: 960242
  • Julia commit: b9aeaf
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["circuitsim", "mctrajectories", "q1001_r1"] 15.031 ms (5%) 500.77 KiB (1%) 18019
["circuitsim", "mctrajectories", "q101_r1"] 167.663 μs (5%) 50.53 KiB (1%) 1818
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 13.410 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 118.051 μs (5%) 256 bytes (1%) 8
["circuitsim", "mctrajectories_union", "q1001_r1"] 12.953 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_union", "q101_r1"] 117.139 μs (5%) 256 bytes (1%) 8
["circuitsim", "pftrajectories", "q1001_r1"] 59.792 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r100"] 173.083 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r10000"] 1.098 ms (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q101_r1"] 5.992 μs (5%) 9.42 KiB (1%) 201
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 44.303 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] 154.218 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.082 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] 5.863 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 4.458 μs (5%) 9.47 KiB (1%) 202
["circuitsim", "pftrajectories_union", "q1001_r1"] 22.602 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r100"] 132.558 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r10000"] 1.054 ms (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q101_r1"] 2.334 μs (5%) 96 bytes (1%) 2
["clifford", "dense", "cnot250_on_dense500_destab"] 11.598 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_dense500_stab"] 5.740 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_destab"] 1.123 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_stab"] 560.608 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot_on_dense500_destab"] 51.988 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_dense500_stab"] 24.546 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_destab"] 26.549 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_stab"] 13.785 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "dense500_on_dense500_destab"] 11.541 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_dense500_stab"] 5.733 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_destab"] 1.123 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_stab"] 560.568 μs (5%) 512 bytes (1%) 8
["clifford", "symbolic", "cnot250_on_dense500_destab"] 1.449 ms (5%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 687.045 μs (5%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 1.158 ms (5%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 570.737 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_destab"] 4.728 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 2.304 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 4.659 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 2.294 μs (5%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 1.798 ms (5%) 1.76 MiB (1%) 42343
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] 4.143 ms (5%) 3.62 MiB (1%) 81606
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 4.549 ms (5%) 3.70 MiB (1%) 82622
["ecc", "evaluate_decoder", "shor_pybp_comm"] 57.878 ms (5%) 4.76 MiB (1%) 144207
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 128.694 ms (5%) 9.57 MiB (1%) 284018
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 171.238 ms (5%) 9.66 MiB (1%) 285104
["ecc", "evaluate_decoder", "shor_table_comm"] 256.099 μs (5%) 241.42 KiB (1%) 4593
["ecc", "evaluate_decoder", "shor_table_naivesyn"] 868.634 μs (5%) 334.20 KiB (1%) 4790
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.263 ms (5%) 427.73 KiB (1%) 5876
["ecc", "evaluate_decoder", "toric8_bp_comm"] 709.219 ms (5%) 13.053 ms 174.38 MiB (1%) 1045311
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 1.449 s (5%) 28.275 ms 348.76 MiB (1%) 2112901
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 1.461 s (5%) 29.249 ms 351.42 MiB (1%) 2146183
["ecc", "evaluate_decoder", "toric8_pybp_comm"] 454.695 ms (5%) 8.11 MiB (1%) 163696
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] 944.429 ms (5%) 18.04 MiB (1%) 360388
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] 940.861 ms (5%) 20.56 MiB (1%) 392802
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] 4.208 ms (5%) 1.95 MiB (1%) 23176
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] 12.468 ms (5%) 5.58 MiB (1%) 79355
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] 20.956 ms (5%) 8.09 MiB (1%) 111769
["ecc", "evaluate_decoder", "toric8_table_comm"] 2.622 ms (5%) 1.64 MiB (1%) 23082
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] 10.162 ms (5%) 4.93 MiB (1%) 79160
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] 18.722 ms (5%) 7.44 MiB (1%) 111574
["pauli", "mul", "100"] 14.416 ns (5%)
["pauli", "mul", "1000"] 18.189 ns (5%)
["pauli", "mul", "100000"] 722.992 ns (5%)
["pauli", "mul", "20000000"] 205.243 μs (5%)
["stabilizer", "canon", "cano500"] 3.044 ms (5%)
["stabilizer", "canon", "diag_cano500"] 708.715 μs (5%)
["stabilizer", "canon", "diag_gott500"] 4.078 ms (5%) 5.19 MiB (1%) 34521
["stabilizer", "canon", "diag_rref500"] 627.043 μs (5%)
["stabilizer", "canon", "gott500"] 6.438 ms (5%) 5.19 MiB (1%) 34531
["stabilizer", "canon", "md_cano500"] 1.533 ms (5%)
["stabilizer", "canon", "md_rref500"] 1.556 ms (5%)
["stabilizer", "canon", "rref500"] 3.120 ms (5%)
["stabilizer", "project", "destabilizer"] 18.585 μs (5%) 288 bytes (1%) 5
["stabilizer", "project", "stabilizer"] 8.906 μs (5%) 80 bytes (1%) 2
["stabilizer", "tensor", "diag_pow5_20"] 1.116 ms (5%) 23.97 MiB (1%) 34
["stabilizer", "tensor", "pow5_20"] 2.805 μs (5%) 6.44 KiB (1%) 31
["stabilizer", "trace", "destabilizer"] 31.319 μs (5%) 80 bytes (1%) 2
["stabilizer", "trace", "stabilizer"] 24.085 μs (5%) 112 bytes (1%) 3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2282 s 0 s 294 s 13346 s 0 s
 #2 0 MHz 2785 s 0 s 306 s 12841 s 0 s...*[Comment body truncated]*

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Apr 14, 2024

Removed unnessary complexity.

Removed the cases where construction method fails as per Steane.

Significantly making is simpler, easy to understand

Removed unnecessary helper function

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Apr 16, 2024

Dear @Krastanov I am looking forward to your comments and review!

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Apr 18, 2024

Showing Table 1 from Steane paper as reference here
The construction method governed by equation [8] of the paper works for k > 0

k t= 1 (Gottesman) , d =3 t= 2 , d =6 t= 3, d =12 t= 4 , d =24 t= 5 , d =48
n= 2 0 (1,2) ✗ - (2,2) ✗ - (3,2) ✗ - (4,2) ✗ - (5,2) ✗
n=3 + (1,3) ✓ - (2,3) ✗ - (3,3) ✗ - (4,3) ✗ - (5,3) ✗
n=4 + (1,4) ✓ 0 (2,4) ✗ - (3,4) ✗ - (4,4) ✗ - (5,4) ✗
n=5 + (1,5) ✓ + (2,5) ✓ - (3,5) ✗ - (4,5) ✗ - (5,5) ✗
n=6 + (1,6) ✓ + (2,6) ✓ 0 (3,6) ✗ - (4,6) ✗ - (5,6) ✗
n=7 + (1,7) ✓ + (2,7) ✓ + (3,7) ✓ - (4,7) ✗ - (5,7) ✗
n=8 + (1,8) ✓ + (2,8) ✓ + (3,8) ✓ 0 (4,8) ✗ - (5,8) ✗
n=9 + (1,9) ✓ + (2,9) ✓ + (3,9) ✓ + (4,9) ✓ - (5,9) ✗
n =10 + (1,10)✓ + (2,10) ✓ + (3,10) ✓ + (4,10) ✓ 0 (5,10) ✗

CHANGELOG.md Outdated Show resolved Hide resolved
Copy link
Member

@Krastanov Krastanov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a lot of code instances that are not tested for correctness. Adding these to the test runner would make it easier to know whether there are errors here. I have not done a very in-depth review of the codes being produced by this yet -- I will wait for the test runner to verify them first.

I made some minor suggestions on formatting. The docstring for the code confuses me a bit, but it seems like only a minor change is necessary there.

Overall this looks pretty good, but let's first see what the test runner says when given a larger family of codes.

src/ecc/codes/steanereedmuller.jl Outdated Show resolved Hide resolved
src/ecc/codes/steanereedmuller.jl Outdated Show resolved Hide resolved
test/test_ecc_base.jl Outdated Show resolved Hide resolved
src/ecc/codes/steanereedmuller.jl Outdated Show resolved Hide resolved
src/ecc/codes/steanereedmuller.jl Outdated Show resolved Hide resolved
src/ecc/codes/steanereedmuller.jl Outdated Show resolved Hide resolved
src/ecc/codes/steanereedmuller.jl Outdated Show resolved Hide resolved
src/ecc/codes/steanereedmuller.jl Outdated Show resolved Hide resolved
@@ -0,0 +1,80 @@
"""The family of non-CSS Steane-Reed-Muller codes, as discovered by Steane in his 1999 paper [steane1999quantum](@cite).

The construction method is effective for k > 0, and generators G = (G_x | G_z) for k < 0 serve as stabilizers (parity check matrices) H = (H_x | H_z) for k > 0 codes.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not understand this sentence. Also, k is not defined.

Also, you have not specified what are the arguments this constructor takes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added the explanation provided by Steane with a bit more description and also explained the parameters as well. I hope that it makes sense now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please let me know does docstring makes sense now? Or I should add more details? I tried to include what was provided in the Steane Paper.

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Apr 18, 2024

There are a lot of code instances that are not tested for correctness. Adding these to the test runner would make it

Thanks for your comments. I will work on the changes. Our limit is from 1 to 7 for both r and t, and most of the code instances, such as entire t =5 given n , t =4 given n have k < 0 so that will give argument error.

I will check all the cases when k > 0 such as (2,5), (2, 6), (2 ,7), (3,7). Also, I will form larger table to see which values give k> 0 when t =6, and t =7

@Krastanov
Copy link
Member

reminder: please do not push after every single small edit. Try to structure your pull request and do pushes only when something substantial is ready.

One way to think about this: do not make pushes until you run the tests locally.

@Fe-r-oz
Copy link
Contributor Author

Fe-r-oz commented Apr 18, 2024

reminder: please do not push after every single small edit. Try to structure your pull request and do pushes only when something substantial is ready.

Thanks for your comments. I have applied the code review suggestions, deleted the steane_convention helper function and adder proper docstring. Locally, there is problem with using `LDPC Decoders "0.3.1" and building the gives error. The problems goes away by reverting back to 0.3.0

ERROR: Unsatisfiable requirements detected for package LDPCDecoders [3c486d74]:
LDPCDecoders [3c486d74] log:
├─possible versions are: 0.1.0-0.3.0 or uninstalled
├─restricted to versions 0.3.1-0.3 by QuantumClifford [0525e862], leaving only versions: uninstalled

@Krastanov
Copy link
Member

The latest version of LDPCDecoders is 0.3.1 https://juliahub.com/ui/Packages/General/LDPCDecoders

Your local environment was probably not updated to know about it. A ] resolve or ] update would probably fix it.

@Fe-r-oz Fe-r-oz marked this pull request as draft April 23, 2024 13:27
@Krastanov Krastanov added the ECC Having to do with the ECC submodule label May 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ECC Having to do with the ECC submodule
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants