Investigate splitting factor operational part from 'constant' part of factor

[Affie]

Why?

• Simplify serialization, pack/unpack factor only depends on the factor and does not need the full factor graph
• Simplify PackedFunctionNodeData{T}, GenericFunctionNodeData{T} where T <: Union{<:AbstractPackedFactor, <:AbstractFactor, <:FactorOperationalMemory}

Questions to be answered
If it can be split so that factor solver data only contains the working memory serialization becomes easy as solver data is just ignored and rehydrated when unpacked for use by the solver and there can possibly be no PackedFactor.

• What fields are always constant?
  • DF: Almost everything in FactorNodeData is constant and can move to FactorFDG -- but, need to checking how CCW is built upon FactorCompute
• Will it impact more complicated factor types?
  No, this should not touch the current usrfun field functionallity.
• Should we build factor data (w.r.t. the compute part FactorOperationalMemory/CCW) lazily from DFG and only "inflate" when used in IIF. That is, move reconstFactorData out of DFG?
  • JT I believe so, that way we don't need a fg to unpack a factor and don't need to calculate ccw if never used (eg plotting).

'constant' in this case means not changed by the solver

In other words:
Should we move non FactorOperationalMemory to DFGFactor:
Maybe: fnc, multihypo, nullhypo, inflation ?
that way we split solverData <: FactorOperationalMemory and constants

[Affie]

Note on CCW: currently, only hyporecipe.certainhypo and usrfnc! are pulled out of CCW to be persisted, the rest are thrown away.
Also code TODO: improve _createCCW for hypotheses and certainhypo field recovery when deserializing reconstitute from stored data

[Affie]

Factor Field Comparison

Field	FactorDFG	FactorCompute
id	Union{UUID, Nothing}	Union{UUID, Nothing}
label	Symbol	Symbol
tags	Vector{Symbol}	Set{Symbol}
_variableOrderSymbols	Vector{Symbol}	NTuple{N, Symbol}
timestamp	ZonedDateTime	ZonedDateTime
nstime	String	Nanosecond
solvable	Int	Base.RefValue{Int}
metadata	String	smallData::Dict{Symbol, SmallDataTypes}
solverData	data::String	Base.RefValue{GenericFunctionNodeData{T}}
fnctype	String	Not present
_version	String	Not present

[Affie]

@kwdef struct CommonConvWrapper{T <: AbstractFactor, VT <: Tuple, TP <: Base.RefValue{<:Tuple}, CT,  AM <: AbstractManifold, HR <: HypoRecipeCompute, MT, G} <: FactorOperationalMemory
  usrfnc::T
  fullvariables::VT
  varValsAll::TP
  dummyCache::CT = nothing
  manifold::AM = getManifold(usrfnc!)
  partialDims::Vector{<:Integer} = collect(1:manifold_dimension(manifold))
  partial::Bool = false
  nullhypo::Float64 = 0.0
  inflation::Float64 = SolverParams().inflation
  hyporecipe::HR = HypoRecipeCompute(;activehypo=collect(1:length(varValsAll)))
  measurement::Vector{MT} = Vector(Vector{Float64}())
  varidx::Base.RefValue{Int} = Ref(1)
  particleidx::Base.RefValue{Int} = Ref(1)
  res::Vector{Float64} = zeros(manifold_dimension(manifold))
  _gradients::G = nothing
end

@kwdef mutable struct GenericFunctionNodeData{T <: Union{<:AbstractPackedFactor, <:AbstractFactor, <:FactorOperationalMemory},}
    eliminated::Bool = false
    potentialused::Bool = false
    edgeIDs::Vector{Int} = Int[]
    fnc::T
    multihypo::Vector{Float64} = Float64[]
    certainhypo::Vector{Int} = Int[]
    nullhypo::Float64 = 0.0
    solveInProgress::Int = 0
    inflation::Float64 = 0.0
end

Overlap: fnc (almost), nullhypo and inflation

[Affie]

It looks like we can look at main differences as:

Field	FactorDFG	FactorCompute
observationFnc	Serialized JSONObject	T <: ObservationFunction
computeMemory	Not Serialized	C <: FactorOperationalMemory*

• *I think FactorOperationalMemory should be lazily initialized

[Affie]

Also rename to _State as part of this refactor. #1121

Does Observation live next to or under State.

FactorCompute

• state
• observation
• computeMemory

or
FactorCompute

• state
  • observation
• computeMemory

Note: some state and observation fields may be copied to computeMemory as one CCW object.

[Affie]

I went through the code and the only thing that might not be ideal is we may need something like computeMem::Base.RefValue{FactorOperationalMemory}

Advantages:

• unblocks defFactorType macro #1077
• unblocks IIFTypes and RoMETypes subprojects
• We can do lazy inflation of OperationalMemory (CCW)
• Simplify serialization, pack/unpack factor only depends on the factor and does not need the full factor graph
• Simplify PackedFunctionNodeData{T}, GenericFunctionNodeData{T} where T <: Union{<:AbstractPackedFactor, <:AbstractFactor, <:FactorOperationalMemory}

The disadvantage is we still have dynamic dispatch or we rebuild the entire FactorCompute when ccw is needed.

• There will still be dynamic dispatch on ccw, FactorOperationalMemory is an abstract type.
• We can maybe just rebuild the entire factor once the type is known. In that case we can have Nothing if it is not yet inflated. It will then be similar to the current state with the type still not know to the compiler.