Pack on virtual fields #1322

acreyes · 2025-09-18T06:26:39Z

PR Summary

This PR allows for type-based packs to pack "virtual" fields that don't exist in memory, but can be evaluated from their own set of dependent variables in the pack by calling a provided evaluate function.

This seems related to #1228, but doesn't provide that capability for outputs as mentioned for ADIOS2.

Also adds SubPacks to provide stencil (0, 1, 2, & 3D) views into a pack centered on a meshblock and ijk index. This can simplify the need to pass around extra indices in the evaluate functions for virtual fields.

PR Checklist

Yurlungur

This is very cool and I like it a lot. It definitely needs a bit of documentation though. Though I suppose we also don't have documentation for type-based variables, which is sorely needed...

Yurlungur · 2025-09-20T21:23:09Z

src/pack/pack_utils.hpp

+template <typename T, int... tuv>
+struct TUV_t<T, tuv...> : impl::TUV_t<T, TopologicalElement::CC, tuv...> {};
+
+template <typename T, TopologicalElement te, int... tuv>
+struct TUV_t<T, te, tuv...> : impl::TUV_t<T, te, tuv...> {};


with the variadics is this going to get correctly resolved?

I think so, since the TE & ints are different types. Happy to add some static asserts like this to the unit tests to demonstrate. Unless there is a different edge case you're thinking of?

Yurlungur · 2025-09-20T21:23:58Z

src/pack/pack_utils.hpp

+// Concept to state that a typed-field is dependent on other
+// fields, and is not itself an actual indexable field.


should the concepts stuff be in concepts_lite or should it be here?

It seemed specialized enough to the pack types that I put it here, but if the convention is to have all concepts in concepts_lite I'm happy to move it

Yurlungur · 2025-09-20T21:28:20Z

src/utils/type_list.hpp

+template <typename, typename, typename...>
+struct UnionTypeLists {};
+
+template <typename... Ts>
+struct UnionTypeLists<TypeList<Ts...>, TypeList<>> {
+  using type = TypeList<Ts...>;
+};
+
+template <typename... Ts, typename V, typename... Vs>
+struct UnionTypeLists<TypeList<Ts...>, TypeList<V, Vs...>> {
+  using TL = TypeList<Ts...>;
+  using type = std::conditional_t<
+      TL::template IsIn<V>(), TL,
+      typename UnionTypeLists<TypeList<Ts..., V>, TypeList<Vs...>>::type>;
+};
+
+template <typename T, typename U, typename V, typename... Args>
+struct UnionTypeLists<T, U, V, Args...> {
+  using type = typename UnionTypeLists<typename UnionTypeLists<T, U>::type, V>::type;
+};


I'm surprised there isn't functionality for this in the standard

Yurlungur · 2025-09-20T21:30:52Z

src/pack/pack_utils.hpp

+// Get a TypeList of all the non-dependent variables that make up the requested types.
+template <typename... Ts>
+using all_dependent_variables_t = decltype(impl::AllDependentVariables::get(Ts()...));


The name and the comment disagree. Is this all dependent variables or all independent variables?

The types should be renamed. It is all the independent variables pulled out from the dependent variables

Yurlungur · 2025-09-20T21:37:56Z

Also changelog and mark non-wip :)

acreyes · 2025-09-20T22:53:37Z

This is very cool and I like it a lot. It definitely needs a bit of documentation though. Though I suppose we also don't have documentation for type-based variables, which is sorely needed...

Happy to add both in this PR.

lroberts36 · 2025-09-22T19:16:35Z

This is very cool! I am wondering though about how complex the template stuff needs to be. Why can't we just do something like

class my_derived_var_t : public variable_names::dependent_var_base_t<...> { 
  ...
  template <class pack_t>
  KOKKOS_INLINE_FUNCTION
  Real evaluate(const pack_t &pack, int b, TopologicalElemet te, int k, int j, int i) {
    return pack(b, var1_t(), te, k, j, i) + pack(b, var2_t(), te, k, j, i);
  }
};

and

template <class TIn, REQUIRES(IncludesType<TIn, Ts...>::value && std::is_base_of_v<dependent_var_base_t, TIn>)>
  KOKKOS_INLINE_FUNCTION Real operator()(const int b, const TE el, const TIn &t,
                                          const int k, const int j, const int i) const {
    return t.evaluate(*this, b, te, k, j, i);
  }

I think there are obviously some circular dependencies that could crop up here (e.g. var2_t is a dependent variable that depends on my_derived_var_t), but assuming the downstream user is responsible this allows things like stencil operations etc. (Caveat: I could be missing some obvious reason this doesn't work since I didn't code it up.)

acreyes · 2025-09-22T19:30:59Z

This is very cool! I am wondering though about how complex the template stuff needs to be. Why can't we just do something like
class my_derived_var_t : public variable_names::dependent_var_base_t<...> { 
  ...
  template <class pack_t>
  KOKKOS_INLINE_FUNCTION
  Real evaluate(const pack_t &pack, int b, TopologicalElemet te, int k, int j, int i) {
    return pack(b, var1_t(), te, k, j, i) + pack(b, var2_t(), te, k, j, i);
  }
};
and
template <class TIn, REQUIRES(IncludesType<TIn, Ts...>::value && std::is_base_of_v<dependent_var_base_t, TIn>)>
  KOKKOS_INLINE_FUNCTION Real operator()(const int b, const TE el, const TIn &t,
                                          const int k, const int j, const int i) const {
    return t.evaluate(*this, b, te, k, j, i);
  }
I think there are obviously some circular dependencies that could crop up here (e.g. var2_t is a dependent variable that depends on my_derived_var_t), but assuming the downstream user is responsible this allows things like stencil operations etc. (Caveat: I could be missing some obvious reason this doesn't work since I didn't code it up.)

That is much simpler and I think would work just fine. My thought was that it would be nice to make it easy to unit test the evaluate methods downstream separate from the packing infrastructure. I think it would be simple enough to mock something that works with the template in your example. This could probably thread later with the sub-pack (#1265 ) infrastructure.

pgrete · 2025-10-06T18:58:05Z

I just got the chance to look at this PR.
It's definitely a nice feature and the user facing interface looks clean and easy to use/write.

Quite of few of our targeted use cases would involve either stencil based ops and/or other information (say an adiabatic index).
Thus the structure @lroberts36 suggested seems more flexible in that regard.

How big of a lift do you think this would be (asking from the point of view whether we should get this interface in as is [pending review comments] or spend "just a little" more time on getting a more flexible version in first place).

acreyes · 2025-10-06T19:58:38Z

I just got the chance to look at this PR. It's definitely a nice feature and the user facing interface looks clean and easy to use/write.

Quite of few of our targeted use cases would involve either stencil based ops and/or other information (say an adiabatic index). Thus the structure @lroberts36 suggested seems more flexible in that regard.

How big of a lift do you think this would be (asking from the point of view whether we should get this interface in as is [pending review comments] or spend "just a little" more time on getting a more flexible version in first place).

I think luke's suggestion would be pretty easy to implement and would be a better starting place for this kind of feature.

…pe-vars

acreyes added the enhancement New feature or request label Sep 18, 2025

Yurlungur reviewed Sep 20, 2025

View reviewed changes

acreyes added 20 commits October 23, 2025 21:27

sketch it out

8aae9ed

working

8fbfddc

use a union for putting together dependent vars

4da3098

derived -> virtual

db4c866

tuv indexing

c146a44

simplify virtual fields to use pack and kji indices

1daa5bc

bring in the subpacks

8f59278

gradient example

98d6640

2d subpacks

4c08d6a

3d subpack

889af25

woops

c0c0a5b

kji->ijk

6e8a034

add tests for subpacks

f4f0e71

clean up independent vs dependent

183dde1

some docs

6b2c3a5

cleanup warnings

a997ab4

check for packed variables

ebdda2f

make evaluates more general

e0508a7

test TE overload

713ae91

Merge remote-tracking branch 'origin/develop' into acreyes/virtual-ty…

b071c5a

…pe-vars

acreyes force-pushed the acreyes/virtual-type-vars branch from 2786e1e to b071c5a Compare December 10, 2025 16:28

acreyes changed the title ~~[WIP] Pack on virtual fields~~ Pack on virtual fields Dec 10, 2025

		// Concept to state that a typed-field is dependent on other
		// fields, and is not itself an actual indexable field.

Pack on virtual fields #1322

Are you sure you want to change the base?

Pack on virtual fields #1322

Uh oh!

Conversation

acreyes commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

PR Checklist

Uh oh!

Yurlungur left a comment

Choose a reason for hiding this comment

Uh oh!

Yurlungur Sep 20, 2025

Choose a reason for hiding this comment

Uh oh!

acreyes Sep 20, 2025

Choose a reason for hiding this comment

Uh oh!

Yurlungur Sep 20, 2025

Choose a reason for hiding this comment

Uh oh!

acreyes Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

Yurlungur Sep 20, 2025

Choose a reason for hiding this comment

Uh oh!

Yurlungur Sep 20, 2025

Choose a reason for hiding this comment

Uh oh!

acreyes Sep 20, 2025

Choose a reason for hiding this comment

Uh oh!

Yurlungur commented Sep 20, 2025

Uh oh!

acreyes commented Sep 20, 2025

Uh oh!

lroberts36 commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

acreyes commented Sep 22, 2025

Uh oh!

pgrete commented Oct 6, 2025

Uh oh!

acreyes commented Oct 6, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

acreyes commented Sep 18, 2025 •

edited

Loading

lroberts36 commented Sep 22, 2025 •

edited

Loading