Guidance on how to compose and lazy load queries in TCA features

[cameronmcefee]

This is somewhat related to #47 but a bit more nuanced.

Core questions

I'm trying to understand

• How do I manage and compose a list of stores that are associated with the results of query?
• How do I lazy load queries in child features?

Explanation

Let's say I'm building a feed reader. I want to create a list of all posts, but I don't want to query all post content all at once. Instead, I want to create a list view that queries all the post ids and then have an item view that handles querying the post itself and holds some other view specific state. There are a couple of reasons I want to do this:

• I've got thousands of records. I don't want to query data I'm probably not going to need. In my actual app, the way things work, without any pagination support, I'm basically just fetching the entire database.
• The content of the records can change. I do want to get updates when they change.

The issue I have is that, as I understand it, the primary guidance for TCA is that a parent should hold its children's stores. However with my current understanding of how to use SQLite Data, that when I create a store for each of the children, it will immediately execute a query even though the view is not visible.

Things I've tried

Regarding how to coordinate the query vs the child state, in my actual code, not my contrived example below, I currently have an .onChange handler that watches the query and if it changes, it checks to see if any ids were added or removed and then updates the list of child state objects. This feels sketchy but I'm not sure what to do otherwise.

For lazy loading, through some testing I figured out that I can wait to .load a @Fetch until .onAppear which gets me the desired behavior (it doesn't fetch until the view appears), but then as far as I know, that query is still actively observing changes even if it's view is not visible. If I scroll away from an item it's still quietly updating. I figured I'd just "unload" the query in a .onDisappear action but I can't figure out a way to do that.

Another thing I tried was keeping the child query outside of the TCA state so it'd be tied to the view lifecycle. This does technically work but then feels like I'm doing something wrong keeping part of my state somewhere else.

I'll admit, I'm still new to this so it's entirely likely I'm thinking about the whole problem wrong and the question I'm asking is not actually the one I need to ask. Can you point me in the right direction? I haven't come across any authoritative info on this topic so far.

Example

For discussion, here's an example of the version of this that eagerly performs all fetches immediately. If you load this preview, you'll see 1000 queries because each child store immediately executes. What I want to see is that there are only a handful of queries for the items that are in the view.

// MARK: - Schema

@Table
public struct Item: Identifiable, Hashable, Sendable {
  public typealias ID = Tagged<Self, UUID>

  public let id: ID
  public let url: String

  public init(id: ID, url: String) {
    self.id = id
    self.url = url
  }
}

extension Item.Draft: Equatable, Sendable {}

// MARK: - Database

public func exampleDatabase() throws -> any DatabaseWriter {
  var configuration = Configuration()
  configuration.foreignKeysEnabled = true

  let database = try DatabaseQueue(configuration: configuration)

  var migrator = DatabaseMigrator()
  migrator.registerMigration("Create items table") { db in
    try #sql(
      """
      CREATE TABLE "Items" (
        "id" TEXT PRIMARY KEY NOT NULL,
        "url" TEXT NOT NULL
      )
      """
    )
    .execute(db)
  }

  try migrator.migrate(database)

  try database.write { db in
    var items: [Item] = []
    for i in 1 ... 1000 {
      items.append(Item(id: Item.ID(UUID()), url: "feed-\(i).html"))
    }

    try db.seed {
      for item in items {
        item
      }
    }
  }

  return database
}

// MARK: - Child Feature

@Reducer
public struct ItemFeature: Sendable {
  public init() {}

  @ObservableState
  public struct State: Equatable, Identifiable {
    public let itemID: Item.ID
    @FetchOne public var item: Item?

    public var id: Item.ID { itemID }

    public init(itemID: Item.ID) {
      self.itemID = itemID
      _item = FetchOne(Item.find(itemID))
    }
  }

  @CasePathable
  public enum Action: Equatable {
    case onAppear
  }

  public var body: some ReducerOf<Self> {
    Reduce { _, action in
      switch action {
      case .onAppear:
        .none
      }
    }
  }
}

// MARK: - Child View

public struct ItemView: View {
  @Bindable var store: StoreOf<ItemFeature>

  public init(store: StoreOf<ItemFeature>) {
    self.store = store
  }

  public var body: some View {
    VStack {
      if let item = store.item {
        Text(item.url)
          .padding()
          .background(Color.blue.opacity(0.2))
          .cornerRadius(8)
      } else {
        Text("Loading...")
          .padding()
      }
    }
    .onAppear {
      store.send(.onAppear)
    }
  }
}

// MARK: - Parent Feature

@Reducer
public struct ListFeature: Sendable {
  public init() {}

  @ObservableState
  public struct State: Equatable {
    @FetchAll public var itemIDs: [Item.ID]
    public var itemStores: IdentifiedArrayOf<ItemFeature.State>

    public init() {
      _itemIDs = FetchAll(Item.all.select(\.id))
      itemStores = []
    }
  }

  @CasePathable
  public enum Action: Equatable {
    case onAppear
    case item(IdentifiedAction<Item.ID, ItemFeature.Action>)
  }

  public var body: some ReducerOf<Self> {
    Reduce { state, action in
      switch action {
      case .onAppear:
        var itemStates: [ItemFeature.State] = []

        for itemID in state.itemIDs {
          itemStates.append(ItemFeature.State(itemID: itemID))
        }

        state.itemStores = IdentifiedArrayOf(uniqueElements: itemStates)

        return .none

      case .item:
        return .none
      }
    }
    .forEach(\.itemStores, action: \.item) {
      ItemFeature()
    }
  }
}

// MARK: - Parent View

public struct ListView: View {
  @Bindable var store: StoreOf<ListFeature>

  public init(store: StoreOf<ListFeature>) {
    self.store = store
  }

  public var body: some View {
    ScrollView(.horizontal) {
      LazyHStack(spacing: 20) {
        ForEach(store.scope(state: \.itemStores, action: \.item)) { itemStore in
          ItemView(store: itemStore)
        }
      }
      .padding()
    }
    .onAppear {
      store.send(.onAppear)
    }
  }
}

// MARK: - Previews

#Preview("ListView") {
  try! prepareDependencies {
    $0.defaultDatabase = try exampleDatabase()
  }

  return ListView(
    store: Store(initialState: ListFeature.State()) {
      ListFeature()
    }
  )
}

[rcarver]

Having modeled things this way, I recommend against it. It seems more efficient, but creating the child features, and then each of them holding an observation to the database becomes expensive.

Instead, bring the parent and child features together.

1. Parent fetches all data. Use a @Selection to narrow to just the fields you need
2. ForEach those rows directly for the view.
3. The parent receives all actions and takes the ID of the row, for example case buttonTapped(Row.ID)

Then, if you truly need pagination here's what I do:

1. Fetch just the id for all records
2. Observe the view for which ids are on screen. I do this in an NSCollectionView which provides that data
3. Fetch full data for that set of ids.
4. Stitch together the id and full data at the view layer. So each row view would have optional full data

[mbrandonw]

I agree with everything @rcarver said above.

This is not the correct way to model things:

@ObservableState
public struct State: Equatable, Identifiable {
  @FetchOne public var item: Item?
  …
}

This executes a SQL query and establishes an observation of the database for every single row of your list. This just isn't going to ever be efficient, and even SwiftData would have the same problem if you used @Query in every single row of a list.

And really, I don't think you even need a TCA feature for each row of your list. The vast majority of apps have quite static content for rows of lists, and it's only when you drill down / present a sheet that you encounter the behavior that would benefit from a full blown TCA feature. And so really the parent feature should just have a @FetchAll and use that data to populate a list of basic SwiftUI views.

[rcarver]

Hard to say without more detail, but it sounds like you need to may need to make a tradeoff of "wide vs deep" if you have a large number of items, each with a large amount of data. For example, to fetch all ids, along with a small set of required data then fetch the rest of the data just for the rows on screen.

Another thing to keep in mind is that your rows will render lazily in many situations (SwiftUI List, LazyVGrid, UICollectionView, etc). Since you're not doing anything with all of the rows other than passing them to the view it might work better than you expect just to load all data. Previously, you had to iterate over the entire dataset and map to child features on every change.

[cameronmcefee]

Thanks, both of you. I think a lot of this is my naive assumptions and feards about performance vs real world experience. I appreciate y'all taking the time to talk it over 🙏

[cameronmcefee]

Actually, I do want to follow up with some more specifics because I got stuck as soon as I started trying to apply this 😬.

My scenario is kind of analogous to the Reminders app, if you were to want to display each reminders list as a disclosure group, so you'd possibly see all reminders, grouped by their associated list.

More concretely:

For the sake of explanation, my app is kind of like instagram. You've got a vertical list of posts and a horizontal list of images in each post. (It does this based on RSS hence my "feed" comments before).

So then I have a structure like:

Gallery([
  Post([
    Image
  ])
])

You should not be performing any kind of filtering or sorting in Swift code.

I'm bringing this up because of that comment. My understanding is that SQL grouping isn't the same as chunking so I can't have SQLite actually group the images together by post. If we've established that lots of little queries aren't desirable, then I think that I'm supposed to fetch all the images for the whole view and then chunk them by post. But then I'm doing filtering in Swift code. What should I be doing?

[rcarver]

Look into jsonGroupArray https://swiftpackageindex.com/pointfreeco/swift-structured-queries/main/documentation/structuredqueriessqlitecore/querycookbook#Pre-loading-associations-with-JSON

[cameronmcefee]

Ah wonderful! I'll dive into that.