Skip to content

Confusing (wrong? buggy?) behavior of sample ranges with chained trees when using dataframes #22741

Description

@acampove

Check duplicate issues.

  • Checked for duplicates

Description

When using dataframes with a set of files, I see different ways

rdfsampleinfo_.EntryRange();

gets computed. This range is either:

Local: From zero to the number of entries in the file containing the entry at which we are.
Global: From the first entry in the file where we are until the last entry in that file, counted as part of the chain.

I get local ranges when using a dataframe constructor. When using FromSpec to use friend trees AND multithreading, I get global ranges. Without multithreading I again get local ranges.

Reproducer

import ROOT

N_FILES          = 4
ENTRIES_PER_FILE = [1000, 1000, 1000, 1000]  # deliberately uneven sizes
FILES            = [f"repro_file_{i}.root" for i in range(N_FILES)]
TREE             = "Events"

def make_files():
    for path, n in zip(FILES, ENTRIES_PER_FILE):
        df = ROOT.RDataFrame(n).Define("x", "rdfentry_")
        df.Snapshot(TREE, path, ["x"])

def run():
    definition = r'''
    auto range = rdfsampleinfo_.EntryRange();
    auto start = std::get<0>(range);
    auto   end = std::get<1>(range);

    std::cout << start << "  " << end << "\n";

    return 3;
    '''

    df = ROOT.RDataFrame(TREE, FILES)
    df = df.DefinePerSample("val", definition)
    df.Count().GetValue()

if __name__ == "__main__":
    make_files()
    run()
    print('---------------')
    ROOT.EnableImplicitMT(2)
    run()

That produces:

0  1000
0  1000
0  1000
0  1000
---------------
0  1000
0  1000
0  1000
0  1000

i.e. with and without multithreading we get the local range.

However:

import ROOT
from ROOT    import RDF # type: ignore
from pathlib import Path

PATH = Path('files')
PATH.mkdir(exist_ok = True)
# -------------------------------------
def _create_files(index : int) -> None:
    main     = PATH / f'main_{index:03}.root'
    friend   = PATH / f'frnd_{index:03}.root'
    if main.exists() and friend.exists():
        return

    df       = ROOT.RDataFrame(1000)
    df.Define('x', f'{index}').Snapshot('Events', str(main))

    df       = ROOT.RDataFrame(1000)
    df.Define('y', f'{index}').Snapshot('Events', str(friend))
# -------------------------------------
def _load_rdf():
    PATH.mkdir(parents=True, exist_ok=True)
    names = dict()
    names['files/main_001.root'] = 1
    names['files/main_002.root'] = 2
    names['files/main_003.root'] = 3
    names['files/main_004.root'] = 4

    definition = r'''
    auto range = rdfsampleinfo_.EntryRange();
    auto start = std::get<0>(range);
    auto   end = std::get<1>(range);

    std::cout << start << "  " << end << "\n";

    return 3;
    '''

    rdf = RDF.Experimental.FromSpec('spec.json')
    rdf = rdf.DefinePerSample('file_id', definition)
    rdf.Count().GetValue()
# -------------------------------------
for ifile in range(4):
    _create_files(index = ifile)

_load_rdf()
ROOT.EnableImplicitMT(4) # type: ignore
print('-----------------')
_load_rdf()

produces

0  1000
0  1000
0  1000
0  1000
-----------------
0  1000
3000  4000
2000  3000
1000  2000

i.e. multithreading (even with one core) gives the global range. The specification file needed to load the trees is:

spec_maker.py

This makes no sense and must be a bug somewhere.

ROOT version

ROOT Version: 6.38.00
Built for linuxx8664gcc on Feb 19 2026, 03:23:03
From tags/6-38-00@6-38-00

Installation method

mamba

Operating system

almalinux 8

Additional context

Are the global ranges meant to belong to a specific file? i.e. Will the global ranges go across file boundaries? From tests it seems that they are meant to be contained to files. I also see that these boundaries do not agree always with the file but they can be file subsets.

I need this to use:

    rdf = rdf.DefinePerSample('file_id', definition)

Where it is not clear if sample refers to files.

Metadata

Metadata

Assignees

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions