[ROOT636] Add test for schema evolution of auto_ptr -> unique_ptr #48817
Conversation
|
cms-bot internal usage |
|
-code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48817/45934
Code check has found code style and quality issues which could be resolved by applying following patch(s)
|
makortel
force-pushed
the
schemaEvolutionTestAutoPtr
branch
from
76ade85 to
15f2fea
Compare
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48817/45936
|
|
Pull request #48817 was updated. |
|
@cmsbuild, please test with cms-data/IOPool-Input#4 This test should fail, but I want it on the record. |
|
-1 Failed Tests: UnitTests The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic: You can see more details here: Unit TestsI found 1 errors in the following unit tests: ---> test TestIOPoolInputSchemaEvolution had ERRORS Comparison SummarySummary:
|
|
Already the splitlevel 0 test failed with |
|
test parameters:
|
cmsbuild
added
requires-external
code-checks-rejected
and removed
code-checks-pending
labels
makortel
force-pushed
the
schemaEvolutionTestAutoPtr
branch
from
1f5301d to
317c8db
Compare
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48817/45941
|
|
Pull request #48817 was updated. |
|
@cmsbuild, please test |
|
+1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0c50cc/47910/summary.html The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
You can see more details here: Comparison SummarySummary:
|
|
FYI @jblomer @pcanal the recipe in https://github.com/jblomer/auto_ptr_demo seems to work, at least in the leading order. |
| <version ClassVersion="4" checksum="3437951039"/> | ||
| <version ClassVersion="3" checksum="1545257825"/> | ||
| </class> | ||
| <class name="std::auto_ptr<edmtest::SchemaEvolutionContained>"/> |
There was a problem hiding this comment.
I still had to keep the std::auto_ptr here, although our original desire would be to not have to do that (#43422), and the recipe doesn't seem to need it (or at least it is not present in https://github.com/jblomer/auto_ptr_demo/blob/main/Event_v3_LinkDef.h while it is in Event_v2_LinkDef.h).
If I remove it, the job reading the data product with auto_ptr fails with
----- Begin Fatal Exception 29-Aug-2025 22:42:57 CEST-----------------------
An exception of category 'FatalRootError' occurred while
[0] Calling EventProcessor::runToCompletion (which does almost everything after beginJob and before endJob)
Additional Info:
[a] Fatal Root Error: @SUB=TClass::BuildRealData
Inspection for auto_ptr<edmtest::SchemaEvolutionContained> not supported!
----- End Fatal Exception -------------------------------------------------
with a call chain
#0 0x00007ffff70f12f1 in __cxxabiv1::__cxa_throw (obj=0x7fffc9fdd780, tinfo=0x7ffff7e9f860 <typeinfo for edm::Exception>, dest=0x7ffff7e65450 <edm::Exception::~Exception()>) at ../../../../libstdc++-v3/libsupc++/eh_throw.cc:81
#1 0x00007ffff1426896 in (anonymous namespace)::RootErrorHandlerImpl(int, char const*, char const*) [clone .cold] () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_ROOT636_X_2025-08-20-2300/lib/el8_amd64_gcc12/pluginFWCoreServicesPlugins.so
#2 0x00007ffff786ad34 in ErrorHandler () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_ROOT636_X_2025-08-20-2300/external/el8_amd64_gcc12/lib/libCore.so
#3 0x00007ffff77b6ce4 in TObject::Error(char const*, char const*, ...) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_ROOT636_X_2025-08-20-2300/external/el8_amd64_gcc12/lib/libCore.so
#4 0x00007ffff788c074 in TClass::BuildRealData(void*, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_ROOT636_X_2025-08-20-2300/external/el8_amd64_gcc12/lib/libCore.so
#5 0x00007ffff0a37a10 in TStreamerInfo::BuildOld() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_ROOT636_X_2025-08-20-2300/external/el8_amd64_gcc12/lib/libRIO.so
#6 0x00007ffff78789a8 in TClass::GetStreamerInfoImpl(int, bool) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_ROOT636_X_2025-08-20-2300/external/el8_amd64_gcc12/lib/libCore.so
#7 0x00007ffff7878c8e in TClass::GetStreamerInfo(int, bool) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_ROOT636_X_2025-08-20-2300/external/el8_amd64_gcc12/lib/libCore.so
#8 0x00007ffff0a1f70c in TStreamerInfo::ForceWriteInfo(TFile*, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_ROOT636_X_2025-08-20-2300/external/el8_amd64_gcc12/lib/libRIO.so
#9 0x00007ffff0e703ba in TTreeCloner::CopyStreamerInfos() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_ROOT636_X_2025-08-20-2300/external/el8_amd64_gcc12/lib/libTree.so
#10 0x00007ffff0e70518 in TTreeCloner::Exec() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_ROOT636_X_2025-08-20-2300/external/el8_amd64_gcc12/lib/libTree.so
#11 0x00007fffcc05a55e in edm::RootOutputTree::fastCloneTTree(TTree*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_ROOT636_X_2025-08-20-2300/lib/el8_amd64_gcc12/libIOPoolOutput.so
#12 0x00007fffcc05abab in edm::RootOutputTree::maybeFastCloneTree(bool, bool, TTree*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_ROOT636_X_2025-08-20-2300/lib/el8_amd64_gcc12/libIOPoolOutput.so
#13 0x00007fffcc05b0bc in edm::RootOutputFile::beginInputFile(edm::FileBlock const&, int) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_ROOT636_X_2025-08-20-2300/lib/el8_amd64_gcc12/libIOPoolOutput.so
#14 0x00007ffff7ca0078 in edm::Schedule::openOutputFiles(edm::FileBlock&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_ROOT636_X_2025-08-20-2300/lib/el8_amd64_gcc12/libFWCoreFramework.so
#15 0x00007ffff7be31ec in edm::EventProcessor::openOutputFiles() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_ROOT636_X_2025-08-20-2300/lib/el8_amd64_gcc12/libFWCoreFramework.so
#16 0x00007ffff7be9b0e in edm::EventProcessor::runToCompletion() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_ROOT636_X_2025-08-20-2300/lib/el8_amd64_gcc12/libFWCoreFramework.so
#17 0x0000000000408556 in tbb::detail::d1::task_arena_function<main::{lambda()#1}::operator()() const::{lambda()#1}, void>::operator()() const ()
#18 0x00007ffff727cf71 in tbb::detail::r1::task_arena_impl::execute (ta=..., d=...) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2022.0.0-79b5a917b0c13f831cd534a5b9f53a95/tbb-v2022.0.0/src/tbb/arena.cpp:821
#19 0x000000000040a283 in main::{lambda()#1}::operator()() const ()
#20 0x00000000004051b8 in main ()
There was a problem hiding this comment.
This is odd. It looks like there might still be a class which has an auto_ptr as a data member.
There was a problem hiding this comment.
It looks like there might still be a class which has an
auto_ptras a data member.
What would be scope of "there"? The libraries being loaded into the job? Anywhere in CMSSW? In any .rootmap file in the $LD_LIBRARY_PATH?
There was a problem hiding this comment.
The scope is any library/dictionary that is being loaded by the failing job. Finding out what is the class is question is 'straightforward' by getting the name of the 'this' in the following stack frames:
#4 0x00007ffff788c074 in TClass::BuildRealData(void*, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_ROOT636_X_2025-08-20-2300/external/el8_amd64_gcc12/lib/libCore.so
#5 0x00007ffff0a37a10 in TStreamerInfo::BuildOld() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_ROOT636_X_2025-08-20-2300/external/el8_amd64_gcc12/lib/libRIO.so
#6 0x00007ffff78789a8 in TClass::GetStreamerInfoImpl(int, bool) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_ROOT636_X_2025-08-20-2300/external/el8_amd64_gcc12/lib/libCore.so
#7 0x00007ffff7878c8e in TClass::GetStreamerInfo(int, bool) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_ROOT636_X_2025-08-20-2300/external/el8_amd64_gcc12/lib/libCore.so
At the very least that information will get us closer to understanding what is going on.
There was a problem hiding this comment.
I misused the debug build from cms-sw/root#222 (comment) (hoping ROOT master and root-project/root#18402 would not interfere :) ) and the this->GetName() at the stack frames 4, 6, and 7 is auto_ptr<edmtest::SchemaEvolutionContained>.
Seeing the TTreeCloner in the stack trace I tested also disabling the fast cloning, but that resulted in another failure, that occurs also without this PR. I opened a separate issue for that #48838.
There was a problem hiding this comment.
So given the stack trace, I expect to see GetName being auto_ptr at stack # 7, what is GetName at stack # 16? (and the result of this->ls())
There was a problem hiding this comment.
I expect to see
GetNamebeingauto_ptrat stack # 7
Correct, this->GetName() frame 7 is auto_ptr<edmtest::SchemaEvolutionContained>
what is
GetNameat stack # 16? (and the result ofthis->ls())
(gdb) p this
$3 = (TStreamerInfo * const) 0x7fffc72677e0
(gdb) p this->GetName()
$4 = 0x7fffc9da74e0 "edmtest::VectorVectorMiddle"
(gdb) p this->ls("")
StreamerInfo for class: edmtest::VectorVectorMiddle, version=3, checksum=0x2d410aab
vector<edmtest::VectorVectorElement> middleVector_ offset= 0 type=300 stl=1 ctype=61
i= 0, middleVector_ type=300, offset= 0, len=1, method=0
$5 = void
There was a problem hiding this comment.
Following @pcanal's suggestion I tested what happens if we would not convert the Inspection for auto_ptr< message to an exception in InitRootHandlers. By effectively ignoring the message, the test succeeds.
In parallel I initiated a ROOT 6.36 debug build in cms-sw/cmsdist#10059 to give a reproducer for @pcanal.
There was a problem hiding this comment.
The conditions necessary for the message to appears includes:
- The
<memory>header is loaded intoCling(i.e.std::auto_ptris known byCling) Contentclass is known or can be known (“autoparsing”) byCling- There is no dictionary for
auto_ptr<Content> StreamerInfoforauto_ptr<Content>is built/used
where 1., 2. and 4. are likely to be true whenever reading an old file that contains anauto_ptr.
"3." can be removed by generating the dictionary for the auto_ptr<Content> but this leads to compiler warnings.
There was a problem hiding this comment.
Thanks @pcanal. We had a chat among @pcanal and @Dr15Jones and decided to proceed by removing the auto_ptr dictionary and ignoring the Inspection for auto_ptr< message in CMSSW InitRootHandlers (at the cost of header parsing when a file containing auto_ptr is read)
| // The following is from an example by Jakob Blomer from the ROOT team | ||
| namespace edmtest::compat { | ||
| template <typename T> | ||
| struct deprecated_auto_ptr { |
There was a problem hiding this comment.
@Dr15Jones We should think if we want to have this class defined in some more central place so that it could be used for both in this test and in SimDataFormats/GeneratorProducts, or just copy the code there in different namespace.
There was a problem hiding this comment.
After a quick chat we're leaning towards copying the class to SimDataFormats/GeneratorProducts.
|
Milestone for this pull request has been moved to CMSSW_16_0_X. Please open a backport if it should also go in to CMSSW_15_1_X. |
PR description:
As a stepping stone to solve #43422 and #43923, this PR extends the schema evolution test to cover a case where an
std::auto_ptrclass member is evolved tostd::unique_ptr. With the first commit only and ROOT older than 6.36 the test will fail.The second commit applies the recipe #43923 (comment) to the test.
Needs the test file update from cms-data/IOPool-Input#4
Resolves cms-sw/framework-team#1451
PR validation:
In ROOT6.36 the schema evolution unit test succeeds.