Use EDM4HEP for standalone event input and hit/mctruth output

[sethrj]

To integrate better into scalable workflows for future experiments, we should consider using the event data model from EDM4HEP for our standalone workflows. This will let us directly use EDM4HEP tooling for analyzing output rather than our own ROOT formats.

[wdconinc]

A question here is whether this is best done as a standalone feature (i.e. celeritas -> edm4hep) or if it makes more sense to get this 'for free' by running celeritas as an engine within dd4hep (i.e. dd4hep -> celeritas -> dd4hep -> edm4hep).

[sethrj]

Good question, but I'm not 100% sure how DD4HEP works... would we have a driver application that uses DD4HEP to load a GDML file, set up SDs, and send events in/read hits out?

If so, I'm a little hesitant to make this the primary It seems like DD4HEP is very ROOT-centric; are there any plans to have a detector framework that doesn't require ROOT for everything? (For example, Uproot lets you use python to load ROOT files, but it relies on off-the-shelf python.) ROOT is such a heavy and fragile dependency that I'm opposed to requiring it for Celeritas to be usable, for two reasons that stem from an ASCR perspective. First, we need to be able to stand up Celeritas quickly on brand new (or unreleased/under NDA!), quirky HPC systems for acceptance testing and performance checks. Because ROOT has such deep nonstandard hooks into LLVM and IO etc., support for new processors and operating systems is often laggy. We couldn't install it onto Summit for a while because of build errors on the new (at the time) IBM PowerPCs. Second, for GPU development, it's occasionally necessary to work with compiler teams to debug problems, and if it's not easier than git clone; cmake .; make install it's not something they're willing to work on.

Sorry to totally take this issue off topic 😄 especially since it seems like ROOT is core to EDM4HEP/Podio...

[wdconinc]

I don't think the ROOT aspect of DD4hep is something celeritas needs to worry about, since this would be another way of providing the geometry (and the standard gdml way would remain). Geant4 has the same considerations as celeritas: it does not want a dependency on ROOT. So, DD4hep handles the ROOT interfacing. What DD4hep does is read the geometry definition (into ROOT TGeo), and setting up this same geometry in Geant4. Then it hands off control to Geant4 for doing its thing, and the SD action calls back into DD4hep with the hit collections. That's when DD4hep takes over again and writes those hit collections in whatever format is desired (ROOT, lcio, sio).

So, I would think that the support for EDM4hep does not have to reside in celeritas if DD4hep can call celeritas in the same way it currently calls Geant4. Maybe that means the development actually needs to happen on the DD4hep side.

[sethrj]

I think it's valuable enough to be able to run "standalone" celeritas—for HPC-oriented execution where we drive the code— that we should be able to do independent I/O with EDM4HEP. Also dd4hep is a many-tentacled dependency monster, requires dlopen for plugins, etc.

Looking at dd4hep's use of edm4hep, I don't think it'll be that hard. It looks like most of the code is https://github.com/AIDASoft/DD4hep/blob/master/DDDigi/io/DigiEdm4hepOutput.cpp and https://github.com/AIDASoft/DD4hep/blob/master/DDDigi/io/DigiEdm4hepInput.cpp .