THttpServer/TRootSniffer: exe.json leaks the object returned by the executed method (e.g. TH2::ProjectionY)

[wiso]

Description

TRootSnifferFull::ProduceExe (the exe.json handler) executes a registered object's method through the interpreter and serializes the returned value, but it never deletes the object the method returns. For a method that returns a freshly allocated, owned object — e.g. TH2::ProjectionX/ProjectionY, which returns a new TH1D — every exe.json request leaks that result.

It is masked when callers reuse the same target name (ProjectionY resets the existing same-named histogram), but with distinct names (or any method that returns a new owned object on each call) the leak is unbounded.

Reproducer

Short, self-contained (THttpServer + a raw-socket HTTP GET, no extra deps):

// ROOT exe.json memory leak: TRootSnifferFull::ProduceExe never frees the
// object returned by the executed method (here TH2::ProjectionY -> a new TH1D).
// Each exe.json call with a fresh target name leaks the result (~3 allocations).
//
// Build: g++ repro.cc $(root-config --cflags --libs) -lRHTTP -lNet -pthread \
//            -fsanitize=address -g -o repro
// Run:   ./repro 200     # LeakSanitizer reports ~3*N leaked allocations
#include "TH2D.h"
#include "THttpServer.h"
#include "TSystem.h"
#include <arpa/inet.h>
#include <netinet/in.h>
#include <sys/socket.h>
#include <unistd.h>
#include <atomic>
#include <string>
#include <thread>

static void http_get(int port, const std::string& path) {
  int fd = socket(AF_INET, SOCK_STREAM, 0);
  sockaddr_in a{}; a.sin_family = AF_INET; a.sin_port = htons(port);
  inet_pton(AF_INET, "127.0.0.1", &a.sin_addr);
  if (connect(fd, (sockaddr*)&a, sizeof(a)) == 0) {
    std::string r = "GET " + path + " HTTP/1.0\r\n\r\n";
    (void)!write(fd, r.data(), r.size());
    char b[4096]; while (read(fd, b, sizeof b) > 0) {}
  }
  close(fd);
}

int main(int argc, char** argv) {
  const int N = argc > 1 ? atoi(argv[1]) : 200;
  const int port = 9410;
  auto* h = new TH2D("h2", "h2", 128, 0, 128, 1024, 0, 1024);
  for (int i = 0; i < 5000; ++i) h->Fill(i % 128, (i * 7) % 1024);

  THttpServer serv(Form("http:%d", port));
  serv.SetReadOnly(kFALSE);                 // allow exe.json method calls
  serv.Register("/", h);

  std::atomic<bool> stop{false};
  std::thread pump([&]{ while (!stop) { serv.ProcessRequests(); gSystem->Sleep(5); } });
  gSystem->Sleep(300);

  for (int k = 0; k < N; ++k)               // distinct target names -> unbounded leak
    http_get(port, Form("/h2/exe.json?method=ProjectionY&name=_s%d&firstxbin=1&lastxbin=1", k));

  stop = true; pump.join();
  return 0;                                  // AddressSanitizer reports the leak at exit
}

Build & run:

g++ repro.cc $(root-config --cflags --libs) -lRHTTP -lNet -pthread -fsanitize=address -g -o repro
./repro 200

Result (ROOT 6.38.04, Linux x86_64, gcc 15, AddressSanitizer)

Distinct target names _s0 .. _sN → 3·N leaked allocations (one TH1D + its two internal arrays per call), i.e. it scales linearly with the number of requests:

requests	leaked
./repro 20	60 allocations, ~186 KB
./repro 200	600 allocations, ~1.86 MB

With a fixed name (name=_s for every call) it is instead bounded at 3 allocations, because ProjectionY finds and resets the existing same-named histogram.

Leak stack (ASan):

operator new
TStorage::ObjectAlloc
TH2::DoProjection
TClingCallFunc::exec / TMethodCall::Execute       (interpreted method call)
TRootSnifferFull::ProduceExe
THttpServer::ProcessRequest
THttpServer::ProcessRequests

Expected

ProduceExe should manage the lifetime of the object returned by the executed method (e.g. delete it after serialization when it owns it and does not keep it reachable in a directory), so repeated exe.json method calls do not leak.

Setup

• ROOT 6.38.04 (system package), Linux x86_64, gcc 15, AddressSanitizer/LeakSanitizer.

[linev]

In the documentation of methods execution via THttpServer:

https://github.com/root-project/jsroot/blob/master/docs/HttpServer.md#methods-execution

There is following remark:

One also used exe.bin method - in this case results of method execution will be returned in binary format.
In case when method returns temporary object, which should be delete at the end of command execution,
one should specify destroy_result parameter in the URL string:

[shell] wget 'http://localhost:8080/Objects/subfolder/obj/exe.json?method=Clone&_destroy_result_' -O clone.json

Therefore, when TH2::ProjectionY method is invoked, one need to add _destroy_result_ to URL.

[wiso]

Thank you very much. This solves my problem. Are there any cases when the user doesn't want this behaviour as the default?

[linev]

Are there any cases when the user doesn't want this behaviour as the default?

It really depends how invoked method is implemented.
Returned pointer can be owned by object itself.
Simple example - TList::First().

I tried to clarify a bit _destroy_result_ option - that it is seen better in documentation.

[wiso]

Thanks, and thanks for clarifying it in the docs.

A small follow-up in case it's useful, from soak-testing this on ROOT 6.38.04 (a THttpServer("http:...") with a registered TH2I, a pump thread calling ProcessRequests(), hammered with exe.json?method=ProjectionY&firstxbin=N&lastxbin=N):

case	RSS growth per call
varying result name, no destroy_result	~112 kB
varying result name, with destroy_result	~78 kB
fixed/reused result name, no destroy_result	~75 kB
fixed/reused result name, with destroy_result	~75 kB

So destroy_result does exactly what you said — with a varying name, it frees ~33 kB/call, which is exactly the returned 4096-bin TH1D. 👍

But there's a residual ~75 kB per call that grows linearly regardless of destroy_result, and is present even when I reuse a fixed result name (so the projection histogram is reset, not re-allocated, and there is nothing for destroy_result to free). It looks like per-call growth in the interpreted method-execution path (TMethodCall::Execute / cling) rather than the returned object.

Is this per-call interpreter growth expected, and is there a way to avoid it for high-frequency method calls (e.g. a precompiled/registered command instead of an interpreted exe.json method)? For my use case, I'm working around it by limiting how often the projection requests are issued, but I wanted to flag it in case it's a separate, fixable leak.

  // th2_projection_exe_soak.cpp
  //
  // Minimal reproducer: repeated TH2::ProjectionY calls through THttpServer's
  // exe.json grow RSS ~linearly, even with `_destroy_result_` and a fixed
  // (reused) result name. `_destroy_result_` only frees the *returned* histogram
  // (visible as a ~32 kB/call drop with a VARYING result name); a per-call
  // residual remains regardless, apparently from the interpreted method-execution
  // path (TMethodCall::Execute / cling).
  //
  // Build:
  //   g++ -O2 th2_projection_exe_soak.cpp -o soak \
  //       $(root-config --cflags --libs) -lRHTTP -lNet -pthread
  //
  // Run (needs `curl` on PATH; the server runs in-process on :8092):
  //   ./soak plain         // fixed reused result name, no _destroy_result_
  //   ./soak destroy       // fixed reused result name, with _destroy_result_
  //   ./soak plainvary     // unique result name per call, no _destroy_result_
  //   ./soak destroyvary   // unique result name per call, with _destroy_result_
  //
  // Observed on ROOT 6.38.04 (Linux), per call:
  //   plain        ~75 kB   destroy      ~75 kB   (fixed name: no difference)
  //   plainvary   ~112 kB   destroyvary  ~78 kB   (_destroy_result_ frees ~32 kB)
  // i.e. a ~75 kB/call residual that _destroy_result_ does not remove.

  
  #include <THttpServer.h>
  #include <TH2I.h>
  #include <TString.h>
  
  #include <atomic>
  #include <chrono>
  #include <cstdio>
  #include <cstdlib>
  #include <string>
  #include <thread>

  static long rss_kb() {
    FILE *f = fopen("/proc/self/status", "r");
    char line[256];
    long kb = -1;
    while (fgets(line, sizeof(line), f))
      if (sscanf(line, "VmRSS: %ld kB", &kb) == 1) break;
    fclose(f);
    return kb;
  } 

 int main(int argc, char **argv) {
    setvbuf(stdout, nullptr, _IONBF, 0);
    std::string mode = argc > 1 ? argv[1] : "plain";
    int N = argc > 2 ? atoi(argv[2]) : 2000;
    bool destroy = mode.find("destroy") != std::string::npos;
    bool vary = mode.find("vary") != std::string::npos;
    
    THttpServer server("http:8092?thrds=3");
    server.SetReadOnly(kFALSE);
    TH2I h2("h2", "h2", 100, 0, 100, 4096, 0, 4096);
    for (int x = 0; x < 100; ++x)
      for (int k = 0; k < 50; ++k) h2.Fill(x, (x * 37 + k * 13) % 4096);
    server.Register("/", &h2);
      
    // THttpServer queues requests; ProcessRequests() executes them (single-thread
    // ROOT access). Pump it from a side thread so main can drive the HTTP client.
    std::atomic<bool> stop{false};
    std::thread pump([&] {
      while (!stop.load()) {
        server.ProcessRequests();
        std::this_thread::sleep_for(std::chrono::milliseconds(2));
      } 
    }); 
    std::this_thread::sleep_for(std::chrono::milliseconds(300));

   printf("# mode=%s (destroy=%d vary=%d) N=%d\n", mode.c_str(), destroy, vary, N);
    const char *suffix = destroy ? "&_destroy_result_" : "";
    long rss0 = rss_kb();
    for (int i = 0; i < N; ++i) {
      int bin = 1 + (i % 100);
      std::string name = vary ? TString::Format("_proj_%d", i).Data() : "_proj_fixed";
      std::string cmd = TString::Format(
          "curl -s -o /dev/null 'http://127.0.0.1:8092/h2/exe.json?method=ProjectionY"
          "&name=%s&firstxbin=%d&lastxbin=%d%s'",
          name.c_str(), bin, bin, suffix).Data();
      if (system(cmd.c_str())) { /* ignore curl rc */ }
      if (i % 250 == 0) 
        printf("  i=%5d  VmRSS=%8ld kB  (+%ld)\n", i, rss_kb(), rss_kb() - rss0);
    } 
    printf("# final VmRSS=%ld kB  (delta=%ld kB over %d calls)\n",
           rss_kb(), rss_kb() - rss0, N);
    
    stop.store(true);
    pump.join();
    return 0;
  } 

[ferdymercury]

Maybe related: #10454