Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Show total resources of multi process applications #301

Open
Nudin opened this issue Nov 3, 2020 · 4 comments · May be fixed by #1338 or #303
Open

Show total resources of multi process applications #301

Nudin opened this issue Nov 3, 2020 · 4 comments · May be fixed by #1338 or #303

Comments

@Nudin
Copy link
Contributor

Nudin commented Nov 3, 2020

Computersystems nowadays have more and more CPU-Cores, due due to this more and more applications use multiprocessing to make better use of the hardware resources. This is increasingly getting an issue with htop (and similar tools).
On my desktop I currently have 35 processes by chromium, 11 by Firefox and 8 by my RSS reader. In university it is common to have machines with 128 CPU cores that are shared by several users who all run computations that may or may not use multiprocessing.
In both these cases it isn't possible anymore to use htop to figure out which applications are using most CPU/memory/….
What would be desirable is to see sum of recourse usage of all the processes that belong to one "application". This is of course not trivial since from the systems point of view all those processes are independent.

How others approach this issue:

  • atop allows you to display the resources joined by user or processes name
  • KDE/gnome is currently working on using cgroups with systemd ([[https://lwn.net/Articles/834329/|article on lwn]])

The first has the limitation that it cannot distinguish two instances of a multiprocessing application running in parallel but joins them to one application.
The second has the problem that 1. it's specific to Linux, 2. it needs systemd or an equivalent daemons to put the applications in cgroups and 3. it only works if the processes have been configured and tagged by the DE.

My proposal would be to use a better heuristic:

  • Two Processes P1 and P2 are joined if:
    • P2 is the child of P1 and
    • P1 and P2 have the same executable and
    • P1 and P2 have the same user and
    • P1 and P2 have the same session

Therefore this process tree:

  • 1 init
    • 2 foo
    • 3 bar
      • 4 bar
      • 5 bar
        • 6 bar
    • 7 baz
    • 8 bar
      • 9 bar

Would be merged to:

  • 1 init
    • 2 foo
    • … bar
    • 7 baz
    • … bar

When enabled the rows off an 'application' that was created by merging 2 or more Processes should show:

  • An indicator that it is not a process but an application instead of the pid in the PID column
  • in columns like USER, PRI, NI, S:
    • if the values of all processes are the same, display it
    • otherwise display an indicator signalling that there are multiple values
  • For fields, that show resource usage, show the combined resource usage
    • For some columns this is the sum of the values (example: cpu usage)
    • For others this might be harder to compute (example: shared memory)

This feature should be an opt-in (default off) in the options.

I know this is a big feature, hard to implement and the heuristic is kinda ugly, but as outlined this problem is already an issue and with the ongoing rise of numbers of cpu cores and containerisation it's likely getting bigger and bigger.

cgzones added a commit to cgzones/htop that referenced this issue Nov 4, 2020
cgzones added a commit to cgzones/htop that referenced this issue Nov 4, 2020
cgzones added a commit to cgzones/htop that referenced this issue Nov 4, 2020
cgzones added a commit to cgzones/htop that referenced this issue Nov 4, 2020
@cgzones cgzones linked a pull request Nov 4, 2020 that will close this issue
cgzones added a commit to cgzones/htop that referenced this issue Nov 4, 2020
cgzones added a commit to cgzones/htop that referenced this issue Nov 15, 2020
cgzones added a commit to cgzones/htop that referenced this issue Nov 15, 2020
cgzones added a commit to cgzones/htop that referenced this issue Nov 16, 2020
cgzones added a commit to cgzones/htop that referenced this issue Nov 16, 2020
cgzones added a commit to cgzones/htop that referenced this issue Nov 17, 2020
cgzones added a commit to cgzones/htop that referenced this issue Nov 23, 2020
cgzones added a commit to cgzones/htop that referenced this issue Dec 10, 2020
cgzones added a commit to cgzones/htop that referenced this issue Dec 10, 2020
cgzones added a commit to cgzones/htop that referenced this issue Jan 21, 2021
cgzones added a commit to cgzones/htop that referenced this issue Mar 19, 2021
cgzones added a commit to cgzones/htop that referenced this issue Mar 19, 2021
cgzones added a commit to cgzones/htop that referenced this issue Dec 15, 2021
@bernd-wechner
Copy link

Dang it, such an awesome feature and so needed as the OP suggests, and 2 years on the PR is in draft mode and no action. Shrug.

@fasterit
Copy link
Member

@bernd-wechner: Rebase the PR onto current main, test it, make it work on non-Linux. You can provide the action.

@bernd-wechner
Copy link

bernd-wechner commented Nov 24, 2022

I might even have considered just that (not that I care one jot about non-Linux myself as I wasn't even aware htop ran on other platforms successfully) but for one little surprise. That I just discovered that the Mint System Monitor - while GUI based which is suboptimal for server use - does just this, it aggregates resource measures when in tree mode and hence delivers the almighty useful information what Firefox, Chrome, Thunderbird and Eclipse are using in tuto (the big hogs on my system) and hence my immediate itch is scratched and along with that plummets any chance I'll look more deeply into this in favour of getting on with the bundle of other to-dos and projects on my plate ;-). Alas, such is the fate of FOSS and why we're all here, with a PR waiting on action (in short, I lay no blame anywhere on that, just shrug - as I know the FOSS deal and story and how thinly we have all become stretched with the explosion this past decade in empowerment that FOSS has delivered and hence opportunities to fix stuff that itches - and now and then we do).

Alas glances shelved the whole Tree view for lack of a maintainer (I like glances a lot, and htop). At least htop still has Tree mode.

@porg
Copy link

porg commented Mar 21, 2024

Annotated screenshot which illustrates the user need

  • @Nudin feel free to paste that into your original post on top.

Process plus sub-processes system utilization

User Experience Design Proposal

  • Proposal: Offer an option which offers that display mode
  • The following modes should be possible:

Default mode for hierarchical view

  • Top row shows the accumulated stats for the parent process and all its spawned child processes
  • Parent stats row shown with special symbol, e.g. (..)
    • Open for suggestions, ideally using established/familiarized conventions.
  • Parent stats row equally among the children sorted by whatever is the current sort order.
PROCESS      CPU ▾   RAM

Process Y    17.0%    44
Parent X     14.5%   470
├─ Child D    7.0%    30
├─ Child A    4.5%   230
├─ (..)       2.0%    50
├─ Child C    1.0%   140
└─ Child B    0.0%    80
Process Z     0.0%    33

Parent stats row on top

  • (..) is always the first row that states the utilization of the parent process itself.
  • Followed by all other child process which follow the sorting convention.
  • Note: The sort indicator slightly changes (from ▾ to ▼).
PROCESS      CPU ▼    RAM

Process Y    17.0%    44
Parent X     14.5%   470
├─ (..)       2.0%    50
├─ Child D    7.0%    30
├─ Child A    4.5%   230
├─ Child C    1.0%   140
└─ Child B    0.0%    80
Process Z     0.0%    33

Parent stats row hidden

  • The utilization of the parent process is not shown.
  • But it is included in the total.
  • Also a valid use case.
PROCESS      CPU ▼    RAM

Process Y    17.0%    44
Parent X     14.5%   470
├─ Child D    7.0%    30
├─ Child A    4.5%   230
├─ Child C    1.0%   140
└─ Child B    0.0%    80
Process Z     0.0%    33

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants