When redirecting output to a log file, it would still be nice to see some progress reporting

[mikix]

When you are redirecting to a log file, rich doesn't really do anything useful for us.

Maybe we could detect this case and print the occasional "1000 records de-identified" or "2 batches written" or something.

Another workaround is running the ETL under something like screen or tmux - which will let you still see the pretty bars even when detached from your original terminal.

So this is a nice-to-have for the log file usecase.

[mikix]

I looked into this a bit.

Current state:

• rich can detect if it's writing to a terminal or file Console.is_terminal and will adjust its own behavior. When it's writing to a file, it waits until the rich.Live area is finished, then writes the final state out (so you do end up seeing the completed progress bar in the file). So we are currently functional, just we don't give any progress feedback).
• That behavior by rich is not adjustable. Well, we could say force_terminal=True which would cause it to write progress bars out - which do look good when viewing the log file in a terminal viewer still - but if you look at the log file in a raw editor, you can see all the backspaces and repeated draws of the bar. But we can't adjust the non-terminal mode behavior.
• We could write a rich.Progress wrapper that intercepted update and advance calls and wrote out something like a series of dots or a percentage occasionally. But... rich isn't helping us here much and it would be annoying.
  • We'd have to keep track of multiple stacked progress bars ourselves (like in the task case of overall read progress + batch write progress) and represent their progress differently.
  • The rich API is not set up to help us with this - most of the good internal state is stored in internal variables. We have to intercept multiple API touch points. We have to look up the task object by iterating, then asking for its percentage - we'll have to keep track of the previous percentage ourselves if we want to wait until it ticks up, etc.
  • We'd have to do adjust how we show the header for a section -- it's currently inside the Live area and we'd probably want to pull it out so it gets printed in both cases.

So for now, I'm putting this back down, but keeping track of the above. We can do it, just might be lightly annoying and involve some light refactoring.