Add klipper_estimator support

[nelsongraca]

Added the ability to pipe gcode through klipper_estimator.

Related feature request: #481

[nelsongraca]

sorry for the ping @Arksine but is there any chance this will be merged?

[CharlemagneLasse]

During my experimentations, I've noticed that the rpi4 is often not able to process the gcode files in 10 seconds. I've now increased it for me to 120s inside the code. Not sure if it would be better to make it configurable from the outside.

[nelsongraca]

I think it's worth adding as a config, I'll add it.

[Arksine]

Thanks, sorry for the delay in responding, time has been somewhat constrained until recently.

I had started some work on this and recently finished it, which is now available in the analysis module. It takes a different approach as it doesn't apply a post-process to the file. The primary issue with a post-process approach that triggers on a filelist_changed event is that it can potentially modify the file while Klipper is attempting to print.

With the [analysis] module Klipper Estimator will automatically perform an analysis as a post-metadata extraction step when enable_auto_analysis is set to true in the configuration. The estimated_time field in the metadata will be replaced with the total_time returned by Klipper Estimator. The analysis module also adds a few more features:

• It attempts to auto detect the platform. If autodetection fails the user can configure it.
• The initial download places the binary in the <data_folder>/tools/klipper_estimator directory.
• There is an option to register klipper_estimator with the update_manager to handle updating.
• There are API endpoints to perform analysis and dump the config
• Klipper Estimator uses a config file rather than Moonraker's URL for peforming estimates. The user can specify the KE config file, however the default is to use a dumped configuration. This makes Klipper Estimator available even if Klippy is not connected to Moonraker. Moonraker will dump the default config if it doesn't exist and can be configured to dump it every time Klipper reports ready.

[freakydude]

@Arksine I read your changeset and configured it with

[analysis]
enable_auto_analysis : true

The estimated_time field now differs from the estimated time in the slicer. So it looks to me like this is working fine so far.
I used the KlipperEstimator before by adding it as a post-progress step in the slicer. It did not change the metadata but updated the M73 gcode commands within the file.

Does the current solution do this? And if not, is it planned?

The reason I ask is that frontends like KlipperScreen calculate the remaining time with one of your methods -> https://moonraker.readthedocs.io/en/latest/external_api/introduction/?h=progress#basic-print-status. And if they use display_status.progress it will be different and not as accurate as KlipperEstimator.

Thanks for your work
Kind regards

[Arksine]

@freakydude That would require a post process. I think it would be better to perform it in the slicer. At this time I don’t have plans to add support for post-processing in Moonraker as there are potential issues with inotify.

[CharlemagneLasse]

@freakydude That would require a post process. I think it would be better to perform it in the slicer. At this time I don’t have plans to add support for post-processing in Moonraker as there are potential issues with inotify.

But aren't you already doing postprocessing for preprocess_cancellation? After I tried to switch from nelsongraca's working implementation to the official one from you, I noticed various problems. Not sure how they are related:

• klipperscreen time is wrong (maybe because m73 are no longer updated)
• fluidd is no longer showing the slicer time during print
• in fluidd, i no longer have the possibility to press "load current file" when opening the webpage/dashboard when a print was started

Of course, I needed to update moonraker and co to get this builtin analysis code. It can therefore be that some other non-analysis change broke these things

[Arksine]

But aren't you already doing postprocessing for preprocess_cancellation? After I tried to switch from nelsongraca's working implementation to the official one from you, I noticed various problems. Not sure how they are related:

The processing done for preprocess_cancellation occurs during the metadata processing step, before the file is moved to folders watched by inotify. It was also meant as a temporary measure to support a new (at the time) feature in Klipper. Most modern slicers now natively support Klipper objects.

I don't use KlipperScreen, but I'll test to see if I can repro some of your issues in Fluidd. Some of what you report doesn't sound related.

Edit: If the M73 time estimates are critical I'll reconsider post-processing support. I'll have to give it some thought about the best way to do it, most likely it will need to be done in metadata.py to prevent issues which is something I wanted to avoid.

[Arksine]

I did decide to change the behavior of the enable_auto_analysis option. It will now result in a post-process just before metadata is extracted from the file, presuming that the file hasn't been previously processed by Klipper Estimator.

@pedrolamas @meteyou This change includes a new metadata field, file_processors (the specification has been updated to refect this). This field will contain an array of application names that have processed the file. Right now it can include klipper_estimator and/or preprocess_cancellation. The [analysis] section has to be present in the config for metadata.py to identify if a file has been processed by Klipper Estimator, but enable_auto_analysis does not have to be set to true. This makes it possible for Moonraker identify files for users who prefer to use Klipper Estimator manually or in a slicer.

I chose this method in case there are further post-processing apps that should be added in the future. The implementation is generic, the component hosting the app (ie: [analysis]) registers a configuration that is passed to metadata.py each time its invoked. The configuration contains the path to the executable, its arguments, its name, the regex used to ID it, etc.

I also added a an endpoint that allows for a manual post-process.

[freakydude]

@Arksine Sounds great – I'll check this out soon. Thanks for reconsidering.

[pedrolamas]

• fluidd is no longer showing the slicer time during print
• in fluidd, i no longer have the possibility to press "load current file" when opening the webpage/dashboard when a print was started

Unrelated, but for the record, that was probably my fault in a botched Fluidd code refactor I did, and not related with the Moonraker changes here...

It should be fixed on Fluidd v1.32.2 though, so please do ensure you update to that version.

[pedrolamas]

@Arksine I've now implemented in the latest moonraker [analysis] changes in Fluidd and everything seems to be working fine!

[freakydude]

@Arksine Seems like it works

Left -> prusa estimation
Center -> moonraker-postprocess
Right -> KlipperEstimator standalone - used the moonraker config api

I really have no idea where the small difference comes from. Does the Moonraker post-processing solution also parse the klipper gcode macros?

But anyways great work!
Thanks

[CharlemagneLasse]

• fluidd is no longer showing the slicer time during print
• in fluidd, i no longer have the possibility to press "load current file" when opening the webpage/dashboard when a print was started

Unrelated, but for the record, that was probably my fault in a botched Fluidd code refactor I did, and not related with the Moonraker changes here...

It should be fixed on Fluidd v1.32.2 though, so please do ensure you update to that version.

I have v1.32.2 according to the "Software Updates" page. But still, I see the problem. But there are a couple of things which could be related:

• I have my printjobs in subfolders
• print from subfolder is runnig
• "load current file" button is not available
• opening the folder (under jobs) with the current printing job makes following available:
  • "load current file" for the gcode preview
  • "Slicer" time in the "Printing" component

[Arksine]

@freakydude Moonraker's automated post-process just executes klipper_estimator. If Klipper Estimator processes macros then they will be included.

The discrepancy might be from post-processing the same file twice. It might be a simple rounding error. I haven't looked into how Klipper Estimator calculates its estimates, its possible that different runs always produce slightly different results.