Skip to content

Commit

Permalink
NGFF-Converter 2.0 (#57)
Browse files Browse the repository at this point in the history
* Indev

* Table Views part 1

* Indev

* Add dialogs

* Task configurators

* Use original file names

* Implement logging & other features

* Job config panel skinning

* Make widgets static, add defaults configuration

* WIP add files dialog, expanded Output module, progressbar listeners

* Shared output module widgets, more css, Output tasks now control working dir.

* Refactor i/o calculation sequence, style primary window

* Bug fixes, improve directory selection widgets

* Add directWrite output option, centralise workflow list, improve workflow name handling, add confirmation dialogs

* Skin additional dialogs, resolve bugs

* Styling tweaks

* UI revisions, status bar, settings import/export

* Fix "open directory" button for Windows

* Style fixes

* Skin about dialog

* Handle jobs failing before start event. Add Taskbar/dock progress display

* Stop a running job without killing entire task list

* Copyright statements

* Add update dialog

* Fix taskbar

* Notify popup on completion

* Use native res logo

* Pin table header heights

* Use glencoe version json for update check

* Implement progressbars

* Update about dialog

* Reset via picocli defaults

* Don't delete initial input files during cleanup when steps were skipped

* Disable debug statements

* Exclude CreateNGFF step from ConvertToTiff when input is NGFF

* Execution system revisions, bug fixes

* Button behaviour tweaks

* Code style and css tweaks

* Completed Todo

* Style progress bars

* Unify progress listeners

* Use exit events rather than Platform when closing GUI

* Update b2r and r2o versions

* Declare bf dependency

* Fix global progress bar

* Replace old b2r/r2o help windows with docs links

* Ignore pre-release tags in version checker

* Increment version

* Jpackage doesn't allow -beta/-rc/-snapshot versions

* GUI tweaks for Windows

* Drop unified stage style

* Clarify logging on aborted runs

* Fix off-by-1 error on conversion counter

* Release drag handler before processing dropped files

* Cleanup

* Apply chosen settings before trying to save defaults/export

* Add "Add Folder" menu option for manually selecting zarrs

* Fix drag/drop on Windows

* Improve "Run Jobs" button state handling

* Drop unused import

* Fix "apply to all"

* Fix window hierarchy

* Fix tiff compression settings loader

* Improve applySettings reliability and add warning system

* Only perform UI updates on UI thread

* Validate output write status

* Fix directwrite mode

* Rename output configurator menu option

* Change b2rhelp link

* Update NoValidFiles error message

* Completed todo

* Explicitly clean zarr folders before trying to delete

* Add exception handler

* Tweak progress listener text

* Fix output write access check

* Link 'Set Default' in AddFilesDialog to 'Do not show again'

* Fix invalid setting export

* Bump bioformats/bioformats2raw/raw2ometiff versions, handle new settings

* Revise file copy system to account for split-planes mode

* Don't set default compressionoptions if missing

* Move notification popup on MacOS

* Improve JSON read error handling

* Tweak job name text overrun

* Drop Output module checking of directwrite results

* Add basic usage docs

* Add program help menu item

* Reduce screenshot size

* Undupe license docs

* Ensure correct extensions when setting custom output file names

* Drop unnecessary semicolon

* Update changelog
  • Loading branch information
DavidStirling authored Feb 16, 2024
1 parent 55fee48 commit a75600f
Show file tree
Hide file tree
Showing 56 changed files with 6,176 additions and 1,226 deletions.
42 changes: 34 additions & 8 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,32 @@
# v2.0.0 (2024-02-16)

### Features:
* Redesigned user interface, modernised layout and design.
* NGFF-Converter now directly interfaces with bioformats2raw/raw2ometiff instead of generating a CLI call.
* Modularised conversion into "Jobs" (internally called "Workflows") comprised of a sequence of "Tasks".
* Added virtual "Output" task to control destination files.
* Added full settings configuration panels for all tasks.
* Added progress bars per-task during execution.
* Revised configuration workflow. Settings are now initialised to user-defined defaults. Jobs can be configured individually or in bulk.
* Job execution now uses a proper queue, so jobs can be started/stopped individually.
* Execution logs are now stored and delivered per-job.
* Added configuration options for working directory and log directory (if logging to file).


### Bugfixes/Maintenance:
* Updated component package versions
* bioformats 7.1.0
* bioformats2raw 0.9.1
* raw2ometiff 0.7.0
* Updated documentation

### Known issues:
* The underlying bioformats package is currently unable to read some formats on ARM-based MacOS systems.
* raw2ometiff execution cannot be reliably interrupted until it finishes the initial scan of a .zarr file.

# v1.1.5 (2023-06-26)

## Bugfixes/Maintenance:
### Bugfixes/Maintenance:
* Update component package versions (#41)
* bioformats 6.13.0
* bioformats2raw 0.7.0
Expand All @@ -9,32 +35,32 @@

# v1.1.4 (2023-01-10)

## Bugfixes/Maintenance:
### Bugfixes/Maintenance:
* Remove year from Copyright in About window (#37)

# v1.1.3 (2023-01-09)

## Bugfixes/Maintenance:
### Bugfixes/Maintenance:
* Temporarily drop support for M1 Macs, breaks x86_64

# v1.1.2 (2023-01-07)

## Bugfixes/Maintenance:
### Bugfixes/Maintenance:
* Fix app image for code signing
* Add limited support for M1 Macs

# v1.1.1 (2023-01-05)

## Bugfixes/Maintenance:
### Bugfixes/Maintenance:
* Switch to app image to aid code signing workflow (#36).

# v1.1.0 (2022-12-15)

## Features:
### Features:
* NGFF-Converter now generates OME-NGFF files according to [version 0.4](https://ngff.openmicroscopy.org/0.4/) of the format specification.
* See the [bioformats2raw changelog](https://github.com/glencoesoftware/bioformats2raw/releases/tag/v0.6.0) for details of new features.

## Bugfixes/Maintenance:
### Bugfixes/Maintenance:
* bioformats2raw, raw2ometiff, bioformats and logback versions have been updated (#27, #30).
* MIRAX .mrxs files can now be converted (#31).
* Fixed an issue which could prevent running conversions from stopping upon exit (#33).
Expand All @@ -43,5 +69,5 @@

# v1.0.3 (2022-09-16)

## Features:
### Features:
* Initial public release
42 changes: 42 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,47 @@
# NGFF-Converter

NGFF-Converter is an open source tool for converting bioimage formats to the [OME-TIFF](https://ome-model.readthedocs.io/en/stable/ome-tiff/) and [OME-NGFF](https://ngff.openmicroscopy.org/) specifications.

Formats [supported by the bio-formats library](https://bio-formats.readthedocs.io/en/stable/supported-formats.html) should be able to be converted by this tool.
Internally NGFF-Converter uses the [bioformats2raw](https://github.com/glencoesoftware/bioformats2raw) and [raw2ometiff](https://github.com/glencoesoftware/raw2ometiff) packages for conversion.
With default settings, the resulting files should automatically have resolution pyramids generated if needed.

## Usage

Running the tool will display a desktop GUI:

![Alt text](./docs/images/launch.png "Initial window")

Files can be dragged and dropped onto this window to add them as conversion jobs. By default you'll see a window requesting
some basic settings when adding jobs.

![Alt text](./docs/images/addfiles.png?raw=true "Add files window")

When conversion jobs are added, you'll see a different interface with two lists. Each conversion **"Job"** is divided into
multiple **"Tasks"** within each job. For example, converting .png to .ome.tiff requires two steps: The PNG is converted to
NGFF, then the resulting NGFF file is converted to TIFF. A third "Output" job offers settings for where to save the final result.

![Alt text](./docs/images/interface.png?raw=true "Main window")

Clicking on the 'Configure' (cog) icon allows you to edit the settings for a specific job. Each task within a job has it's own configuration options.

For example, configuring the "Output" task will allow you to change the destination directory or filename.

![Alt text](./docs/images/configure.png?raw=true "Job configuration window")

From the main window, jobs can be started using the "Run" buttons. Only a single job will execute at a given time, any others
will be queued. Job progress will be displayed next to the individual tasks. If all goes well, the job's status will change to
"completed".

![Alt text](./docs/images/completed.png?raw=true "Finished job")

If anything goes wrong a "Failed" status will display. Each job has it's own individual execution logs which may provide
detail on what prevented the conversion. Hovering over the status label will also attempt to display any error messages.
Please note that it is possible to configure settings of individual tasks in ways which are incompatible with later workflow
steps. The "help" icon next to each setting provides warnings about particularly disruptive settings.

If you need further help, please feel free to [raise an issue](https://github.com/glencoesoftware/NGFF-Converter/issues).

## License
NGFF-converter is distributed under the terms of the GPL license. Please see LICENSE.txt for further details.

Expand Down
20 changes: 12 additions & 8 deletions build.gradle
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
import java.text.SimpleDateFormat

plugins {
id 'application'
id 'org.openjfx.javafxplugin' version '0.0.10'
id 'org.beryx.runtime' version '1.12.5'
}

version = '1.1.5'
String bfversion = "6.13.0"
String b2rversion = "0.7.0"
String r2oversion = "0.5.0"
version = '2.0.0'
String bfversion = "7.1.0"
String b2rversion = "0.9.1"
String r2oversion = "0.7.0"

mainClassName = 'com.glencoesoftware.convert.Launcher'
applicationName = 'NGFF-Converter'
Expand All @@ -34,7 +36,7 @@ sourceCompatibility = 16
targetCompatibility = 16

javafx {
version = '17.0.2'
version = '17.0.9'
modules = [ 'javafx.controls', 'javafx.fxml', 'javafx.graphics' ]
}

Expand All @@ -49,12 +51,14 @@ dependencies {
implementation group: 'ch.qos.logback', name: 'logback-classic', version: '1.4.4'
implementation group: 'ch.qos.logback', name: 'logback-core', version: '1.4.4'
implementation group: 'ch.qos.logback', name: 'logback-access', version: '1.4.4'
implementation group: 'org.controlsfx', name: 'controlsfx', version: '11.1.2'
implementation group: 'com.google.guava', name: 'guava', version: '32.1.3-jre'
runtimeOnly "org.openjfx:javafx-graphics:$javafx.version:win"
runtimeOnly "org.openjfx:javafx-graphics:$javafx.version:linux"
runtimeOnly "org.openjfx:javafx-graphics:$javafx.version:mac"
}

configurations.all {
configurations.configureEach {
exclude group: 'edu.ucar', 'module': 'cdm'
}

Expand All @@ -64,7 +68,7 @@ jar {
"Created-By": "Gradle ${gradle.gradleVersion}",
"Build-Jdk": "${System.properties['java.version']} (${System.properties['java.vendor']} ${System.properties['java.vm.version']})",
"Built-By": System.properties['user.name'],
"Built-On": new java.text.SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSZ").format(new Date()),
"Built-On": new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSZ").format(new Date()),
"Implementation-Build": "git rev-parse --verify HEAD".execute().getText().trim(),
"Implementation-Title": 'NGFF-Converter',
"Implementation-Version": archiveVersion,
Expand All @@ -77,7 +81,7 @@ jar {
}

runtime {
options = ['--strip-debug', '--compress', '2', '--no-header-files', '--no-man-pages']
addOptions('--strip-debug', '--compress', '2', '--no-header-files', '--no-man-pages')

def currentOs = org.gradle.internal.os.OperatingSystem.current()

Expand Down
Binary file added docs/images/addfiles.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/completed.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/configure.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/interface.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/launch.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
77 changes: 66 additions & 11 deletions src/main/java/com/glencoesoftware/convert/App.java
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
/**
* Copyright (c) 2022 Glencoe Software, Inc. All rights reserved.
*
* Copyright (c) 2023 Glencoe Software, Inc. All rights reserved.
* This software is distributed under the terms described by the LICENSE.txt
* file you can find at the root of the distribution bundle. If the file is
* missing please request a copy by contacting [email protected]
Expand All @@ -12,39 +11,86 @@
import javafx.fxml.FXMLLoader;
import javafx.scene.Parent;
import javafx.scene.Scene;
import javafx.scene.control.Alert;
import javafx.scene.control.ButtonType;
import javafx.scene.image.Image;
import javafx.stage.Stage;

import java.io.IOException;
import java.util.Objects;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.ThreadPoolExecutor;

/**
* JavaFX App
*/
public class App extends Application {

private static Scene scene;
private static PrimaryController controller;
public static Scene scene;

public static PrimaryController controller;

public static String version;

public static Image appIcon = new Image(Objects.requireNonNull(App.class.getResourceAsStream("main-icon.png")));

static {
version = App.class.getPackage().getImplementationVersion();
if (version == null) { version = "DEV"; }
}

@Override
public void start(Stage stage) throws IOException {
FXMLLoader fxmlLoader = new FXMLLoader(App.class.getResource("primary.fxml"));
Parent primary = fxmlLoader.load();
scene = new Scene(primary, 800, 550);
scene = new Scene(primary, 1024, 600);
controller = fxmlLoader.getController();

Thread.setDefaultUncaughtExceptionHandler((Thread t, Throwable e) -> {
if (Platform.isFxApplicationThread()) controller.showExceptionDialog(e);
else System.err.println("An unexpected error occurred in " + t);
});

stage.setScene(scene);
stage.setTitle("NGFF-Converter");
Image icon = new Image(Objects.requireNonNull(getClass().getResourceAsStream("main-icon.png")));
stage.getIcons().add(icon);
stage.setOnCloseRequest(event -> Platform.exit());
stage.setTitle("NGFF-Converter - %s".formatted(version));
stage.getIcons().add(appIcon);
stage.setOnCloseRequest(event -> {
if (controller.jobsRunning()) {
// Confirm exit if tasks are running
Alert choice = new Alert(Alert.AlertType.CONFIRMATION);
choice.initOwner(getScene().getWindow());
choice.setTitle("Close NGFF-Converter");
choice.setHeaderText("Conversions are still running!");
choice.setContentText(
"File conversions are still running, are you sure you want to quit?"
);
choice.getDialogPane().getStylesheets().add(
Objects.requireNonNull(App.class.getResource("Alert.css")).toExternalForm());
choice.showAndWait().ifPresent(response -> {
if (response == ButtonType.CANCEL) {
event.consume();
} else {
stop();
Platform.exit();
}
});
} else {
stop();
Platform.exit();
}
}
);
stage.show();
}

@Override
public void stop() throws InterruptedException {
if (controller.isRunning) {
public void stop() {
if (controller.jobsRunning()) {
controller.runCancel();
}
executor.shutdownNow();
}

public static Scene getScene() {
Expand All @@ -55,4 +101,13 @@ public static void main(String[] args) {
launch();
}

public static final ExecutorService executor = Executors.newFixedThreadPool(1, r -> {
Thread t = new Thread(r, "Converter");
t.setDaemon(true);
return t ;
});

private static final BlockingQueue<Runnable> queue = ((ThreadPoolExecutor) executor).getQueue();

public static int queueSize() { return queue.size(); }
}
Loading

0 comments on commit a75600f

Please sign in to comment.