Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

5 basic image handling #7

Merged
merged 48 commits into from
Oct 31, 2023
Merged

5 basic image handling #7

merged 48 commits into from
Oct 31, 2023

Conversation

AvocadoMoon
Copy link
Collaborator

Fully working ImageJ plugin to handle N5 files both locally and remotely, with tests included. The tests for s3 can be more extensive but as of right now its serviceable.

AvocadoMoon and others added 30 commits August 3, 2023 07:39
Able to open N5 images with the N5 utils library, and display them with ImgJ I/O library.
Made a basic GUI to the N5 reader that allows for users to open a window manager and select what file they want.
Can now open any N5 file that is given to the script, rather than manually inputting a string into the script.
Added basic test case and an N5 file to use with it. Need to make the test case use multiple N5 files all with different attributes and the resource folder in the project.
Made a resources folder for test objects, and utilize these objects within my test cases.
Made the GUI dynamically update the list of datasets within an N5 file. Refactored the code so that there is a stronger separation of GUI and functional code, this should both allow test cases to no longer be dependent on GUI libraries allowing Github actions to work, and make the code more clean.
Changed the current tests to be more concise and less vague about what they are testing.
Previously N5 files would statically assign the data type for every dataset to Unsigned short, but now it can dynamically change to any type of data set available.
Instead of adding some separate look and feel to the GUI just leave it as default.
Added an okay button to the GUI for users to use and authenticate their decision on a file to choose. Also added a remote files button that will be used to open remote files when that finally works. Changed the library that will be opening images in ImageJ from N5IJ to N5 because N5IJ causes some error not allowing Blosc compressed images to be opened. This will mean that some boiler plate code will have to be written, but at least it won't raise errors. Finally some testing with S3 capabilities have been played with recently. No full implementations have been made, but few attempts where given.
The GUI that appears for setting the credentials and configuration for accessing remote files.
Remove all dependency on the N5-IJ library within the plugin.
Added a GUI that allows for users to enter information regarding S3 credentials and URI.
Added a test case to ensure that the image plus class returned has the appropriate dimensions and channels. Also made the option available to open the image fully in memory or in virtual memory (only important slices of the image are in memory).
Ricky's java artifacts seem to not be working so I'm uploading my jar file to git where he can take it and test if it works on his system.
Made it so that the plugin can now list the datasets in a remote N5 bucket using S3 API. It can also take a chosen dataset and display the resulting image in ImageJ.
Making sure all current changes are within the remote repo so that when I change my desktop to Linux I can pick up where I left off.
Wrote some S3 tests that ensure the remote storage option is working properly. Also redid the local storage tests so that both S3 and local storage do the same form of tests on the resulting images.
It seems to be the S3 proxy dependency that is causing all of the issues in the CI so I changed its scope to only apply to the tests in this project.
Previously the tests would extend the test class making it unproductive to work with. Now decorators are used to specify what each function is.
Before S3 Proxy was a service that was being ran within the Java environment, but now its going to be ran outside of the java environment and instead within the git hub actions VM.
There must be a more elegant way to allow the required files to be copied over into the directory where the volume mount for the docker container is, but this hopefully will just work.
S3 Proxy was still in the POM file which caused the build event to fail.
The errors thrown in the tests currently arent that good so hopefully this will change that.
Checks if the N5 bucket is available.
Made every command line action have sudo such that no permission errors can be raised.
Instead of using the shells command line to copy the resource files use the AWS CLI.
All of it is defaults except for the the first few variables.
Set some fake credentials for the container and hopefully it'll just work and I won't have to setup actual credentials for it.
It seems that the client using the CLI still requires a set credentials, but hopefully now that its none anyone can use it.
@AvocadoMoon AvocadoMoon self-assigned this Oct 2, 2023
- Made its region dynamic
- Fixed no endpoint and credentials branch
- Remove unneeded key in hash map
Setting the environmental variables in actions VM so that more thorough S3 tests can be done.
Make the code slightly more clean, removing unused variables.
Copy link
Member

@jcschaff jcschaff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approved, more detail reviews will follow.

@AvocadoMoon AvocadoMoon merged commit 3386674 into main Oct 31, 2023
1 check passed
@AvocadoMoon AvocadoMoon deleted the 5-basic-image-handling branch October 31, 2023 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants