BigWarp, get transformations back into BDV #93

martinschorb · 2020-03-03T15:40:05Z

Hi,

is there a way to get the transformations that I create with bigwarp associated to my selected sources and then back into "our" bdv?

NicoKiaru · 2020-03-03T16:42:33Z

Not yet, but that's doable. I'll give it a shot

martinschorb · 2020-03-06T14:20:58Z

I do see a xxxx_xfm when I launch BigWarp appearing in my sources list. Is that it?

martinschorb · 2020-03-06T14:25:33Z

Also, I noticed that BigWarp's "Export as ImgPlus" does produce entirely empty images.
Before it crashed (see saalfeldlab/bigwarp#88) but I got a new snapshot that fixed at least this...

bogovicj · 2020-03-06T14:35:54Z

Also, I noticed that BigWarp's "Export as ImgPlus" does produce entirely empty images.

Can you elaborate? Obviously, it's not supposed to do that. What's your setup like?

bogovicj · 2020-03-06T14:51:15Z

@NicoKiaru
Regarding getting bigwarp transformed images into bdv, can you give this a try

LandmarkTableModel ltm = new LandmarkTableModel( src.getSource(0, 0).numDimensions() );
try {
	ltm.load(landmarks);
} catch (IOException e) {
	e.printStackTrace();
}

WarpedSource<T> srcOut = new WarpedSource<>( src, src.getName() + "_transformed");
srcOut.updateTransform( new ThinplateSplineTransform( ltm.getTransform() ));
srcOut.setIsTransformed( true );

BdvFunctions.show( srcOut );

martinschorb · 2020-03-09T14:06:36Z

Minimum working example:
https://oc.embl.de/index.php/s/fFxYlIqFmULJspK

select fm_ch0 as moving and m_120 as target
display fused image (in fixed image viewer)
export as ImgPlus (Target,Target)

BdvHandlePostprocessor:BdvHandle found.
BdvHandlePostprocessor:BdvHandle found.
Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
	at bigwarp.BigWarpExporter.<init>(BigWarpExporter.java:98)
	at bigwarp.BigWarpRealExporter.<init>(BigWarpRealExporter.java:58)
	at bigwarp.BigWarpRealExporter.<init>(BigWarpRealExporter.java:70)
	at bigwarp.BigWarpExporter.getExporter(BigWarpExporter.java:639)
	at bdv.ij.ApplyBigwarpPlugin.runExport(ApplyBigwarpPlugin.java:740)
	at bigwarp.BigWarp.exportAsImagePlus(BigWarp.java:1071)
	at bigwarp.BigWarp.exportAsImagePlus(BigWarp.java:818)
	at bigwarp.BigWarpActions$ExportImagePlusAction.actionPerformed(BigWarpActions.java:1012)
	at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
	at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
	at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
	at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
	at javax.swing.AbstractButton.doClick(AbstractButton.java:376)
	at javax.swing.plaf.basic.BasicMenuItemUI.doClick(BasicMenuItemUI.java:842)
	at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(BasicMenuItemUI.java:886)
	at java.awt.Component.processMouseEvent(Component.java:6539)
	at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
	at java.awt.Component.processEvent(Component.java:6304)
	at java.awt.Container.processEvent(Container.java:2239)
	at java.awt.Component.dispatchEventImpl(Component.java:4889)
	at java.awt.Container.dispatchEventImpl(Container.java:2297)
	at java.awt.Component.dispatchEvent(Component.java:4711)
	at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904)
	at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535)
	at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476)
	at java.awt.Container.dispatchEventImpl(Container.java:2283)
	at java.awt.Window.dispatchEventImpl(Window.java:2746)
	at java.awt.Component.dispatchEvent(Component.java:4711)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
	at java.awt.EventQueue$4.run(EventQueue.java:733)
	at java.awt.EventQueue$4.run(EventQueue.java:731)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

This happens in both moving and target windows.

Tested on Linux & Windows.

bogovicj · 2020-03-09T14:35:10Z

Thanks - I also get this behavior.
I have an idea what this is about, but can you try something to confirm you get the same thing as me:

Open the images in the same way as before.
Control-click several times in the image (at least 4). (This explicitly creates an identity transformation.)
export as ImgPlus (Target,Target), like before.

You'll hopefully not get an error. A resulting image will appear, but it probably will be empty :(

NicoKiaru · 2020-03-09T14:37:42Z

FYI, if relevant, the pom file has these explicit versions:

<bigdataviewer-core.version>7.0.0</bigdataviewer-core.version>
		<bigdataviewer-vistools.version>1.0.0-beta-15</bigdataviewer-vistools.version>
		<bigwarp_fiji.version>5.0.0</bigwarp_fiji.version>

martinschorb · 2020-03-09T14:43:48Z

1. Open the images in the same way as before.

2. Control-click several times in the image (at least 4).  (This explicitly creates an identity transformation.)

3. export as ImgPlus (Target,Target), like before.
You'll hopefully not get an error. A resulting image will appear, but it probably will be empty :(

Exactly. Empty image.
The error message appears when I try to export without doing anything.

bogovicj · 2020-03-09T14:45:43Z

Perfect, I'll get you a patch as soon as I can.

To clarify - you're using this to export an affine-transformed moving image into target space, right?
Is that correct?

martinschorb · 2020-03-09T14:48:02Z

Both images come affine transformed from @NicoKiaru 's multi-source viewer. I use BigWarp to refine this transformation. This works: the resulting transformed moving image is transformed correctly. I just realized that the export to ImagePlus seems to be broken in this case. When I load the pixel data of the two images into BigWarp the "traditional" way, the export works.

NicoKiaru · 2020-03-09T15:04:58Z

This repo uses a different 'manual transform' than the one from bigdataviewer.
There are 2 ways to store a manual transform (append and mutate), and the way the 'storage' is done depends on the original source class (TransformedSource and AbstractSpimSource supported).

Case 1 : mutate a TransformedSource : it concatenate the transform to the current transform of the source -> you cannot retrieve the original transform. The output source is still the initial TransformedSource object
Case 2 : append a TransformedSource : it wraps the original transformed source into a new transformed source -> you keep the previous transformed source. The output source is a NEW TransformedSource object
Case 3 : mutate an AbstractSpimSource : it concatenate the transform to the last transform taken contained in the model of the AbstractSpimData object -> you cannot retrieve the original transform. The output source is still the initial AbstractSpimSource object
Case 4 : append an AbstractSpimSource : it adds a transform to the last transform taken contained in the model of the AbstractSpimData object. The output source is still the initial AbstractSpimSource object

All these modes are in the SourceAndConverterUtils class :

bigdataviewer-playground/src/main/java/sc/fiji/bdvpg/sourceandconverter/SourceAndConverterUtils.java

Lines 441 to 497 in add256d

    
               /** 
        
                * if a source has a linked spimdata, mutates the last registration to account for changes 
        
                * @param affineTransform3D 
        
                * @param sac 
        
                * @return 
        
                */ 
        
               public static SourceAndConverter mutateLastSpimdataTransformation(AffineTransform3D affineTransform3D, SourceAndConverter sac) { 
        
                   assert SourceAndConverterServices 
        
                           .getSourceAndConverterService() 
        
                           .getSacToMetadata().get(sac).containsKey(SPIM_DATA_INFO); 
        
                   assert SourceAndConverterServices 
        
                           .getSourceAndConverterService() 
        
                           .getSacToMetadata().get(sac).get(SPIM_DATA_INFO) instanceof SourceAndConverterService.SpimDataInfo; 
        
                   SourceAndConverterService.SpimDataInfo sdi = ((SourceAndConverterService.SpimDataInfo) 
        
                           SourceAndConverterServices.getSourceAndConverterService() 
        
                                   .getSacToMetadata().get(sac).get(SPIM_DATA_INFO)); 
        
                   // TODO : find a way to pass the ref of starter into this function ? but static looks great... 
        
                   BdvHandle bdvHandle = SourceAndConverterServices.getSourceAndConverterDisplayService().getActiveBdv(); 
        
                   int timePoint = bdvHandle.getViewerPanel().getState().getCurrentTimepoint(); 
        
                   ViewRegistration vr = sdi.asd.getViewRegistrations().getViewRegistration(timePoint,sdi.setupId); 
        
                   ViewTransform vt = vr.getTransformList().get(vr.getTransformList().size()-1); 
        
                   AffineTransform3D at3D = new AffineTransform3D(); 
        
                   at3D.concatenate(vt.asAffine3D()); 
        
                   at3D.preConcatenate(affineTransform3D); 
        
                   ViewTransform newvt = new ViewTransformAffine(vt.getName(), at3D); 
        
                   vr.getTransformList().remove(vt); 
        
                   vr.getTransformList().add(newvt); 
        
                   vr.updateModel(); 
        
                   try { 
        
                       Method updateBdvSource = Class.forName("bdv.AbstractSpimSource").getDeclaredMethod("loadTimepoint", int.class); 
        
                       updateBdvSource.setAccessible(true); 
        
                       AbstractSpimSource ass = (AbstractSpimSource) sac.getSpimSource(); 
        
                       updateBdvSource.invoke(ass, timePoint); 
        
                       if (sac.asVolatile() != null) { 
        
                           ass = (AbstractSpimSource) sac.asVolatile().getSpimSource(); 
        
                           updateBdvSource.invoke(ass, timePoint); 
        
                       } 
        
                   } catch (ClassCastException e) { 
        
                       e.printStackTrace(); 
        
                   } catch (Exception e) { 
        
                       e.printStackTrace(); 
        
                   } 
        
                   return sac; 
        
               }

bogovicj · 2020-03-13T17:56:19Z

Quick update -
I'm confident the cause of the error is fixed, but pretty sure there is another problem which causes the image to be empty (for the test above).

I'm looking into it, and will keep you posted.

NicoKiaru · 2020-03-13T17:58:02Z

@martinschorb , are sources 2d or 3d ?

martinschorb · 2020-03-14T09:23:57Z

@martinschorb , are sources 2d or 3d ?

these are 2D.
I can try with 3D as well but due to Fiji currently not starting with BDV Playground (#98) in all except one remote installation it is a bit tricky to test... Will keep you posted.

martinschorb · 2020-04-01T09:27:40Z

Hi,

It seems that the new sources that are added somehow are screwed up in terms of ViewSetupID...
Wrong Colour. It also does not remember the Projection mode.
When I have Views fused in the BigWarp Window, also the fixed source appears (with assigned Colour and Contrast settings for one moving...)

martinschorb · 2020-04-01T09:43:13Z

Minimum example:

files:
https://oc.embl.de/index.php/s/KihC2SEFIsCBYB1

empty BDV (mixed proj)
fm->Sum ; m120 -> Avg
Show, change colour fm ->red

Launch BigWarp, fm... moving

landmarks.zip

transform (T)
show fused (F)

now I have:

I have to change Projectionmode of fm_xfm.. to Sum
Show new source
AutoContrast,AdjustView does not matter...

[ERROR] Module threw exception
java.lang.NullPointerException
	at sc.fiji.bdvpg.scijava.services.SourceAndConverterBdvDisplayService.show(SourceAndConverterBdvDisplayService.java:181)
	at sc.fiji.bdvpg.scijava.command.bdv.BdvSourcesAdderCommand.run(BdvSourcesAdderCommand.java:31)
	at org.scijava.command.CommandModule.run(CommandModule.java:199)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

NicoKiaru self-assigned this Mar 3, 2020

NicoKiaru added the bug Something isn't working label Jun 7, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BigWarp, get transformations back into BDV #93

BigWarp, get transformations back into BDV #93

martinschorb commented Mar 3, 2020

NicoKiaru commented Mar 3, 2020

martinschorb commented Mar 6, 2020

martinschorb commented Mar 6, 2020

bogovicj commented Mar 6, 2020

bogovicj commented Mar 6, 2020

martinschorb commented Mar 9, 2020 •

edited

Loading

bogovicj commented Mar 9, 2020

NicoKiaru commented Mar 9, 2020

martinschorb commented Mar 9, 2020

bogovicj commented Mar 9, 2020

martinschorb commented Mar 9, 2020

NicoKiaru commented Mar 9, 2020 •

edited

Loading

bogovicj commented Mar 13, 2020

NicoKiaru commented Mar 13, 2020

martinschorb commented Mar 14, 2020

martinschorb commented Apr 1, 2020

martinschorb commented Apr 1, 2020

BigWarp, get transformations back into BDV #93

BigWarp, get transformations back into BDV #93

Comments

martinschorb commented Mar 3, 2020

NicoKiaru commented Mar 3, 2020

martinschorb commented Mar 6, 2020

martinschorb commented Mar 6, 2020

bogovicj commented Mar 6, 2020

bogovicj commented Mar 6, 2020

martinschorb commented Mar 9, 2020 • edited Loading

bogovicj commented Mar 9, 2020

NicoKiaru commented Mar 9, 2020

martinschorb commented Mar 9, 2020

bogovicj commented Mar 9, 2020

martinschorb commented Mar 9, 2020

NicoKiaru commented Mar 9, 2020 • edited Loading

bogovicj commented Mar 13, 2020

NicoKiaru commented Mar 13, 2020

martinschorb commented Mar 14, 2020

martinschorb commented Apr 1, 2020

martinschorb commented Apr 1, 2020

martinschorb commented Mar 9, 2020 •

edited

Loading

NicoKiaru commented Mar 9, 2020 •

edited

Loading