Creating your own constants file

SuperSegger segments the image into cells using 1) segmentation parameters such as intensity thresholds and cell length and 2 ) a local scoring function for the segments (region boundaries) and 3) a global scoring function for the regions (areas surrounded by segments) to choose the correct cell boundaries.

The scoring functions use parameters for the segments and regions such as intensity, area, length. The weights for the scoring function are obtained by optimizing a neural network (searching for the global minimum of an energy function). To obtain the weights the user trains the algorithm by identifying segments and complete regions that are believed to be true. The software then updates the weight in the cost function to best match the true boundaries in the training data. Before you start a segmentation process you load these weights from files saved in the settings folder, through loadConstants.m. Since different species of bacteria and bacteria growing under different growth conditions (media, temperature) have slightly different shape and growth characteristics, the parameter values sometimes must be calculated for each condition.

Opening the program

When you type trainingGui in the command window you should see something similar to the image below. Using the folder icon on the left you can locate the folder where your images are located.

Selecting the constants and modifying the intensity thresholds :

The first step is to select the closest constants from the already existent constants. Click on modify constants to try all the constants and modify segmentation parameters (intensity thresholds and cell lengths). A new gui will open allowing you to do that.

Use the folder icon on the toolbar to load an image with your cells. We recommend an image with multiple cells in close contact. Click try all constants to decide on the constants that better suit your cells. Choose the constants that look the best from the list.

If you wish to further modify the values you can use the sliders and then click run. The image on the left shows the result after the segment optimization state only (no region optimization). You can see how the background mask and the segments look (red are permanent segments, orange are segments rated as true by the software, blue are segments rated as false by the software). The result you want at the end of this is for the algorithm to be able to find all the segments that would separate the cells. Do not worry if there are some extra segments or their identification of true or false (orange or blue) is incorrect, that will be our next step. Here is what the different parameters do :

• Halo intensity: anything above this value in the minimum contrast filtered phase image is removed from the mask. This value needs to be about the intensity of the halos around the cells.
• Mask thresh 1: intensity threshold used to find the cell clumps in the initial background mask. Anything above this is removed from the image.
• Mask thresh 2: intensity threshold used to remove regions between cells in the initial background mask. Anything above this is removed from the image. (Usually, we set it to 0 because the halo intensity below does that)
• Blur width: width of gaussian blur used on the image before the minimum contrast filter is applied.
• Cell thresh: intensities below this are set to 0. This is to remove differences in the intensity of the interiors of a cell so that fewer boundaries are found inside the cells.
• Filter radius: the size of the minimum contrast filter. Needs to be about the size of the width of the cells.
• Max width: cells wider than this are further refined by iteratively adding more segments. If
• Region Optimizing values : The effect of these values are not shown in the image.
• Min length: cells with a length smaller than this are further optimized.

Save your constants by clicking save constants so you can load them in the other gui.

Providing the algorithm with examples :

To optimize the neural network you need supply the software with examples. This means that you need the *_seg.mat files made during the segmentation process. Therefore you need to start with already segmented data. The basic strategy is to provide the algorithm with enough data such that it has to try to segment as many different conditions as possible but not so much data that you’ll be reasonably unable to police all the segmentation problems. Usually one uses a time-lapse series, crops out a small region of cells, uses a subset of the images of about 30-50 frames with cells in close contact, and segments the subset with the closest constants possible.
The first step is to crop the images. To do that zoom in at the size you would like the image to be cropped and click copy and crop data. Here is a reasonably sized field of view :

Click ‘load constants’ to load the segmentation constants of your choice. Then click 'start segmentation'. You can set the frame skip parameter to every how many frames you would like to segment. A reasonable amount of frames is ~50 frames. If you have much more than that it will be harder to catch all the mistakes.

Delete dust, bubbles and other bad regions

Once segmentation is done the actions panel should change like is shown below. (If it doesn't load your directory or directly the 'xy1/seg' folder in the training gui, and select your constants). You can now go through every frame using the arrows and remove regions that you do not want to train the software on.

To do this you can either click on ‘delete square’ and click on the top right and bottom left to make a square. Everything within is deleted. Or you can click delete region and the region that you clicked on is deleted. If you make a mistake you can click ‘undo’

Training for the segments :

In segment optimization the algorithm needs to be able to distinguish between false and true segments. The point of training the algorithm is so that what the user thinks is a true segment agrees with what the algorithm identifies as a good segment and vice versa. At this step the software should show an image similar to the one below :

Regions (closed areas) are indicated in black and segments (boundaries between regions) are indicated in red, orange, light blue, and dark blue. The segment colors are defined as follows: User and SuperSegger/Computer say on : Red User says on and SuperSegger/Computer say off : Orange User and SuperSegger/Computer say off : Blue User says off and SuperSegger/Computer say on : Light Blue

As you can see, the images are almost always over-segmented during the watershed process, which is why the segment optimization of true and false segments is used. The segment colors give a quick summary of how well the model and user agree on segment choices. Ideally, once the algorithm is sufficiently trained there will only be dark blue and red segments in the image, indicating perfect agreement between user and computer.

In the above example, there are three segments that are in disagreement. By looking at the phase image, we can see that the segments are in fact a poor choice (i.e. it is splitting a cell in half that shouldn’t be). To modify segments, i.e. to change the User decision, click on a segment. It should change to the associated color, e.g. red to orange.

In this case I’ve flipped both disagreeing segments, and everything looks copacetic. You’ll have to do this for every frame in your timelapse, which is why the training data sets should be kept relatively small. Note that this data set was already trained fairly well, so there are very few disagreements. A new set will likely require much more fiddling. Optimize the neural network

Now that you’ve fixed all the segments to your liking, the computer will now try to adjust the weights on the parameters of a neural network to best match your choices of true and false segments. A window should open showing the process. You don’t have to do anything at this point except wait for it to complete.

Training for the regions :

• Train the software to identify true and false region

Regions are defined as areas entirely closed by ON segments. Ultimately, the cell masks will be defined by regions, so we want them to be as good as possible. Optimizing a region goes one step further than segment optimization in that it uses non-local information to make decisions, e.g. turning off a segment on one side of the region may make the overall shape better even though the energy of the segment is positive. Below is a region view of our data set. The color scheme is essentially the same as before with lighter colors indicating disagreements.

Click on ‘all good regions’ to set all the regions on the frame as good regions. You can then click on the ones that are not well shaped so that the software can distinguish good from bad regions.

• Create bad examples of regions

Since there are so fewer regions than segments to train, it is helpful to generate intentionally bad data and see how well the algorithm can recognize it. Click on create bad regions. For each image you have, this will create three extra images with incorrect regions by randomly turning on and off segments as shown below on the right panel.

Notice that many of the regions are now combinations of true regions or sections of the true regions. Again, dark blue represents an agreed OFF region, dark red represents agreed ON region, light blue is disagreed user says OFF region and light red disagreed user says ON region. At this point, you should go through all the images again and mark any region that is incorrect (merged or cut regions) as OFF, and any region that has a correct shape as ON.

• Optimize the region parameters

Next, click on train regions. The computer will now try to adjust the weights of a second neural network on the region parameters to best match your choices of true and false regions. A window similar to before should open showing the process.

Using the new constants

• Saving the new constants

Now that you have shown the computer a thing or two, it’s time to save the new constants file you have created. Click ‘save new constants’ to save the new constants. To load the constants save your constants in the folder superSeggerRelease/settings/. Choose a name that starts with the resolutions of your camera and describes the trained cells eg ‘60XBThai.mat’.

• Loading the new constants

To load the constants CONST = loadConstants([constname]). In the example above we would load them by calling CONST = loadConstants(‘60XBThai’). They should already appear in the list box in the superSeggerGUI when you try to segment your data.

• Changing constants values not included in the GUI

It may be necessary to modify some values in the segmentation parameters that can not be modified through the GUI. To do that follow the instructions in changing specific values in the constants..

Some values that may need to be modified are the following (Default values shown are for 60XEc) :

If cell regions are not identified at all, they may be removed during the debris removal. Here are the relevant values :

• CONST.superSeggerOpti.remove_microcolonies, you can set it to zero if whole micro-colonies are incorrectly removed. It uses the intensity difference between the outside and the outline of the micro-colony (Default 1).
• CONST.superSeggerOpti.remove_debris, you can set it to zero for no debris removal. The PEBBLE_CONST and INTENSITY_DIF will not be used (Default 1).
• CONST.superSeggerOpti.PEBBLE_CONST, you can decrease the 'cell-texture' constant (Default 3).
• CONST.superSeggerOpti.INTENSITY_DIF, you can decrease the intensity difference constant (Default 0.15).

If cell regions are merged or deleted during linking/segmentation:

• CONST.trackOpti.SMALL_AREA_MERGE, merges regions with areas next to them if they are below this area (Default 55 pixels).
• CONST.trackOpti.MIN_AREA_NO_NEIGH, deletes regions that have no neighbors and area smaller than this (for bubbles etc) (Default 30 pixels).
• CONST.trackOpti.MIN_AREA, deletes regions with an area smaller than this (for bubbles etc) (Default 5 pixels).