Microtime Image Analysis (MIA)

The MIA window can be opened from the PAM main window by clicking on Advanced Analysis -> MIA or from the command window by typing in Mia.

MIA is used for a variety of different image analysis methods, mainly via image correlation techniques. An introduction and description of the various techniques can be found in:

Image Correlation Spectroscopy (ICS): [Petersen1993]

Image Cross-Correlation Spectroscopy (ICCS): [Wiseman2000]

Raster Image Correlation Spectroscopy (RICS): [Digman2005] and [Digman2009a]

Spatio-Temporal Image Correlation Spectroscopy (STICS): [Toplak2012]

Image Mean-Square Displacement (iMSD): [DiRienzo2013]

Number & Brigthness (N&B): [Digman2008] and [Digman2009b]

Combination of Pulsed Interleaved Excitation and Fluctuation Imaging (PIE-FI): [Hendrix2013] and [Hendrix2015]

Arbitrary Regions for Image Correlation Spectroscopy (ARICS): [Hendrix2016]

_images/main_gui1.png

MIA main gui

Data loading and handling

Currently, MIA supports four types of input data.

In the first case MIA accesses the files currently loaded in PAM to generate an image series. This can be done by selecting Load -> … data from PAM. Hereby it uses the PIE-channels defined in the Channels tab.

The second and more common case is to load TIFF stacks via Load -> … single color TIFFs. In this case, the program opens a window to select the TIFF files to be loaded (multiple files will be concatenated). The file selection window will be opened a second time to select the files for the second channel. Press cancel if only one color is needed.

If you have analog data, you can click the y axis label in the count rate plot to display the axis in counts instead of absolute photon counts (kHz). Alternatively, you can convert analog to photon counting images.

Alternatively, you can load RLICS and RSICS data with the Load -> … weighted TIFFs option. This works the same as standard TIFF data, but extracts re-scaling parameters to account for the non-integer values saved with an integer file format. RLICS and RSICS TIFFs contain the unfiltered data in the first half and the filtered frames in the second half. This option loads only the filtered frames (the raw data can be loaded with the other option).

The last type is to load custom data formats via Load -> … custom data format. This case depends on the particular format selected. To change the custom read-in to the desired format, go to the Options tab. and change the Custom Filetype dropdown menu.

All data is stored in the global variable MIAData and can be easily accessed from the MATLAB command line.

Options tab

_images/MIA_options.png

Options tab in Mia

Miscellaneous options are grouped here.

Change orientation of channel 2:
These options can be used to flip or rotate an image in case the cameras were rotated in respect to each other.
Custom filetype:
Here, the user can defined specialized read-in routines (temporary solution!)
S & Offset:
Here, the parameters from [Dalal2008] can be entered to convert analog to photon counting images.

Converting analog to photon counting data

For determining absolute brightnesses (kHz) with e.g. RICS and N&B using imaging data recorded in analog mode, and for determining absolute number concentrations with N&B using analog data, the images need to be converted: (Image-offset)/S, with the parameters defined in [Dalal2008]. Here is a brief protocol on how to record the parameters.

  • Put a drop of MQ on a coverslip, or MQ in the wells you use for measuring your cells.
  • You measure 20 frames with your normal RICS settings 256x256, pixel dwell time 8-20 µs, pixel size ~50nm, BUT instead of using your laser (“unclick” to use the laser), you are creative with light that has different intensity.
  • Use roomlight / no light / cellphone light / a bike lamp / … shining in various directions towards your sample. Try to obtain a nice spread in lamp intensity.
  • For focussing, (if needed, put something in the well next to it) turn on the laser, 10x higher than usual, put the LUT to see dim signals, and you should find the position where the laser is reflecting the glass, go ~10 µm upwards.
  • For analysis, open the file in Mia and simply copy paste the following code to the Matlab command:
global MIAData

disp(['there are ' num2str(size(find(MIAData.Data{1}>255),1)/numel(MIAData.Data{1})*100) ' % saturated pixels']);

int = mean(mean(mean(double(MIAData.Data{1}),3)))

var = mean(mean(var(double(MIAData.Data{1}),0,3)))
  • make a variance vs. intensity plot
_images/VarInt.png

Variance vs. Intensity plot

  • Fit a straight line to the data. Slope = S, intercept = offset. Enter those values into Mia. As the data is immediately converted, all downstream calculations are now done on photon counting data.
  • Importantly, RICS and N&B analysis should now provide exactly the same number and brightness.
_images/NBvsRICS.png

Comparison of RICS and N&B using analog => photon counting converted data (Olympus Fluoview FV1000)

Main Image Tab

The central controls and settings of MIA are found in the Image tab. The two leftmost plots show the mean frame intensity and a photon counting histogram. The green lines correspond to the first and the red to the second channel. The dotted lines represent the data for the full stack, while the solid line represents the selected pixels in the region of interest (ROI) after correction. A mouse click on the y-label toggles different displaying options.

The four images in the center show the full frame (left) and the selected and corrected ROI (right) for the first (upper images) and second (lower images) channels.

Hint

All images in MIA can be exported as TIFFs by clicking on them with the middle mouse button of by shift-clicking. The TIFFs will be saved as 8bit RGBs, with the same contrast as used for displaying.

All settings for displaying, correcting and analyzing the images can be found in the various tabs at the right of the window, and will be individually discussed in the next sections.

_images/image_tab.png

The image tab

To export the rightmost image as a TIF file, right click it.

Channels

_images/channels_tab.png

The channels tab

This tab contains mainly setting for displaying the images and switching between frames. From top to bottom these are:

Image Colormap:
This changes the colormap for the channels between different options to standard MATLAB colormaps (Gray, Jet, Hot and HSV). The HiLo colormap is the same as the Gray colormap with the exception that all pixels below and above the manual scaling are displayed in blue and red, respectively. The Custom colormap uses a single color (standard green and red) for displaying. The color can be changed with the middle mouse button.
PIE Channel to load:
This function selects the PIE channel that will be used when data is loaded from the PAM main window. For the second channel Nothing can be selected for one channel analysis.
Show in second plot:
This option switches between different options to display in the right images. Original and Dynamic show the ROI before and after the corrections are applied, respectively. Static show the corrections that were applies, i.e. the difference between Original and Dynamic.
Scale Image:
This switches between different scaling options. Autoscale Frame scales the image to the min/max of the displayed image, while Autoscale Total scales to the min/max of the full stack. Manual Scale allows the users to set their own fixed scaling limits.
Link channels:
This checkbox (un)links the two channels, so that the frames can be changed individually or together.
Show Frame:
These edit boxes and sliders are used to switch between different frames of the stack. When frame “0” is selected, the program displays the mean of the stack. With the Use checkbox the user can unselect individual frames for some further analysis.

Image

This tab contains the most important information about the image stacks, like frame, line and pixel time and the pixel size. These values are used for certain calculations and data processing. If the TIFFs were created with PAM, these values will be updated automatically.

ROI

_images/ROI_tab.png

The ROI tab

This tab contains options for selecting ROIs and discarding pixels vie arbitrary region ICS. From Top to bottom:

ROI size and position:
These edit boxes define the size and position of the selected ROI for both channels. Alternatively, the user can click on the left images to define the ROI. Right clicking allows the user to draw a box to select the ROI. Left clicking shifts the center of the current ROI without changing the size.
Frames to use:
This allows the user to define the frames used for most further data processing. This uses standard MATLAB notation for array slicing.
Arbitrary region ICS popupmenu:

This popupmenu switches between (1) using all frames, (2) using only a selected range of frames or (3) using arbitrary region ICS for further data processing. Arbitrary region ICS ([Hendrix2016]) allows discarding individual pixels from the correlations to remove the influence of unwanted objects and artifacts on the analysis. There are different ways of unselecting pixels, which can be combined.

Manual selection (this option might not work on every device!):
Here, regions are (un)selected by manually drawing the regions on the images. This is accessed by holding down and dragging with the right mouse button on the left images. The selection applies to the pixels throughout the whole stack.

For all arbitrary region ROIs, unselected pixels are highlighted in the right images, based on the selected colormap.

Intensity [kHz]:
Pixels are discarded if their mean intensity throughout the whole stack falls outside the range defined by MIN and MAX. A value equal to zero for MIN and MAX turns it off. This is especially useful for discarding e.g. the cell exterior. Note: 2 edit boxes are visible for MIN and MAX. The left one is for channel 1, the right one is for channel 2!
Subregions [px]:
Further options below employ the mean property of pixels in one or two small ROIs within the image. The size of those small ROIs is defined by these two edit boxes. The left size always has to be smaller than the right size.
Intensity [fold] or Variance [fold]:
The intensity or intensity variance inside the small ROI is compared with the large ROI. If the ratio between those values is below or above the range defined by the values in the edit boxes, the pixel is discarded. For pixels at the edges the ROI cannot be created. These pixels are therefore automatically discarded. Note: These options are performed on a frame to frame basis, so pixels are not automatically removed from the whole stack. Pixels that use discarded pixels for a moving average subtraction/addition are also discarded.
Individual Channels popupmenu:
Selects which channel(s) are use for calculating the arbitrary region.
Framewise Int. Threshold:
Pixels are discarded if their mean intensity in small subregion (defined by the left edit box in Subregions [px]) falls outisde the range defined by MIN and MAX. Note: The intensity in the small subregion is rescaled (/(mean int.frame)*(mean int.frame1) to account for intensity fluctuations such as drift or photobleaching.
Median Filter:
Smoothen the arbitrary ROI by applying a 3x3 median filter.
Get background from ROI pushbutton:
For some applications (e.g. RICS or N&B) it might be useful to subtract the background from the corrected image prior to further analysis. Pushing this button displays the average count rate (in photons per pixel dwell time) in the currently selected ROI. Alternatively, if you know the background count rate from a control experiment, you can also write the value directly in the edit box. The left edit box is channel 1, the right one is channel 2. The values displayed in the edit boxes are subtracted from the images after any correction (see Correction tab) has been carried out.

Correction

This tab has options to apply corrections to the data, e.g. to get rid of an immobile fraction for (R)ICS. It includes various options for subtracting from and adding to the image, including a moving average (both pixels and frames), and pixel, frame and total mean. For the calculation of these values the discarded pixels from the arbitrary region selection are not used.

N&B and RICS calculation tab

_images/processing_tab.png

The Processing tab

At the bottom right of the main window is a tab that contains all settings and buttons for applying the various processing methods like (R)ICS, TICS, STICS and N&B.

In the first popupmenu, the user can choose to calculate only a single autocorrelation function for the selected channel, or both auto- and crosscorrelation functions. Additionally, the user can choose for each method whether and how to save the calculated data.

Two extra edit boxes are present: Window and Offset.

(R)ICS using the ‘save blockwise analysis as .miacor’ option.

  • Window is the window size (in frames) in which correlation data need to be averaged
  • Offset is the offset (in frames) of the next window with respect to the previous.
  • For analyzing adjacent windows, Offset = Window
STICS/iMSD
  • Window is the maximum number of lags (in frames) that have to be calculated.

Additional information tab

This tab contains a couple of plots that show general information about the loaded files in greater detail. The left hand side shows the data as a frame average, while the right side shows the time average.

(R)ICS tab

The third window is reserved for displaying the image correlation data and a quick first evaluation of the data quality. Proper analysis is done using the MIAFit sub-program by selecting to save the data as a MATLAB file with a .miacor ending. It is also possible to save the data as TIFFs to load and plot them with standard image analysis programs like ImageJ.

The autocorrelations are plotted in the right and left plots, while the crosscorrelation is shown in the center plots. Hereby, the first row displays the correlations as and image plot. In the second row, it is plotted as a surface plot. The last row displays data related to the fit, either the fit or the residuals as a surface plot. The third option plots the data as a surface and the residuals in blue and red. The last option shows the on-axis data and fit.

_images/rics.png

The (R)ICS tab

The setting in the top left define the size of the correlation to be used, the colormap for plotting and which frame to display. When frame “0” is selected, the program displays the average of the frames defined in the Frames to use edit box. Use standard MATLAB notation.

TICS tab

The TICS window is reserved for temporal correlation data and for a quick first evaluation of the data quality. Proper analysis is done using the FCSFit sub-program by selecting to save the data as a matlab file with a .mcor ending. To limit the contribution of undersampling, the correlation is only calculated up to one tenth of the maximum movie length.

The plot at the top of the window shows the pixel averaged auto- (green and red) and crosscorrelation (yellow). The table on the left can be used to perform a very simple 3D diffusion FCS fit.

The three images at the bottom show different calculated correlation parameters for the individual pixels for the two auto- (left and right) and the crosscorrelation (center). The user can select regions of the image by right clicking on them and export the averaged correlation for only the selected region.

_images/tics.png

The TICS tab

STICS/iMSD tab

MIA can not only perform spatial or temporal correlation but also spatiotemporal correlation. The full STICS data can be properly analyzed with MIAFit when saved as a MATLAB file with the .stcor extension. In the STICS/iMSD tab the user can look at the data as a function of lag time in the three images at the bottom (left and right for the auto- and the central for the crosscorrelation).

_images/stics.png

The STICS tab

A simplified way of analyzing spatio-temporal correlation data is called iMSD. Here, the square of the width of the Gaussian used to fit the correlation is plotted vs. the time lag. This way of plotting the data contains in principle the same information as mean-square displacement plots acquired by tracking. The iMSD data can be calculated, plotted and fit with the controls at the top of the window. For a more detailed analysis, the user can select to save the iMSD data during correlation as a MATLAB file with the .mcor extension. Since iMSD gives 1D temporal data, it can be analyzed with an appropriate fit in FCSFit.

Number and Brightness tab

N&B analysis is still under development, so use with caution!

The absolute number and brightness are calculated in Mia from absolute photon counting images using Eqs. 8 and 9 from [Digman2008], with the difference that a one-photon PSF shape correction factor = sqrt(8) is included (Eqs. 16-17 in [Hendrix2015]) to be consistent with RICS analysis.

For analog or pseudo photon counting (e.g. Olympus Fluoview FV1000) data, instead of using Eqs. 8 and 9 from [Dalal2008], in Mia images are converted to absolute photon counting after which Eqs. 8 and 9 from [Digman2008] can still be used.

Data correction:

The N&B program uses the formulas for absolute photon counting A procedure for correcting data prior to calculating N&B can be found in [Hendrix2015]. In brief, for N&B, all slow fluctuations that are not caused by diffusing molecules need to be removed, but, in contrast to RICS, spatial image inhomogeneities due to e.g. concentration differences do not need to be removed. Ideally, subtracting a pixel’s moving average followed by adding the pixel’s average throughout the video would work. In our experience, however, we find it better to subtract the moving average of the ‘local spatial average of a pixel’, followed by adding the pixel’s average throughout the video. Therefore, on the Correction tab in Mia, select

  • subtract moving average: pixels = 3, frames = 3
  • add pixel mean
Notes:
N&B calculation:

On the do N&B tab in Mia, the following options are visible:

_images/doNB.png

The N&B calculation tab

  • Channel 1 popupmenu: select which data channel you want to use for normal N&B [Digman2008], or whether you want to do cross-NB [Digman2009b].
  • Average popupmenu: For N&B, the program will calculate the mean and variance image. This option allows you to additionally spatially average (‘smooth’) these images prior to calculating the N&B images.
  • Radius: select the smoothing filter radius in pixels.
  • Median filter checkbox: applies a [radius x radius] median filter to the N&B images.
  • Detector deadtime (ns): as described in [Hendrix2013], detector/TCSPC dead time will cause artifacts in N&B due to the fact that the highest pixel counts are systematically missing some counts, skewing the PCH and thus the N&B data. If you don’t use TCSPC, a value of 0 will probably be ok. If you use TCSPC, look up the device’s dead time. APDs also have a dead time, so take whichever dead time is highest (TCSPC or APD). Hybrid and normal PMTs have very low dead time (~1 ns). If you’re unsure about dead time artifacts, compare the N&B brightness of different concentrations of the same sample (e.g. 10, 50, 250, 1000 kHz). If the brightness artificially goes down, try correcting the dead time to see if the effect disappears.
N&B data display:

On the N&B tab, the calculated N&B data can be displayed and manipulated:

_images/NB.png

The N&B tab in Mia

  • Intensity, number and brightness images display the corresponding parameters in each of the images.
  • Intensity, number, brightness and PCH histogram displays the corresponding informations.
  • The 2D histogram allows plotting different parameters against each other.
  • You can set the thresholds bottom right on the user interface to make the display of the images and histogram look better.
  • Note: for the correct conversion of pixel counts to kHz, the correct pixel dwell time is needed!
General notes on N&B:
  • For a dataset of 100 frames, try to have brightnesses at least >5 kHz, otherwise the data is too noisy for decent N&B.
  • Increasing the moving average or filter averaging radi will improve data quality.

References

[Petersen1993]Petersen, N.O., et al. Quantitation of membrane receptor distributions by image correlation spectroscopy: concept and application. Biophys. J. 65(3), 1135-1146 (1993)
[Wiseman2000]Wiseman, P.W., et al. Two-photon image correlation spectroscopy and image cross-correlation spectroscopy. J. Microsc. 200, 14–25 (2000)
[Digman2005]Digman, M.A., et al. Measuring fast dynamics in solutions and cells with a laser scanning microscope. Biophys. J. 89(2), 1317-1327 (2005)
[Digman2009a]Digman, M.A., et al. Detecting protein complexes in living cells from laser scanning confocal image sequences by the cross correlation raster image spectroscopy method. Biophys. J. 96(2) 96707–716 (2009)
[Toplak2012]Toplak, T., et al. STICCS reveals matrix-dependent adhesion slipping and gripping in migrating cells. Biophys. J. 103(8), 1672–1682 (2012)
[DiRienzo2013]Di Rienzo, C., et al. Fast spatiotemporal correlation spectroscopy to determine protein lateral diffusion laws in live cell membranes. Proc. Natl. Acad. Sci. 110(30) 12307-12312 (2013)
[Digman2008](1, 2, 3, 4) Digman, M.A., et al. Mapping the number of molecules and brightness in the laser scanning microscope. Biophys. J. 94(6), 2320-2332 (2008)
[Dalal2008](1, 2, 3) Dalal, R.B., et al. Determination of particle number and brightness using a laser scanning confocal microscope operating in the analog mode. Microsc. Res. Tech. 71(1) 69-81 (2008)
[Digman2009b](1, 2) Digman, M.A., et al. Stoichiometry of molecular complexes at adhesions in living cells. Proc. Natl. Acad. Sci. 106(7), 2170-2175 (2009)
[Hendrix2013](1, 2) Hendrix, J., et al. Pulsed interleaved excitation fluctuation imaging. Biophys. J. 105(4) 848-861 (2013)
[Hendrix2015](1, 2, 3) Hendrix, J., et al. Live-cell observation of cytosolic HIV-1 assembly onset reveals RNA-interacting Gag oligomers. J Cell Biol 210(4) 629-646 (2015)
[Hendrix2016](1, 2) Hendrix, J., et al. Arbitrary-Region Raster Image Correlation Spectroscopy. Biophys. J 111(8) 1785-1796 (2016)