Processing pipeline

Data loading, localization, tracking, tracks compute, high-resolution visualization, graph visualization, and gallery generation are the main steps of the PALM processing pipeline. These steps are executed in the process() method according to the interface settings.

Step 1: Stack loading

According to the batch defined in the interface settings, stacks are loaded into memory for processing.

Several options are available:

  • A single stack is loaded; in this case, the selected mode has no impact.

  • Multiple stacks are loaded; in this case, the selected mode affects the behavior:

    • Only one: only the selected stack is used.

    • Each File separately: stacks are processed one after another.

    • All in One: stacks are concatenated into a single one.

Step 2: Saving setting and metadata files

In the output folder — defined by the stack name with the suffix _PALM_Tracer (example: stack.tif → stack_PALM_Tracer) — the following files are saved:

  • A setting file settings-<timestamp>.json containing the settings used for processing.

  • A metadata file meta-<timestamp>.csv containing the stack metadata (dimensions, calibration, etc.).

  • A log file log-<timestamp>.log containing the processing logs.

Step 3: Localization

Localization is executed according to the interface settings. If disabled, precomputed and unfiltered localizations can be loaded, provided they are present in the output folder.

If filters are selected, they are applied to the localizations to retain only those matching the defined criteria. If the save filtered option is selected, the filtered localizations are saved in a file localizations_filtered-<timestamp>.csv.

Note

If localization is disabled and no precomputed localization is present in the output folder, localization is considered empty and the following steps are executed with empty localizations.

Note

Loading a precomputed localization is always performed using the most recent unfiltered version.

Step 4: Molecule tracking

Tracking is executed according to the interface settings. If disabled, precomputed and unfiltered tracking data can be loaded, provided it is present in the output folder. Tracking uses as input the filtered version of the previous localizations (computed or loaded). If no filtered version exists or if filtering removed all localizations, the unfiltered version is used.

If filters are selected, they are applied to the tracking results to retain only tracks matching the defined criteria. If the save filtered option is selected, the filtered tracking is saved in a file tracking_filtered-<timestamp>.csv.

Tracks reconnection

If the reconnection option is enabled, tracks are reconnected to attempt reconstruction of longer tracks. Reconnected tracks are saved in a file tracking-reconnected-<timestamp>.csv. It uses the unfiltered tracking version as input.

If filters are selected, they are applied to the tracking results to retain only tracks matching the defined criteria. If the save filtered option is selected, the filtered and reconnected tracking is saved in a file tracking_filtered_reconnected-<timestamp>.csv.

Note

If tracking is disabled and no precomputed tracking is present in the output folder, tracking is considered empty and the following steps are executed with empty tracking data.

Note

Loading precomputed tracking is always performed using the most recent unfiltered version, and the reconnected one if it exists.

Step 5: Tracks compute

Calculations can be performed on tracks to extract information (MSD, instantaneous diffusion, model fitting). These computations are executed according to the interface settings.

Calculations use as input the filtered and reconnected version of the previous tracking (computed or loaded). If no filtered or reconnected version exists or if filtering removed all tracks, the unfiltered version is used.

If filters are selected, they are applied to the tracking results to retain only tracks matching the defined criteria. If the save filtered option is selected, the filtered and reconnected calculations are saved in three files: tracking_MSD_filtered-<timestamp>.csv, tracking_InstantD_filtered-<timestamp>.csv, and tracking_Fit_filtered-<timestamp>.csv.

Note

If calculations are disabled, the following steps will be able to compute the required elements quickly.

Warning

Removed tracks are also deleted from the original tracking table, and the file tracking_filtered-<timestamp>.csv is rewritten.

Step 6: Visualizations

The visualization step includes the generation of a localization ROI gallery as well as high-resolution renderings and graph visualizations. These are automatically generated using simple default settings (e.g., number of ROIs, ROI size, etc.) for a first rapid data exploration, especially during large batch processing. The use of dedicated viewers is recommended for more in-depth and customized data exploration.