Imaging framework for light-sheet microscopy of organoid growth
We developed a multiscale imaging framework that comprehends acquisition, pre-processing, automated tracking, segmentation with further feature extraction as well as visualization dedicated for 3D live imaging (Fig. 1a). In this work, we applied our framework to live intestinal organoid light-sheet recordings performed with a dual-illumination inverted light-sheet6 microscope, which utilizes a multi-positioning sample holder system (Fig. 1b). In order to image organoid development, we followed previously published protocols6, FACS sorting single cells (Supplementary Fig. 1a) from mature organoids and mounting them as 5 uL mix drops with Matrigel on top of a ca. 50 um thick fluorinated ethylene propylene (FEP) foil, which are then covered in medium (Fig. 1c left and Methods Section). To stabilize the imaging, we patterned the FEP foil used for mounting in order to create small wells (Supplementary Fig. 1b–d, Supplementary Note 1), allowing better control of the sample position within the holder, while improving reproducibility of experiments by preventing drops from being washed away during medium change of fixation procedures. As previously demonstrated, the microscope we utilized is capable of imaging live intestinal organoids for long periods of time6,8, as well as acquiring time-lapses of mouse embryonic and gastruloid development18,19. However, one important drawback of the system was that the alignment of the illumination beams is done only once, prior to the experiment and irrespective of the position of the sample in the dish or holder. Although sufficient in certain situations (e.g., mouse embryo imaging), imaging of samples embedded and distributed inside a gel suffer from refractive index mismatch between water and Matrigel, as well as from the presence of other obstacles in the light-path (other organoids or debris) and from the curved shape of the sample holder itself. Therefore, to improve recording conditions in every individual sample, we developed a position dependent illumination alignment step. This allows to fine tune the alignment of each of the illumination sheets in respect to the detection plane for every sample position so that best image quality possible can be achieved throughout (Fig. 1b right and Supplementary Note 1).
To minimize storage needs and improve SNR, acquired images are cropped using a dedicated tool that automatically corrects for 3D sample drifting (Fig. 1d upper row). The cropped images may also be further pre-processed through denoising and deconvolution steps. Denoising is performed using the Noise2Void scheme20, with its output sent to a tensor-flow based image deconvolution21 (Fig. 1d lower row) using measured PSFs from beads (Supplementary Note 2 and Methods Section).
With these first modules at hand, we imaged organoids expressing Histone 2B and mem9 membrane peptide tagged with mCherry and GFP respectively, recording the growth and development of several organoids starting from single cells or 4-cell spheres (Fig. 1e and Supplementary Movie 1) every 10 min throughout the course of around 4 days. The collected data comprised of organoids that form both budding and enterocyst phenotypes: whereas budding organoids grow from single cells into mature organoids with both crypt and villus structures, enterocysts, comprised of terminally differentiated enterocytes, do not have crypts as they do not develop Paneth cells required for the establishment of the stem cell niche, a necessary step for crypt formation6,8.
For the analysis, we initially performed single-cell partial semi-automatic tracking using the Fiji plugin Mastodon (https://github.com/mastodon-sc/mastodon) on seven datasets. After that, we extracted features based on organoid and single cell segmentation and plotted this data over time (Fig. 1f and Supplementary Table 1). For example, we noticed large variability in cell division synchronicity, as in some datasets the nuclei number growth over time loses the typical staircase-like behavior already early during the first day of recording. Although epithelium volume growth curves follow that of nuclei number, with the characteristic exponential behavior, nuclei density slightly increases over time. Mean cell volume showed characteristic mitotic peaks, with overall cell volume decrease over time, matching the increase in nuclei density. Interestingly, although initially cell to nuclei volume ratio vary, all datasets converge to common steady state values where the cell volume is ca. 3–4 times larger than the nuclear volume. We also observed a consistent change in organoid volume due to medium change during the live recordings (Supplementary Fig. 2). As this initial assessment of our imaging data showed consistent and reproducible results, and to handle larger dataset more rapidly and consistently, we developed an integrated and automated approach to turn the imaging into digital organoids with a visualization tool.
Dedicated image processing workflow
To make the entire analysis and visualization tools directly accessible, we incorporated all image processing and data analysis modules into a unified workflow named LSTree, having most processing and training steps implemented using Luigi-based tasks, and the rest as jupyter notebooks for cropping and segmentation evaluation. The workflow along with juypter notebook and two example datasets are provided as a documented Github repository with step-by-step guide (see Methods Section and Supplementary Notes 2–4 for more information).
In the first pre-processing step, the user selects which organoid needs to be cropped. This automatically generates minimal bounding boxes per time-point as well as global bounding box (Fig. 2a). The workflow also has an interactive tool to review the crops and perform few manual corrections, e.g., to account for large displacement between consecutive frames (Supplementary Fig. 3). Next, if needed, denoising and deconvolution of cropped and registered movies is performed as one combined step. Important to note that we chose to denoise and deconvolved our datasets as the image quality usually decays quite heavily at later timepoints. However, this is not a requirement, and the prediction models can also be trained based on good quality unprocessed datasets. More details on how to bypass the denoising and deconvolution steps are discussed through the example datasets provided in the GitHub documentation.
For the segmentation of organoids, as well as their cells and nuclei, we adopted different segmentation strategies all relying on existing convolutional neural networks (Fig. 2b). Our main initial motivation was to test whether we could incorporate the spatial information from the lineage trees spots for training segmentation models. To that end, we decided to use the RDCNet instance segmentation network as a base22, taking advantage of its inherent recursive architecture. First, nuclei are segmented in 3D following a deep learning model trained with a mix of complete and partial annotations. A small subset of the frames is fully annotated by manually expanding the labels to the full nuclei, whereas partial annotations rely on the initial tracking performed with Mastodon by drawing spheres at the position of tracked nuclei. (Fig. 2b upper row). Jupyter notebooks for interactive visualization and correction of the predicted segmentation are also part of the framework and added onto the GitHub, which allows improving the model accuracy with minimal annotating time. To check whether this approach was valid, we compared the trained network output with randomly selected hand-annotated image volumes, yielding very good results (see Supplementary Note 5 and Supplementary Fig. 4a, b).
Motivated by the initial results with nuclei segmentation based on sparse annotations, we took a similar approach for cell segmentation. To this end, organoid and cell segmentation also use RDCNet and leverages the pre-computed nuclei segmentation to avoid manual annotations of individual cells. At the same time, we added a constraint based on lumen and epithelium segmentation, to avoid that cell labels spread outside of the epithelial layer. To subdivide the epithelium mask into cells, the previously segmented nuclei are used as partial cell annotations under the assumption that they are randomly distributed within the cell compartment (Fig. 2b lower row, Supplementary Note 3). Finally, in addition to the segmentation volumes and nuclei number (Fig. 1e), several different features are extracted such as nuclear distance to apical/basal membranes, fluorescence intensity, distance to parent node and number of neighbors per cell (For a complete list of features with short explanations see Supplementary Table 2).
Deep learning model for automated lineage tracing
Although suitable for estimating lineage trees for few datasets, semi-automated tracking of many datasets with the Mastodon Fiji plugin can be time consuming, as different datasets may require different setting parameters often break when cells are too packed or with low signal-to-noise. To significantly improve this process, we trained and refined a deep learning model on the available tracked datasets aiming at automatic generation of candidate trees that only require minimal corrections (Fig. 2c, d, Supplementary Note 3). To avoid usage of tracing algorithms that enforce a complex set of rules23,24,25, we developed a joint segmentation-tracking approach that simultaneously predicts matching nuclei labels on 2 consecutive frames. To this end, we extended the RDCNet instance segmentation model to predict pseudo 4D labels (3D convolutional network with time axis as an additional image channels) mapping correspondences between nuclei in two consecutive frames (Fig. 2c). Predicting linked nuclei segmentations has the advantage to enforce constancy over the entire nuclear volume rather than relying on an ambiguous center, as well as implicitly enforcing rules such as minimum cell distance or plausible nuclei volume constraints in a data-driven manner. This method keeps the number of manual hyper-parameters tuning to a minimum and can be improved over time as more validated and corrected datasets are incorporated in the training set. In a complementary manner, this method can be used together with other deep learning strategies such as Elephant in a complementary and modular manner, in which curated trees via Elephant can be used for training of more generalized tracking models based on RDCNet, or even directly used for nuclei/cell segmentation training/prediction.
To assemble the predicted tree in the framework, nuclei labels in each frame are connected to their parents in the previous frame by finding the linking label with the maximum overlap (Fig. 2c, Supplementary Fig. 5 and evaluation for dataset 006 in Supplementary Table 3). The predicted tree is then saved with the structure of a MaMuT.xml track file, which can be then imported into Mastodon for further correction if necessary (Fig. 2d). As a direct consequence of the joint segmentation, additional information, such as the nuclei volume, can be overlaid on the predicted trees to aid in the curation process (Supplementary Fig. 5, Supplementary Note 3). For instance, jumps in nuclear volume highlight positions where tracks should be merged or split. The manual curation time ranges from minutes to a couple hours on the most challenging datasets (e.g., low SNR images, abnormal nuclei shape). In summary, the here developed lineage tree-prediction approach allows high quality prediction of intestinal organoid lineage trees with long tracks spanning multiple division cycles (up to five generations in this work) enabling tracked data to cross spatiotemporal scales. To further challenge our tracking prediction strategy, we have also tested it outside of our main focus on live imaging of intestinal organoid and used trained models to validate prediction accuracy on mouse embryo datasets from published work (Supplementary Fig. 4c, e, Supplementary Table 4, and discussions on Supplementary Note 5), also comparing it to output from trained Elephant Tracker models (all trained models can be found in the Supplementary Software).
Digital organoid viewer
With the lineage trees and the deep learning 4D segmentation of organoid, lumen, cells and nuclei at hand, we developed a multiscale digital organoid viewer to explore and perform in-depth data mining. The viewer combines both lineage trees and segmented meshes, facilitating the direct comparison of different features within a multiscale digital organoid framework. We have added it to our LSTree Github repository along with example data, also including the possibility to overlay recorded images with the corresponding meshes allowing a direct inspection of the predicted segmentation (Fig. 3a, Supplementary Movie 2). As can be seen through the example datasets present in the repository, this interactive viewer allows associated features to be displayed, selected nodes to be interactively highlighted on the meshes, and color coding of both trees and meshes to be assigned independently. This way same or complementary features can be visualized at once (All currently extracted features are discussed in Supplementary Note 4 and Supplementary Table 3)
As an example of the image-analysis and visualization tools presented in the framework, nuclear volume quantifications can be evaluated directly onto the tree of a specific dataset (Fig. 3b). Using this approach, it is possible to observe and quantify how much nuclei volumes change with each generation and over time, with the smallest volumes observed right after division. Similarly, we observe that the nuclear distance to the basal membrane (Fig. 3c) increases due to interkinetic nuclear migration toward the apical side. Combining the same visualization procedure with the segmented meshes, we render the nuclei or cells in 3D, using the same color-coding as for feature on the trees (Fig. 3d). Last but not least, we also compare the extracted features against the general trend from all other datasets combined, allowing us a direct evaluation of variability across experiments (Fig. 3e, f), evaluating the increased distancing of nuclei from the apical membrane, a known effect due to epithelial polarization (Fig. 3f).
In summary, this is a broad set of tools embedded under the same workflow which allows not only multiscale segmentation of organoids along with lineage tree predictions, but also the simultaneous visualization of both trees and segmented meshes into a unified web-viewer. All steps of the process are implemented to keep storage, memory, and manual tuning requirements to a minimum, making this a powerful and yet easily accessible part of the light-sheet framework.
Functional imaging through fixation and backtracking
Next, we analyzed functional information on the tracked cells and organoids contained in the lineage trees. Although our imaging framework allows the visualization and quantification of a large number of features at the cellular and organoid levels throughout organoid growth, functional information remains dependent on fluorescent reporter organoid lines. The easiest way to theoretically approach this is to perform stable multicolor live imaging for long periods of time. However, overlapping emission/excitation spectra limits the total number of fluorescent reporters and concerns regarding interference with the normal cell function, signal-to-noise sensitivity for low abundant proteins, photostability and general phototoxicity due to laser illumination limit the use of fluorescent reporters. To overcome this, we fixed and added an immunolabelling steps at the end of the live recordings to assess the end state of the cells. Then we tracked the immunolabelled cells back through the lineage tree (Fig. 4a), using LSTree for further visualization and analysis.
In details, we fixed the sample at the end of the recording with 4% PFA, to then perform the immunolabelling protocol (see Methods Section). The pre-patterning of the FEP foil holding the sample was crucial, as without it the Matrigel drops were washed out. To account for organoid drifts, we imaged the entire fixation procedure, so that the organoids could be tracked during fixation, leading to a recovery of more than 80% (for more detailed information please see Supplementary Note 6). To register the fixed organoids to the last time-point of the live recording we used similarity transformations implemented in ITK and available via Elastix26,27 (used as a stand-alone tool, as exemplified in Fig. 4b. For more information, please refer to the Methods Section and Supplementary Note 6). To test this approach, we imaged H2B-mCherry, mem9-GFP organoids until day 3 (Supplementary Movie 3 left). It has been shown that between around day 1.5 intestinal organoids break symmetry through the appearance of the first differentiated cells of the secretory lineage (Paneth cells, Lysozyme). Preceding the appearance of Paneth cells there is the local establishment of a Notch-Delta lateral inhibition event, with future Paneth cells being typically Delta Like Ligand 1 positive (DLL1+)6. To analyze symmetry breaking, we fixed the organoids at 56 h and stained for DLL1-Alexa488 and Lys-Alexa647 (Fig. 4c–e-, Supplementary Movie 3 right). Intriguingly, the two DLL1+ cells are two sister cells that were formed at the end of the division from generation 5 to generation 6, around 10 h before fixation.
To follow cellular dynamics and changes of features of these specific cells in their spatial environment we analyzed nuclei and cell volumes (extracted with LSTree) per generation of the backtracked cells from generation 5 and 6 and compared them to all the other cells during the same generations (Fig. 4f). Interestingly, cellular, and nuclear volumes of the backtracked cells do not seem to deviate relative to each other during generation 5. After cell division and entering generation 6, however, the nuclei volumes of both DLL1+ sister cells show an increased relative difference to one another, with the Lys+ cell having a slightly larger nucleus. Changes in nuclear volume related to appearance of DLL1+ signal was also observed in other datasets (Supplementary Fig. 6).
Next, we evaluated the dynamics of neighbor exchange by cross-checking the closest cells to a backtracked cell(s) of interest at each time-point with LSTree. We examined if the progeny of these two sister cells had high level of mixing with other cells during cyst growth. From the visualization of the tracked neighbors on the lineage tree and segmented meshes (Fig. 4g), it is apparent that neighbor exchanges, although distributed across the tree, do not happen often nor with many different cells, keeping an average of five cells.
The above results show that, by combining our light-sheet framework with standard fixation and registration techniques we can broaden the level of functional information, bridging it to the dynamical processes during live imaging. Consequently, we were capable of dissecting some initial dynamical elements preceding the formation of DLL1+ and Paneth cells in the context of the entire organoid development, analyzing the process of symmetry-breaking events across biological scales.
Nuclei merging events during organoid growth
From our backtracking example it became apparent that one cell undergoes multiple rounds of failed divisions, with two daughter cells merging before a new division starts (Fig. 5a). Upon further inspection of the other lineage trees, we realized that most of the datasets contained at least one merging event during early phase of organoid growth whereby two sister nuclei, at the end of their cell cycle, divided again into two instead of into four nuclei (Supplementary Fig. 7). To investigate whether a failed division during the previous mitotic cycle was causing these nuclei merging, we examined the last step of the previous cell division. In all cases there was a problem during late cytokinesis, with the two sister cells never fully separating (Fig. 5a, b). Nuclear volume for all the daughters arising after the merging event is clearly increased and cell volume followed the same behavior, roughly doubling in tetraploid cells (Fig. 5c, d). To dismiss the possibility that these mitotic failures are caused by phototoxic effects of the imaging itself, we performed a time-course experiment with wild-type organoids grown from single cells under same medium conditions as the live recordings. We fixed organoids at days 2 and 3, staining them for e-Cadherin and DAPI. Despite the lack of continuous illumination, the resulting data showed many cysts with polynucleated cells, as well as cells with enlarged nuclei (Fig. 5e).
Using the framework we were able to follow the fate of the progeny of a cell that underwent cytokinesis failure and surprisingly we noticed that they can lead to cells that remain part of the epithelium until the end of the recordings without dying, when we can observe fully budded organoids or mature enterocysts (end of day 4) (Fig. 5f, g). Yet, comparing to unaffected parts of the trees, this binucleation progeny typically has higher probability to be extruded into the lumen (~46% for merged progeny against ~5% for other cells). Another intriguing observation is that the remaining 54% of the cells are never localized to the crypt but to the villus (Fig. 5f, Supplementary Fig. 8). This is an interesting result, as it suggests that the cells that undergo cytokinesis failure, and might have chromosomal defect, do not migrate or differentiate into niche cells (Stem cells and Paneth cells) but stay as villus cells that are shorter lived. This might mean that there are mechanisms to maintain cellular integrity in the stem cell niche avoiding damaged cells in the crypt.
Molecularly, it is known that the Large tumor suppressor kinase 1 (Lats1) can influence cytokinesis failure via lack of inhibition of Lim kinase 1 (Limk1)28,29,30. This poses an interesting hypothesis on the role of mitotic failure in intestinal regeneration as Lats1 and Yap1 are master regulators of regenerative response of the intestinal epithelium31,32. Analysis of RNAseq from previous studies6, show decrease in Lats1 expression during initial days of organoid growth that mimic the regenerative response of the intestinal epithelium6. This regenerative response is achieved by downregulating Lats1 as a negative regulator of Yap1. We initially stained for Limk1 and could see cell-to-cell variability of its expression (Supplementary Fig. 9a,b). To further analyze the role of Lats1 and Limk1 in the regulation of cytokinesis failure in the early days of organoid formation we perturbed Lats1 and Limk1 activity. Time-course imaging of Lats1 double knockouts6 showed several cysts with double nucleated cells that result from mitotic failures (Supplementary Fig. 9c). Moreover, inhibition of Lats1 and Limk1 with chemical inhibitors (Truli and Damnacanthal, see Methods Section and Supplementary Fig. 9), increased and decreased the number of bi-nucleated cells, respectively (Fig. 5h, i). Lats1 inhibition also display an increase of Yap1 activation. This shows that in intestinal organoids formation, cytokinesis failure is regulated by Lats1 activity that in turn is a negative regulator of Limk1.
Taken together, through the multiscale approach of 3D segmentation, feature extraction and lineage tree analysis we were able to identify consistent polyploidy events during early intestinal organoid development and the fate of their progeny. Our framework allowed us to bridge the observed mitotic defects across scales toward the tissue scale, showing the end fate of the merged cells progeny and spatially locating them onto the mature organoid morphology. This is shedding new light on the robustness of a regenerative YAP cellular state, questioning the role of polyploidy in intestinal regeneration.