Transfer Learning and Cellpose

Caroline Baker

Supported by Duke’s URS Summer Idea Grant, I performed transfer learning on cell segmentation models for data analysis on cardiac tissue images. My tasks for analyzing these images were to identify all cell nuclei, cardiomyocyte (CM) nuclei, and endothelial cell (EC)nuclei, as well as proliferating CMs and ECs. This is the final step in my research lab’s LVAD project, which involves determining the relationship between the use of a left ventricular assist device (LVAD) and CM and EC proliferation. I began working with Cellpose, a recently developed, quick, and robust Python-based cell segmentation algorithm. I used a training set with masks manually created in ImageJ, hoping that this model would be able to accurately create masks for a testing set of un-masked images. After weeks of tweaking parameters, Cellpose was not performing at the levels I expected for each of the nuclei types. I then trained u-nets, another cell segmentation model, for the majority of these analysis tasks. I am still working on this project and am using ensemble learning with a combination of the Cellpose and u-net models to provide a robust, accurate, and fast cell segmentation model for the analysis of approximately250,000 images. I would not be able to conduct this segmentation without the help of the speed and graphics of Google Colab Pro and 2 TB of data storage on Google Drive. Thank you to theURS Office for giving me this opportunity to further my project in the Karra Lab and helping me learn a vital skill for my future research laboratories and biomedical engineering coursework.

Image caption: This image shows the accuracy with which Cellpose was able to analyze unseen images in the LVAD Project. The “Input" is the original image. The “Ground Truth” is the manually masked image (true output). “Prediction” is Cellpose’s guess for what the output should be. The last image shows differences between the true output and Cellpose’s prediction.