Medical Design Briefs - April 2022 - 14

Image Cytometry
Machine Vision
Camera
CXP Bank A
CXP Bank B
CXP Bank C
CXP Bank D
C-Mount Adapter
DMA Cyclic Buffer
DMA Cyclic Buffer
DMA Cyclic Buffer
DMA Cyclic Buffer
stAPI
Capture
Microscope
Image Stitch
stAPI Storage
Manager
Display
Stream Pointer
Synch
Real-Time
Processing
LED Fiber Light
Microfluidic Chip
NVMe Sets
Fig. 4 - Block diagram showing the hardware and software architecture in the real-time image
cytometry setup. The machine vision camera streams images out from the microscope/cytometry
setup and into the backend hardware. Note: FG in the schematic denotes " frame grabber. "
was generated showing specific particles
that traversed the field of view (see
Figure 3). This data also detailed cell
size, shape, and color, along with position
and speed.
Backend PC and Results. The system
processed an average time per frame of
6.2 µs, which permits speeds of ~100 kfps
at a selected resolution of 640 × 200.
Most calculations were performed on
unstitched images, and image classification
was performed using selective ROI
stitching. Given that the main processing
cycle runs on a single core, the other
cores were used to perform classification
and calculations and to display results.
See Figure 4 for an overview of the hardware
and software architecture.
In this system, it was possible to store
data in a 30 minute cyclic buffer while
performing calculations, allowing the
user to review portions of the content in
real time. Image portions of the detected
cells were stored in HSV format in up to
256 GB of RAM to help the classification
algorithms. Based on preliminary results,
each particle and/or cell used an average
of 2 KBytes (8 bits) in HSV format. This
setup achieved more than 500,000 samples
in 1 GB. Such high sample rates
enable very detailed classification and
allow for parallel statistical calculations.
The system can also be configured to save
the image data in a form of RAW pixel
14
Cov
values that can be directly discovered and
read into software such as CellProfiler.7
With this architecture, it is possible to
send event-signals from cell presence,
cell count, and cell image/phenotype,
in addition to many other possible identification
triggers. Calibrated triggers
that match an event with time and position
are also an option. This is ideal for
researchers who would like to filter,
destroy, and/or sort specific microparticles
in real time. The maximum latency
in the system is 300 µs, or ~15 frames at
52,000 fps. In this case, since it takes a
particle or cell ~2,400 frames to cross the
entire field of view, there is more than
enough time to perform a real-time
action on the cell and observe the result
within the channel for validation and
troubleshooting purposes.
Conclusion
The aim of this study was to set a benchmark
for what is possible in real-time
image cytometry. It is hoped that re -
searchers will better understand what is
possible for the next generation of biological
applications that demand single-cell
precision and rapid real-time results. This
model system demonstrates a level of flexibility
that may accommodate diverse
cytometry applications. It is important to
note that this technology is not limited to
image cytometry but can be expanded to
www.medicaldesignbriefs.com
ToC
other applications that demand real-time
analysis on the order of Gpixels*s-1.
Discussion of Fluorescence. In fluorescence
applications, frame rates are typically
restricted to below ~1,500 fps. As
was briefly mentioned above, this is due
to the lack of photons available from the
fluorescence event, which puts a hard
limit on acceptable exposure times. If
such an experiment were attempted,
signal-to-noise ratio (SNR) would likely
be too low to generate an image of
usable quality and clarity. Furthermore,
if exposure time was increased to incorporate
more photons in an attempt to
achieve higher SNR, the cells would
exhibit severe motion blur. To overcome
these issues, researchers may implement
the use of image intensifiers or make use
of slower flow speeds coupled with time
delayed integration (TDI).
Machine Learning. Given that the software
system provides acquisition and
recording, real-time detection, and classification,
it is possible for software functions
to be executed in parallel. As a
result, this architecture is amenable to
machine learning for determination of
trigger-based patterns or more complex
conditions, and it can also be used to
refine blob analysis and detection algorithms.
The software allows external ac -
cess, so it can be controlled via separate
programs built in Python or similar languages,
making it easier to integrate into
other AI systems.
References
1. Y. Han, et al., " Review: Imaging Technologies
For Flow Cytometry, " Lab Chip,
2016, 16, 4639-4647.
2. H. Mikami, et al., " High-Speed Imaging
Meets Single-Cell Analysis, " Chem, 4, 11
2018, 2278-2300.
3. D. R. Gossett, et al., " Hydrodynamic
stretching of single cells for large
population mechanical phenotyping, "
PNAS, 2012, 7630-7635.
4. M. Doan and A. E. Carpenter, " Leveraging
machine vision in cell-based diagnostics to
do more with less, " Nature Materials 18,
2019, 410-427.
5. D. D. Carlo, " A Mechanical Biomarker of
Cell State in Medicine, " Journal of
Laboratory Automation, 17, 2012, 32-42.
6. Y. J. Heo, et al., " Real-time Image
Processing for Microscopy-based Label-free
Imaging Flow Cytometry in a Microfluidic
Chip, " Scientific Reports 7, 2017, 11651.
7. T. R. Jones, et al., " CellProfiler Analyst:
data exploration and analysis software for
complex image-based screens, " BMC
Bioinformatics, 2008, 9, 482.
This article was written by Kyle D. Gilroy,
PhD, Field Applications Engineer for Vision
Research, Wayne, NJ. For more information,
visit http://info.hotims.com/82320-341.
Medical Design Briefs, April 2022
FG1
FG2

http://info.hotims.com/82320-341 http://www.medicaldesignbriefs.com

Medical Design Briefs - April 2022

Table of Contents for the Digital Edition of Medical Design Briefs - April 2022