Traditionally, biological visualisation was limited to two-dimensional displays only. Until recently, using three-dimensional displays for quantitative signal assessment, including precise signal selection, processing of multiple signals and determining their spatial relationship to one another has received fairly little attention. However, recent developments in the virtual reality (VR) arena have seen the development and use of lightweight headsets, offering high resolution, low latency head tracking, and a large field of view.
These developments and innovation make VR an attractive form of technology to use in biological data visualisation. Biological data visualisation as a branch of bioinformatics entails the application of computer graphics, scientific visualisation, and information visualisation to different areas of the life sciences. Software tools that can be used for visualising biological data include simple, standalone programs and complex, integrated systems. It allows immersive three-dimensional visualisation, compared to a three-dimensional rendering on a two-dimensional display. Since the visualisations are based on a true three-dimensional awareness of the sample, they not only offer a clear representation but also a more intuitive process of interaction which can ultimately aid the scientific investigation and discovery process.
Background
One key factor that has, in the past, limited the utility of microscopy volume visualisation was its low rendering speed. “Just eight years ago”, the authors explain, “it was challenging to exceed 20 frames per second (10 frames per second per eye) on consumer computing equipment.” This is far from enough for VR, which requires a consistent frame rate of no less than 60 frames per second per eye to ensure an immersive experience. Not maintaining this rate may result in simulation sickness. However, with the current advances in consumer graphics performance, the frame rates necessary for VR can be achieved with reasonable rendering quality.
In this regard, confocal microscopes deliver comprehensive three-dimensional data that are instrumental in biological analysis and research. Usually, this three-dimensional data is rendered as a projection onto a two-dimensional display. The research describes the system that renders such data using a VR headset. Sample manipulation is furthermore possible “by fully immersive hand-tracking and also by means of a conventional gamepad”. In this research, this approach is applied to the specific task of colocalization analysis – an essential tool for analysis in biological microscopy – and evaluated through user trials.
To evaluate the quality, suitability, and characteristics of this system, a fluorescence-based three-dimensional biological sample obtained from a confocal microscope was used. It should be noted, however, that performing a confocal microscopy-based three-dimensional sample investigation and conducting subsequent colocalization analysis remains challenging. The video below demonstrates the use of virtual reality for this purpose using two different interfaces using either a traditional gamepad or a leap motion hand tracking system using a mammalian cell as an example.
Results and conclusions
The user trials indicate that, despite inaccuracies that still plague the hand tracking, this is the most productive and intuitive interface. However, these inaccuracies still lead to users perceiving productivity as low, which results in a subjective preference for the gamepad. Fully immersive manipulation was shown to be especially effective when defining a region of interest for colocalization analysis.
VR offers an attractive and powerful means of visualization for microscopy data. Fully immersive interfaces using hand tracking demonstrate the highest levels of intuitiveness and resulting productivity. However, current inaccuracies in hand tracking performance still lead to a disproportionately critical user perception that should be addressed.
You can find the original research here:
https://www.youtube.com/watch?v=ajdNKnAHFMw&t=2s
http://dsp.sun.ac.za/~trn/reports/theart+loos+niesler_bmcbioinf_2017.pdf
https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-016-1446-2
