Back to Front Architecture for Diagnosis as a Service

Software as a Service (SaaS) is a cloud computing model in which a provider hosts applications in a server that customers use via internet. Since SaaS does not require to install applications on customers' own computers, it allows the use by multiple users of highly specialized software without extra expenses for hardware acquisition or licensing. A SaaS tailored for clinical needs not only would alleviate licensing costs, but also would facilitate easy access to new methods for diagnosis assistance. This paper presents a SaaS client-server architecture for Diagnosis as a Service (DaaS). The server is based on docker technology in order to allow execution of softwares implemented in different languages with the highest portability and scalability. The client is a content management system allowing the design of websites with multimedia content and interactive visualization of results allowing user editing. We explain a usage case that uses our DaaS as crowdsourcing platform in a multicentric pilot study carried out to evaluate the clinical benefits of a software for assessment of central airway obstruction.


I. INTRODUCTION
Nowadays it is common practice in clinical environments to have a dedicated PC (often with expensive licenses) that is used by the professional to assist in diagnosis based on the analysis of medical images and volumes. On the other hand, researchers in the field of medical images are developing software and rapidly come up with new methods based on images that are provided by the professional. However, to obtain these images most often the researchers personally have to visit the hospital to extract the image form the specific medical device (by means of a portable hard drive). In a more flexible scenario the expert could directly send the image to the researchers computer to process the image and obtain the result using a SaaS client-server architecture.
Since there are many researchers working in the field of medical imaging, it would be very interesting that the SaaS could process different kind of medical images using different methods at a single spot. The main difficulty is that each and every researcher feels comfortable with his/her favorite tools and development environment, i.e. the methods probably consists of a suitable combination of several Matlab and Python scripts and C/C++ programs. In order to manage this situation, one could think of having a Virtual Machine for each method running on the host. A virtual machine creates the idea/illusion that one has access to an operating system different from Spanish Projects DPI2015-65286-R, 2014-SGR-1470, Fundació Marató TV3 20133510 and FIS-ETES PI09/90917. D.Gil is a Serra Hunter fellow. Titan X donated by the NVIDIA Corporation. C.Sanchez is suported by H-2020-Generalitat de Catalunya Marie Skodowska-Curie No 712949 the host with its own dedicated resources [1]. However, we have come to the conclusion that it is more natural to make use of the docker (container) technology [2], [3]. Dockers creates the perception of an isolated exclusive environment for each application with resources allocated by the host, while in traditional virtual machines the application is executed in a virtual operating system with own resources where it coexists with other applications. A docker acts as a container [4] in which an application together all its dependencies can be encapsulated or stored. The goal of the docker is to achieve true independence between applications on the one hand and infrastructure on the other hand so that in the end the docker can either work on a standalone PC or in the cloud.
A SaaS architecture tailored for clinical use opens up the validation of the diagnosis tool using expert-users as a crowd. Crowdsourcing [5] uses a client-server architecture and separates extensive tasks into many micro-tasks that demand few time. The micro-tasks defined for the crowd can be performed using internet and flexible interfaces that demand few time (less than 1 minute) to end each micro-task. This allows users to end the whole validation test at their own place using any available free time they might have. Although crowdsourcing has become a popular efficient methodology to gather annotated data and validate computational systems [5], its potential for validation of the clinical benefits of computational decision support systems has not been explored so far. This paper focuses on the development of SaaS computational technologies tailored to provide DaaS. We present a baseline client-server architecture allowing, both, the execution of medical imaging methods as DaaS and on-line validation of such methods using expert crowds. The server for method processing is based on dockers to allow easy execution of algorithms implemented in different programming languages and auxiliary softwares. Concerning validation, we present a platform for web editing that incorporates elements for interactive visualization of videos and correction of segmentation algorithms. Finally, we explain a usage case that uses our DaaS as crowdsourcing platform to evaluate the clinical benefits of an own-designed software for assessment of central airway obstruction (CAO) in videobronchoscopy.

II. BACK-END/FRONT-END ARCHITECTURE FOR DAAS
Our back-end/front-end solution is depicted in the scheme of Fig.1. The back-end server has a docker architecture with 3 layers: a server managing connection to internet, an API Web service that sends internet queries to the computing layer and, finally, the computing layer managing the execution of methods. The front-end provides with a web interface that allows the upload of new cases, as well as, interactive visualization of methods output.
In our back-end architecture, each computational method is encapsulated in a docker image which has an Apache Server to interact with our central server that receives requests from the outside world (front-end applications). These requests are then translated by the Flask FrameWork API Web service which converts the web requests into the specific format that can be understood by the method processing. Finally, methods are executed in the computer layer that contains all necessary software and library dependencies to execute each method. In order to handle protection of personal data and privacy policy, we use a dedicate server that only accepts requests from registered users (certificate of authenticity). Moreover, images will be encrypted and they will be deleted after being processed.
Concerning the front-end interface, we have developed an editing platform on content management system (CMS) [6] based on PhP, HTML5, CSS and mySQL database for easy implementation and design of websites for data gathering and interactive visualization of results. To such end, besides the standard tools available in common web editing platforms like wordpress, Drupal or Joomla, we have developed 5 specific plug-ins for crowd-sourcing task creation that could process and validate medical imaging algorithms. Each plug-in gives custom functions and specific settings enabling researchers to tailor their sites to their specific needs and methods. The specific settings include the layout of the element (size, position, etc), output folders and particular settings for input data files. The 5 plug-ins (summarized in Table I) are: 1) Video visualization. This plug-in shows a video with a sliding bar allowing interactive navigation and a button for frame selection. The input settings are the label of the videos to be displayed. The outputs are the time spent by the user and the selected video frame. 2) Image Gallery. This plug-in shows a gallery of images allowing interactive navigation using a slide bar and a button to select one of them. The input settings are the name of the directory containing the images. The outputs are the time spent by the user and the selected image. 3) Multimedia Processing. This plug-in allows to upload either a video or an image to be processed by one of the docker methods which is selected using an option list. 4) Curve/point editing. This plug-in is specific for segmentation methods and it shows either an image or video frame with the contours segmented by the method for their manual correction. The input settings are the name of the directory containing the images and the method output points to be displayed. The outputs of this component are the time spend for correction and the edited points. 5) Questionnaire: This plug-in manages opinion surveys with queries including existing elements of HTML forms for specific answer type (text,textbox, group radiobutton, option list). For this plug-in, aside standard layout settings, the number of questions, question text and answer type. The output of this component is the collection of all answers. A DaaS web page combines several plug-ins with specific settings (position inside the page, associated text, output) that can be encapsulated in standard HTML forms. According to data collection requirements, these forms have been customized to include PhP instructions managing data storage. For each DaaS page, its composition of plug-ins with their specific settings are stored in a MySQL database in order to allow modifications. The logic design of the database for restoration and modification of DaaS web pages is shown in fig.2

III. USAGE CASE: CROWDSOURCING VALIDATION OF A DIAGNOSIS SUPPORT TOOL
Our DaaS has been used to run and evaluate an owndesign System for Endoscopic Stenosis Assessment (SENSA) of central airways developed in cooperation with the "Hospital de Bellvitge". SENSA computes the degree of central airways obstruction (CAO) from standard bronchoscopic explorations. Degree of obstruction was computed using the Stenosis Index (SI) as the ratio between healthy and obstructed cross sectional areas (CSA) [7]. Areas are obtained from contours of tracheobronchial rings and obstructed lumen automatically computed by SENSA in videobronchoscopy explorations [8].
We used a crowdsourcing platform to evaluate SENSA benefits for obtaining a more efficient and accurate standardized CAO in a group of experts and non experts pulmonologists. For our crowdsourcing approach we defined the following friendly-user micro-tasks 1 (see Fig.3) using our web editing tool: 1) µTask1: SI visual assessment. A short video (2-4 seconds) obtained during bronchoscopy procedure is shown. During this video, non-obstructed and obstructed segments are visualized by the clinicians. Videos can be stopped and replayed as many times they like. Pulmonologist have to visual estimate SI and write it down to the textbox. Finally, by ending the micro-task, time required as well as SI are saved. 2) µTask2: Visual SI using SENSA contours. SENSA CSA contours displayed on the frames selected by the expert were shown to users for visual SI estimation.
To allow for an easier comparison of stenotic and healthy CSA, an image displaying both contours on the obstructed frame was also provided. As before, pulmonologists were asked to write the estimated SI into a textbox. 3) µTask3 SENSA contours manual correction. The frames of the previous task were shown to pulmonologists for correction of CSA contours by clicking image 1 available on-line at http://ivendis.cvc.uab.es/Dades/SENSAValidationDEMO/ points on the right contour every time SENSA CSA was considered to be sub-optimal. In order to make manual edition efficient, we used a computational algorithm that allowed corrections using a minimal set of points.
Pulmonologists enrolled in the study were provided with a user and password for a secure log in to the validation webpage. A short manual was designed to guide physicians through each task. Once pulmonologists had logged in, they chose the case they wanted to validate from the cases they had not validated yet. After ending a case (the 3 micro-tasks), SI, time and corrected points were stored at the central server and the case was removed from the to-do-list shown at the log-in site. Once they had finish the validation of all cases, they were asked to answer two questions: 1) Evaluate SENSA as a support tool to calculate SI (from 1 to 10); 2) Indicate a positive and a negative aspect of the DaaS platform as a tool for clinical validation (free text).

A. Results
A retrospective review of videos from patients with CAO was performed. Bronchoscopic videos were obtained in patients with benign CAO at the Respiratory Endoscopic Unit from Hospital Universitari Bellvitge. In each video, the proximal view of the tracheal stenosis and distal normal airway were visualized. Videos were recorded using an Olympus BF P160 and BF T180 videobronchoscopes. A total of 12 videos were included in the study.
A first sequence from each video was chosen where the scope was seen proximal to stenotic area, moving inside the stenotic area and crossing it to show distal lumen. Two more sequences, lasting 3 seconds each, were compiled from each video, one from healthy anatomy, one from the obstructed one where the videobronchoscope was positioned 1cm proximal to each area. These 3 seconds were converted into frames that were processed by SENSA using our DaaS.
For the crowsourcing validation, frames showing healthy and obstructed segments with SENSA contours drawn in two different colours were displayed in two different windows of a web page. One of the experts browsed through the healthy sequence as well as through the stenotic one using a slide bar to select the frames best suited for stenosis computation. A total number of 13 pulmonologists (7 interventional and 6 SENSA crowdsourcing web page was available for 2 months. A total of 12 cases were downloaded for validation in the webpage and all 13 pulmonologist completed the 3 microtask for each case. Most users (10/13) could do the tasks for all cases in a single connection. The time required to obtain a reference SI by correcting SENSA contours was in the range 36.5 ± 35 seconds for each case (7.3 ± 7 minutes in total) and no differences between experts and non-experts pulmonologist (p=0.365, according to a t-test) were found.
Regarding the opinion survey, only half of the participants answer it. SENSA as a tool for standardized CAO assessment was rated with a 10 by all experts. The possibility of managing tasks using web applications able to run in portable devices (like tablets or smartphones) was highly appreciated by the participants and considered a friendly way of performing validation tasks. Crowdsourcing was also well accepted as a means of doing time demanding tasks in a flexible way and saving time.

IV. DISCUSSION AND CONCLUSION
A client-server architecture enables simultaneous use of a unique license of the system installed at a central server. This allows a multicentre testing of new methods skipping in-situ deployment of the software and extra use of the clinical institutions resources. Another advantage is that all measures and data are stored in a central server with a unique format ready for statistical analysis and validation trials. The docker technology provides flexibility and portability because applications can either be executed in a physical machine or in a (private) cloud. Because of its nature (a container) it is fairly easy to transfer the container/docker from one environment to another without almost any effort. Such flexibility makes our platform highly scalable to other problems or even fields.
Our usage case points out the potential benefits of using crowdsourcing techniques for the validation and development of clinical decision support systems. Aside training and validation, there are 2 major bottlenecks in the development of new clinical decision support systems. First, assessing the system clinical benefits in terms of improved clinical decisions and outcomes and reduction of clinicians learning curve once the system has been trained, validated and iterated. Second, disseminating and promoting its use among the medical community as a tool to standardized tracheal stenosis measurement.
The use of a flexible web interface allowing annotation of visual data and execution of methods in the cloud could alleviate such shortcomings by using the medical communities and societies as expert crowd. Besides, a side benefit of using a crowdsourcing targeted to clinicians is to let them know about the advantages of using new processing clinical algorithms. Finally, since crowd sourcing allows many different users to have access to a new technology, crowd sourcing could be a good mean to diffuse processing algorithms diagnostic tool and, thus, could be useful as marketing tool to introduce technology among clinicians.
We conclude that DaaS offers affordable multicentric deployment of new developments for diagnosis support and opens up their testing and diffusion among the clinical community. Also DaaS used as crowdsourcing platform allows easy gathering of data as well as experts' input useful to carry out multicentric studies.