Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
FACULDADE DE E NGENHARIA DA U NIVERSIDADE DO P ORTO Signal processing and machine learning methods and algorithm analysis for classication of lung function on a smartphone app EEC0035 - Preparação da Dissertação João Pedro Fonseca Teixeira Mestrado Integrado em Engenharia Electrotécnica e de Computadores FEUP Supervisor: Luís Teixeira External Supervisor: Miguel Coimbra February 26, 2014 c João Pedro Fonseca Teixeira, 2014 Signal processing and machine learning methods and algorithm analysis for classication of lung function on a smartphone app EEC0035 - Preparação da Dissertação João Pedro Fonseca Teixeira Mestrado Integrado em Engenharia Electrotécnica e de Computadores February 26, 2014 Contents 1 PDI report intro 1.1 Project objectives and motivations . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 2 Problem statement 2.1 Defining the problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Refining the problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 3 3 Literature Review 3.1 Introduction . . . . . . . . . . . . . . 3.2 Spirometry Technology . . . . . . . . 3.2.1 Spirometers and Sensors . . . 3.2.2 Research and Development . . 3.2.3 Games and Applications . . . 3.3 Methods and Algorithms . . . . . . . 3.3.1 Spirometric Parameters . . . . 3.3.2 Signal Processing . . . . . . . 3.3.3 Machine Learning . . . . . . 3.4 Discussion and Decision . . . . . . . 3.4.1 Signal Processing Discussion 3.4.2 Machine Learning Discussion 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 5 5 5 9 9 10 10 11 13 16 16 17 Work Plan 4.1 Calendarization . . . . . . . . . . . . . 4.2 Methodology . . . . . . . . . . . . . . 4.3 Technologies, tools and work platforms 4.4 Concluded tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 19 20 21 21 References 23 i ii CONTENTS List of Figures 3.2 3.3 Common Flow Sensing Devices (Pneumotachometers). A-Fleisch type sensor, B-Heated wire sensor, C-Pitot tube sensor, D-Turbine sensor, E-Ultrasonic sensor Flow vs Volume representations for different lung functions . . . . . . . . . . . . Perceptron diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 11 15 4.1 Time distribution diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.1 iii iv LIST OF FIGURES List of Tables 3.1 Distance metric functions between instances (x and y). . . . . . . . . . . . . . . 16 4.1 Temporal distribution of the project’s tasks. . . . . . . . . . . . . . . . . . . . . 20 v vi LIST OF TABLES Abbreviations and Symbols Medical Related COPD FEF25%−75% FEM FEV1 /FVC FEVt FVC MMV PEF RV SVC TD VC VTG Chronic Obstructive Pulmonary Disease Forced Expiratory Flow between 25% and 75%, average flow during FVC maneuver Forced Expiratory Maneuver Tiffeneau-Pinelli index Forced Expiratory Volume in the first t seconds Forced Vital Capacity Maximum Voluntary Ventilation Peak Expiratory Flow Residual Volume Slow Vital Capacity Tidal Volume Vital Capacity Thoracic Gas Volume Engineering Related ADC ANN APF BPF CART DIY DFT FFT HPF kNN LPF ML SL SVM UL USB Analog to Digital Converter Artificial Neural Network All Pass Filter Band Pass Filter Classification And Regression Tree Do It Yourself Discrete Fourier Transform Fast Fourier Transform High Pass Filter k-Nearest Neighbors Low Pass Filter Machine Learning Supervised Learning Support Vector Machines Unsupervised Learning Universal Serial Bus CAGR Compound Annual Growth Rate vii Chapter 1 PDI report intro This document aims to present the work developed throughout the curricular unit Preparação da Dissertação. Chapter 2 contains a description of the problems that will be approached during the project. In chapter 3 the State of the Art concerning the relevant topics for the development of the project will be presented. Also, the methods that will be researched in finer detail will be summarized. Those include: • Signal processing methods of forced expiration recordings. • Feature gathering on original and processed signals. • Machine Learning algorithms for input classification. Finally, in chapter 4 the work plan for the semester will be presented . In addition, the platforms and technologies which are expected to be used will be described. 1.1 Project objectives and motivations This project aims to develop a mobile and easy to use platform that can help evaluate a patient’s respiratory function. This kind of technology can also enable monitoring systems for patients with lung problems such as asthmatics, COPD patients and people with other obstructive and/or restrictive airway diseases. The following requirements are also desirable for the platform: – Self contained, – Detached from external systems (no external sensors), – Fast enough (Timely results), – Precise in the classification, 1 2 PDI report intro The mobile and the self contained characteristics are important in order to reach the most isolated places, which generally have worse medical care, if any. It not only makes sense the use of smartphones in isolated places as it has been studied that areas in relatively severe poverty have had a booming growth in smartphone use. Africa and Middle East is expected to increase from the current 10% of share of smart devices and connections to 36% until 2018. The mobile data traffic CAGR will also grow to 70% (Cisco, 2014). Currently about 1 in 5 Africans have access to a smartphone (AfricanTelecomsNews, 2012). The possibility of reaching such a number of people with less access to medical facilities would certainly improve respiratory health care and awareness. Chapter 2 Problem statement 2.1 Defining the problem Lung function evaluation methods often revolve the forced expiratory maneuver (FEM). Since the ultimate goal consists in the development of an Android app, the problem is to collect breathing data in a way that the data resembles the FEM’s. Though the cooperation of the recording patients and the correctness of the maneuver is important, only the topics concerning the development of the application will be addressed. The available data to work with will be an air pressure waveform which will be transformed in a classification of the lung function. 2.2 Refining the problem Since the problem is quite complex it urges to follow a divide and conquer strategy. In this sense, the main problem is broken down to a few simpler and more contained problems: – Physical Model of the Signal Propagation: Most of the spirometric relevant measures are based on air flow and volume. The microphone is only capable of picking up data in the form of pressure. In addition, the measurable pressure is attenuated and dispersed comparing with the produced by the vocal tract. Therefore it might be beneficial to develop/assume a radiation model and a conversion model from pressure to flow. – Signal Processing: The signals, either previously processed or not, should go through a series of processes that should have significant meaning in terms of audio. Processes may include: FFT, Hilbert Transform, Filtering. – Signal Feature Extraction: All the audio should be analyzed and distinct features should be passed on to the classification stage. Features may include: FEV1 , FEV1 /FVC, PEF and audio related measurements. – Optimization and Classification: The features will be used to train a supervised learning model of how to produce a characteristic label of the inputs. 3 4 Problem statement – Method set comparison: Finally, several different method sets have to be evaluated on which gives the best results. Chapter 3 Literature Review 3.1 Introduction In this chapter, a literature review deemed relevant for a better understanding of the problem under study will be presented. A brief description of current spirometry technology is shown on section 3.2. In section 3.3 some algorithms for Signal Processing and Machine Learning that are currently available are presented. Finally, section 3.4 will present a brief discussion of which methods might be best for the problem at hands. 3.2 Spirometry Technology The precursor of the modern spirometer was introduced by Hutchinson around 1844. It was a volume displacement water-sealed device. In recent years the sensor paradigm has shifted towards the flow measure type thanks to the evolution and wide-spreading of electronic devices. Also, it became possible to measure lung function parameters, using several other methods. 3.2.1 Spirometers and Sensors During the course of lung function testing history many types of systems have been developed. Some of them enable exams that complement each other and others are specific for providing data from particular respiratory related diseases. A short presentation of each medically accepted spirometry device is provided as described on Ruppel (2008): 3.2.1.1 Pneumotachometers From Greek, pneuma, "wind" or "breath", thacos, "speed" and metron, "measure", pneumotachometer means that it measures breath speed. "The flow sensors that historically have been, and continue to be, the mainstay of the respiratory laboratory utilize flow resistors with approximately linear pressure–flow relationships. These devices are usually referred to as pneumotachometers" (Macia, 2006). 5 6 Literature Review Volume Displacement Spirometers As the name implies, these spirometers measure directly the shifting fluid volumes. Since the designs of this kind of devices were made during a time prior to the electronic massive growth they are mainly based on mechanical actuation. The movement caused by the breathing expirations is transferred to a stylus which could write on a moving chart. More recently, with the development of computers, potentiometers were incorporated so the signal could be better analyzed. Water-Seal Spirometer: This device consists of a large bell suspended in a container with its open end submerged. Breathing into the spirometer moves upwards the bell by an amount proportional to the volume expired. During an inspiration the bell submerges beyond the stable level. For many years the bell had a pen attached, that moved sympathetically with the bell. The pen recorded on a rolling chart, the kymograph, the respective volume. In recent years the bell activates a potentiometer handle that varies a DC voltage output. This signal could later be digitized with an ADC and processed by a computer. Dry Rolling-Seal Spirometer: This spirometer is cylindrically shaped holding a lightweight piston inside. A flexible plastic seal couples the piston to the cylinder wall. The seal rolls on itself when the piston moves slightly with the gas’ volume displacement. The piston shaft rides on a linear bearing which actuates a rotary potentiometer. This generates an analog signal for flow and volume that can be fed to an ADC and analyzed. Bellows-Type Spirometer: This kind of device uses a collapsible bellow that folds or un- folds depending on the fluid’s flow direction. The common bellow design resembles a flexible accordion. One end is fixed and the other moves freely with the expirations. This movement actuates either a pen that writes on a moving chart or a potentiometer similar to the other volume displacement spirometers. Flow-Sensing Spirometers The following spirometers measure air flow based on several different principles. All of them include the respective sensor and a tube to contain the expiration. Figure 3.1 illustrates the following sensor types. Turbine: The turbine, or respirometer, is an instrument that "consists of a vane connected to precision gears". The vane’s rotation is triggered by the gas flowing through it. This sensor is more suited to measure SVC. The sensor’s configuration can also use a light source and a photoreceptor to retrieve the number of rotations per second, just like an odometer for motor controllers. Pressure Differential Flow Sensors: This types of sensors make a pressure differential measurement. The sampling openings are placed one before and another after a resistive element, in the middle of the sensor tube. The element accelerates the gas by narrowing the gas flow 3.2 Spirometry Technology 7 Figure 3.1: Common Flow Sensing Devices (Pneumotachometers). A-Fleisch type sensor, BHeated wire sensor, C-Pitot tube sensor, D-Turbine sensor, E-Ultrasonic sensor1 . opening, while reducing the pressure. This is called the Venturi effect. While changing the gas’ velocity and pressure, the element also ensures laminar flow. The resistive element can either be a bundle of capillary tubes, the Fleisch type, or a mesh screen or membrane, the Silverman or Lilly type. The Fleisch type is more reliable than the Lilly type. Though the Lilly type is better suited for measuring widely varying flows. The disadvantages of these kinds of measurement devices are that they are very sensitive to local atmospheric pressure, temperature and humidity. As a consequence, they need to be calibrated daily. Heated-Wire Flow Sensors: The heated wire method is based on the cooling effect of a flowing gas. The wire is preheated to a fixed amount and, during the expiration, the gas’ flow reduces the wire’s temperature. The instantaneous energy consumption to reheat the wire gives a flow value at that time. Pitot Tube Flow Sensors: The Pitot tube sensors are based on the principle that states that the pressure of a fluid is related to it’s density and velocity. In general, a spirometry Pitot tube sensor has, at least, two sets of small tubes, one facing each flow direction for measuring both expiration and inspiration. The tubes are connected to pressure transducers. Ultrasonic Flow Sensors: A sound wave is a compression wave that is influenced by the means through it travels. The flowing of gas can increase or decrease a sound’s speed depending of 1 Image taken from Ruppel (2008). 8 Literature Review the relative direction it travels. The sensor consists of two oppositely faced ultrasound transducers, angled with the tube. By measuring the sound’s traveling time the flow measurement can be integrated into volume. There is also a different type of gas flow measurement that can be obtained by quantifying the ultrasound’s frequency shift, based on the Doppler effect. The design includes a disposable tube which is introduced between the sensors. This allows a great advantage over several other spirometer types since it needs not calibration and solves the problem of patient cross-contamination. 3.2.1.2 Peak Flow Meters This instrument was designed to measure Peak Expiratory Flow (PEF). It provides a simple measurement that aids asthma monitoring with an inexpensive device. The meters themselves tend to be made disposable. These meters work by forcing air through a a resistor or a flow tube with a movable indicator. 3.2.1.3 Plethysmographs From Greek, plethysmos, "increase" and graphein, "to write". The plethysmograph is an instrument that measures the change in volume within an organ or body. There are two types of body plethysmographs: the constant-volume variable-pressure and the flow or variable volume plethysmographs, Pressure plethysmograph and flow plethysmograph, respectively. Both are devices that include a sealed box which the patient has to be in. They are used to measure the thoracic gas volume (VTG ), the specific airway resistance (sRaw) and its derivatives. Pressure Plethysmograph: This spirometer is based on an adaptation of Boyle’s law. The change in volume inside the chamber is inversely proportional to the pressure variation. The pressure change is caused by compressing and decompressing the atmosphere inside the box, including the patients chest. If the temperature remains constant, the pressure change corresponds to a specific volume change. Since the chamber has rigid walls and is sealed, its free volume experiences the same, mirror-image shift volume as the lungs (Criée et al., 2011). Flow Plethysmograph: This kind of plethysmograph uses a flow transducer in the chamber’s wall. This device measures the volume changes inside the box. As gas is compressed and decompressed, flow passes through the chamber’s opening. That flow is integrated and volume change is recorded as the sum of the volume compressed and the one that left through the opening. With this device there are also some other tests that can be conduced using a pneumotachometer connected to the outside room. 3.2 Spirometry Technology 3.2.2 9 Research and Development There is a significant effort to develop spirometric devices focused on the availability of materials or cost efficiency. This aims to reach areas with less access to health care and in isolation as described on 1.1. A team from Penn State University developed a Lilly-Pneumotach type spirometer from easily accessible materials (der Mauer et al., 2013). This Mashavu project prototype was designed as a DIY concept enabling the wide-spreading of the design. It used a pressure differential electronic sensor to measure the airflow and was made with PVC tubing, coffee stirrers, protoboard as a mesh along with cotton cloth for filtering. Another relevant project came from a multidisciplinary team of students from the University of Washington in St.Louis. They worked towards a low cost spirometer (Brimer et al., 2012). Their solution consisted of using a microphone as a pressure transducer and coupling it to a custom designed breathing piece. The piece was relatively cheap to produce, could be easily sterilized and was of robust manufacturing. One project that motivated this dissertation was the SpiroSmart application developed by Larson et al. (2012) addressing the Ubiquitous Computing field. The application consists of a somewhat complex software that records the patient’s breath while giving incentive and showing the performance. It employs signal processing and machine learning methods to measure lung function. A later project developed by van Stein (2013) for his Master’s thesis was based on SpiroSmart but primarily focused on robustness to noise. Additionally, by using offline methods it could enable better results since the methods were more powerful and compute-intensive than real-time methods such as those used by SpiroSmart. 3.2.3 Games and Applications Throughout spirometry history there have been several technological advances. As described on 3.2.1, sensory methods have shifted from purely mechanical to partially electronic. As a consequence, spirometry apparatuses have become computerized and became able to include secondary functions or add-ons. Computers have maintained the tendency of getting smaller, following a corollary of Moore’s law2 . This means that there has been a reduction of the size of the platforms where spirometry related software run. In recent years, significant effort has been focused around smartphones since they are continuously carried around and include a significant amount of electronic devices. Games for compliance: The forced expiration maneuver is somewhat difficult to execute cor- rectly. This case is specially true for children that need to make a lung function evaluation. 2 Moore’s law is an empirical prediction which states that within roughly 18 months a chip performance will double, due to the miniaturization of transistor technology. 10 Literature Review Therefore pulmonary function laboratories commonly have computer programs that guide children through the process, most of the time without them even noticing. Usual tests comprise computer games that are played using breathing apparatus, such as breathing forcefully through a tube. This enables a better cooperation from the patient since kids rarely wish to lose a game. Games for entertainment: With the growth in smartphone use and a rising concern about respi- ratory diseases, several independent application developers have created sets of games that relate to spirometry. Many measure the loudness recorded by the smartphone’s microphone to actuate something. Some of them actually advertise spirometry, though they appear under the game application category. Mobile health applications: There is a significant amount of developers that create smartphone applications that measure spirometric parameters. Those applications usually have an external sensor that is connected by Bluetooth, Wi-Fi or USB to the smartphone. The application becomes a mean to relay data, observe it and update a patient’s profile. Some solutions are also compatible with multiple sensors for different health problems such as heart rate monitors, pulse glucose and cholesterol meters, oximeters and so on. 3.3 3.3.1 Methods and Algorithms Spirometric Parameters Methods are only successful if interesting features can be extracted from data. Standard spirometers measure flow of air that goes through the sensor to generate Flow vs Time plots. The flow can then be integrated to achieve Volume vs Time plots that provide medical doctors another way to visualize the information. Also, by mapping air volume to time, Flow vs Volume plots can be made. From this last plot several quantities are measured: • Forced Vital Capacity (FVC): total expelled volume during the expiratory maneuver. • Forced Expiratory Volume in one second (FEV1 ): lung volume displaced in the first second of the maneuver. • FEV1 /FVC: the ratio of the aforementioned values, also called the Tiffeneau-Pinelli index. • Peak Expiratory Flow: the maximum flow velocity present in the maneuver. "The most common clinically-reported measures are FEV1 , FVC, and FEV1 /FVC as they are used to quantify the degree of airflow limitation in chronic lung diseases such as asthma, COPD, and cystic fibrosis" according to Larson et al. (2012). These features try to summarize the expiration curve’s characteristics, though a physician can retrieve a great deal of information from analyzing the curve itself. 3.3 Methods and Algorithms 11 Figure 3.2: Flow vs Volume representations for different lung functions3 . Though the curve also provides relevant data, the study will focus on extracting these features based on curves approximations. A completely accurate curve shape regression will not be developed. 3.3.2 Signal Processing Signal Processing (SP) is a field of engineering and applied mathematics that focuses on manipulating signals to extract features or to turn them into a more desirable form for other applications. The forced expirations recorded by a smartphone’s microphone will have certain characteristics that prevent a meaningful extraction of parameters of interest. As a consequence, those signals have to be transformed in order to get relevant data. Some pertinent SP methods are summarized below. • Fast Fourier Transform • Up and Down sampling • Signal Filtering • Hilbert Transform • Linear Regression 3.3.2.1 Fast Fourier Transform The Fast Fourier Transform (FFT) is an algorithm commonly used in sound processing and analysis. It is an efficient way to compute the Discrete Fourier Transform (DFT) and its inverse. This process can be viewed as the transformation of an amplitude and time signal into its frequency power spectrum contents. 3 Image taken from http://www.frca.co.uk/images/flow_volume.gif. 12 Literature Review Let x0 , ..., xN−1 be complex numbers: N−1 DFT : Xk = ∑ xn e−i2πk n N k = 0, . . . , N − 1. n=0 Here n stands for a descretized instant of time and k is a descretized frequency. In many applications the signal is "windowed" in overlapping chunks in order to analyze how the frequencies vary with time. This enables frequency tracking which is useful for characterizing both the signal and its source. 3.3.2.2 Up and Down Sampling Downsampling, or decimation, is a process which reduces a signal’s sampling rate. The decimation rate usually is an integer or a rational fraction greater than one. When the rational is used, first the signal is upsampled by the fraction’s numerator and then downsampled by its denominator. Upsampling is the opposite process, that adds samples between the original signal ones, also known as interpolation. Generally, the downsampling process is used to change a signal to a fixed lower sampling rate that a signal processor will use. The upsampling is used to change back a processed signal, making it more adequate for the processor’s external environment. 3.3.2.3 Signal Filtering Filters are systems that change both a signal as its frequency power spectrum with respect to the filter’s frequency response. They remove unwanted features and components from a signal and can be described as to their frequency response as: • Low Pass Filters: remove higher frequencies maintaining lower frequencies almost untouched. Can be perceived as a shifting mean. • High Pass Filters: remove signal’s lower frequencies keeping the higher frequencies almost intact. • Band Pass Filters: remove all frequencies outside a specific frequency band. It is a minimum second order filter since it can be viewed as a combination of a LPF and a HPF. • All Pass Filters: are a different kind of filter that maintain equally the frequency gains. They change the phase of the different frequency components modifying the time series and not the frequency response. The expression removal of frequencies might not be completely accurate since most filters tend to attenuate deeply the frequencies cut, thus not completely removing them. If the intended response of the filter is complex it could be implemented as a series of simpler filters. 3.3 Methods and Algorithms 3.3.2.4 13 Hilbert Transform The so called Hilbert Transform is a linear operator that results in the convolution of a real valued signal with the signal −1/(πt). The process is a basic tool in Fourier analysis and can calculate the harmonic conjugate of a given function or Fourier series. If this conjugate is added back to the original signal and processed using a LPF it is possible to extract the original signal’s envelope. 3.3.2.5 Function approximation In order to extract breath parameters robustly, one could approximate the expiratory sound envelope by a high degree function (van Stein, 2013). By using the signal’s approximation much of undesired noise is removed and the general form of the flow-volume curve is greatly preserved. There are several methods of approximation such as: • Linear regression: First order approximation. The data becomes represented by a linear function. • Polynomial regression: A Nth order polynomial function approximates a noisy signal. • Smoothing Spline: The spline methods consist on segmenting a signal into several smooth polynomial functions. If the functions degree is greater than one then each segment connects to the next polynomial such as that the functions derivatives have the same value on the connection. For higher degrees, this gives better results than simple polynomial regression since it can avoid Runge’s phenomenon4 . • Orthogonal polynomials: are families of polynomials with consecutive pairs of functions that are orthogonal to each other under some inner product. A signal can be represented by a linear combination of those functions. 3.3.3 Machine Learning Machine Learning (ML) is a branch of Artificial Intelligence which develops models and studies systems that can learn from data. These methods are widely applicable since they are abstract concerning data representation and utility. They can be roughly categorized in two sets based on the desired outputs or the inputs available on the training set. The Supervised Learning (SL) methods are algorithms that use both inputs and expected outputs, provided on the data set, to replicate the intended model. The Unsupervised Learning (UL) methods are used when the desired output is unknown. They operate on unlabeled data and have to devise the system’s behavior. The field of ML is vast and growing. Therefore the scope of the methods to be researched and developed needs to be narrowed. One important aspect of the project is to be considered. To aid the learning process a series of recordings of real patients will be made. The smartphone 4 Using equidistant points, with the increase of the polynomial function degree the interpolation fitting error might not tend to 0. 14 Literature Review recordings will be accompanied by a commercial spirometer from which a medical diagnosis will be made. This way, the ML method choice problem is reduced to Supervised Learning processes. Some of these methods are shown below: • Decision Trees • Perceptron-based techniques • Instance Based-Learning • Support Vector Machines 3.3.3.1 Decision Trees A tree is a structure that consists of a root node and several others attached to it in some kind of hierarchical fashion. The decision tree’s nodes represent decisions, separating data according to the node’s function. The terminal leaf nodes are the data labels. This kind of structure is transparent to the user since it progressively sorts the data based on some feature until it is completely defined. The tree, or in fact the node’s functions, have to be generated based on the training data. There is significant effort to find the best and fastest ways to divide the training data. The termination of the dividing algorithm generally isolates many data entries to their own leaf. This causes problems concerning the overfitting of the method. Many variations were proposed to reduce this problem such as terminating the algorithm before the tree grows to full depth or pruning some of the outlier branches. There are also ensembles of methods that provide a better accuracy. A known technique of variance reduction that diminishes overfitting problems is Bagged Decision Trees. By using n random subdatasets, with replacement, to generate trees and enabling a voting process of new data it can enhance local adaptivity of the boundary (Rao and Potts, 1997). 3.3.3.2 Perceptron-Based techniques A single layer perceptron is a system that has N inputs and one output. First, the inputs are linearly combined according to each input’s weight (typically real values between [-1, 1]). This output is then fed to an adjustable threshold. Above it the output is 1; else it’s 0. Figure 3.3 shows a system representation of a perceptron. The learning algorithm changes the branches’ weights until the desired input/output behavior is reached. If the inputs are not linearly separable with respect to the correct categories then a perceptron will not find a solution that fits all available data. In order to solve this problem multilayered perceptron (ANNs) methods were developed (Rumelhart et al., 1985). ANNs consist of several perceptrons, otherwise called nodes or units, arranged such as to produce weighted links between each others’ inputs and outputs. They can be separated in three 3.3 Methods and Algorithms 15 Figure 3.3: Perceptron diagram5 categories according to their interface: the input layer nodes receive and process the system’s inputs; the output layer nodes are the last stage of data processing and generate the system’s outputs; the hidden layer nodes are perceptrons that connect input and output layer nodes by means of activation values. As with a single layer method, a succession of training instances are exposed to the net, and comparing the input/output response the weights are slightly adjusted in the direction of error reduction. One of the main problems of ANNs resides on the size/number of layers of the hidden set of nodes. According to the desired application, too few nodes can result in a poor approximation and generalization capabilities. If too much nodes are considered then overfitting can occur and finding a global optimum can be proven difficult. 3.3.3.3 Instance Based-Learning Instance based learning algorithms are what can be called as lazy algorithms. The majority of the computation is made on the classification and not at the training phase. One of the most known methods is k-Nearest Neighbours (kNN). The idea behind this algorithm is based on the instances’ proximity. Instances with the similar proprieties tend to be closer to each other than to other instances on the dataset. A new, unclassified, instance is assigned the most frequent label of its neighbors. This relative distance between instances, this neighborhood, can be defined by a distance metric. Some of the most popular distance metrics are presented on table 3.1. 3.3.3.4 Support Vector Machines SVMs are focused on separating labeled data using hyperplanes. Their aim is to maximize a "margin" between the boundary of different classifications and the instances of either side. Minimizing the Lagrangian in 3.1 with respect to the weight vector w and bias b would get the best hyperplane cut. 5 Image taken from http://homepages.gold.ac.uk/nikolaev/311perc.htm. 16 Literature Review Table 3.1: Distance metric functions between instances (x and y). r 1/r Minkowski: D(x, y) = (∑m i=1 kxi − yi k ) Manhattan: D(x, y) = ∑m i=1 kxi − yi k Chebyshev: D(x, y) = maxm i=1 kxi − yi k 2 1/2 Euclidean: D(x, y) = ∑m i=1 kxi − yi k kxi −yi k Camberra: D(x, y) = ∑m i=1 kxi +yi k Kendall’s Rank Correlation: i−1 2 D(x, y) = 1 − m(m−1) ∑m i= j ∑ j=1 sign(xi − x j )sign(yi − y j ) There are some variations of the method for solving problems such as data containing misclassifications and non-separable cases. There are even methods that remap non-linearly-separable data in order to enable a hyperplane boundary and consequently to use SVMs. The SVMs training algorithms necessarily reach the optimal separation, avoiding local minimums unlike ANNs for example. LP ≡ N N 1 kwk − ∑ αi yi (xi · w − b) + ∑ αi 2 i=1 i=1 3.4 Discussion and Decision 3.4.1 Signal Processing Discussion (3.1) The Hilbert Transform seems to be a crucial process since it provides a simple way to compute the envelope of the breath signal. Another way to compute an envelope is tracking the major frequencies of the signal, with a FFT, and regenerating a time series. Envelope signals might need to be filtered of noise in order to extract correctly the features. Therefore, it could prove beneficial to use LPF. On the other hand, the envelope can be made immune to noise if a function approximation is computed. Of those methods listed on subsection 3.3.2.5, some might work better then others for the specific case. The Linear Regression probably would not have good results since the FEM does not follow a linear function. A Polynomial Regression seems to a good candidate though it has the instability problem for higher orders. On the other hand, it could have good results for an order between 3 and 5, which seems to be enough to represent the FEM accurately (van Stein, 2013). Splines have a track record of lower order requirements and have similar results to Polynomial Regression. Though, since the polynomial order seems to be relatively low it might not be justifiable. Orthogonal Polynomials have a significant number of different families and need to be researched in more detail to assess their relevance for the project. It was defined that one of the requirements of the application is to have a processing time relatively small so not all the methods mentioned should be implemented. Additionally, the results of some methods might be similar to the degree that feature extraction does not monitor significant change. The methods will be subject to a deeper study. 3.4 Discussion and Decision 3.4.2 17 Machine Learning Discussion An instance based algorithm such as kNN does not seem suited to the projects requirements. First, since a great deal of computation is made during the classification phase, the algorithm takes a significant amount of time to calculate the output, specially if the training data has a considerable size. Furthermore, this also means that the method has a large storage requirements. Another significant problem concerns the sensitivity of the kNN to the distance metric function. Finally, the number of neighbors, k, is a difficult variable to optimize. Perceptron based techniques such as Neural Networks have a great classification speed and can tolerate relatively well highly interdependent attributes. Though they have a poor tolerance to missing values and do not deal well with overfitting issues. Therefore, ANN methods do not seem very adequate for the problem at hand. Decision trees have generally good characteristics such as transparency, speed of classification and reduced model parameter set. Their main problem concerns noisy data and the risk of overfitting. Though, as discussed on subsection 3.3.3.1 there are simple methods that diminish the consequences of such situation. CART and Bootstrap aggregation methods (Bagging) seem good candidates for tackling the problem. Finally, SVMs have generally a very good accuracy, are quick to classify data and tolerant to irrelevant attributes. They also have a fairly good success rate tolerating redundant and highly interdependent attributes. Their shortcomings concern the transparency of the method and handling model parameters. Though, given the problem at hand, they are also strong candidates. An overview of several methods can be found on (Kotsiantis, 2007). 18 Literature Review Chapter 4 Work Plan Planning is one of the most important stages of any project and contributes significantly to its success. It enables the best allocation of resources while keeping in focus small and medium objectives. It also states a rate at which those objectives need to be completed. In section 4.1 the tasks and small objectives are presented along with the respective timeline and description. The methodology which will be used is described in section 4.2. In section 4.3 the tools and technologies that will enable to develop the project are stated. Finally, in section 4.4 will be shown the tasks already completed. Since every project is subject to fluctuation when it comes to task conclusion the calendarization presented below is a mere proposal and is expected to be somewhat flexible. In fact, allocated time could be shifted between tasks in order to ameliorate the time compromise. 4.1 Calendarization Figure 4.1: Time distribution diagram In table 4.1 the main steps of the project are shown along with the respective expected start and end dates plus their duration. Below, a short summary of those tasks is provided. 19 20 Work Plan Table 4.1: Temporal distribution of the project’s tasks. Task Personal Web page development Data gathering Methods analysis and Development Methods Simulation Application architecture and specs Writing the Dissertation Preparing the Presentation Initial date 10-2-14 14-2-14 10-2-14 18-3-14 22-4-14 12-5-14 25-6-14 Duration (weeks) 18 6 7 5 2 5 1 Ending date 30-6-14 28-3-14 28-3-14 21-4-14 2-5-14 24-6-14 1-7-14 1 Personal Web page development: During the projects development it is expected to present several documents namely minutes from meetings with the advisors and weekly reports. Therefore, it is needed to develop a personal website to make those documents available to the interested parties. 2 Data gathering: In order to teach the optimization models real life data is needed. During the course of several weeks, recordings of forced expiratory maneuvers are going to be made. 3 Methods analysis and Development: The various methods of signal processing and machine learning currently available will be analyzed in depth. It will culminate on the selection of the set which will be developed and evaluated. 4 Methods Simulation: The methods set will be implemented in Matlab and compared with each other to verify which combination of algorithms works best. They will be tested using a data set not previously used. 5 Application architecture and specs: Once the best conjugation of methods is chosen, the specification of the smartphone application will be stipulated. In addition, a concept of how the program should be implemented will be developed. 6 Writing the Dissertation: The last stage of the project will consist in writing a report compiling all relevant data, work developed and findings. 7 Preparing the Presentation: The dissertation will be accompanied by a presentation that demonstrates the work done. 4.2 Methodology During the course of the project it is intended to follow this methodology: – Gather forced expiratory recordings from real patients with different types of lung function characteristics/diagnosis. 4.3 Technologies, tools and work platforms 21 – Research the Signal Processing methods currently used and relevant for the data type. – Research the Machine Learning and Optimization techniques suitable for Supervised Learning methods. – Evaluate which methods are interesting enough to analyze in grater detail. – Implement the more promising methods in Matlab and simulate in various configurations. – Compare results and chose the best set of methods and configuration. – Specify and design an architecture for the android app. – Write the final report and presentation. 4.3 Technologies, tools and work platforms The implementation and simulation of the methods is expected to be done in Matlab. Has a requirement of Preparação da Dissertação, the periodically generated documents will be uploaded to a personal website, which is to be developed. The final report, presentation and weekly reports will be done in LATEXsince it is an important document editing tool that, once configured, doesn’t require much maintenance concerning text formatting. 4.4 Concluded tasks In Preparação para a Dissertação several initial milestones were achieved: • Initial research of the topic. • Understanding of commercial grade spirometry sensors. • Market survey of commercial spirometers. • Development of a smartphone application for enhanced maneuver recording. • Development of some methods (such as Hilbert Transform and classification tree generation) • Familiarization with LATEX. 22 Work Plan References AfricanTelecomsNews. Africa Mobile Factbook 2012. Technical report, Africa & Middle East Telecom-Week, 2012. URL http://www.mikekujawski.ca/2012/05/30/ finally-some-2012-statistics-for-the-african-mobile-phone-market/. Andrew Brimer, Abigail Cohen, Olga Neyman, Braden Eliason, and Charles Wu. Low Cost Spirometer - DEBUT. Technical report, Washington University in St.Louis, St.Louis, 2012. URL http://s3.amazonaws.com/challengepost/zip_files/production/ 3622/zip_files/Low-CostSpirometerReport-DEBUT.pdf?1338659900. Cisco. Cisco Visual Networking Index : Global Mobile Data Traffic Forecast Update , 2013 – 2018. Technical report, Cisco, 2014. C P Criée, S Sorichter, H J Smith, P Kardos, R Merget, D Heise, D Berdel, D Köhler, H Magnussen, W Marek, H Mitfessel, K Rasche, M Rolke, H Worth, and R A Jörres. Body plethysmography–its principles and clinical use. Respiratory medicine, 105(7):959–71, July 2011. ISSN 1532-3064. doi: 10.1016/j.rmed.2011.02.006. URL http://www.ncbi.nlm. nih.gov/pubmed/21356587. Stefanie Auf der Mauer, Samantha Chan, Peter Chhour, Josh Homa, Vy Pham, and Karthik Pisupati. Mashavu Spirometer Project, 2013. URL https://decibel.ni.com/content/ docs/DOC-5837. S. B. Kotsiantis. Supervised machine learning: A review of classification techniques. informatica 31:249–268, 2007. Eric C Larson, Mayank Goel, Gaetano Boriello, Sonya Heltshe, Margaret Rosenfeld, Shwetak N Patel, and Computer Science. SpiroSmart: Using a Microphone to Measure Lung Function on a Mobile Phone. In 14th ACM International Conference on Ubiquitous Computing, Pittsburgh, Pennsylvania, USA, 2012. ISBN 9781450312240. N. F. Macia. Pneumotachometers. In J. G. Webster, editor, Encyclopedia of Medical Devices and Instrumentation, Vol. 5, chapter Pneumotachometers, pages 367–379. Wiley, New York, 2nd edition, 2006. J. Sunil Rao and William J.E. Potts. Visualizing Bagged Decision Trees. In KDD, pages 243– 246, Providence, Rhode Island, USA, 1997. URL http://www.aaai.org/Papers/KDD/ 1997/KDD97-050.pdf. D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning internal representations by error propagation. In D. E. Rumelhart, James L. McClelland, and CORPORATE PDP Research Group, editors, Parallel Distributed Processing Explorations in the Microstructure of Cognition, volume 1, chapter 8, pages 318–362. MIT Press, Cambridge, MA, 23 24 REFERENCES 1985. URL http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix= html&identifier=ADA164453. Gregg L. Ruppel. Manual of Pulmonary Function Testing. Mosby, 9th edition, 2008. ISBN 9780323052122. Bas van Stein. A Mobile Smart Care platform Home spirometry by using the smartphone microphone. Master’s thesis, Leiden University, Leiden, The Netherlands, 2013.