Combining two open source tools for neural computation (BioPatRec and Netlab) improves movement classification for prosthetic control
© The Author(s) 2016
Received: 15 April 2016
Accepted: 19 August 2016
Published: 31 August 2016
Controlling a myoelectric prosthesis for upper limbs is increasingly challenging for the user as more electrodes and joints become available. Motion classification based on pattern recognition with a multi-electrode array allows multiple joints to be controlled simultaneously. Previous pattern recognition studies are difficult to compare, because individual research groups use their own data sets. To resolve this shortcoming and to facilitate comparisons, open access data sets were analysed using components of BioPatRec and Netlab pattern recognition models.
Performances of the artificial neural networks, linear models, and training program components were compared. Evaluation took place within the BioPatRec environment, a Matlab-based open source platform that provides feature extraction, processing and motion classification algorithms for prosthetic control. The algorithms were applied to myoelectric signals for individual and simultaneous classification of movements, with the aim of finding the best performing algorithm and network model. Evaluation criteria included classification accuracy and training time.
Results in both the linear and the artificial neural network models demonstrated that Netlab’s implementation using scaled conjugate training algorithm reached significantly higher accuracies than BioPatRec.
It is concluded that the best movement classification performance would be achieved through integrating Netlab training algorithms in the BioPatRec environment so that future prosthesis training can be shortened and control made more reliable. Netlab was therefore included into the newest release of BioPatRec (v4.0).
KeywordsProsthetics Upper limb amputation Machine learning Pattern recognition Neural computation
Performance of machine learning algorithms are constantly compared with one another to improve the classification of motion based electromyographic (EMG) signals in order to control a prosthetic device. Since mechatronically the number of controllable joints has increased and simultaneous control is replacing sequential control, the limiting factor to be improved in the future is the human-machine-interface.
Since this improvement is a challenging task, conventional control strategies for myoelectric prostheses have not changed much over decades . Two bipolar electrodes are placed on one of the amputee’s residual muscles each and a sufficiently high amplitude of the EMG signal triggers a threshold detection method which then activates a prosthetic movement [2, 3]. This conventional control strategy, even though proven to be reliable [3, 4], turns out to be tedious and slow when there is more than one joint to be controlled. To change a currently selected degree of freedom (DoF), e.g. from wrist to elbow, both electrodes need to be activated simultaneously. This manual switching between joints is far from the natural movement of a hand and prostheses users are quickly frustrated with the limitations of their device .
The machine learning approach for prosthetic control makes use of an array of electrodes instead of just two control sites and considers patterns of EMG activation to increase the number of motion classes. EMG pattern classification in its early stages focused mainly on controlling one DoF at a time , which was not very different from the user experience of the conventional control. In the last years, however, pattern recognition has been extended to concurrent classification of motion intent [6–8]. Each activation pattern received by the electrode array around the amputees residual muscles corresponds to either an individual prosthetic motion, or a simultaneous one that involves multiple DoF. This way, the pattern recognition approach enables simultaneous control and thus promotes natural interaction with the environment.
To achieve a high classification accuracy, several types of classifiers for myoelectric prosthetic control have been examined such as artificial neural networks (ANN) [9–11], linear discriminant analysis (LDA) [12–15], support vector machines , k-nearest neighbour clustering  and unsupervised clustering . High classification accuracy is important to prevent misinterpretations of the prosthesis user’s motion intent.
Previous studies proposed a multi-layer perceptron (MLP) classification strategy that outperformed an LDA based approach and improved the state-of-the-art classification . However, the error rate was still high with 5 % which has also been reproduced in this study. In another study, accuracies of 96.3 % have been reached with a Gaussian mixture model (GMM) based classifier compared to LDA with 95.6 % . They also compared GMM to MLP but did not specify the MLP training algorithm and network properties. Since the MLP only achieved an accuracy of only 95.4 %, it suggests they did not use an optimized training algorithm such as scaled conjugate gradient (scg). Another research group achieved accuracies of up to 97.6 % comparing a self enhancing linear approach with a standard LDA that reached 94.1 % accuracy , which corresponds to the LDA accuracy achieved within this study. However, because both groups used their own data set which is not accessible, the results are not directly comparable.
This study aims to evaluate and compare the performance of four open-source, Matlab-based pattern classifiers taken from BioPatRec and Netlab on correctly categorizing offline EMG signals across different movements.
The classifiers used in previous studies by BioPatRec are (amongst others) LDA and MLP. Therefore this study extends the previous studies [7, 20] using optimized algorithms for MLP and generalized linear models (GLM) taken from the Netlab toolbox to improve classification accuracy. Regularly, many studies use their own data set to test algorithms, which limits generalization of their results across methods. To grant comparability across methods, this study works on the same openly available data set .
BioPatRec is a useful environment for pattern classification already operated in a clinical setting . Because training the classifier took very long and the resulting accuracy looked improvable, other options to increase classification accuracy were considered. To keep the initial set-up, these other options should consequently be implemented into the BioPatRec environment. The goal of this study is to find the algorithm with the highest classification accuracy and least computational complexity and therefore lowest training time that increases the performance of a myoelectric prosthetic controller. Since the newest release of BioPatRec, Netlab has been successfully integrated.
A linear and non-linear classifier each, taken from both toolboxes BioPatRec and Netlab were compared with one another regarding offline movement classification performance and training time. The fastest algorithm within BioPatRec was LDA, which also is often used by researchers. The classifier reaching the highest accuracy within BioPatRec was MLP. Both classifiers were compared with Netlab’s highly optimized training algorithms iteratively reweighted least squares (irls) and scaled conjugate gradient for GLM and MLP.
The main benefit of linear methods such as LDA and GLM is their low complexity and quick training [5, 12–14, 23]. Artificial neural networks such as MLP depend on specific training algorithms and are inherently capable of simultaneous predictions. They can still be cost effective despite their increased complexity [10, 11, 19, 24].
To enable repeatability, the algorithms were compared using an open access data set from the BioPatRec data repository. This set has already been used for comparing different classifiers in previous evaluations [25, 26]. Three DoF are available for motion classification and used for individual and simultaneous control strategies, resulting in 26 possible movement labels and one no-movement label, in which no intentional EMG signal occurs. The resulting algorithm accuracy reflects the correspondence of the instructed user motion intend to the recognized movement by the classifier. The higher the accuracy the more reliable is the movement recognition. It is calculated by the number of correct classifications divided by the overall number of performed classifications.
Netlab is an open source software toolbox with highly optimized training algorithms for data analysis, neural computation and neural network simulations which requires the Matlab environment to run [27–29]. The library includes a variety of implementations for data analyses and neural network simulations.
BioPatRec is an open source Matlab-based research environment for development and evaluation of pattern recognition algorithms for prosthetic control. Matlab’s statistics toolbox is required. It provides tools for data acquisition, signal processing, feature selection and extraction, pattern recognition and real-time control and was developed by one of the authors—Max Ortiz-Catalan .
A. Data source and treatment
The data set which was used for comparing classification consists of 17 EMG recordings, one from each of the 17 healthy participants (mean age of 25.9 ± 4.9 years) and is available within the BioPatRec data repository . Data was recorded by eight bipolar surface electrodes attached around the forearm. From each of the eight electrode channels, four time domain signal features (mean absolute value, waveform length, zero crossing and slope sign change) were extracted from 121 fixed time windows for each movement (200 ms with a time increment of 50 ms) which are used as feature vectors to feed the classifiers . During recording the participant received a visual indication to perform a specific movement. Each movement was repeated 3 times with a recording time of 3 s. The first and last 15 % of that recording were removed to make sure only isotonic contraction will remain. Thus, a total of 121 windows were generated per movement. Out of these 121 windows per movement, 72 were randomly selected as training set and 49 windows were set aside as a validation set to evaluate generalization after the training phase and were not previously used for training. Performance accuracy was averaged over 100 iterations per subject (cross-validation), with a different random splitting into training and test set. All data was pre-processed and treated within BioPatRec.
Without normalization, features with different standard deviations would be weighted differently, error surfaces would be distorted and learning algorithms would not sufficiently converge. As a preliminary measure and because BioPatRec offers the possibility to choose among different normalizations, two commonly used linear normalizations in data processing were compared across all models: standard normalization \(\mu\) = 0, \(\sigma\) = 1 and a transformation to [−1;1] called “midrange 0 range 2”. The comparison table can be found in the supplementary material. Due to equal or better performance of the standard normalization all data was treated with standard normalization.
Netlab is able to use multiple processor cores with an average workload of 80–100 % on a quadcore CPU. BioPatRec MLP uses one processor core with an average workload of 25 % on a quadcore CPU. The PC used in this study has an i5-2500K processor running at 3.3 GHz when multiple cores are occupied, and at 3.7 GHz when just one core is active.
B. Movement classification
There are 27 possible labels for classification which derive from six individual hand and wrist movements as well as their simultaneous combination and an additional no-movement state the participants were instructed to perform. The individual movements were hand open/close, hand flexion/extension and wrist pronation/supination. Possible output types are (a) one single active unit, that can either represent one individual movement or a simultaneous movement consisting of a combination of individual movements; and (b) multiple active units in which each output unit represents one individual movement. Simultaneous movements are a combination of multiple output units.
C. Network architecture of BioPatRec and Netlab
BioPatRec was used as a platform to execute both its own inherent classifiers (LDA and MLP) and the additionally implemented Netlab classifiers (GLM and MLP). Both models employ the two forms of output types mentioned above (single and multiple), that are implemented in Netlab with softmax and logistic output functions respectively. The advantage of using a softmax output function is an increase in accuracy, when there is a single classification problem.
Linear classification method of BioPatRec and Netlab BioPatRec’s LDA is taken from the Matlab statistics toolbox. Because this LDA can not generate multiple outputs as a single classifier, a BioPatRec inherent multi class problem approach was used that consists of one LDA for each degree of freedom. The individual LDAs possess four output neurons, two are for classifying the agonistic and antagonistic movement of the respective DoF, one is for every other movement and the last one is for a no-movement state.
The Netlab GLM uses scaled conjugate gradient or iteratively reweighted least squares as training algorithm. Regarding multiple outputs, Netlab GLM also uses the same configuration as BioPatRec’s LDA with one net for each degree of freedom and four output units.
Non-linear classification method of BioPatRec and Netlab The BioPatRec MLP is a feedforward artificial neural network (ANN) with backpropagation as supervised learning and gradient descent as training algorithm with a logistic activation function . The standard configuration of BioPatRec consists of 32 input neurons, two layers of 32 hidden neurons each and a variable number of output neurons corresponding to the number of classes [7, 25]. However, the number of hidden layers as well as hidden units (HU) can be adapted. For comparison, this study employs one and two hidden layers with 32 hidden neurons each and the number of neurons as evaluated by cross validation for each model.
The Netlab MLP is also a feedforward ANN, but with scaled conjugate gradient as training algorithm. It consists of 32 input neurons, one hidden layer with 32 hidden neurons and variable output neurons. Because the Netlab architecture is based on Bishop , only one hidden layer is available. Output function used by the MLP was either softmax or logistic with a 0 to 1 output range.
Both training algorithms are limited to a maximum number of 200 iterations.
The Wilcoxon Sign-Rank test with a significance level of \(\alpha\) = 0.05 is applied to all algorithms and their respective parameters to evaluate significance for each movement classification accuracy. The mean accuracy (Acc), standard deviation (SD) and p value of the models, as well as training time in seconds, will be given in the tables. The SD was calculated over the averaged iterations and across subjects and movements. Although training time is not regarded as an important aspect for offline classification, short training time has advantages in practical applications such as immediately available prosthesis control without a delay caused by the computing training algorithm. Fast training would also be a requirement for prospective online adaptive learning.
A. Performance comparison of linear models
Comparison of linear models
Netlab GLM netopt (scg)
Netlab GLM train (irls)
B. Optimal number of hidden units
Because of time concerns, hidden unit evaluation was performed for Netlab. The Wilcoxon Sign-Rank test evaluated if an increase in the number of hidden units significantly improved the accuracy. The smallest number of hidden units out of 100 maximum permissible, for which no significantly higher accuracy could be observed are: (a) 64 hidden units for a single output type and (b) 74 hidden units for the multiple output type. However, from 30 to 40 hidden units on, the accuracy only slightly increases. The peaks seen on the curves cannot be attributed to statistical error but are due to systematic variance between the number of hidden units.
As the network for the multiple output (74 HU) problem is more complex than for single output (64 HU) it could be assumed that the multiple output problem is more complex.
C. Performance comparison of neural networks
Performance comparison of non-linear models and optimal number of hidden units (HU)
2 × 32 HU
64 /74 HU
64 /74 HU
E. Training time
Training time (in s)
BioPatRec (gradient decent)
MLP 2 × 32 HU
MLP 64 HU
MLP 74 HU
MLP 64 HU
MLP 74 HU
Netlab’s GLM with irls training algorithm performed significantly better than BioPatRec’s LDA and still maintained short training time. The performance accuracies of BioPatRec’s MLP are best with two hidden layers of 32 neurons. However, Netlab’s MLP consisting of one layer of 32 hidden neurons performed significantly better. During testing of different training algorithms for MLP which all had a maximum number of iterations of 200, the results showed that the scg learning algorithm for MLP was superior in accuracy. These findings correspond to the results of previous benchmarkings of scg against the standard backpropagation, the conjugate gradient backpropagation and the quasi-Newton algorithm . The Netlab MLP with optimized number of hidden units achieved 98.7 % accuracy compared to 95.1 % of the standard BioPatRec MLP, which in terms of error rate is an improvement from 4.9 to 1.3 %. When comparing normalizations for preprocessing standard normal distribution was found to be superior to a transformation to mid-range of 0 and range of 2.
Multiple output types do not work well with linear methods because they seem to require non-linear separation (as can be seen in the classification accuracies) and single output types perform well with linear methods, however, non-linear algorithms are still considerably superior. Single output type always performs better than multiple output type (with BioPatRec MLP being the only exception). This suggests that the single output type is a more simple solution to the same problem, although multiple output type has the advantage to recognize combined movements with only having the single movements available. Future research could focus on the issue of recognizing combined movements after training only single movements.
Regarding training time, Netlab was possibly faster because it is a sophisticated toolbox especially optimized for neural computation and its utilization of hyperthreading.
However, it is important to not only look at offline performances, but also to take into consideration that several factors challenge the robustness and reliability of pattern recognition algorithms in real-time. Electrode condition and displacement as well as sweat and muscle fatigue influence conductivity of the electrodes [31–34]. Therefore this study’s results will be incorporated into further movement classification assessments in real-time.
Integrating Netlab’s efficient training algorithms for artificial neural networks and linear models into the BioPatRec environment resulted in an improvement of offline classification accuracies and training time. The Netlab toolbox for neural computation has been successfully implemented into the newest releaese of BioPatRec (v4.0).
artificial neural network
linear discriminant analysis
multi layer perceptron
Gaussian mixture model
general linear model
scaled conjugate gradient
iteratively reweighted least squares
All authors contributed equally to this article. All authors read and approved the final manuscript.
The authors would like to thank the Technical University of Vienna for funding this publication, and the Christian Doppler Forschungsgesellschaft.
The authors declare that they have no competing interests.
Availability and requirements
Project name: BioPatRec
Project homepage: https://github.com/biopatrec
Data Repository: https://github.com/biopatrec/biopatrec/wiki/Data_Repository.md
Operating system: Windows 7+
Programming language: Matlab
Other requirements: Matlab Statistics Toolbox
Any restrictions to use by non-academics: no
The data sets supporting the results of this article are available in the BioPatrec repository at GitHub: https://github.com/biopatrec/biopatrec/wiki/Data_Repository.md.
Consent to publish
All authors give their consent to publish.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Parker P, Scott R. Myoelectric control of prostheses. Crit Rev Biomed Eng. 1985;13(4):283–310.Google Scholar
- Fougner A, Stavdahl O, Kyberd P, Losier YG, Parker PA. Control of upper limb prostheses: terminology and proportional myoelectric control-a review. IEEE Trans Neural Syst Rehabil Eng. 2012;20(5):663–77.View ArticlePubMedGoogle Scholar
- Roche A, Rehbaum H, Farina D, Aszmann O. Prosthetic myoelectric control strategies: a clinical perspective. Curr Surg Rep. 2014;2(44):1–11.Google Scholar
- Scott R, Parker P. Myoelectric prostheses: state of the art. J Med Eng Technol. 1988;12(4):143–51.View ArticlePubMedGoogle Scholar
- Scheme EJ, Englehart K. Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. J Rehabil Res Dev. 2011;48(6):643.View ArticlePubMedGoogle Scholar
- Geng Y, Tao D, Chen L, Li G. Recognition of combined arm motions using support vector machine. In: Yang D, editor. Informatics in control, automation and robotics, lecture notes in electrical engineering, vol. 133. Berlin: Springer; 2012. pp. 807–14.Google Scholar
- M. Ortiz-Catalan, R. Branemark, and B. Hakansson, Evaluation of classifier topologies for the real-time classification of simultaneous limb motion, 35th annual international conference of the IEEE engineering in medicine and biology society; 2013.
- Young A, Smith L, Rouse E, Hargrove L. Classification of simultaneous movements using surface emg pattern recognition. IEEE Trans Biomed Eng. 2013;60:1250–8.View ArticlePubMedGoogle Scholar
- Hudgins B, Parker P, Scott RN. A new strategy for multifunction myoelectric control. IEEE Trans Biomed Eng. 1993;40(1):82–94.View ArticlePubMedGoogle Scholar
- Amsüss S, Paredes LP, Rudigkeit N, Graimann B, Herrmann MJ, Farina D. Long term stability of surface emg pattern classification for prosthetic control. In: 35th annual international conference of the IEEE engineering in medicine and biology society (EMBC); 2013, pp. 3622–3625.
- Amsüss S, Goebel PM, Jiang N, Graimann B, Paredes L, Farina D. Self-correcting pattern recognition system of surface emg signals for upper limb prosthesis control. IEEE Trans Biomed Eng. 2014;61(4):1167–76.View ArticlePubMedGoogle Scholar
- Englehart K, Hudgins B. A robust, real-time control scheme for multifunction myoelectric control. IEEE Trans Biomed Eng. 2003;50(7):848–54.View ArticlePubMedGoogle Scholar
- Huang Y, Englehart KB, Hudgins B, Chan AD. A gaussian mixture model based classification scheme for myoelectric control of powered upper limb prostheses. IEEE Trans Biomed Eng. 2005;52(11):1801–11.View ArticlePubMedGoogle Scholar
- Huang H, Zhang F, Sun YL, He H. Design of a robust emg sensing interface for pattern classification. J Neural Eng. 2010;7(5):056005.View ArticlePubMedPubMed CentralGoogle Scholar
- Chen X, Zhang D, Zhu X. Application of a self-enhancing classification method to electromyography pattern recognition for multifunctional prosthesis control. J Neuroeng Rehabil. 2013;10(1):1.View ArticleGoogle Scholar
- Oskoei MA, Hu H. Support vector machine-based classification scheme for myoelectric control applied to upper limb. IEEE Trans Biomed Eng. 2008;55(8):1956–65.View ArticlePubMedGoogle Scholar
- Peleg D, Braiman E, Yom-Tov E, Inbar GF. Classification of finger activation for use in a robotic prosthesis arm. IEEE Trans Neural Syst Rehabil Eng. 2002;10(4):290–3.View ArticlePubMedGoogle Scholar
- Momen K, Krishnan S, Chau T. Real-time classification of forearm electromyographic signals corresponding to user-selected intentional movements for multifunction prosthesis control. IEEE Trans Neural Syst Rehabil Eng. 2007;15(4):535–42.View ArticlePubMedGoogle Scholar
- Ortiz-Catalan M, Hakansson B, Branemark R. Real-time and simultaneous control of artificial limbs based on pattern recognition algorithms. IEEE Trans Neural Syst Rehabil Eng. 2014;22(4):756–64.View ArticlePubMedGoogle Scholar
- Ortiz-Catalan M, Branemark R, Hakansson B. Biopatrec: a modular research platform for the control of artificial limbs based on pattern recognition algorithms. Source Code Biol Med. 2013;8(11):1.Google Scholar
- Ortiz-Catalan M. Biopatrec, https://github.com/biopatrec/biopatrec/wiki. 2015.
- Ortiz-Catalan M, Sander N, Kristoffersen MB, Hakansson B, Branemark R. Treatment of phantom limb pain (plp) based on augmented reality and gaming controlled by myoelectric pattern recognition: a case study of a chronic plp patient. Front Neurosci. 2014;8:246.View ArticleGoogle Scholar
- Zhang H, Zhao Y, Yao F, Xu L, Shang P, Li G. An adaptation strategy of using lda classifier for emg pattern recognition. In: 2013 35th annual international conference of the IEEE engineering in medicine and biology society (EMBC); 2013. pp. 4267–4270.
- Caetano D, Mattioli F, Nogueira K, Lamounier E, Cardoso A. Comparison between mlp and lvq neural networks for virtual upper limb prosthesis control. In: International conference on intelligent data engineering and automated learning. Berlin: Springer; 2012. pp. 382–389.
- Ortiz-Catalan M, Branemark R, Hakansson B. Real-time classification of simultaneous hand and wrist motions using artificial neural networks with variable threshold outputs. In: Proceedings of the XXXIV international conference on artificial neural networks (ICANN); 2013. pp. 1159–64.
- Khong L, Gale D, Jiang J, Oliver J, Ortiz-Catalan M. Multi-layer perceptron training algorithms for pattern recognition of myoelectric signals. In: Biomedical engineering international conference (BMEiCON-2013); 2013. pp. 1–5.
- Nabney I. NETLAB: algorithms for pattern recognition (advances in computer vision and pattern recognition). 3rd ed. Springer; 2008.
- Bishop C, et al. Pattern recognition and machine learning, vol. 1. New York: Springer; 2006.
- Tarassenko L. A guide to neural computing applications. Elsevier; 2004.
- Moller M. A scaled conjugate gradient algorithm for fast supervised learning. Neural Netw. 1993;6(4):525–33.View ArticleGoogle Scholar
- Jiang N, Dosen S, Mueller K, Farina D. Myoelectric control of artificial limbs—is there a need to change focus? IEEE Signal Process Mag. 2012;29(5):150–2.Google Scholar
- Sensinger JW, Lock BA, Kuiken TA. Adaptive pattern recognition of myoelectric signals: exploration of conceptual framework and practical algorithms. IEEE Trans Neural Syst Rehabil Eng. 2009;17(3):270–8.View ArticlePubMedPubMed CentralGoogle Scholar
- Hargrove L, Englehart K, Hudgins B. A training strategy to reduce classification degradation due to electrode displacements in pattern recognition based myoelectric control. Biomed Signal Process Control. 2008;3(2):175–80.View ArticleGoogle Scholar
- Nishikawa D, Yu W, Maruishi M, Watanabe I, Yokoi H, Mano Y, Kakazu Y. On-line learning based electromyogram to forearm motion classifier with motor skill evaluation. JSME Intern J Ser C. 2000;43(4):906–15.View ArticleGoogle Scholar