Development of the Hardware and Software Complex Controlling Robotic Devices by Means of Bioelectric Signals of the Brain and Muscles
Aim — to develop a hardware-software complex with combined command-proportional control of robotic devices based on electromyography (EMG) and electroencephalography (EEG) signals.
Materials and methods. EMG and EEG signals are recorded using our original units. The system also supports a number of commercial EEG and EMG recording systems, such as NVX52 (MCS ltd, Russia), DELSYS Trigno (Delsys Inc, USA), MYO Thalmic (Thalmic Labs, Canada). Raw signals undergo preprocessing and feature extraction. Then features are fed to classifiers. The interpretation unit controls robotic devices on the base of classified EEG- and EMG-patterns and muscle effort estimation. The number of controlled devices includes mobile robot LEGO NXT Mindstorms (LEGO, Denmark), humanoid robot NAO (Aldebaran, France) and exoskeleton Ilia Muromets (UNN, Russia).
Results. We have developed and tested an interface combining command and proportional control based on EMG signals. We have determined the parameters providing optimal characteristics of classification accuracy of EMG patterns, as well as the speed and accuracy of proportional control. Also we have developed and tested a BCI interface based on motor imagined patterns. Both EMG and EEG interfaces are included into hardware and software system. The system combines outputs of the interfaces and sends commands to a robotic device.
Conclusion. We have developed and approved the hardware-software system on the basis of the combined command-proportional EMG and EEG control of external robotic devices.
1. Delsys® Trigno™ Wireless Systems and Smart Sensors. URL: http://www.delsys.com/products/wireless-emg/
2. Lobov S, Mironov V, Kastalskiy I, Kazantsev V. A Spiking Neural Network in sEMG Feature Extraction. Sensors. 2015;15(11):27894-27904. doi:10.3390/s151127894.
3. Chowdhury RH, Reaz MB, Ali MABM, Bakar AA, Chellappan K, Chang TG. Surface electromyography signal processing and classification techniques. Sensors 2013; 13(9): 12431-12466, DOI:10.3390/s130912431.
4. Lyons KR, Joshi SS. A case study on classification of foot gestures via surface electromyography. Proceedings of the RESNA Annual Conference; 2015 June 10-14; Denver, USA; 2015.
5. Aszmann OC, Roche AD, Salminger S, Paternostro-Sluga T, Herceg M, Sturma A, Christian H, Farina D. Bionic reconstruction to restore hand function after brachial plexus injury: a case series of three patients. Lancet 2015; 385(9983): 2183-2189, DOI:10.1016/S0140-6736(14)61776-1
6. Nicolelis M.A. Brain-machine interfaces to restore motor function and probe neural circuits. Nat Rev Neurosci 2003; 4(5): 417–422.
7. Kaplan AYa, Lim JJ, Jin KS, Park BW, Byeon JG, Tarasova SU. Unconscious operant conditioning in the paradigm of brain-computer interface based on color perception. Int J Neurosci 2005; 115: 781–802.
8. Kleih SC, Kaufmann T, Zickler C, Halder S, Leotta F, Cincotti F, Aloise F, Riccio A, Herbert C, Mattia D, Kubler A. Out of the frying pan into the fire the P300 based BCI faces real world challenges. Prog Brain Res 2011; 194: 27–46.
9. Do AH, Wang PT, King CE, Schombs A, Cramer SC, Nenadic Z. Brain-computer interface controlled functional electrical stimulation device for foot drop due to stroke. Conf Proc IEEE Eng Med Biol Soc 2012: 6414–6417.
10. Sellers EW, Vaughan TM, Wolpaw JR. A brain-computer interface for long-term independent home use. Amyotroph Lateral Scler 2010; 11: 449–455.
11. Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. Brain-computer interfaces for communication and control. Clin Neurophysiol 2002; 113: 767–791.
12. Mironov V, Lobov S, Kastalskiy I, Kazantsev V. Myoelectric Control System of Lower Limb Exoskeleton for Re-training Motion Deficiencies. Neural Information Processing. 2015:428-435. doi:10.1007/978-3-319-26561-2_51.
13. Bortole M, Venkatakrishnan A, Zhu F, Moreno JC, Francisco GE, Pons JL, Contreras-Vidal JL. The H2 robotic exoskeleton for gait rehabilitation after stroke: early findings from a clinical study. J Neuroeng Rehabil 2015; 12: 54, DOI:10.1186/s12984-015-0048-y
14. Singh RM, Chatterji S, Kumar A. Trends and Challenges in EMG Based Control Scheme of Exoskeleton Robots – A Review. International Journal of Scienctific and Engineering Research 2012, 3(8): 1-8.
15. Roche AD, Rehbaum H, Farina D, Aszmann OC. Prosthetic Myoelectric Control Strategies: A Clinical Perspective. Curr Surg Rep 2014; 2: 44, DOI:10.1007/s40137-013-0044-8
16. Ison M, Artemiadis P. Proportional myoelectric control of robots: muscle synergy development drives performance enhancement, retainment, and generalization. IEEE T Robot 2015; 31(2): 259-268, DOI:10.1109/TRO.2015.2395731
17. Myo™ Gesture Control Armband – Wearable Technology by Thalmic Labs. URL: https://www.myo.com/
18. Lobov SA, Mironov VI, Kastalskiy IA, Kazantsev VB. Combined Use of Command-Proportional Control of External Robotic Devices Based on Electromyography Signals. Sovremennye tehnologii v medicine. 2015;7(4):30-38. doi:10.17691/stm2015.7.4.04. (in Russ.).
19. Lobov SA, Krilova NP, Kastalskiy IA, Kazantsev VB, Makarov VA. A human-computer interface based on electromyography command-proportional control. Proceedings of the 4th International Congress on Neurotechnology, Electronics and Informatics. 2016.
20. Kaplan AYa., Shishkin SL, Ganin IP, Basyul IA, Zhigalov AY. Adapting the P300-based brain‑computer interface for gaming: a review. IEEE Trans on Comput Intelligence and AI in Games. 2013; 5(2): 141–149.
21. Blankertz B, Tangermann M, Vidaurre C, Fazli S, Sannelli C, Haufe S, Maeder C, Ramsey L, Sturm I, Curio G, Muller KR. The Berlin brain-computer interface: Non-medical uses of BCI technology. Front Neurosci 2010; 4: 198.
Gordleeva SYu, Lobov SA, Mironov VI, Kastalskiy IA, Lukoyanov MV, Krilova NP, Mukhina IV, Kaplan AYa, Kazantsev VB. Development of the Hardware and Software Complex Controlling Robotic Devices by Means of Bioelectric Signals of the Brain and Muscles.Science & Innovations in Medicine. 2016(3):77-82.
Send an online application form to the publicationSend