Paper Info
Title
Fingerspelling Alphabet Recognition Using Cnns With 3d Convolutions For Cross Platform Applications
Abstract
The proposed communication technology is developed with cross-platform frameworks and consists of two parts: modeling and recognition of Ukrainian dactyl signs. Modeling is performed using realistic 3d hand model with animations of dynamic gesture and transitions between gesture, developed in Unity3D framework. User is able specify different words in the user interface and adjust number of polygons and step of animation to get satisfying performance. The computations can be done both on the device or in web. Recognition model training and serving is done with Tensorflow, which allows to deploy the model on different devices, including mobile, or to perform the model prediction on server in cloud. The dataset with Ukrainian dactyl signs was collected with 50 persons and 1500 images per each gesture, which allowed to train the model with high enough accuracy and robust in different environment conditions. The model is based on the MobileNetv3 convolutional neural network architecture, and with the optimal configuration of layers and network parameters, also in order to take into account temporal data, 3d convolutions were used. On the collected test dataset, which is 10% of the overall augmented dataset, and is 15000 images, with different light, noise and blurring condition and different personas hands, accuracy of over 98% is achieved.
Year
DOI
Venue
2020
10.1007/978-3-030-54215-3_37
LECTURE NOTES IN COMPUTATIONAL INTELLIGENCE AND DECISION MAKING (ISDMCI 2020)
Keywords
DocType
Volume
Cross platform, Sing language, Dactyl modeling, Mobilenet, Dactyl recognition, Convolutional neural networks
Conference
1246
ISSN
Citations 
PageRank 
2194-5357
0
0.34
References 
Authors
0
4
Name
Order
Citations
PageRank
Serhii Kondratiuk100.34
Iurii Krak203.72
Anatolii Kylias300.34
Veda Kasianiuk400.34