Radio Deep Learning Efforts Showcase Presentation November 2016 hume@vt.edu www.hume.vt.edu Tim O Shea Senior Research Associate
Program Overview Program Objective: Rethink fundamental approaches to how we do radio signal processing from a data-centric machine learning driven perspective provide large steps forward in system performance, generalization and autonomy! High Level Motivation: We have seen major changes in how signal processing is done in the vision, voice and text fields over the last 5 years due to major advances of techniques in the field of deep learning Learn from and adopt many of these methods to greatly advance our radio capabilities! Move from expert based features to learned features Move from rule based control system to learned control systems Focus on methods which generalize well and learn directly from data Leverage architectures which work well in vision/voice and adopt them to the radio domain Convolutional networks, Recurrent networks, Attention Models, Deep Reinforcement Learning Program Personnel Tim O Shea, Research Faculty Kayla Brosie, Graduate Student Seth Hitefield, Graduate Student
Technical Focus Areas Specific Focus Areas Signal Sensing and Spectrum Awareness Signal Detection and Separation Signal Modulation and Protocol Identification Control System Learning and Resource Optimization Anomaly Detection and Semi-Supervised Learning of Emitters Learning to Communicate Learning New Physical Layer Representations for Radio Transport Learning from Sparse Structure on Legacy Physical Layers Network Optimization Methods Hyper-parameter Optimization, architecture search methods Network parameter learning, probabilistic alternatives to back-prop Current deep dive areas
Signal Modulation and Protocol Classification Dataset Generation GNU Radio based labeled synthetic dataset with known ground truth and realistic channel effects (fading, Doppler, noise) Working to expand modulations, variation and channel effects More information available from GRCon Paper and at http://radioml.com http://pubs.gnuradio.org/index.php/grcon/article/view/11 Modulation Recognition Demonstrated Conv-Net based classifier learning on raw RF data which outperforms most current day expert classifiers With some tuning, complexity vs state of the art DARPA expert system is 4-6x lower in FLOP count for forward classification (no on-line learning) Full paper @ https://arxiv.org/abs/1602.04105 Looking at leveraging more recent advances from vision domain
Signal Modulation and Protocol Classification Semi-Supervised Learning How can we learn from non-labeled radio data? Since this is most of the world? Organize and identify classes autonomously based on sparse representations and clustering methods Bootstrap from similar supervised learned features and unsupervised feature learning Raw RF Anomaly Detection Applications of reconstruction-based anomaly detection to raw wide-band radio data to detect anomalies (malicious users, interference, failures, etc) Leverage recurrent sequence prediction models in place of Kalman predictor In error vector distribution based novelty detection method
Hyper-Parameter Optimization Methods Hyper-Parameter Search In neural networks we typically have two kinds of parameters to define the network Weight Parameters: Used in neuron to define the transfer function from input to out, tuned using back-propagation of loss function through network gradients Fully connected layer weight matrix, bais matrix Convolutional layer filter weights Recurrent layer weights and biases Hyper-Parameters: Chosen to define how the network is structured typically before training begins, defines how many actual parameters gradient based training must optimize Network depth, width, layer types, connections, activation functions, convolutional filter configurations, weight initializations, learning rate, dropout rate, regularization rates Since gradient descent for a single model can be very expensive we need a lot of concurrency to be able to efficiently search the parameter space and the hyperparameter space Ongoing work on developing an open distributed architecture for performing hyper-parameter searches https://github.com/osh/dist_hyperas Runs on top of Keras (Theano and TensorFlow backends) Meta-Model + Dataset Hyper-Param Search Controller Node (SGD Model Optimization) Performance Ranking Node (SGD Model Optimization) GPU GPU GPU GPU