Hyperdimensional Representations (Funded in part by grants from the National Science Foundation and the National Institutes of Health). Recent years have seen unprecedented advances in deep learning algorithms for a variety of applications including computer vision, natural language processing, multi-modal representation learning, node labeling and link prediction in social networks, molecular structure classification, among others. However, such methods are data hungry, require extensive hyperparameter tuning, and achieve their superior performance at considerable computational cost, resulting in a large carbon footprint with attendant adverse environmental impacts. Hence, there is an urgent need for computationally efficient and consequently, environmentally more sustainable alternatives to deep learning across a broad range of AI applications.
Against this background. We explore hyperdimensional computing as a brain-inspired alternative to deep neural networks for artificial intelligence and machine learning applications. Hyperdimensional representation maps each object using a sufficiently high-dimensional random encoding to produce a binary or bipolar vector. All computations are performed on the objects thus encoded using simple operations, e.g., element-wise additions and dot products. Computations on HD vectors, because they yield binary or bipolar vectors, can be realized using low-precision, fast, low-power, energy-efficient hardware. Our recent work has resulted in:
- A Hyper-dimensional Graph Learning (HDGL) algorithm. HDGL leverages the injectivity property of node representations of a family of Graph Neural Networks (GNNs) to map node features to the HD space and then uses HD operators such as bundling and binding to aggregate information from the local neighborhood of each node. The resulting latent node representations support both node classification and link prediction tasks, unlike typical deep learning methods, which often require separate models for these tasks. The results of our experiments using widely used benchmark datasets which demonstrate that, on the node classification task, HDGL yields performance that is competitive with the state-of-the-art GNN methods at substantially reduced computational cost. Furthermore, HDGL is well-suited for class incremental learning where the model has to learn to effectively discriminate between a growing number of classes. Our experiments also show that the HD representation constructed by HDGL supports link prediction at accuracies comparable to that of DeepWalk and related methods. Thus, HDGL offers a computationally efficient alternative to graph neural networks for node classification, especially in settings that call for class-incremental learning or in applications that demand high accuracy models at significantly lower computational cost and learning time than possible with the state-of-the-art graph neural network methods.
- A hyper-dimensional causal effect estimation method. Matching is one of the simplest approaches for estimating causal effects from observational data. Matching techniques compare the observed outcomes across pairs of individuals with similar covariate values but differing in their treatment status in order to estimate causal effects. However, traditional matching techniques are unreliable when faced with high-dimensional covariates due to the infamous curse of dimensionality. To overcome this challenge, we propose a simple, fast, yet highly effective approach to matching using Random Hyperplane Tessellations (RHPT). We prove that the RHPT representation is an approximate balancing score – thus maintaining the strong ignorability assumption – and provide empirical evidence for this claim. We report results of extensive experiments showing that matching using RHPT outperforms traditional matching techniques and is competitive with state-of-the-art deep learning methods for causal effect estimation. In addition, RHPT avoids the need for computationally expensive training of deep neural networks.
Work in progress is aimed at developing hyperdimensional representation based algorithms for (i) continual learning (ii) geometric learning; and (iii) multimodal data fusion.