Exploration Techniques in Active Learning in Classification

Active Learning is the process of selectively querying unlabelled data to be classified by an expert for efficient supervised learning on small data to, among other things, reduce labelling costs. While many works are concerned with the notion of uncertainty quantification as a measure to query informative data points, less work has been dedicated to the preceding exploration phase. In this article, we introduce techniques which can be used to explore the unlabelled data before switching to an exploitation algorithm such as query by uncertainty. Besides benchmarking various algorithms, a novel "fake class"-technique is introduced, which is specialized on discovering new classes within the unlabelled dataset, but also generally improves the performance of the algorithms.