Active Learning: From Linear Classifiers to Overparameterized Neural Networks

07 Oct 2020 - Arun Balajiee

Talk Speaker: Robert Nowak, UW Madison

Talk Date: 2020-10-07

In this interesting research talk of the week, Dr. Nowak introduced the growing field of research in developing heuristics for Active Learning – a branch of Machine Learning that concentrates on developing tools to be make more accurate models by allowing human interaction in the process of prediction by a ML model. In this talk, Dr. Nowak introduced some of his work from the past 15 years of reach and the more recent work as well. Specifically, he talked about a MaxiMin heuristic function to introduce allow active label selection at decision boundaries of the model. Further, he also talked about using this heuristic in developing a model that works in non-parameterized setups. All this was discussed in the context of image classification with labeled and unlabeled dataset. In summary, his contributions would be the development of theory and methods for active learning in linear classifiers and the complete understanding of this. Also, development of fundamental notion that classical theory woulnd’t be useful in developing new methods for active learning in non-parameterized. Finally, a new framework for active learning in a non-parameterized settings. Some of hte open challenges would be to make these theoretical implementations into more practical setups and make the Maximin heuristic into a more efficient method.

The key takeaways for me from the talk would be to be able to develop natural language annotation techniques that could not just depend on a human annotator expert for labelling the dataset, but also use active learning heuristics to be able to selectively learn form the sampel dataset, reducing the dataset size and training time greatly. Further, the aspect of interactive nature of the model itself seems like a concept that could be explored in depth in the field of HCI. I think there are possible directions in this area which I will actively look at by reading the papers written by Dr. Nowak to understand this topic in more detail. Dr. Nowak also recommended some reading material at the end of his talk, which I could possibly read to further understand this. Another takeaway from the presentation would be his style of presentation. The beginning of the talk introduced the topic to even a person completely new to the topic – however it didn’t dwell too long before spending equally sufficient amount of time in taking the audience through a roller coaster ride of statistical and mathematical theory behind the models developed by Dr. Nowak and his collaborators in an equally interesting way for a person completely new to the topic. Finally, Dr. Nowak drove home the points to his contributions and what the audience could take away from his talk as well. Throughout the presentation, Dr. Nowak was aware of the complexity of the topic and took excellent care in unravelling it just right amount so keep the interest going, but also giving just enough information so that the audience could also actively think about the problem and come up with many thoughts comments and questions. A clear and concise topic well presented!