Bayesian Models

I’ve given an invited talk at ACML in Beijing November:  see Invited Speakers at the ACML website.

I talked about the state of Machine Learning, contrasting the old with the new, and discuss where we may head next.  Moreover, I gave some warnings about some problems we are currently facing.  PDF slides for the talk are here.  Abstract is given below.  Prof. Jun Zhu (Tsinghua U.) has had some similar ideas so we conferred afterwards.

Several of us from Monash went:  in the picture are Ye Zhu, Wray Buntine, Lan Du, Yuan Jin and He Zhao.

Something Old, Something New, Something Borrowed, Something Blue

Something Old: In this talk I will first describe some of our recent work with hierarchical probabilistic models that are not deep neural networks. Nevertheless, these are currently among the state of the art in classification and in topic modelling: k-dependence Bayesian networks and hierarchical topic models, respectively, and both are deep models in a different sense. These represent some of the leading edge machine learning technology prior to the advent of deep neural networks. Something New: On deep neural networks, I will describe as a point of comparison some of the state of the art applications I am familiar with: multi-task learning, document classification, and learning to learn. These build on the RNNs widely used in semi-structured learning. The old and the new are remarkably different. So what are the new capabilities deep neural networks have yielded? Do we even need the old technology? What can we do next? Something Borrowed: to complete the story, I’ll introduce some efforts to combine the two approaches, borrowing from earlier work in statistics.