KDD Workshop on Programming Language Processing

August 17, 2021
CodeBERT from EMNLP 2020

I attended this workshop, held as part of KDD 2021 organised by Aussie’s Chang Xu (U.Syd), Siqi Ma (UQ) and David Lo (SMU). Gave my introductory talk (PDF), which was putting deep learning into context and explaining the background to language models and how they could be used. Then Nan Duan (Microsoft Research Asia) told everyone how his 20 person international team had been doing just that for several years. Their work is well advertised on the usual places (ZDNet, The Verge, etc.) and is absolutely amazing stuff. This will revolutionise coding the way deep language models have revolutionised NLP.


  1. Thanks Wray for an interesting talk. Back in 2015-2016 some of us Aussies plotted a full attack on code modeling using deep learning (https://truyentran.github.io/world.html#Software_analytics). However, Australia wasn’t ready for it.

    • Yeah … not uncommon! Great forward thinking by your team, 5 years ahead of your time!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: