Archive for the ‘students’ Category

h1

Machine Learning Research Tutorials

March 8, 2020

Machine learning has become one of the hottest areas in computer science and technology. Both industry and academia have gone gaga. Big tech companies send 100’s each to the top research conferences and the conference numbers are increasing in size so they are now beyond capacity. But, how do you learn about machine learning in the first place? Assuming you have a strong STEM undergraduate degree and are research savvy, this page points to some appropriate resources for research. These are intended for starting PhD students.

If you are more interested in learning the basics as a potential user, then you will need to find different resources such as the blogs up on https://medium.com/search?q=machinelearning&ref=opensearch or at the MOOCS such as Coursera.

University Classes

Places like Stanford and CMU have very good advanced masters-level classes ideal for starting PhD students. Slides and oftentimes lectures are online for the public. e.g., deep networks for NLP http://web.stanford.edu/class/cs224n/

See also Lex Fridman’s seminars up at https://deeplearning.mit.edu/ . Very good overview of capabilities and directions for a general overview.

Good Venues

Excellent tutorials are available recently at the major conferences, oftentimes with vidoes and/or slides on the website, although sometimes you have to hunt through the author’s webpages. The top conferences include AI&Stats, IJCAI, ICML, ACL … be warned, some tutorials are a bit specialised or advanced.

Machine Learning Summer School (MLSS)

This series is managed by venerable machine learning researchers and only has a few per year internationally. Their list of venues is at http://mlss.cc/ . You have to go to each and navigate disparate and sometimes wacky layouts to locate slides and/or videos.

AutoML

The Freiburg-Hannover group has a great sequence of tutorials on AutoML and learning to learn:

VideoLectures.net

An initiative of the Jozef Stefan Institute, Ljubljana, records many great tutorials, but coverage not as good recently. Go seaching for your favorite subjects:

Others

h1

Resources for Research Students

March 1, 2020
Monash FIT Postgrad Workshop

On 12th February, Reza Haffari and I organised a workshop to support research students in their journey. It seems our Monash faculty each has their own special superpower, and their quality and relevance blew me away:

  • Maria Garcia De La Banda was a careers expert
  • Christoph Bergmier was an efficiency expert
  • Reza Haffari pondered the big philosophical questions
  • Dinh Phung thought about being the best researcher you could be

Anyway, they presented lots of good material which is on the Monash hard drives. But here I’ve summarised the main resources we all mentioned.

Famous Tutorials/Papers/Books for Research Students

Living productively

(resources from Profs Dinh Phung and Reza Haffari at Monash FIT)

Writing

  • Strunk & White’s “Elements of Style”, a short book summarising good writing, with 5 editions since 1920, considered one of the 100 best English language books ever written, often bought with “On Writing Well”
  • William Zinsser’s “On Writing Well”, a classic guide to writing non-fiction, often bought with Strunk & White
  • Jacque Barzun’s “Simple & Direct”, a writer’s guide, https://www.amazon.com.au/Simple-Direct-Jacques-Barzun/dp/0060937238

Research writing (including for nonnative speakers of English)

(resources from Julie Holden at Monash FIT)

  • Cargill, Margaret, and Patrick O’Connor. Writing scientific research articles: Strategy and steps. John Wiley & Sons, 2013.   ( an updated version is being currently written)
  • Glasman-Deal, H. (2010). Science research writing for non-native speakers of English. London: Imperial College Press. Hargrave-Andrew Library 808.0665 G548S2010 and Monash University Library ebook)
  • Swales, J., and Feak, C. (2012). Academic writing for graduate students: A course for nonnative speakers of English ( 3rd ed.).  Ann Arbor: the University of Michigan Press. (Hargrave-Andrew Library 808.042 S971A 2004)
  • Swales, J., and Feak, C. (2000). English in today’s research world: A writing guide. Ann Arbor: The University of Michigan Press. (Matheson Library 808.042 S971E 2000)
  • Weissberg, R., and Buker, S. (1990). Writing up research: Experimental research report writing for students of English. Englewood Cliffs, NJ: Prentice Hall Regents. (Hargrave-Andrew Library 808.0666 W432W)
  • Graff, G., and Birkenstein, C. (2017). “They say / I say”: The moves that matter in academic writing. New York: Norton & Company. (Caulfield and Matheson Libraries 808.042 G736T 2017).  The full text of the 2010 edition is also available to download at https://www.iss.k12.nc.us/cms/lib/NC01000579/Centricity/Domain/2741/They%20Say%20I%20Say%20Full%20Text.pdf

Resources on quality conferences/journals

Understanding reviewers

The journal editorial process 

(didn’t have “known” resources here, so went Googling … these seemed reasonable)

h1

Caitie on Einstein A Go Go

October 27, 2019

Caitie and Wray at 3RRR

Caitie Doogan, through her Twitter activies, got invited to talk on a science radio programme on 3RRR:  Einstein A Go Go – 27 October 2019 (our section starts at 25:15 in). Dr Krystal, Dr Ray & Dr Shane were the three scientists asking really relevant questions.  Caitie’s work is at the interface of applications and machine learning, so is far more accessible to the science public.  I came along for the ride, and discussed my work with Turning Point.

h1

Visiting Zhengzhou University

September 27, 2019

Workshop at Zhengzhou, 2019

Faculty and some students after the workshop, 27/09/19.

Dr. Ming Liu of Deakin organised for Dr. Lan Du and I to give a series of lectures on machine learning and natural language processing at Zhengzhou University in Henan province, from 25-27th September.   I gave versions of some previous talks, as well as presenting a new talk on some of the fundamental principles behind some of the new techniques in deep neural networks like pre-training.

The photo above shows me, with Lan and Ming on my left and Prof. Zan, our principal host, on the right.  Several other Zhengzhou faculty are in the front row and some of the masters + phd students in the back row.   Zhengzhou is a large university with the best students in Henan.  The food was, of course, fabulous.  We had a small dinner with the Dean of the entire Engineering Faculty (seen cut-off, on far left), no less, on Monday night, and I was introduced to the toasting customs of Henan with their tiny 10ml shot glasses!

Dinner at Zhengzhou University

At the faculty club 25/09/19, with the Dean on the far left.  Lan is having his soup and Ming is waving.

h1

Southeast Asia Machine Learning School

July 9, 2019

Very fortunate to be asked to give a lecture on “Foundations of Supervised Learning” at SEA-MLS in Jakarta on 8th July.  The school was co-organised by Google, so opening talk by Google and a member of the Indonesian government.  A big crowd too!  Everyones slides are up on the schedule page, and mine are copied here.

Never been to Jakarta so an exciting opportunity meet some colleagues, some students, in a lovely environment. Monash has a school at Malaysia so a few Malaysia Monash folks turned up too.  Here we are after my lecture.

SEAMLS-Monash2

Monash Malaysia and Melbourne students at SEA-MLS.

h1

Graduating MDS students

May 25, 2018

Our first larger batch of MDS students graduating.   Here are some who attended the ceremony.  Really great students!

MDS_GradMay2018

MDS Graduation May 2018

h1

Trying out DataCamp this semester

February 21, 2018

Our Master of Data Science students explore a lot of things and discuss.  I got a lot of requests to include the excellent material from DataCamp:

DataCamp logo

DataCamp – who support data science education for free

So we’ll see how it goes.  Not sure how well I’ll get to integrate it, because this semester I’m working more on our introductory statistics class.

h1

Good Health for Students!

January 10, 2018

So enrolment sessions start soon for our incoming Master of Data Science students.  I know its a stressful time for some students in terms of “life”.  I usually talk briefly about staying healthy, and Monash offers various services to support this.  But for PhD students I think its important to take this on as a lifestyle objective.  They are undertaking a knowledge intensive career path and brain health will be critical for their future career.

Disclaimer:  Now, this page is full of opinions and pointers to, in some case, controversial material.  I’m just a little old computer science professor, so my opinions have no real backing, and I have no recognised expertise. All care but no responsibility for what I say! 

Just to put things into perspective, here’s me after shopping one Saturday.  What is missing?  Packaged food!  Rarely buy it except for pasta, spices, etc.  If you can whip up a good salad, home made vege soup, a great pasta dish, or breakfast eggs in 5 minutes, its the best.  Read more to find out.

One week's shopping.

Taken in Canberra, about 2013 I think. Canberra had a great farmer’s market just one suburb away from me.

The fact is, keeping healthy and understanding how to keep healthy in the modern world is a subject fraught with challenges.  To understand this, consider the following:

  1. The official Australian government position on colds and flu prevention, and the official USA government position:  hygiene and vaccines.  What’s missing:  discussion of healthy diet, exercise and other lifestyle factors to strengthen and repair the immune system.
  2. The Time magazine reports extensive research shows vitamin D helps prevent colds and flu, so some sunshine is also important.  No mention of this in the official government positions above!
  3. Believe it or not, in the USA prescription drugs are the third leading cause of death!  There is a larger issue here in that most published medical research findings are false.  Note, I see this is a systemic thing not limited to medical research.  However, the medical research community has extensive, concerted efforts like systematic reviews to address the issue.
  4. Tobacco science is a term used to describe fake science protecting an industry.  Read about the Tobacco Institute and see the movie The Insider.  How much of this goes on in the food and drugs industry?  Lots according to Nina Teicholz, see point 8 below.
  5. Sugar is now known to be very damaging to the health.  Here is a hard hitting discussion about it, though note quite a few of these claims are considered controversial.  But it is known that sugar suppresses the immune system.  Figuring out your sugar consumption is challenging.   There are rumors (in movie form) of tobacco science going on here too.
  6. Energy drinks rot the teeth, like soft drinks.  Its due to the high acid content.  Its certainly not clear they give any energy.
  7. Artificial sweeteners are not a substitute, in fact evidence suggests they have poor health impacts, and they mess up the brains analysis of your food intake.
  8. Fats are the subject of a massive onslaught from advertisers.  For years we were told to avoid butter and use margarine instead, but now it seems butter is good, or at least no worse than a good margarine.  The rather hilarious and utterly confusing history of health advice about butter is in this Butter Studies Roundup. Current conflicting advice is now being broadcast about the humble coconut.
  9. See Nina Teicholz’s talk about the complex history of our understanding of fats in her TEDx seminar, based on her best selling book, “The Big Fat Surprise”.  She is now on a crusade to get scientific results into government dietary guidelines.  Is she right?  Well, its all too detailed for us mere observers to really know, so we have to suffer being jossled around by deceptive techniques of propaganda (“appeal to authority”, etc.) used in heavy doses by many concerned.
  10. The health of organic produce is currently a propaganda battleground.   None other than former tobacco scientist Henry I Miller (he was a founder of TASSC) has claimed its an expensive scam.  Hint:  organics are also lower in toxic pesticide residue, which is why I would get them.
  11. The commercial world has taken on healthy eating big time, and it is the fastest growing segment of the food industry.  Monash University has done a wonderful job of getting really good fast food vendors at the Caulfield campus food court.  If you’re an old time traveller, you should also be aware of the huge changes in international airport food courts and tourist spots like London when it comes to healthy eating options.

Summary:  There is lots of conflicting and bad advice out there!  Heck, even the government websites seem to have errors of omission.

Now, if we consider the specific position of someone who wants their brain to function well, then consider the following:

  1. Exercise is known to boost mental performance, and boost a whole lot of other things including mood.
  2. Meditation and mindfulness is also known to boost performance in exams.
  3. Long term sitting is considered to be as bad for health as smoking!  Here is a poster of the dangers.
  4. There are also lifestyle recommendations about studying from scientists:  don’t cram for subjects, learn slowly over the semester.
  5. Recent studies show the brain can be encouraged to grow new cells, with advocates claiming fasting and different kinds of exercise can help.
  6. The brain is mostly fat, so we need healthy fats to work well.  Don’t believe a lot of what you read about fats!  Cholesterol, for instance, is important for maintaining the brain.
  7. Sugar consumption (e.g., soft drinks, commercial juices, commercial cereals, flavoured yogurts, etc. etc. etc.) is bad for the brain, as well as the immune system.
  8. Canola oil is bad for the brain.  This one is important because most cheap salad oils, margarines and many food products are loaded with it.
  9. All sorts of chemicals and medicines can be bad for the brain.  Here’s a TEDx talk on details.  Note TEDx means not official (is this a reliable information source?).
  10. Deep sleep is the basis for memory, learning and health.   In particular, without deep sleep, your brain will not be functioning properly and your memory will be impaired.  Here is a disquieting Google talk on health and sleep (along the lines of the hideous anti-smoking adverts some countries have), but there are many more on this.
  11. Adult neurogenesis is the process by which we adults gain new brain cells.  Not surprisingly, this is very popular amongst the Silicon Valley crowd, and I suspect is also a domain where snake-oil salesman like to peddle.  Nonetheless, here is a video on it:  a TED talk.

Note, for each of these, there are 10’s-100’s of good articles and scientific literature to back it up, though oftentimes conflicting scientific literature as well.  I’m just giving generally readable and somewhat respectable accounts.  A lot of these issues remain controversial, and possibly there is some tobacco science going on, but its hard for us non-experts to really know.

Anyway, I hope from this you understand the complexity of trying to stay health, and trying to keep your brain functioning well in the modern world.

Thoughts on eating and food:

  • Benjamin Franklin said “An ounce of prevention is worth a pound of cure.” and my Dad lived by it.  I agree heartily.   So eating well and living a healthy lifestyle is better than loading yourself up with drugs to maintain performance.

  • Try and cook your own meals from real ingredients.  After a while, it becomes easy and its a great way to wind down with friends.
  • If someone’s great grandmother (anyone’s, Fiji, Vietnam, Sweden, …) didn’t make the food 100 years ago, its probably not good for you.
  • Don’t take dietary or health advice from Big Food.  In fact, looking at the government advice (listed above) on the flu, I’d say their’s is missing some major points too for some issues.
  • Try and avoid packaged meals, fast food, and canned and bottle drinks.  Likewise, avoid most commercial fruit juices which have way too much sugar and have lost too much of the fabulous nutrients in the original fruit due to being pasteurised or reconstituted.
  • Go low sugar, low refined carbohydrates and healthy fats.  Its a lifestyle thing, not a diet.  Once you do, you’ll discover all the amazing subtle flavours you’ve missed from traditional foods and realise how horrible standard breads, sweet deserts, snack bars and cakes really are:  the sugar masks the real flavour, and it gives you a longer term bad after taste.  Refined carbohydrates have removed a lot of the flavours and its replaced it with a few bland substitutes.
    • Healthy fats is challenging to maintain because Big Food likes to put unhealthy canola oil in everything:  most salad oils, hummus and deli mixes are mostly canola oil, as is margarine.
    • Well made healthy bread is truly remarkable in flavour, but it costs more and you cannot find it at the big chains.  Have it with a thick spread of (grass-fed) Tasmanian or New Zealand butter.  Nothing better!
  • Just avoid artificial sweeteners.  Once you’ve gone cold turkey and got off the sugar addiction you wont be craving sugar and you’ll feel better for it.
  • Health slogans on food products, “low fat”, “low cholesterol” often mean its bad for you!   Low fat usually means high sugar, for instance.

About other aspects of health:

  • Get exercise, and make it a lifestyle thing.  When you’re older, you’ll discover you cannot function well as a knowledge worker without it.
  • Don’t sit at your desk for long hours.  You need to get up and move around every hour!  Also, become aware of your posture.   Don’t become a hunchback!  Some 2nd year masters students are already heading that way.  When I go to conferences and chat with the other grey-haired wrinkly folk of my era, its scary to see how many have really bad backs.
  • When you’re mentally worn out, a quick nap or a brisk walk does wonders, and both have scientific backing.
  • Make sure you are getting proper sleep.  That can mean organising your assignments and study properly so you don’t need a to do a bunch of all-nighters to get through.  But it also means setting up the right environment at home for sleep.
  • I know of few cases where drugs or alcohol support good health or brain functioning, including many so-called smart drugs.  Most are dangerous to the liver, as are many medicines.  Headache and pain medicine is far more dangerous and damaging than many other things!  Good food, exercise and sleep is how you increase performance.
  • Note there is a whole field of nootropics which is emerging as an alternative therapy (not condoned by medical science).  Human biology is extremely complex, and the scientific method is fairly crude as a developmental model of knowledge, especially when its constantly interfered with by commercial interests.  I expect some good natural supplements to eventually emerge here where they naturally enhance human biology.
  • Routine … that’s what the body needs.  For sleep, for eating, for study, for exercise, routine is critical part of making it function well.

Anyway, I have said too much already.  In case you’re wondering, I wrote this initially while on holidays, and update it sometimes.  No time for a Data Science professor to talk about this stuff during semester!   But I’ve been updating it ever since.  Students badly need some good advice in this area.  The modern world is a minefield for those wanting to stay healthy.

But bear in mind, I have no official qualifications or expertise when it comes to health.   But I have applied my cynical, technical mind to the subject because I found that simple “appeal to authority” was leaving too many obvious conflicts.

h1

Picking Conferences

January 7, 2018

As a PhD student starting out, you do have some career options.  Likewise, as a typical junior academic, with limited budget and research time, you have similar career options.  A main one which I’ll discuss here is:  Which conference(s) should I got to?  This is peculiar to computer scientists whose conferences are competitive publications (say 20-25% acceptance rate) and count as publications.

So you only get time to attend a few conferences.  Likewise, you only get time to write papers for a few.  So you want them to count.  Conferences each have their own style.  Best way to think of it is that a conference is a tribe where membership is part-time.  You have to take time to learn about the habits and preferences of the tribe, i.e., in terms of paper content.  If the tribe always starts off with 20% of detailed theoretical definitions then you have to as well.  If they do certain kinds of experiments, then so should you.  Think of these sorts of things as tribal markings.  To be innovative, you generally need to do so from inside the system.  I know this sounds conformist, and belief me, I am completely non-conformist myself, but generally its how conferences work, largely as a result of the reviewing system. If a trusted member of the tribe starts quoting classical, venerated philosophers, so will the others.  If a complete unknown person submits a paper quoting venerated philosophers, then it’ll be viewed as weird unless they have enough other tribal markings on their work to accepted.

I have a number of conferences I really like where I understand the general tribal markings and am happy to live with them.  So SIGKDD has solid experimental work, ICML has innovative new methods, ACL has applications of machine learning to real linguistic problems.  They sometimes have additional tribal markings that can be more or less problematic.

Anyway, as a junior academic, you have to target a few conferences and learn to become a reliable tribal member.  You might want to pick a few authors and build on their work.  Or you might want to pick a specialised problem.  Regardless, to publish in particular venues you’ll have to get to know the tribal preferences and adhere to them.  Doing good research is one thing, and really good research will usually speak for itself, but if your contribution is not outstanding, say “merely” at the top 25 percentile of work, then you have to follow the tribe to be accepted into the tribe.  That takes time.

Moreover, the vibe at the conference is always much, much more than the printed proceedings.  You need to be there:  hear the questions, watch the audience, chat to others in the breaks, see the quality of the presenters.  What is important and influential?  What is losing out, perhaps because it was trendy rather than productive?  All this happens at the conference.  You need to be there to see it.  Otherwise, you’ll be a year behind the others … new ideas for next year’s conference are often the germ of an idea at this year’s conference.  Moreover, it always helps to see the movers and shakers in action.  What sort of people are they?  How do they present their work?

So what does this mean to the junior academic?  You need early on to target a particular conference, subject or influential author’s/group’s body of work, and learn what it is they do.  That’ll take time.  So if you don’t see yourself as being involved in that community 5 years down the track, you probably shouldn’t be making that effort.  If you think their research doesn’t have a good future, then again, you probably shouldn’t be making that effort.  Pick some conferences with this in mind, and try and go along semi-regularly to keep track of things and pick up the vibe.

h1

MDSS Seminar Series: Doing Bayesian Text Analysis

August 4, 2017

Giving a talk to the Monash Data Science Society on August 28th.  Details here.  Its a historical perspective and motivational talk about doing text and document analysis.  Slides are here.