Image Analysis Course Recap, Fall 2022

This past term, Fall 2022, I taught for the second time a course I created on image analysis. (Syllabus.) The structure and content were similar to last year’s — last year went well, and I put lots of work into it, so “if it ain’t broke…”

Here’s a summary of things in the 2021 course recap that also apply this year; below, I’ll note new comments.

• Though the bulk of the class is on computational algorithms and methods, we also explore the physics of imaging, especially related to noise, resolution, and image formation.
• the course involves a lot of programming. The assignments are all programming assignments and mostly involve writing code from scratch (i.e. not filling in blanks in a Jupyter notebook — more on that later).
• The topics include filtering, localization algorithms, deconvolution, compression, a little bit of machine learning (see below), and more. Localization-based super resolution microscopy is my favorite module; it’s a fascinating subject, and it combines physics, computation, and statistics in elegant ways.

Some comments for this year:

Programming

Though students could use Python or MATLAB for assignments (or something else if they didn’t mind that I might not help them), everyone used Python. Like last year, I made sure that I could write solutions to all the exercises in Python, and that I could anticipate problems and offer guidance on structure and methods. (Guidance like: Write functions rather than repeating code, and For-loops make angels weep.) This went well. Thanks to the course as well as other exercises I’ve been doing, I now feel bilingual in Python and MATLAB.

Having a decent ability to program is a prerequisite for the course. The preceding summer, I helped a student prepare for the course by meeting weekly to go over these exercises I wrote, which could perhaps be useful for others to learn from. We — the broader “we” of university science faculty, at least here at the University of Oregon — don’t have a good way of teaching or assessing the programming abilities that students should have. This comes up in many courses and leads to either spending a lot of time teaching basics that many students already know or being inaccessible to potentially interested students. One might think that particular course prerequisites are the solution, but: (i) Introductory computer science courses are long and, for many people, excruciatingly slow. (Plus, they seem to focus on odd / impractical things.) (ii) Many students are self-taught programmers, and we don’t have a process for assessing this. Of course, assessment is possible — it’s commonplace when applying for programming jobs to take tests! Universities, though, are strangely loathe to evaluate students’ skills. (iii) Having gone through a programming course isn’t a guarantee that one can program. This I’ve been surprised by. I think it arises in part from forgetting in the absence of continued practice, and in part from classes that rely too much on students filling in pieces of largely preexisting code rather than writing from scratch. This distinction is like the difference between learning vocabulary in a foreign language, or how to conjugate verbs, rather than learning the more difficult and deeper skill of putting together whole sentences de novo. As a result, students can be lost when facing a new task for which there isn’t a scaffold, which comes up all the time in science.

Machine Learning

Last year I wrote that I’d like to increase the coverage of machine learning. I did this, but not as much as I should have. It’s well known that machine learning can do amazing things with images, recognizing faces, deblurring, creating complex scenes given text-only prompts, and more. It is conceivable that in the near future (5-10 years?) a course like mine would be irrelevant. We’re not there yet, though, and being able to write image analysis code remains very useful in many areas of science. Unlike the millions of training images Google uses to distinguish trees from stop signs, or cats from dogs, one’s datasets of images of mineral slices or cells on a slide are far smaller in size, probably insufficient to train a neural network. In addition, even if one can use machine learning it’s useful to do some intelligent pre-processing of images, for example filtering or finding features. Finally, another reason I didn’t do much with machine learning in the course is that the term is only 10 weeks long!

I did, however, spend more time on machine learning than I did last year. I created a homework assignment based on support vector machines, a feature-based method for classifying objects that one doesn’t hear about much these days with the advent of neural networks, but that requires less training data and whose output is more interpretable. It wasn’t my favorite homework assignment, but it was something. We also spent over two hours in class on neural networks. I did not succeed in creating a good homework assignment: tractable, fitting into the term, and not a “cookie cutter” activity. Coincidentally, I’ve been working off and on for the last several months on neural network homework exercises as a student, studying this “Deep Learning for Computer Vision” course. I’ve so far watched half the lectures and done 3 of 6 homework assignments, which are very difficult, and perhaps I’ll write more about this later. (I wrote about my earlier machine learning learning here.) I thought my class session on neural networks — on the history and basic architecture — went well. My goal was that the students would understand, for example, how convolutional networks differ from other networks (e.g. fully connected networks), how the convolutional part has similarities to filtering methods discussed earlier in the course, and why convolutional layers have far fewer parameters to learn but cost many more calculations than fully connected layers in a network.

Students

As before, the course was a mixed undergraduate / graduate course. There were only three undergraduates enrolled, even though the course now counts as an elective for the University of Oregon’s new Data Science major. Many of us remain rather confused about how the new major is structured and how students are being advised, which is something I hope to poke at in the future.

Evaluations

Students seemed to like the course a lot, as did I. From the end-of-term evaluations, “Not only are the assignments actually fun, I feel like I learn a lot through them” and “It builds more than just imaging skills [image analysis skills, presumably] and is helpful for all physicists.”

For the future

I hope to teach the course next year, but this remains to be determined. If I do, some improvements I’d like to make are pretty evident from what I’ve listed above: more machine learning and better advertising. Other things on the list: a more systematic approach to conveying the uses of Fourier analysis; more structure for the students’ final projects; maybe quizzes?

Artwork

Like last year, I started each class with an “artwork of the day,” which leaned heavily toward 20th century pieces, in some cases with features that reminded me of the day’s topic. This was fun, and became educational in unexpected ways. It was clear that students had no idea how much famous contemporary art can cost, being orders of magnitude off when guessing auction prices for works like Jeff Koons’ Balloon Dog:

(Price: $58.4 million in 2013) Speaking of which…

Today’s illustration…

Watercolor isn’t the ideal medium for reproducing Jeff Koons’ “Balloon Dog,” but this turned out better than I expected. It’s mostly based on a photo I took at The Broad Museum in Los Angeles in 2018, along with other photos.

— Raghuveer Parthasarathy, December 18, 2022