Altdeep.ai Newsletter

Share this post

Christmas Trees and Invariance in Machine Learning

newsletter.altdeep.ai

Christmas Trees and Invariance in Machine Learning

Rotation invariant Christmas trees are trending this year.

Robert Osazuwa Ness
Dec 26, 2021
2
1
Share this post

Christmas Trees and Invariance in Machine Learning

newsletter.altdeep.ai
White House Christmas Tree Melania - christmasthing

Machine learning experts often talk about “invariance.”

“Invariance” is a term borrowed from math. In math, you say a property of a mathematical object is “invariant” if it remains unchanged after operations or transformations of a certain type are applied to the objects.

In machine learning, the language describes why a particular learning algorithm works so well. For example, “It works because it is invariant to ….”

For example, here’s a useful illustration of invariance in computer vision I found on StackExchange.

machine learning - What is translation invariance in ...

Suppose you have a computer vision algorithm trying to identify objects in images in this example. You want the learning algorithm to know that operations on the object in the image, such as translating it, rotating it, changing its size, or changing the illumination, don't change the object's identity.

The problem is that the algorithm can learn many various statistical patterns from data and that some patterns help identify the object, and some do not. We describe the patterns that are not useful in terms of invariance.

Am I the only one that feels like this is a weird way of getting machines to learn? I feel odd trying to represent something in terms of what it isn’t rather than what it is.

Imagine learning what a Christmas tree is like this.

What’s a Christmas tree?

This is a Christmas tree.

A Christmas tree

Also, note that Christmas trees are invariant to being indoors or outdoors.

An indoor tree and an outdoor tree.
My dear algorithm, the Christmas tree-ness of a thing does not depend on whether or not it is outdoors.

And they are size invariant.

A tiny Christmas tree
My dear algorithm, the Christmas tree-ness of a thing does not depend on its size.

But not proximity invariant. That’s pine needles.

pine needles
Wait a sec, my dear algorithm, that’s too close.

And not ornament invariant. That’s just a pine tree.

An unadorned Scotts pine tree
Wait a sec, my dear algorithm, I think it doesn’t count if it doesn’t have ornaments.

Wait, except when indoors. And there’s presents. That’s still a Christmas tree.

Unadorned Christmas tree inside with presents
My mistake, dear algorithm. A Christmas tree can be unadorned if it is inside and there are a bunch of presents and stuff.

And I guess they are rotation invariant?

A Christmas tree suspended upside-down
Ok, dear algorithm, it’s weird but I think upside-down Christmas trees are a thing.

And color invariant…

A set of variously colored Christmas trees
Dear algorithm, color doesn’t matter but, you know, they are usually green.

Except for these. These are just messed up…

Blood red Christmas trees organized by Melania Trump in the White House
No idea, dear algorithm. I think maybe these are some sort of coniferous Truffula?

Share

Leave a comment

1
Share this post

Christmas Trees and Invariance in Machine Learning

newsletter.altdeep.ai
1 Comment
Richard Wang
Dec 27, 2021

This shows how little the machine really knows about Christmas tree, only use spurious corrleations that it has seen in the past to draw conclusion! That's the problem of covariate shift/Out of distribution. Can causal inference make machine learn better from the "true" concept that is invariant?

Expand full comment
Reply
TopNewCommunity

No posts

Ready for more?

© 2023 Robert Osazuwa Ness
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing