AI and America’s Racial "One Country, Two Systems" Policy

Plus, recent research on transformer networks and generative ML.

Happy Thursday. This week’s condition comes to you during a time of public outcry and protests in the U.S. precipitated by the recent killings of George Floyd, Brianna Taylor, Ahmaud Arbery, and Sean Reed. If you are a researcher and were working towards the Neurips2020 deadline this week while peaceful protests, police clashes, and/or rioting was happening down the block, you might have felt like that dog-in-burning house meme. You weren’t alone.

America’s One Country, Two Systems policy

With Hong Kong protests back in the news, and as an avid China watcher, I’m reminded of the phrase “One Country, Two Systems”, the term the CCP uses to normalize the apparent contradiction of Hong Kong and Macau having drastically different governance relative to Mainland China, particularly when it comes to rule-of-law and civil liberties.

Recent events reveal the U.S. has its own race-based “One Country, Two Systems” policy. The videos shine a light about the contradiction inherent in such a policy. They also bring to the surface the pent-up frustration about being gaslit about the existence of that contradiction despite a decade of viral video evidence and pesky things like American history.

Tech and ML look inward

For the tech, data science, and machine learning communities, there was a wake-up call in Amy Cooper’s use of the age-old trope of black male aggression against helpless white femininity to weaponize the police against the bird watcher who wanted her to leash her dog. Amy Cooper is an investment banker. Like members of our community, she is well-educated and numerate. She is an elite, and she i̶s̶ was working at an elite institution that hires data scientists. When she pulled racial rank on that bird watcher in Central Park, that rank was part of that elite identity and role that she and many others play in elite quantitation-heavy institutions.

This is forcing us to hold a mirror to our own institutions, as well as ask ourselves how the tools we are building might contribute to institutional racism more broadly. I’m sharing some links this week, new and old, that connect along those lines. Of course, some members of the community have been sounding the alarm on these issues for a while. Hopefully, this time we might hit a tipping point for those ideas leading to actual institutional change.

Racial disparities in tech

Facial Recognition and Race

Policing and Data

Help me get the word out. Share this post if you think its useful.


Ear to the Ground — Recent advances in Transformer Networks

Transformers are the most significant advancement in deep learning in the last four years. In that time, they have emerged as the architecture of key models such as OpenAI’s GPT, Microsoft’s Turing-NLG, and Google’s BERT.

The success has lead to a race for size. Larger models have consistently outperformed smaller alternatives across many tasks. Turing-NLG used 17 billion parameters, OpenAI’s new GPT-3 model uses 175 billion.

This publication has argued that there are some bad consequences of this trend.

  1. It biases the articulation of problems to their amenability to solving with deep learning. If it is infeasible to solve a problem this way (e.g., it is not feasible to get the data within a problem domain curated, structured, and labeled enough to train 175 billion parameters, that problem will be ignored).

  2. It increases the gravitational pull the few corporations that can afford to train such models, sucking in greater amounts of research resources, IP, and talent. These are the same organizations that we worry have too much control over our culture, political processes, and privacy.

On the flip side, I love the idea of 175 billion parameters. It’s brute-forcing a problem, showing the frontiers of what can be done with raw compute. Once we’ve found that frontier, we can fiddle with the black box and learn how it works. I jibes with my intuition as an engineer — get something that solves the problem, optimize later.

People still don’t know what to generate

With all the advances in generative ML, people seem not to know what to generate aside from Democracy-threatening deep fakes and sexy people who don’t exist.

Deep Learning Solves the Uncanny Valley Problem ...

I’ve seen some noble efforts. For example, some have tried to use GANs and other approaches to generate novel molecules.

My experience in materials science and drug discovery has taught me that the bottleneck in those fields is not generating new molecules. The bottleneck is figuring out which molecule will be the next blockbuster. How would this molecule behave in a polymer, or in an adhesive used in humid environments? What would this compound’s binding affinity be to a particular cellular receptor? Would it even make it to the cell if orally ingested? This is the real prediction challenge. Generating new things is useless if we can’t predict how useful they’d be.

This Summer I’m working through the generative ML literature, mostly VAEs and GANs. I’m interested in particular in models where the latent representation is semantically meaningful, not merely collections of curves and ovoids (to use computer vision as an example).

If you have any suggestions or want to do some readings together, please reach out.

Leave a comment