Undiscovered #058: AI Model Engineers Biology , Gwern's Latest Interview, "Schools Were Never Optimized for Learning"


#058: AI Model Engineers Biology , Gwern's Latest Interview, "Schools Were Never Optimized for Learning"

Hi All!

We are pleased to welcome you to this week's edition of Undiscovered, a newsletter with exclusive resources and insights expanding from the material found on our main site - becketu.com.

This week, we will take a look at a new AI model that can engineer biology, Andrej Karpathy's syllabus for his AI course, how schools were "never optimized for learning", and more.

Let's dive in:

AI Model Learns Biology Like a Language

A new research article in the publication, Science, just presented a groundbreaking discovery. "Evo" is an AI model that learns biology like a language, from single DNA mutations to entire genomes. In other words, it's the clearest proof yet that AI can engineer biology.

It can predict the effects of mutations across all layers of cellular regression, accelerate disease diagnosis and treatment, design new biological systems including synthetic CRISPR, model realistic, genome-length sequences over one megabase in length, and more.

While the potential is exciting for Evo, there are ethical concerns surrounding the biosecurity of the creation of these sequences. With implications such as faster drug recovery, new gene therapies, better understanding of evolution, and even the potential for creating new lifeforms, this will surely be a monumental discovery that will be followed closely for years to come.

A Better Wikipedia for Exploration

Globe has long been on our radar, starting with the excellent launch of their product Globe Explorer. Explorer touts itself as a 'discovery engine'. You search a subject of interest, and Explorer will create a table of contents for the topic with complimentary visuals to help you navigate what you'd like to learn.

This was already a useful tool for exploration and discovery, but their latest product may be more ideal for those who appreciate more direct learning.

Globe Index is their attempt at building a GPT that progressively answers your questions, while being mindful of the whole conversation. Sometimes with classic ChatGPT or Perplexity, we find that each individual query can be interpreted as separate questions, when we intend for them to be interconnected.

This makes Index ideal for a longer discussion that you know will require a more complicated set of follow up questions. Its minimalist design reduces potential distractions, and we can't wait to try it out further in other projects we are building.

Andrej Karpathy's Syllabus for AI Course

Andrej Karpathy is a widely respected computer scientist and AI researcher. He was a co-founder at OpenAI and the Director of AI and Autopilot Vision at Tesla.

In July 2024, Karpathy announced the launch of his latest initiative - an AI+Education company named Eureka Labs. The first goal of Eureka Labs is to create "obviously the best AI course" which will be undergraduate-level, designed by humans, and guided by an AI teaching assistant.

The full course isn't quite finished yet, but we do have a syllabus for what all will be covered:

Chapter 01 Bigram Language Model (language modeling)
Chapter 02 Micrograd (machine learning, backpropagation)
Chapter 03 N-gram model (multi-layer perceptron, matmul, gelu)
Chapter 04 Attention (attention, softmax, positional encoder)
Chapter 05 Transformer (transformer, residual, layernorm, GPT-2)
Chapter 06 Tokenization (minBPE, byte pair encoding)
Chapter 07 Optimization (initialization, optimization, AdamW)
Chapter 08 Need for Speed I: Device (device, CPU, GPU, ...)
Chapter 09 Need for Speed II: Precision (mixed precision training, fp16, bf16, fp8, ...)
Chapter 10 Need for Speed III: Distributed (distributed optimization, DDP, ZeRO)
Chapter 11 Datasets (datasets, data loading, synthetic data generation)
Chapter 12 Inference I: kv-cache (kv-cache)
Chapter 13 Inference II: Quantization (quantization)
Chapter 14 Finetuning I: SFT (supervised finetuning SFT, PEFT, LoRA, chat)
Chapter 15 Finetuning II: RL (reinforcement learning, RLHF, PPO, DPO)
Chapter 16 Deployment (API, web app)
Chapter 17 Multimodal (VQVAE, diffusion transformer)

"Schools Were Never Optimized for Learning"

Justin Skycak Is the Chief Quant and Director of Analytics for Math Academy, the resource we recommend the most for those serious about improving their math skills. His blog is incredible, but we've been blown away by some of his brilliant tweets recently on learning.

The first tweet covers his philosophy on solving problems:

"If you have difficulty in solving problems, then you have not understood the concepts yet. Conceptual clarity is tested by problems."

Saying "I understand everything but I just can't solve the problems" is the equivalent of "I am really strong but I just can't lift heavy weights."

You are only as strong as the feats of strength you can pull off. You only understand as much as you can evidence that understanding by solving problems.

He goes on to explain in further tweets how schools were never optimized for learning and that there are plenty of well-known learning strategies that have been known for decades, but aren't being implemented.

Justin is an excellent voice the field of education, and well worth listening to.

The Anonymous Writer Who Predicted the Path to AI

"The goal of these pages is not to be a model of concision, maximizing entertainment value per word, or to preach to a choir by elegantly repeating a conclusion. Rather, I am attempting to explain things to my future self, who is intelligent and interested, but has forgotten."

These are the opening words from the blog of pseudonymous polymathic writer, Gwern Branwen. Gwern has been gaining notoriety in tech circles for years, due to being an early adopter of Bitcoin and sharing correct predictions on Artificial Intelligence.

Gwern was also recently featured on the Dwarkesh Podcast (with a very cool virtual Avatar and voice actor used to protect his anonymity). Be sure to listen and check out his fascinating story.


P.S Can you please respond to this email and bring it into your 'primary' inbox? You can say 'Hi!', tell us the last book you read recently, or what your favorite resource was from above.

It would also help if you add us as a contact on your mailing list, with a visual tutorial below:

We appreciate any feedback you are able to provide here. What do you want more or less of? Other suggestions? Feel free to reach out to us on Instagram and give us a follow there, tag your friends on our posts, and please forward this newsletter along to anyone else who would enjoy it.

Disclaimer: Becket U is an Amazon Associate and purchases through Amazon links may earn a small affiliate commission, but the price is the same for you. We only recommend books we love and think you would love, too.

Always wishing you the best,

J.B.

Becket U

Becket U curates the best resources in Math, Physics, Computers, Microeconomics, Game Theory, and Persuasion. With this knowledge, you will understand how the world works.

Read more from Becket U
Becket U - The Best Resources for Learning STEM

#062: Roadmap to Making Real Friendships, Build Your Dream App in 2025, Story About a Contractor Happy Holidays and Hello to All! We are pleased to welcome you to this week's cheerful edition of Undiscovered, a newsletter with exclusive resources and insights expanding from the material found on our main site - becketu.com. This week, we will take a look at practical roadmaps for building your dream app in 2025, a guide to making authentic friendships, a book written by one of the recent lead...

Becket U - The Best Resources for Learning STEM

#061: Multiverse Theory Confirmed?, iPod Classic Emulator, Science of Short-Form Attention Hi All! We are pleased to welcome you to this week's edition of Undiscovered, a newsletter with exclusive resources and insights expanding from the material found on our main site - becketu.com. This week, we will take a look at a breakthrough quantum computing chip, an iPod Classic emulator for iPhone, the science of why short-form video is so engaging, and more. Let's dive in: Google's Breakthrough...

Becket U - The Best Resources for Learning STEM

#060: Restoring the Ability to Walk, Bigger Than Going Viral, Handwriting Increases Brain Connectivity Hi All! We are pleased to welcome you to this week's edition of Undiscovered, a newsletter with exclusive resources and insights expanding from the material found on our main site - becketu.com. This week we will take a look at an incredible breakthrough for patients suffering from paralysis, a search engine that will help you navigate the world of academia, a time capsule that takes us to...