clever techniques + missing out

· Zi Wang · 1 min read

Z / Runner 🏃🏻‍♂️

🏃🏻‍♂️ listen; dario "cleverness, all the techniques, all the "we need a new method to do something", that doesn't matter very much. There are only a few things that matter. I think I listed seven of them…

  1. Raw Compute: Total FLOPs/hardware power.
  2. Data Quantity: The volume of tokens or samples.
  3. Quality & Distribution: Broad, high-fidelity datasets for generalization.
  4. Training Duration: Total steps and time allocated.
  5. Scalable Objective Functions: Mechanisms like next-token prediction or RL (math/coding/RLHF) that scale indefinitely.
  6. Normalization: Structural stability (e.g., LayerNorm) for deep architectures.
  7. Conditioning/Numerical Stability: Ensuring "laminar" gradient flow to prevent mathematical collapse in massive compute blobs.

Stephen / Basketball 🏀

🏀 much prefer elon's.

🏀 "do i contradict myself? very well then i contradict myself"

  • z, your (5 examples of) ai agents for (fine) dining? for (grocery) shopping (vs josey baker)?
  • "joy of missing out": agent for poking friends, collecting (voice) messages.. assembling into weekly podcasts + paper yearbooks.