Here are the sections I have done since September 2023.

I am a software engineer in Seattle. In the last three years, I have worked on multiple projects in ML compilers, runtimes, and frameworks, and left some random notes in corporate accounts instead of this WordPress account. Now, I am restarting my WordPress account to write down everything here and potentially share it with other learners worldwide.

From my last updates, my interests have shifted from the system domain (compilers and architectures) to the machine learning domain, with a focus on model compressions (pruning/quantization/nature architecture search/distillation/etc.), large-scale model training, and serving, aiming to increase the impact of machine learning to people’s daily life by making it affordable, accessible and accountable.

• Summaries
  • Efficient Compute for ML
  • Triton
    • API Notes
  • JAX
    • Notes
• Papers
  • Keep a record of papers, books and courses read/taken.
• Courses
  • Generative AI with Large Lanugae Model
    • Week 1
    • Week 2
    • Week 3
  • Deep Reinforcement Learning Into
    • Unit 1-3. Deep Reinforcement Learning & Q-Learning
    • Unit 4. Policy-Gradient Method
    • Unit 6. Actor-Critic Method
    • Unit 8. Proximal Policy Optimization (PPO)
  • System Design Interview By Alex Xu
    • Chapter 1: Scale from Zero to Millions of Users
    • Chapter 2: Back-of-The-Envelope Estimation
    • Chapter 3: A Framework for System Design Interviews
    • Chapter 4: Design a Rate Limiter
    • Chapter 5: Design Consistent Hashing
    • Chapter 6: Design A Key-Value Store
    • Chapter 7: Design A Unique ID Generator In Distributed Systems
    • Chapter 8: Design A URL Shortener
    • Chapter 9: Design A Web Crawler
    • Chapter 10: Design A Notification System
    • Chapter 11: Design A News Feed System
    • Chapter 12: Design a Chat System
  • Machine Learning System Design Interview By Alex Xu
    • Chapter 6: Video Recommendation System
• Talks
  • Meta AI Infra Talk 2023

Here are the sections I have done prior to June 2020.

I am a software engineer in New York. I think I have tons of space to learn and grow. I am writing some technical posts here, not to help others, but to help myself. Writing these posts helps me summarize my knowledge, make them structured, and improve my writing skills.
Here are some things I care about:

• Machine Learning
  I have taken statistics courses and a statistical machine learning course in college. I also have taken the Neural Networks for Machine Learning course from the University of Toronto in Coursera. However, that’s too shallow and outdated. I am currently taking Machine Learning Foundations from National Taiwan University and Introduction to Deep Learning from CMU. I hope they can give me a more comprehensive understanding of both classic machine learning and deep learning. At the meanwhile, I will post my notes here and post my homework on Github.
  • Machine Learning Foundations from National Taiwan University
    • Note 1 – The Learning Problem
    • Note 2 – Learning to Answer Yes/No
    • Note 3 – Types of Learning
    • Note 4 – Feasibility of Learning
  • Introduction to Deep Learning from CMU
    • Lecture 1 The Perceptron
    • Lecture 2 The Universal Approximation Theorem
    • Lecture 3 Learning, Empirical Risk Minimization, and Optimization
    • Lecture 4 Learning the network: Backprop
    • Lecture 5 Convergence, Learning Rates, and Gradient Descent
    • Lecture 6 Convergence, Loss Surfaces, and Optimization
    • Lecture 7 Acceleration, Regularization and Normalization
    • Lecture 8 Batch Normalization, Dropout and other Regularization methods
    • Lecture 9 Convolutional Neural Networks (1/3)
    • Lecture 10 Convolutional Neural Networks (2/3)
    • Lecture 11 Convolutional Neural Networks (3/3)
    • Lecture 12 Recurrent Neural Networks (1/5)
  • Tensorflow Note
• Data Structure And Algorithms
  I have taken a data structure and algorithm course in college. I find it fun but I am not obsessed with it. I think it is not really helpful with my career except that it helps me go through interviews.
  • Algorithms
    • Topology Sort
    • Binary Search Tree
    • Binary Search – The Limits
    • Dynamic Programming – The Intervals
  • Typical Questions
    • The Calculator Problem
    • The Sliding Window Problem
• System Design
  Similarly to the above topic, except that I am less prepared.
  • System Design Overview & Design News Feed
  • Design User System
  • Consistent Hashing & Design Tiny URL
  • Design Distributed File System
  • Design Web Crawler & Typeahead
  • Design Location-Based Service
  • Design WhatsApp / Messenger
  • Design Rate Limiter / Data Dog
• Programming Language: C++ / Python / Haskell
  These are the programming languages that I love. I hope I can write something about them when I finish other things.
• Computer System: Operating System / Compiler / Architecture
  Nowadays, no many people care about these topics. However, I still find some of them fascinating to me.
• Hot Techniques / Tools
  Follow the trends.
  • TensorRT Notes (1)
  • TensorRT Notes (2)
  • CUDA Programming Note
• Math / Finance
  Keep a sharp mind.