These 94 lines of code are everything that is needed to train a neural network. Everything else is just efficiency. This is my earlier project Micrograd. It implements a scalar-valued auto-grad engine. You start with some numbers at the leafs (usually the input data and the neural network parameters), build up a computational graph with operations like + and * that mix them, and the graph ends with a single value at the very end (the loss). You then go backwards through the graph applying chain rule at each node to calculate the gradients. The gradients tell you how to nudge your parameters to decrease the loss (and hence improve your network). Sometimes when things get too complicated, I come back to this code and just breathe a little. But ok ok you also do have to know what the computational graph should be (e.g. MLP -> Transformer), what the loss function should be (e.g. autoregressive/diffusion), how to best use the gradients for a parameter update (e.g. SGD -> AdamW) etc etc. But it is the core of what is mostly happening. The 1986 paper from Rumelhart, Hinton, Williams that popularized and used this algorithm (backpropagation) for training neural nets: cs.toronto.edu/~hinton/absps… micrograd on Github: github.com/karpathy/microgra… and my (now somewhat old) YouTube video where I very slowly build and explain: youtube.com/watch?v=VMj-3S1t…

Do you have ADHD or trouble staying focused while reading a book? Here's a simple hack I created that changed my life. It's called a "Distraction Catcher":

Overthinkers get luckier in life once they notice they didn’t lack intelligence, but the courage to act.

How to do hard stuff: write down a 1 page doc with your key assumptions, the hypothesis/goal you want to test, and your plan to do it now read it with a maximally critical eye…does it make sense? are there gaps? iterate until you can’t find any now go execute like crazy

When you don't take venture capital: - You can be profitable and proud - You are debt free - You can sell your business for $5M and make life changing money - You don't have to care about valuation - You decide if and when to grow - You make all the decisions not a Silicon Valley VC who spends 1 hour a week on your business - You reward yourself and your team through dividends - You don't need to send updates to investors - You have a stable business for your team and customers - You put customers first, always - You can change your business without asking anyone - You choose when or if you want to sell without any approvals - You can pass on the business to your family - You keep your company's culture the way you like it - You aim for a long-lasting business without being rushed - You operate without the weight of the exit goalpost, allowing the business's essence to evolve naturally - You have no bosses - You might make less, but it's your baby - You can do what you think is right and that's everything - You don't have to be feel pressured to spend your capital - You're building without strings attached - You're chasing fundamentals never funding The next generation of entrepreneurs, solopreneurs and multipreneurs (@MultipreneurGuy) value freedom and sustainable growth over quick hits of VC dollars This fires me up... you?

People think they struggle to grow because they are not good at learning, they are actually not good at unlearning. Their old identity is in the way, their inner insecure voice keeps telling them what they cannot do. They must learn to ignore it in order to rebuild themselves.

“give yourself a lot of shots to get lucky” is even better advice than it appears on the surface. luck isn’t an independent variable but increases super-linearly with more surface area—you meet more people, make more connections between new ideas, learn patterns, etc.

Hang out with self-confident people who have no reason to envy you, you will avoid a lot of drama.

I built many projects as a developer. I made $0. Then I learned marketing. Now I own a $350k+ a year SaaS business. Here is how you can do the same if you're a developer 🧵👇

operate from the assumption that people have no idea what you do

Opportunities come all the time, people just miss them because they think they are “not ready.” Jump, you will learn on the go.

10 days of free game ↓ leveragedcreator.com

Don’t be busy, be productive.

You are sabotaging your own personal and professional growth if you're not finding time to read. These are the books that changed my life, and what I learned from them:

Stop worrying about people who don't even care about you.

most of the madness you see on the internet can be explained by the simple fact that an enormous percentage of jobs are absolutely pointless roleplaying activities

Your mind isn’t agitated, your body is just underworked. The gym fixes this.

After arriving late to his MBA class in 1972, Elon Musk copied from the blackboard three problems thinking they were homework, and then solved them They were actually the three most difficult to execute business models of all time, which became PayPal, Tesla, and SpaceX Legend

Overnight success takes about 10 years. With $1M+ NFT sales now occurring regularly, it can be easy to feel like you are doing something wrong and being left behind if your pieces aren’t selling. The artists achieving success now have been working at their craft for a long time.

Logical fallacies are errors in reasoning that undermine the quality of an argument. THREAD: 20 common logical fallacies to learn, identify, and avoid: