(neuro/ai) I make technical deep learning tutorials đş
Montreal Canada
- Tweets 18,686
- Following 1,831
- Followers 30,580
- Likes 54,184
Pinned Tweet
if there is one thing that you must not do is surrender
donât surrender your dreams, your passion, your curiosity or your freedom
never
28
62
886
216,674
from now on whenever someone ask what I use as a whiteboard during my live paper review session I'll just link them this
4
1
33
2,997
man looking back galactica was kind of a great idea
Remember October 2022 when you doused Galactica with vitriol?
Galactica was a 120b-parameter LLM-based system from Meta-FAIR designed to help scientist write papers.
It was open sourced (link below).
A mob of haters, including Michael, claimed it was dangerous and toxic and was going to destroy Science.
The small team at FAIR couldn't sleep at night and took down the demo website (they kept the GitHub and paper up).
Then, only 3 weeks later, ChatGPT was released and was welcomed as the second coming of the Messiahđ¤
The vitriol dousers were silent.
technologyreview.com/2022/11âŚ
github.com/paperswithcode/gaâŚ
2
1
19
4,974
I think code + writing quality would be greatly improve if we let agent write stuff inside the documents that they can read and we can't see
7
41
2,086
almost done upgrading my humble little business website so I don't look jobless and in need for a free latte
6
3
108
6,229
one of the most impressive full-growth-high-agency-just-do-hard-thing-no-limitâ˘ď¸ mindset I've ever seen
She left Waterloo without graduating. Now she's a Member of Technical Staff at @OpenAI.
@BrookeaJoseph joined a 12-person startup called @skybysoftware as an intern. Six months later, OpenAI acquired it. Now she does agent post-training research on computer use full-time, teaching models to click, type, and operate any software a human can.
I sat down with her in Washington Square Park to get into:
â˘Why she takes jobs she's "unqualified" for and how you build that confidence
â˘What makes software feel magical
â˘Why computer use is the road to AGI
â˘When you should not let an agent use your computer
â˘How she learns and why she still does it by hand
đđđđđđđđđđ
(00:00) Intro: Washington Square Park
(02:00) Her First Job: Her Dad's Produce Store
(03:43) Why She Takes Roles She's "Unqualified" For
(07:12) Why Sweden: Building Edge ML for Volvo & Siemens
(11:18) What Made Sky Magical
(14:19) Ari, Konrad, and Asking "Why" Relentlessly
(17:28) Six Months In: OpenAI Acquires Sky
(18:54) Computer Use, Explained
(20:14) Why Computer Use Is the Road to AGI
(25:14) When Not to Let an Agent Use Your Computer
(29:32) The "Never Satisfied" Mindset
(35:40) Her Love of Math, and Mr. White
(45:32) Her Curiosity Deep-Dive System
(52:29) Post-AGI Plans: A Bakery in Paris
8
49
1,139
105,841
one thing that always surprise me with RL over LLM is how much everything kind of works like training them with 400 gradient steps lag
alright guys I'll have the pleasure this week to interview @mrdrozdov from databricks on all thing retrievals like:
- RL for knowledge agents
- learning representation program for retrieval
- place of harness in all that
if you have any questions on retrieval shoot it my way
1
8
133
7,830
well well well would you look at what I found in my garden
exactly the paper I need to understand deeply tuning-free retrieval augmentation using program synthesis at indexing time
alright guys I'll have the pleasure this week to interview @mrdrozdov from databricks on all thing retrievals like:
- RL for knowledge agents
- learning representation program for retrieval
- place of harness in all that
if you have any questions on retrieval shoot it my way
3
1
47
3,232
great short blog post by andrew on doing a PhD (but also just generally being a good scientist)
I truly resonated with pursuing positivity as some of the worst papers that ever blessed the field still had lots of useful nuggets of knowledge hidden in the garbage.
8
68
944
51,944
alright guys I'll have the pleasure this week to interview @mrdrozdov from databricks on all thing retrievals like:
- RL for knowledge agents
- learning representation program for retrieval
- place of harness in all that
if you have any questions on retrieval shoot it my way
7
25
353
29,595
is most of the gain from the frontier about bringing the world in-distribution?
if you are building a product using AI, you should be spending >25% of your time making benchmarks and trying to get the model labs to care about said benchmarks
easiest path to accelerate your progress as a company
1
2
12
3,880
on a more pragmatic note here is a 3h discussion with my man @justinskycak on how to learn technical topic like math 100% on your own
a lot of students have a hard time with this so let me spell it out:
- class is for asking dumb ass questions to someone that knows the material
- learning happens on your own outside the classroom
- office hours is to ask adversarial questions to the main grader about exams
7
87
1,043
46,823