LLMs are shifting the paradigm of information retrieval and search. But this computational wizardry comes at a significant energetic cost.
👉 exponentialview.co/p/ev-444
In the latest Sunday newsletter, we look at @AINowInstitute new report by @sarahbmyers@JaaiVipra and the burgeoning issue of big tech’s consolidation of power within the AI sector.
💬 Narrated by AI via @substack
Exponential solar 💚 Although it only comprised 2.06% of global energy consumption in 2022, solar PV’s share is bound to grow fast. In the United States, more than half of new electricity-generating capacity in 2023 will be solar.
And finally, LK-99 got us excited for a week! Athough independent validation failed, the scientific community rose to the occasion. The collective intelligence of scientists around the world testing and broadcasting in real-time was a marvel to observe.
NVIDIA’s A100 GPUs have been instrumental in propelling recent AI advancements. Building on this legacy, NVIDIA’s H100 GPU emerges as a powerhouse, training models like GPT-3 four times faster than the A100s.
A pre-print we reviewed on August 13 revealed that LLMs have political biases: GPT-4 leans towards libertarianism, while LLaMA is more authoritarian. To unearth these biases, researchers posed questions about social and economic ideologies.
Renewable energy performance continues to amaze. In the last week of August, we spotted that the learning rates of solar PV are the best we’ve ever seen. In 2000 it took a year to add 1GW of solar PV globally; today it takes less than a day! via @_HannahRitchie
We stepped into August following the hottest month on record globally. Every day of July starting on July 3rd broke the modern-day record high of 16.924°C(62.46°F), which was set on July 24, 2022.
"The combination of pressures led to Eldrup committing in 2009 that then-DONG Energy will change its energy mix from 85% fossil fuels and 15% clean energy to 85% clean energy and 15% fossil fuels by 2040. [...] In fact, Orsted achieved Eldrup’s target 20 years earlier."
4⃣ The global increase in temperatures is felt with particular intensity in big cities, where extreme heat waves are becoming common. Phoenix now deals with four times as many searing heat days as in the 1950s, Paris eight times as many, and London ten times as many.
3⃣ Sea ice is shrinking, fast. The amount of Arctic sea ice is currently the 12th lowest on record, and Antarctic ice extent is a record low. The reality is that we’re entering uncharted waters.
1⃣ July 2023 has been a month of relentless record-breaking. After the hottest day and week, we are now experiencing the hottest month globally in 120,000 years.
If superconductivity becomes possible at room temperature, it will open up new applications ranging from quantum computers to magnetically levitating trains.
6⃣ June was a busy month for regulators. @Europarl_EN proposed the first comprehensive set of regulations for AI. We praised the fact that rules are being seriously considered, but we concluded that the rush to put rules around foundation models could be problematic.
5⃣ While today’s state-of-the-art models have limited or no open access, open-source is catching up. Open-source was a topic of discussion at the US Congress with @ClementDelangue last month.
4⃣ Back in May, we highlighted one of the first studies to tackle the effects of AI on labour and the economy. @erikbryn, Raymond and Li show that generative AI can boost productivity by 14% in some jobs, and enhance productivity in employees who are underperforming.
3⃣ The context windows of LLMs are growing rapidly. In Q2, @AnthropicAI released a new, 100k-token version of its model Claude. Thanks to its massive context window, #Claude can process the entirety of The Great Gatsby and answer questions about it in 30 seconds.
2⃣ Remember Chegg? In May, Wall Street wiped over 40% off this online education company’s stock price in a single day, valuing it at ⅓ of what it was worth at the time of ChatGPT’s initial launch. This probably makes Chegg the first company to have directly suffered from genAI.
1⃣ AI eating software for lunch: This is a forecasted decline in the cost of computation once natural language programming makes inroads into the profession of software engineering. #CodeInterpreter
Can #China reach its #innovation potential and overtake the #US as the main global economic superpower by 2050? Probably not. Although it spends an increasing amount of R&D, its research is not as good, and it will struggle with its demography.
cc @jjding99@niubi
China’s birth rate has been decreasing for decades. If it wants to keep growing its economy and catch up to the US, China needs to deal with its impending population decline.
#China#demographics#populationdecline
Researchers at Stanford tested major foundation models against the rules and found that they largely do not comply. The most compliant models are the open-source ones. #ai
The Apple 2, IBM PC, and Compaq Portable were all over $5,000 (inflation-adjusted). The iPhone did cost much less ($730 adjusted), but @Apple isn’t expecting the Vision Pro to have the same mass adoption yet.
"Chips are important. They are the steam engines of the 21st century. The Chinese know this. The Europeans are starting to get it. And the US certainly do. But I wonder whether the British government does." — @azeem on the UK Gov't semiconductor strategy
Three powerful conversations on @azeem's podcast with @sama@demishassabis@GaryMarcus exploring AI development and humanity's future. Links to all three👇🧵
Context windows for large-language models (LLMs) are growing rapidly. @AnthropicAI Claude’s new 100,000-token capacity means it can easily process the entirety of The Great Gatsby novel, and answer queries about it in under 30 seconds.
Within this competitive landscape, OpenAI boasts the highest valuation. @azeem spoke with @sama@openAI when GPT-3 first launched about broader implications of generative AI for humanity.
Full conversation: podcasts.apple.com/rs/podcas…