The Humans in the Loop: Diminishing Returns
This Week in AI for Devs: Models Can Only Get So Large
Welcome to The Humans in the Loop, your executive summary of AI news for devs. I’m Andrew, the [human] author, and I’d love your feedback on this newsletter. Please feel free to email me at andrew@heavybit.com with your thoughts. Thanks!
Our top story: Are LLMs Hitting the Wall?
Major AI vendors may be seeing diminishing returns training the next generation of LLMs. Reports suggest quiet frustration at OpenAI with the lack of progress in getting the vendor’s next model, GPT-5, to show quantum leaps in performance, perhaps due to a diminishing amount of publicly available training data. META reportedly plans to deploy as many as 500,000 H100 chips to train its next major model, Llama-4. But like many large foundation model vendors, META is bumping its head against limitations like training data availability, power needs, and overall costs. (Meanwhile, available public data on the Web may not be growing fast enough to provide noticeable training benefits, even as publishers like Reddit and Twitter—annoyed by AI scraping—add API restrictions that are decreasing publicly available data.) Researchers like former OpenAI scientist Ilya Sutskever suggest the path forward may lie in more-efficient inference methods like test-time compute, which, unlike costly initial pretraining on billions of parameters, focuses on iterative improvements with each data job run through a model. If language models can’t get much larger, the next frontier may be somewhere other than pretraining. Maybe new opportunities will come from efforts to boost overall model efficiency, or from exploring alternative architectures, like space-state models rather than traditional transformer-based models.
Coming Up: The bottom line on this week’s AI news.
💻 Development
New Open Coding Models: QWEN 2.5 Coder and OpenCoder
New SOTA models for coding include the newly open-sourced Alibaba QWEN 2.5 Coder and the OSS coding assistant OpenCoder.
[GH] LLMs From Scratch: BYO Using PyTorch
Haven’t built your own LLM yet? Try this repo.
Python Is #1 Thanks to AI Projects
Heard about this “AI” thing? It’s such a big deal that Python, used for many AI projects, has become 2024’s #1 coding language.
[GH] LLMariner: Run GenAI on Kubernetes
This open-source platform helps you manage GenAI workloads on K8s.
[GH] GenAIScript: JavaScript-ish LLM Scripting
MSFT’s new JS-ish environment offers “automatable GenAI scripting.”
🤔 Interesting AI Projects, Research, and Updates
Open-Source Reasoning: Steiner and Nous Reasoning API
We’re not quite at open-source o1-level reasoning yet, but Steiner is an OSS QWEN-based model with multiple reasoning paths, and French researchers Nous have released their new Reasoning API.
[GH] Datachain: Data Warehouse for AI
This Python-based OSS project was specifically built to transform and analyze unstructured AI data.
[GH] Inferit: Visual Comparisons of Model Outputs
This project provides at-a-glance, side-by-side comparisons of how different models perform.
💼 Hiring and Community
Startups Hiring This Week:
- Sr. Frontend Engineer → Imagen
- Data Scientist → Strider
Mid-Markets Hiring This Week:
- ML Research Engineer → Coactive
- Sr. Full-Stack Engineer → Labelbox
Enterprises Hiring This Week:
- Sr. Staff Engineer → Aurora
- Sr. Engineering Manager → Torc
💡 Spotlight: Newly-Launched AI Startups
- Decart: $21M from Sequoia for AI model training infrastructure
- Thesys: $4M from Together Fund for AI-generated UI
- Deckmatch: $3.1M from Alliance VC for product-led VC deal flow
- Connecty AI: $1.8M from Market One for enterprise data agents
🏭 Industry: M&A, Launches, Trends
Of Course AGI Will Arrive Soon, Say AI CEOs
Hadn’t you heard? Why, just ask Anthropic’s CEO, who predicts a 2026 AGI debut, or OpenAI’s CEO, who calls it for 2025.
NVDA: World’s Most Valuable Company as Investments Skyrocket
After election night, the chipmaker soared to a $3.6T valuation, ending 2024 by backing 40+ startups at an average deal size of $60M+.
AWS Launches New Inference Chip + $110M Bounty to Research It
AWS is serious about its Trainium 2 inference chip as competition for NVDA, and is offering north of $100M for research support.
2024: The Year the Defense Industry and AI Got Together
Government AI contracts increase 1200%, with defense contractors like Palantir adopting Claude and AI vendors like META signing contracts with Anduril, BAH, and LMT.
⚖️ Copyright, IP, Licensing, and Regulation
Trump to Roll Back Biden AI Regulation
The next president will roll back the previous one’s AI executive order—and may look to deregulate AI developments further.
US Commerce Department Blocks Chips to China
The US has made its competitive concerns against China clear by blocking chip giant TSMC from sending AI chips to Chinese markets.
OpenAI Actually Wins a Lawsuit
After claims by independent news outlets, the frequently-sued AI vendor saw one case actually get tossed out by the judge.
UK Government Launches AI Safety Platform
The new safety platform will evaluate AI products for bias and risk analysis.
About the Author
Hi there. My name is Andrew, and I work at Heavybit, the leading VC for developer-first startups. As the Editorial Lead, my goal is to find the most valuable and important AI news for developers and founders. The idea is to curate and bottom-line emerging trends from our perspective in 10+ years of coaching and funding developer-facing companies. Email me and let me know what you think.
That’s all for this edition. In the meantime, please feel free to follow The Humans in the Loop on Twitter and LinkedIn.