The Humans in the Loop: AI Fought the Law, Did the Law Win?
This Week in AI for Devs: AI Runs Afoul of Johnny Law
Welcome to The Humans in the Loop, your executive summary of AI news for devs. I’m Andrew, the author, and I’m trying out a new look and feel for the newsletter. Thoughts? Email me at andrew@heavybit.com.
Our top story: The Many Legal Challenges Against AI
While recent talk on Wall Street has been about skepticism over ROI, new AI legal and regulatory battles are popping up everywhere. AI vendors are cheering the 11th-hour edits made to California bill SB 1047, which fundamentally weaken legal protections for AI customers. And OpenAI declares success against what it calls “a covert Iranian influence operation” that utilitized ChatGPT accounts to allegedly affect the 2024 election. But other courtroom challenges aren’t going as well—a copyright infringement case against text-to-image generators Stability and Midjourney has “advanced all copyright infringement and trademark claims” against the vendors and may implicate others too. Anthropic is also embroiled in a new class-action lawsuit from three authors who allege the vendor “misused their books and hundreds of thousands of others to train its AI-powered chatbot Claude.” And a San Francisco case has called attention to the darker side of AI, with the City Attorney filing suit against the owners of non-consensual deepfake pornography websites. Maybe the next big boom in AI is going to be hiring a lawyer.
Next Up: More of the week’s AI news and what it means for busy devs.
💻 Development
OpenAI Launches Updated Benchmark: SWE-bench Verified
OpenAI claims this human-validated subset “more reliably evaluates AI models’ ability to solve real-world software issues.”
Guide: Free ML Tools for Beginners
If you’ve been too busy to get that PhD in machine learning, here’s a quick overview of free tools to help you ramp faster.
[GH] Intel RAG Foundry: Stop Reinventing the Wheel
Intel’s new repo offers an open-source framework that puts the standard components of the RAG dev lifecycle into a single pipeline.
Model Merging: Theory and Execution
This report covers the state of model merging—combining models for stronger performance—and this repo fuses 6 LLMs to try now.
Benchmarking CodeGen on HumanEval
This leaderboard rates the codegen abilities of various LLMs against the HumanEval benchmark. Repo here and full research paper here.
🤔 Interesting AI Projects, Research, and Updates
Guide: Model Calibration
This guide covers calibration (improving predictions), which differs from fine-tuning (improving performance using datasets).
Hermes 3: The First Full-Parameter Fine-Tuned Llama-3.1 405B
Shortly after the launch of Meta’s latest heavy-duty model, the researchers at Nous have fine-tuned the living daylights out of it.
Llama-3.1 8B → NVDA 4B: Chopping Small Models in Half
NVDA walks through the process of refining the 8B version of Llama-3.1 down to 4B using structured compression.
Pipelines for Pre- and Post-Training
Researcher Sebastian Raschka walks through the latest developments in pre- and post-training pipelines.
Using LLMS to Classify Every PDF on the Internet
Say, do you like the internet? This undergrad classified every PDF online using LLMs—an alternative to more-established methods like TF-IDF.
💼 Hiring and Community
Startups Hiring This Week:
- VP of AI → Infactory
- Sr. Research Engineer, Model Inference → Otter.ai
Mid-Markets Hiring This Week:
- Sr. Data Scientist/LLM Engineer → Qventus
- Sr. Eng Manager - AI → Discord
Enterprises Hiring This Week:
- Sr. Staff Engineer, GenAI → UBER
- Dir of Engineering, Core AI → RBLX
💡 Spotlight: Newly-Launched AI Startups
- DefConAI: $44M from Bessemer for military logistics
- Reliant AI: $11.3M from Inovia Capital for GenAI analytics
- BeyondMath: $8.5M from UP.Partners for AI engineering simulation
- FutureAI: $5.8M from PivotNorth Capital for UX generation
- Gradient Labs: $3.6M from LocalGlobe for AI support agents
🏭 Industry: M&A, Launches, Trends
AMD Acquires AI Server Maker ZT Systems for $4.9B
To compete with AI chip leader NVDA, AMD has placed a huge bet, likely using ZT’s data center expertise to accelerate product development.
GPT-4o Fine-Tuning Is Here
With millions of free tokens for both GPT-4o and 4o-mini until Sep 23.
Claude Cuts Costs with Prompt Caching
The newest feature for Anthropic’s model caches context across frequent API calls, which may save users 85%-90%.
MSFT to Train AIs On Your Apps; META Scraping the Web Today
MSFT will slurp up data from Copilot/Bing/MSN in October unless you opt-out. META just quietly launched a new web scraper for AI.
RAND Corporation: Why 80% of AI Projects Fail
Report: The majority of AI projects fail due to poor project scoping, lack of data, or a failure to focus on appropriate problems to solve.
MSFT’s New Small Models: Phi 3.5 Launches
3.5-mini-instruct, 3.5-MoE-instruct, and 3.5-vision-instruct are already hitting top performance benchmarks for small models.
⚖️ Copyright, IP, Licensing, and Regulation
Universal and META Sign “Expanded Global Agreement” for AI
While details are scarce, the formal announcement states the companies will partner to manage unauthorized AI-generated content.
Wyoming Mayoral Candidate Promises to Use AI to Run a City
In a new entry to our “Bad Ways to Run a City” series, Cheyenne mayoral candidate Victor Miller has vowed to use a customized GPT to govern.
About the Author
Hi there. My name is Andrew, and I work at Heavybit, the leading VC for developer-first startups. As the Editorial Lead, my goal with The Humans in the Loop is to find the most valuable and important AI news for developers and founders. The idea is to curate and bottom-line emerging trends from our perspective in 10 years of coaching and funding developer-facing companies. Email me and let me know what you think.
That’s all for this edition. In the meantime, please feel free to follow The Humans in the Loop on Twitter and LinkedIn.