Issue #02 | Reasoning, Rethinking & Research

About This Newsletter

This is a weekly newsletter that I hope to release mid-week, with a focus on amplifying relevant news and discussions around model choice and the AI engineer experience. I want to balance professional content with personal perspectives - and it will take a while to find my voice, but I hope you find the information useful.

Note that while I do promote content from my work, the perspectives shared (and any errors made in this context) are mine alone.❤️

What's New This Week

Want to connect with the community around Azure AI Foundry? You can join the Discord to meet others, share projects and get help. Browse the Forum to find upcoming AMAs.

In Azure AI Foundry

1. Sarah Bird , CPO of Responsible AI at Microsoft, shared the 2025 Reponsible AI Transparency Report which provides insights into how we build AI responsibly, how we make decisions, support customer adoption, and learn & evolve with usage.
2. Chang Liu announced the availability of Safety Leaderboards in Azure AI Foundry - making model safety a first-class criterion (alongisde quality, cost, and throughput) for helping developers make model choice decisions.
3. Rory Preddy☕️ released a new Java Spring Boot sample that showcases video generation using Azure OpenAI's Sora model. Go from prompt to playback with a user-friendly web application interface and built-in controls for resolution and duration!
4. Govind Kamtamneni and team announced general availability of Azure OpenAI codex-mini in Azure AI Foundry models - providing fast, scalable code generation from the CLI! Check out this detailed guide from the author, to try it out yourself.
5. Seth Juarez is back with the AI Show live every Monday! Want to learn about Model Router (in preview)? Watch my segment on the show as I walk through a demo that shows why smarter models (multi-model + smart routing) simplify model selection.

In GitHub Models

Did you know that GitHub has a Models Marketplace powered by Azure AI Foundry Models? It allows you to get started exploring cutting-edge models with just a GitHub account. Want a 15-minute quick intro? Check out my Model Mondays episode then read on for updates.

1. Jun 6, 2025 - You can now run model evaluations with the Models CLI
2. Jun 12, 2025 - AI prompt editor & evals tooling w/ multi-turn convos.
3. Jun 18, 2025 - Use multiple custom variables in AI evalutions tooling
4. Jun 24, 2025 - GitHub Models now supports moving beyond free limits

Spotlight: Reasoning Models

Reasoning models are a new class of language models that are trained using large-scale reinforcement learning, to think deeply before they respond. They can break down complex problems into simpler tasks that can be executed by general purpose models, making them useful for agentic AI. As with any model, you need to assess tradeoffs between token usage and response accuracy - having an applied understanding of the space, helps.

We covered this in two past episodes on Model Mondays! In March, Jennifer Marsman gave an introduction to reasoning models using a complex scheduling problem as a use case - and solving it with the Azure OpenAI o-series of reasoning models.

Last week, we were joined by Marlene Mhangami as she walked us through the steps for building a Deep Researcher agentic application using the DeepSeek-R1 model with the LangGraph framework for orchestration. Check out the Deep Research sample here - and watch the talk below for links to a hands-on lab guide to help you explore it on your own.

And next week, we'll be joined by Mojan Javaheripi, Ph.D. from the Microsoft Research team as we take a look at Phi-4 reasoning models suitable for use in resource-constrained devices. These models are trained with supervised fine-tuning of Phi-4 and show strong (comparable) performance to larger models, across a wide range of reasoning tasks.

1. Register here to join us on Monday for the livestream
2. Register here to join us on Friday for the AMA
3. Read the Technical Report Here on Phi-4 reasoning

In Other News

Here are some of the other items that caught my eye this past week.

Paper To Read. Speaking of reasoning models and research, the paper causing the most buzz in this context comes from Apple research.

Published earlier in June, "The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity" paper states this:

"Through extensive experimentation across diverse puzzles, we show that frontier LRMs face a complete accuracy collapse beyond certain complexities. Moreover, they exhibit a counter-intuitive scaling limit: their reasoning effort increases with problem complexity up to a point, then declines despite having an adequate token budget".

This has led to numerous opinions being written - some arguing that this shows reasoning models are ineffective, while others argue that the paper should be read in the context of the benchmarks and constraints (token costs) that informed the results. Here are some perspectives that I found useful to get a broader context:

1. Andreas Horn (IBM) reviews this in the context of an Apple vs. Anthropic debate.
2. Xavier (Xavi) Amatriain (Google) provides context and links to a detailed rebuttal
3. Ashish Bhatia (ex-MSFT) has a great summary with similar areas for disagreement
4. Ethan Mollick (Wharton) links to a great thread on twitter on why this is a token issue

Paper To Read. Vibe coding can generate technical debt - but does AI usage also accumulate cognitive debt? This paper created a lot of buzz and quite a lot of misinterpretation of results. Learn more.

Published as a preprint in June, "Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task", the abstract contained this takeaway: "While LLMs offer immediate convenience, our findings highlight potential cognitive costs. Over four months, LLM users consistently underperformed at neural, linguistic, and behavioral levels." leading to a lot of hot takes.

The author Nataliya Kosmyna, Ph.D also published this takeaways post along the brainonllm.com website to create clarity - I recommend you read the FAQ before you weigh in yourself. This is a very early research study still to be peer-reviewed and with a limited number of participants - so jumping to conclusions feels premature. My takeaway is that this is a fertile area for research from both HCI (user experience) and behavioral psychology (the motivation to work) to understand how the future of work looks like in the era of AI.

Talk To Watch: Andrej Karpathy "Software in the era of AI"

I posted about this here minutes after the talk went live - this was the analogy he gave that resonated with me in terms of understanding the evolution of software.

• Software 1.0 = code programming computers = GitHub
• Software 2.0 = weights programming neural nets = Hugging Face
• Software 3.0 = prompts programming LLMs = "The latest programming lang is English"

From a business adoption perspective, his comparison of the AI ecosystem to a "utility" like elecricity - was eye-opening. If we think about tokens as the new billing metric for the utility, we can really use familiar constructs to understand implications for scale adoption. Highly recommend watching the talk.

Today I Learned: "Context Engineering >> Prompt Engineering"

Speaking of Andrej Karpathy, I was reminded of this post from Ethan Mollick yesterday stating "The hottest discussion in AI is about context engineering" and referencing this post from Karpathy. The takeaway is that context is about providing AI the right information at the right time, to make the right decisions. And in an agentic world where autonomy in decision making is key, the ability to provide the right context is critical.

This is driving the popularity of standards like Model Context Protocol (the subject of my next newsletter) but is also driving a lot of discussions around architectures, design patterns, pros and cons. Two posts that I read recently that made me think:

1. Don't Build Multi Agents - Cognition.AI presents a framework for AI Agents guided by 2 Context Engineering principles: share context and actions carry implicit decisions.
2. rUv Cohen - wrote a good post about the article above and articulating his ideas on agent swarms with emphasis on guided orchestration with timely context sharing.

As we drive deeper into agentic AI, the notion of Context Engineering - and tools, principles, and practices behind it, will take shape.

Essaying Forth

My observation is that we learn more by doing than by reading. So I want to end each issue with one task or exercise you can try to build your model IQ. Essay forth - and conquer it!

Want to learn about reasoning models with a hands-on lab using the OpenAI o-series of models? Visit https://aka.ms/discuss/build25-lab333 to get the slides, and links to a self-guided lab that will leave you with a sandbox to explore more. Here's what you'll learn!