Trends in AI and LLM Development

AI is rapidly moving from passive text generators to active decision-makers. To understand where things are headed, it’s important to trace the stages of this evolution. 1. 𝗟𝗟𝗠𝘀: 𝗧𝗵𝗲 𝗘𝗿𝗮 𝗼𝗳 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗙𝗹𝘂𝗲𝗻𝗰𝘆 Large Language Models (LLMs) like GPT-3 and GPT-4 excel at generating human-like text by predicting the next word in a sequence. They can produce coherent and contextually appropriate responses—but their capabilities end there. They don’t retain memory, they don’t take actions, and they don’t understand goals. They are reactive, not proactive. 2. 𝗥𝗔𝗚: 𝗧𝗵𝗲 𝗔𝗴𝗲 𝗼𝗳 𝗖𝗼𝗻𝘁𝗲𝘅𝘁-𝗔𝘄𝗮𝗿𝗲 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻 Retrieval-Augmented Generation (RAG) brought a major upgrade by integrating LLMs with external knowledge sources like vector databases or document stores. Now the model could retrieve relevant context and generate more accurate and personalized responses based on that information. This stage introduced the idea of 𝗱𝘆𝗻𝗮𝗺𝗶𝗰 𝗸𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗮𝗰𝗰𝗲𝘀𝘀, but still required orchestration. The system didn’t plan or act—it responded with more relevance. 3. 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗔𝗜: 𝗧𝗼𝘄𝗮𝗿𝗱 𝗔𝘂𝘁𝗼𝗻𝗼𝗺𝗼𝘂𝘀 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 Agentic AI is a fundamentally different paradigm. Here, systems are built to perceive, reason, and act toward goals—often without constant human prompting. An Agentic system includes: • 𝗠𝗲𝗺𝗼𝗿𝘆: to retain and recall information over time. • 𝗣𝗹𝗮𝗻𝗻𝗶𝗻𝗴: to decide what actions to take and in what order. • 𝗧𝗼𝗼𝗹 𝗨𝘀𝗲: to interact with APIs, databases, code, or software systems. • 𝗔𝘂𝘁𝗼𝗻𝗼𝗺𝘆: to loop through perception, decision, and action—iteratively improving performance.    Instead of a single model generating content, we now orchestrate 𝗺𝘂𝗹𝘁𝗶𝗽𝗹𝗲 𝗮𝗴𝗲𝗻𝘁𝘀, each responsible for specific tasks, coordinated by a central controller or planner. This is the architecture behind emerging use cases like autonomous coding assistants, intelligent workflow bots, and AI co-pilots that can operate entire systems. 𝗧𝗵𝗲 𝗦𝗵𝗶𝗳𝘁 𝗶𝗻 𝗧𝗵𝗶𝗻𝗸𝗶𝗻𝗴 We’re no longer designing prompts. We’re designing 𝗺𝗼𝗱𝘂𝗹𝗮𝗿, 𝗴𝗼𝗮𝗹-𝗱𝗿𝗶𝘃𝗲𝗻 𝘀𝘆𝘀𝘁𝗲𝗺𝘀 capable of interacting with the real world. This evolution—LLM → RAG → Agentic AI—marks the transition from 𝗹𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝘂𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱𝗶𝗻𝗴 to 𝗴𝗼𝗮𝗹-𝗱𝗿𝗶𝘃𝗲𝗻 𝗶𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲.

A lot has changed since my #LLM inference article last January—it’s hard to believe a year has passed! The AI industry has pivoted from focusing solely on scaling model sizes to enhancing reasoning abilities during inference. This shift is driven by the recognition that simply increasing model parameters yields diminishing returns and that improving inference capabilities can lead to more efficient and intelligent AI systems. OpenAI's o1 and Google's Gemini 2.0 are examples of models that employ #InferenceTimeCompute. Some techniques include best-of-N sampling, which generates multiple outputs and selects the best one; iterative refinement, which allows the model to improve its initial answers; and speculative decoding. Self-verification lets the model check its own output, while adaptive inference-time computation dynamically allocates extra #GPU resources for challenging prompts. These methods represent a significant step toward more reasoning-driven inference. Another exciting trend is #AgenticWorkflows, where an AI agent, a SW program running on an inference server, breaks the queried task into multiple small tasks without requiring complex user prompts (prompt engineering may see end of life this year!). It then autonomously plans, executes, and monitors these tasks. In this process, it may run inference multiple times on the model while maintaining context across the runs. #TestTimeTraining takes things further by adapting models on the fly. This technique fine-tunes the model for new inputs, enhancing its performance. These advancements can complement each other. For example, an AI system may use agentic workflow to break down a task, apply inference-time computing to generate high-quality outputs at each step and employ test-time training to learn unexpected challenges. The result? Systems that are faster, smarter, and more adaptable. What does this mean for inference hardware and networking gear? Previously, most open-source models barely needed one GPU server, and inference was often done in front-end networks or by reusing the training networks. However, as the computational complexity of inference increases, more focus will be on building scale-up systems with hundreds of tightly interconnected GPUs or accelerators for inference flows. While Nvidia GPUs continue to dominate, other accelerators, especially from hyperscalers, would likely gain traction. Networking remains a critical piece of the puzzle. Can #Ethernet, with enhancements like compressed headers, link retries, and reduced latencies, rise to meet the demands of these scale-up systems? Or will we see a fragmented ecosystem of switches for non-Nvdia scale-up systems? My bet is on Ethernet. Its ubiquity makes it a strong contender for the job... Reflecting on the past year, it’s clear that AI progress isn’t just about making things bigger but smarter. The future looks more exciting as we rethink models, hardware, and networking. Here’s to what the 2025 will bring!

Recent research is advancing two critical areas in AI: autonomy and reasoning, building on their strengths to make them more autonomous and adaptable for real-world applications. Here is a summary of a few papers that I found interesting and rather transformative: • 𝐋𝐋𝐌-𝐁𝐫𝐚𝐢𝐧𝐞𝐝 𝐆𝐔𝐈 𝐀𝐠𝐞𝐧𝐭𝐬 (𝐌𝐢𝐜𝐫𝐨𝐬𝐨𝐟𝐭): These agents use LLMs to interact directly with graphical interfaces—screenshots, widget trees, and user inputs—bypassing the need for APIs or scripts. They can execute multi-step workflows through natural language, automating tasks across web, mobile, and desktop platforms. • 𝐀𝐅𝐋𝐎𝐖: By treating workflows as code-represented graphs, AFLOW dynamically optimizes processes using modular operators like “generate” and “review/revise.” This framework demonstrates how smaller, specialized models can rival larger, general-purpose systems, making automation more accessible and cost-efficient for businesses of all sizes. • 𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥-𝐀𝐮𝐠𝐦𝐞𝐧𝐭𝐞𝐝 𝐑𝐞𝐚𝐬𝐨𝐧𝐢𝐧𝐠 (𝐑𝐀𝐑𝐄): RARE integrates real-time knowledge retrieval with logical reasoning steps, enabling LLMs to adapt dynamically to fact-intensive tasks. This is critical in fields like healthcare and legal workflows, where accurate and up-to-date information is essential for decision-making. • 𝐇𝐢𝐀𝐑-𝐈𝐂𝐋:: Leveraging Monte Carlo Tree Search (MCTS), this framework teaches LLMs to navigate abstract decision trees, allowing them to reason flexibly beyond linear steps. It excels in solving multi-step, structured problems like mathematical reasoning, achieving state-of-the-art results on challenging benchmarks. By removing the reliance on APIs and scripts, systems like GUI agents and AFLOW make automation far more flexible and scalable. Businesses can now automate across fragmented ecosystems, reducing development cycles and empowering non-technical users to design and execute workflows. Simultaneously, reasoning frameworks like RARE and HiAR-ICL enable LLMs to adapt to new information and solve open-ended problems, particularly in high-stakes domains like healthcare and law. These studies highlight key emerging trends in AI: 1. APIs and Simplifying Integration: A major trend is the move away from API dependencies, with AI systems integrating directly into existing software environments through natural language and GUI interaction. This addresses one of the largest barriers to AI adoption in organizations. 2. Redefining User Interfaces: Traditional app interfaces with icons and menus are being reimagined. With conversational AI, users can simply ask for what they need, and the system executes it autonomously. 3. Tackling More Complex Tasks Autonomously: As reasoning capabilities improve, AI systems are expanding their range of activities and elevating their ability to plan and adapt. As these trends unfold, we’re witnessing the beginning of a new era in AI. Where do you see the next big research trends in AI heading?

🧠 The evolution of AI reasoning is fascinating - and with all the buzz about AI Agents, we're seeing a rapid shift from "fast thinking" to "deliberate reasoning" in Large Language Models. Most LLMs today operate like System 1 (from Kahneman & Tversky's seminal work) thinking in humans - quick, intuitive, and sometimes prone to errors. But 2024 has brought exciting developments in pushing these models toward System 2 thinking - the slow, methodical reasoning we use for complex problems. 📝 Chain of Thought was the first breakthrough - imagine teaching someone by saying "show your work." Instead of jumping to answers, we prompt LLMs to write out their step-by-step reasoning. Simple but powerful: "First, I'll calculate X... Then, considering Y..." This dramatically improved accuracy on complex tasks. 🌳 Graph of Thought took this further - instead of a linear path, it explores multiple reasoning routes simultaneously. Think of it like brainstorming where you map out different approaches to a problem, evaluate each path, and choose the most promising one. This helps catch errors and find innovative solutions. 🎲 And now, researchers have introduced Monte Carlo Tree Search for LLM reasoning. Think of it like a chess grandmaster exploring possible moves, but instead of game positions, we're dealing with reasoning steps. Each potential path is tested hundreds of times, and the most promising ones are explored further. The implications? We're getting closer to AI systems that can tackle complex reasoning tasks with the kind of methodical approach that humans use for critical thinking. We're already seeing this with models like o1 that crush benchmarks for PhD level reasoning compared to GPT 4-class models, but the use cases for these will be different - as in, these models aren't just a 'more powerful GPT 4' but are useful for a different set of problems or applications, especially those requiring precise logical reasoning or complex problem-solving. And as GPT 4-class models start getting commoditized, more model providers will lean into developments in this area.

AI is racing ahead, working its way into every part of how we work, live, and innovate. But here’s the kicker: AI isn’t a one-size-fits-all solution. Instead, it’s about using the right tool for the right task.    Deloitte’s Tech Trends 2025 report (https://deloi.tt/41Ze6bE) highlights some of the ways we can expect AI to evolve in the coming year:    🟢 Large Language Models: An estimated 70% of surveyed organizations are actively exploring or implementing LLM use cases. LLMs remain the gold standard for big-picture tasks like general-purpose chatbots or complex simulations (think scientific research or space exploration).    🟢 Small Language Models: More efficient, cost-effective, and perfect for targeted tasks than their larger counterparts, SLMs are trained by organizations for tasks like summarizing inspection reports or quickly retrieving insights from business data.     🟢 Agentic AI: AI agents aren’t just answering questions, they’re taking actions with tasks like preparing financial reports, booking flights, or applying for grants— all on their own. As we shift from augmenting knowledge to augmenting execution, “There’s an agent for that” may be the new “There’s an app for that!”    Great collaborating with Bill Briggs, Kelly Raskovich, Mike Bechtel, Abhijith Ravinutala, Nitin Mittal, Lou DiLorenzo, and more on this!

Large language models (LLMs) are deeply trained models powered by artificial intelligence (AI), whose applications are expanding in the medical field, in which they exhibit impressive performance to process tasks in natural language such as answering questions, summarizing clinical situations, or even doing a diagnosis from a case. Multidisciplinary team meetings (MDTs) are a cornerstone in oncology where several specialists discuss patients' cancer cases but are also time- and resource-consuming. Recent improvements of LLMs could represent an opportunity to augment these meetings by leveraging AI to analyze patient data and suggest personalized treatment options. We hypothesized that LLMs can provide treatment recommendations comparable with those emitted by medical doctors during MDTs. This study aimed to assess the accuracy of LLMs to generate appropriate treatment options for patients with early breast cancer (BC) on the basis of their medical records. Interesting study in ASCO Clinical Cancer Informatics evaluating the accuracy of LLM-based treatment recommendations for patients with early-stage breast cancer: https://lnkd.in/ePK44kkx Additional resources on companies working in Biomedical LLM development for those interested: Hippocratic AI John Snow Labs Insilico Medicine AION Labs GenBio AI DeepSeek AI Verily Life Sciences Cerebras Systems Movano Health Microsoft BioNTech SE and InstaDeep OpenAI Anthropic Perplexity Google DeepMind 

🚀 AI Agents: 4 Trends to Watch in 2025🌍💡 AI agents are revolutionizing industries, moving beyond copilots to autonomous digital workers 🤖. As we enter 2025, four key trends are shaping the AI agent landscape: 1️⃣ Big Tech & LLM Developers Dominate General-Purpose Agents 🔹 Tech giants (OpenAI, Anthropic, etc.) are driving AI advancements, making agents cheaper, more powerful, and widely available. 🔹 400M weekly users on ChatGPT showcase the massive distribution advantage. 🔹 Enterprise adoption is increasing, but big tech’s dominance pressures startups to specialize. 2️⃣ Private AI Agent Market Moves Toward Specialization 🔹 Horizontal AI applications (customer support, software development) are crowded – differentiation is key. 🔹 Industry-specific AI agents in healthcare, finance, compliance, and logistics are poised for growth. 🔹 Deeper workflow integrations & leveraging proprietary data will create competitive moats. 3️⃣ AI Agent Infrastructure Stack Crystallizes 🔹 The AI agent ecosystem is evolving into a structured stack with specialized solutions: ✅ Data curation (LlamaIndex, Unstructured) ✅ Web search & tool use (Browserbase) ✅ Evaluation & observability (Langfuse, Coval) ✅ Full-stack AI agent development platforms gaining traction 4️⃣ Enterprises Shift from Experimentation to Implementation 🔹 63% of enterprises place high importance on AI agents in 2025. 🔹 Challenges remain: Reliability & security (47%), Implementation (41%), Talent gaps (35%). 🔹 Solutions: Human-in-the-loop oversight, stronger data infrastructure, and enterprise-grade agent platforms. 🚀 2025 is a breakout year for AI agents – the shift from copilots to autonomous digital workers is happening now! 📈 #AIAgents