Jože Rožanec

NVIDIA have had a big week on the research front, first in LLM memory, now they look like they made a breakthrough when considering latency with CoT models… If the research holds true, then our frontier CoT models may be about to get a lot faster. The paper, Fast-ThinkAct, tackles one of the biggest practical bottlenecks in embodied AI: chain-of-thought helps generalisation… but it’s way too slow for real-time control. Instead of generating ~250 tokens of verbose reasoning every step, Fast-ThinkAct compresses “thinking” into a tiny latent plan (e.g., 6 continuous tokens) that still captures the critical spatial + temporal structure needed to act. What’s the trick? ✅ Verbalisable latent reasoning Latents are decodable into text via a “verbaliser” during training, so, the model learns compact internal thoughts that are still aligned with high-quality reasoning. ✅ Preference-guided distillation (teacher → student) A teacher generates reasoning traces of varying quality; the student learns the good patterns and suppresses the junk via preference-style optimisation. ✅ Trajectory-aligned visual planning They don’t just distil language, they align visual plan representations too, transferring the “where to move” signal directly. The result: reasoning that’s actually deployable 📉 Up to 89.3% lower inference latency 🏎️ ~9.3× faster than ThinkAct …and performance still holds (or improves) on long-horizon manipulation + reasoning benchmarks. 🤓Ok cool, but what does it mean?😎 Basically, if I’m reading this right, through distillation, the big model is used to “model” CoT reasoning, with the distilled model essentially mirroring the behaviour, instead of repeating it. The way I picture it: The teacher teaches how to build a map, the student then learns how to build compressed “tiny” maps as required… Let me know what you think. More info here: https://lnkd.in/gaiqzMAM #EmbodiedAI #Robotics #VisionLanguageAction #Reasoning #InferenceEfficiency #NVIDIA #LLMs #VLMs #ai #sovereignai Sovereign Australia AI

16

2 Comments

This is the fundamental design flaw of the LLM model, just as it is with human beings. I don't know everything, even though I have likely received far more input from my senses than even the best LLM model. There are area I have strong expertise in, there are areas when I may have some working knowledge, there are some areas where I am completely ignorant, to the extent of not even knowing the degree of my ignorance. There are some variations in capacity of the human brain, and people who train in cross-discipline thinking often have better success at maintaining a holistic view of the world because they are better at drawing inferences and metaphors, but there is still very much an upper limit. This limit exists primarily because it takes time and energy to think. If you view your brain as a graph of connections. In graph theory, this means that the number of nodes that you can traverse remains relatively constant. Some people develop deep specialisations in certain areas, at the expense of more limited general knowledge, some develop broad generalisations but very shallow specialisation. Most are somewhere in the middle. People with higher intelligence may be able to branch out to more nodes, but the mechanisms generally don't change - they just become more holistic. This will ALWAYS be a tradeoff. The number of nodes that can be explored is limited. A generalised LLM (what AGI is trying to produce) has a limit to the depth that it can explore before it branches off to irrelevancies. A specialised LLM is good in its curated domain, but is an idiot outside of that domain. This is one reason why you're seeing more networks of LLM working in concert; just as with humans, specialised LLMs can often cover a broader spectrum than a single latent space can, at the expense of holistic coverage and cost. Ontological grounding with knowledge graphs can also serve much of this role.

49

2 Comments

This diagram illustrates the Retrieval-Augmented Generation (RAG) pipeline, detailing how Large Language Models (LLMs) overcome their inherent memory limitations to provide grounded, relevant, and personalized answers. The Four Stages of Semantic Memory Vectorization (Encoding): All incoming data—whether it's a user query, a document, or a past interaction—is passed through an Embedding Model (Eactor Tilon). This process converts the text into Magical Vectors, which are high-dimensional numerical coordinates (e.g., [0.7, 0.3, -0.9, ...]). The key is that semantic meaning is preserved: concepts like "Dog" and "Cat" are positioned closer together in the Vector Space than "Dog" and "House." Storage & Indexing: These vectors, representing the LLM's long-term knowledge, are permanently stored in the Vector Database. This database is indexed for speed, essentially organizing related concepts so they can be retrieved efficiently based on their numerical proximity. Vector Database (Retrieval): When the user asks a new question ("What are pets?"), the system performs a Similarity Search. It looks for the stored vectors closest to the new query vector. The result is the retrieved context (e.g., from an internal document or past memory), represented here as a treasure chest of relevant information. Context Injection (RAG) & Answer: The retrieved context is instantly delivered to the LLM's Brain. The LLM then combines its own static, generalized knowledge with the fresh, targeted information from memory to generate a Tailored Answer. This guarantees the response is both accurate and relevant to the user's specific history or knowledge base. #cleanethicalcode #RAG #AIMemory #LLM #VectorDatabse #Encoding #Storage

4

🚀 Exploring PINNeAPPle: A Collaborative Approach to Physics-Informed Machine Learning 🚀 I recently worked through a notebook demonstrating PINNeAPPle, a library designed for Physics-Informed Machine Learning (PIML). It’s structured for researchers and developers looking to integrate physical simulation and machine learning in a practical workflow. Here are some technical highlights and experiences from the exploration: Geometry and Data Preparation - PINNeAPPle Geom: Creation of simple 3D geometries (e.g., boxes with holes) and conversion into PINN-compatible datasets. - Boundary Condition Automation: Facilitates the transition from CAD models to physical simulation. Models and Architectures - Inverse PINN: Learning unknown physical parameters. - PIELM (Physics-Informed Extreme Learning Machine): Applied to the Burgers’ equation for inverse parameter identification. - Vanilla PINN: Base framework for ODEs with physical constraints. - X-TFC (eXtreme Theory of Functional Connections): Integration of “hard” boundary conditions. - Neural Operators (FNO & DeepONet): Learning mappings between function spaces, demonstrated on 1D Maxwell wave dynamics. Training and Solvers - PINNeAPPle PINN Compiler: Automatic generation of loss functions for PDEs, simplifying the training pipeline. - PINNeAPPle Train: Training models with combined physical and supervised losses. - PINNeAPPle Solvers: Integration of classical methods like FEM (1D Poisson) and FVM (1D advection), allowing comparisons with ML-based approaches. Data Management and Integration - PINNeAPPle PDB + Data: Collection of scientific data (e.g., NASA POWER), conversion into PhysicalSample, and efficient storage in Zarr for large-scale use. Example Applications - 3D Heat Conduction Problems: Setting up volumetric heat sources, direct integration with physical models, and automatic learning of solutions. 💡 This notebook serves as a starting point for exploration and collaboration in PIML projects, allowing testing of different architectures, solvers, and datasets in an integrated manner. It’s a useful tool for scientists and engineers working at the intersection of physical simulation and machine learning. 📎 Access the full notebook here: https://lnkd.in/dFmcsQid #PINNeAPPle #PhysicsInformedML #MachineLearning #DeepLearning #PDEs #Simulation #Python #CollaborativeResearch

130

7 Comments

🧬 DeepMind’s EvoDiff: The Dawn of Generative Protein Design After transforming structure prediction with AlphaFold, DeepMind is once again expanding biology’s horizons, this time with EvoDiff, an AI model that doesn’t just predict protein structures… it creates them. ⚙️ What makes EvoDiff revolutionary: EvoDiff applies the power of diffusion models, the same technology behind generative AI in images and text, to the world of proteins. But instead of generating pictures or sentences, EvoDiff generates entirely new protein sequences that are both plausible and functional. 🧠 How it works: EvoDiff learns from evolutionary patterns in natural proteins, capturing how amino acids change while preserving structure and function. Using this learned “evolutionary grammar,” it can design novel proteins that nature hasn’t yet explored, opening new frontiers in synthetic biology, enzyme engineering, and biotherapeutics. Unlike traditional design tools, EvoDiff doesn’t need structural templates; it designs from first principles, guided by the logic of evolution itself. 🌍 Why it matters: EvoDiff represents a major leap toward AI-driven molecular creativity, where we move beyond prediction to purposeful invention. It blurs the line between biology and computation, allowing scientists to explore the vast space of possible proteins faster than evolution ever could. We’re not just decoding life anymore, we’re designing its next generation. #EvoDiff #AIBiology #ProteinDesign #StructuralBiology #Bioinformatics #SyntheticBiology #DeepLearning #MolecularDesign #ComputationalBiology #AIInnovation https://lnkd.in/dt7iGnSQ

13

Should we trust Science or Data? For a long time, we acted like we had to choose. The latest developments in AI are proving that this is a false dichotomy. We are entering an era where the two don't just coexist—they amplify each other. Physics-Informed Neural Networks (PINNs) are a prime example. While traditional Machine Learning can sometimes get lost in "local optima" or ignore real-world constraints, PINNs anchor data-based learning to the laws of physics. We saw this struggle early on during my time at John Deere. We constantly asked: How do we balance human intuition, the science of agronomy, and AI-driven insights? That same debate is happening in every boardroom today. LLMs are already closing the gap between experiential-knowledge and science. PINNs are the next step in grounding digital predictions in physical reality. Check out this visual aid below - it perfectly captures what we gain when these worlds converge. #AI, #DataScience

59

8 Comments

Monte Carlo simulations are computational methods that model uncertainty and variability through repeated random sampling. They enable researchers to estimate probabilities of success and predict outcomes in complex biological systems where deterministic approaches fall short. By performing over 10,000 iterations per scenario, these simulations provide robust estimates with statistical confidence intervals, typically at 95%

1