Ep548_The Pixel Path: From Perception to Action, and the Future of Intelligent Robots with Nizar

Stewart Alsop interviews Nizar, CEO of Pixel Robotics, on the Crazy Wisdom Podcast to explore the intersection of AI, robotics, and perception. The conversation covers a wide range of technical topics including how transformers enable multimodal representation across text, images, and voice, the role of world models in predicting physical interactions, the advantages of diffusion models over traditional LLMs for certain applications, and the challenges of achieving real-time processing for robotics applications. Nizar explains Pixel Robotics' work on creating accurate 3D meshes from smartphone cameras for companies like L'Oréal, moving away from specialized sensors to make the technology more accessible through sophisticated algorithms, and discusses the future of robotics as closing the perception-action loop to enable robots to perform real tasks beyond simple demonstrations. To find out more visit Pixel Robotics' website . Timestamps 00:00 Stewart welcomes Nizar, CEO of Pixel Robotics, discussing what a pixel is as the smallest visual unit on screens composed of red green and blue colors 05:00 Discussion of perception systems and how logarithmic laws help compress signals in both human and artificial systems, exploring normalization layers and sigmoid functions in deep learning 10:00 Exploring how transformers unified different data modalities including text voice and images, creating common representations through methods like contrastive learning 15:00 Nizar explains transformers as brute force learning systems with room for improvement through focused attention mechanisms and knowledge graphs rather than processing everything 20:00 Conversation about loss functions local minima versus global minima and how mixture of experts uses specialized small models instead of one massive generalist network 25:00 Discussion of deterministic versus probabilistic systems and how explicitly defined task graphs often outperform orchestrator-based approaches in AI systems 30:00 Exploring world models as predictive physics-based systems that learn environmental flows and transformations, complementing rather than replacing language models 35:00 Nizar discusses real-time processing challenges for robotics requiring millisecond responses with small memory footprints using vision transformers for faster experimentation 40:00 Pixel's work creating three d meshes from smartphone cameras for companies like L'Oreal, moving away from specialized sensors toward accessible software-based solutions 45:00 Explanation of different three d representations including voxels point clouds and meshes, with meshes being optimal for manipulation and rendering in applications 50:00 Future direction involves closing perception-action loops in robotics, moving beyond dancing toy robots toward practical multimodal systems that perform real tasks 55:00 Pixel's goal is democratizing high-quality three d scanning through smartphones, making mesh creation accessible to unlock applications in gaming cinema and virtual showrooms Key Insights 1. Pixel Robotics derives its name from combining perception and action in robotics, where the pixel represents the digital perception component and robotics represents the physical action component. The pixel serves as a metaphor for how robots must quantize and digitize continuous analog information from the real world into discrete units that computer systems can process, similar to how pixels are the fundamental building blocks of images on a screen. This quantization process is essential because numerical systems cannot work with truly continuous data and must convert reality into tractable digital representations that algorithms can manipulate. 2. The transformer architecture has created a fundamental unification in how different types of data can be represented and processed across mu