Pixel 6 AI explained | Lex Fridman

here's the new pixel 6 pro from google you're now seeing the result of it running a computationally intensive neural network in real time that i put on there for testing purposes it's using tensorflow lite and the new tensor chip that is optimized for ai i am unboxing this ai because uh as a robotics and ai person i'm interested in seeing how innovation in ai hardware and software is increasingly taking over the smartphone space also unboxing ai makes me think of pandora's box the myth that serves as the metaphor for the mystery and the power of ai i know this is just a phone but let's pause for a second to think this computer has over 200 000 times the processing power of the computer that first landed humans on the moon and over 2 million times the ram we are engineering our way to super human intelligence one phone at a time one small step for phone and soon enough one giant leap for uh hybrid of machine and mankind let's talk about the specs here's the comparison of the pixel 6 to the pixel 6 pro across various specs highlighting what to me are the key differences in yellow and in green what are the key similarities so key differences 300 in price also the pixel 6 pro has a slightly higher display slightly higher resolution and a 120 hertz refresh rate versus the 90 hertz refresh rate though honestly i've been using both phones for almost a week now and i don't feel any difference between them the key similarity to me on the ai and the computational side is that they both have the same soc system on a chip the google tensor to me the ram and storage are very important but once you get to a certain threshold it really doesn't matter 12 gigabytes feels the same as 8 gigabytes and the same goes for the storage both phones have a 50 megapixel wide angle now that produces a 12 megapixel image because google combines the 2x2 pixel groups into single effective pixel to cut noise and improve color and dynamic range there's also the 12 megapixel ultrawide that's used to decrease noise for both the photos and the videos and the pro has a 48 megapixel telephoto lens which results in one key difference i think it provides a 4x optical zoom plus a 20x digital zoom via machine learning via their super resolution algorithm the rest to me is pretty much the same both phones feel amazing in my hand but of course me being who i am i care mostly about what's on the inside and that's the google tenser now let's talk about the tensor chip my main flagship phone for machine learning applications this year has been the samsung galaxy s21 ultra 5g pictured here with the pixel 6 and the pixel 6 pro the galaxy brain is powered by snapdragon 888 the pixel brain is powered by the new tensor chip both are truly amazing machines i think ai innovation in both hardware and software will be what matters in flagship smartphones over the next decade this is where the battle is let's now look at the details of the technical specs of the tensor system on a chip and also the philosophical vision behind its architecture the key components of the architecture of the tensor system on a chip are the cpus the gpu isp tpu contacts hub and the titan m2 depending on the application various components of this chip can be used at the same time leading to what google is calling heterogeneous computing for the cpu there is two big cpu cores with the cortex x1 there's two medium cpu cores with the a76 and there's four small cpu cores with the a55 this is in contrast with the most common design for the flagships which is one big cortex x1 core and three medium a78 cores it's funny that google says that having one big cpu core is good for benchmarks but uh not good for the experience it's funny because as you'll see in the benchmarks the pixel 6 actually performs really well on the single core geekbench 5 test performance wise the benefit of having two cortex x1 is that you can distribute a thermal budget across them so there's less overheating on intensive tasks like 4k 60fps video so in the initial test there's a lot less overheating so you'll be able to shoot video for much longer besides the cpus there's the gpu there's an upgrade in that there's the isp image signal processor that's optimized for image and video processing in terms of machine learning and then there's the more general machine learning engine that's the tensor processing unit the context hub does ultra low power ambient computing and the titan m2 does hardware security like i said i've been using the galaxy s21 ultra with the snapdragon 88 for many months now and so it's nice to take a look at some benchmarks for the cpu gpu and mpu for these two flagship chips now the big caveat here as you probably know is that benchmarks often don't reflect real-life performance so arguably that don't actually matter but the main takeaway story here is that these are both amazing chips in geekbench 5 cpu benchmark pixel 6 outperforms the galaxy s 21 on single core test and uh the galaxy s21 out performs pixel 6 on the multi-core test for the geekbench machine learning benchmark the snapdragon wins on the cpu and the gpu and the pixel 6 wins on the npu the geekbench machine learning benchmark by the way uses tensorflow lite and there's also the ai benchmark 4 that's specialized for machine learning it runs a huge number of different neural networks on the devices and there once again pixel 6 far outperforms galaxy s21 in fact it leads every other smartphone on the current ai benchmark for leaderboard again i think the takeaway here in terms of benchmarks is that pixel 6 does well on machine learning tasks and snapdragon does well on cpu gpu-centric tasks but they're both again incredible machines i think the important thing here is what does this heterogeneous computing enable in terms of software features and the pixel 6 provides a huge number of seamlessly integrated machine learning algorithms increasing the vibrancy of the color with the hdr plus for the images and hdr net for the video improving the accuracy and the efficiency of the face detection again both for images and video and then there's just a huge number of cool features like face on blur motion mode that adds blur to moving objects there's the magic eraser that's actually shown here on screen where you can select certain parts of the object they can be removed and then intelligently filled based on what the background is and for images there's real tone that's looking at skin color making sure this shows up looking great on photos honestly the video is where the fun is like i said hdr net that's an incredible use of neural networks i actually personally think super resolution algorithms are one of the coolest applications of computer vision in terms of its uh maybe simplicity and usefulness and impact and there's a huge number of applications outside of visual domain so speech automatic speech recognition you're talking about deployment of state-of-the-art asr algorithms that pay attention to context pauses is able to do noise removal and on the language side there's neural machine translation obviously google is taking natural language nlp really seriously in both the textual domain in speech that's audio and again back to images and video this is incredible leveraging to do heterogeneous computing on ai hardware to enable all kinds of cool computational photography features okay let's look at some takeaways my two favorite ai chips for android now are the google tensor and the snapdragon 888 time will tell which wins for which applications but for now competition in the space is great for everyone if you want me to talk about other ai systems or about running machine learning code on this and on other phones let me know i'll close with a quote from eliezer ytkowski by far the greatest danger of ai is that people conclude too early that they understand it thanks for watching and hope to see you next time you