Build an AI Robot Using Android Phone with ESP32 & micro:bit

Bring your own personal artificial intelligence companion to life! In this hands-on course, you’ll learn how to build an interactive, expressive robot called SAM (Simulated Autonomous Multi-Intelligence) powered by an Android smartphone and an ESP32 development board, or, a Micro:bit kit.  Using MIT App Inventor as the core development tool, you’ll discover just how easy it is to build complex AI applications through its intuitive block-based coding system—perfect for beginners and accessible to anyone without prior programming experience.

The Android phone will serve as the “brain” of your robot, enabling speech recognition, natural conversation, personality, text-to-speech, facial tracking, object recognition and facial expressions displayed right on the screen. The ESP32 (or Micro:bit aka Microbit ) acts as the robot’s body, controlling motors and simple hardware that bring your creation to life. You’ll only need a handful of inexpensive, easily available components to assemble the physical robot, making this project both affordable and beginner-friendly.

All software used in this course is completely free.

MIT App Inventor and Microsoft MakeCode are web-based tools that run directly in your browser and do not require installation. The Arduino IDE is also free to download and use. For advanced AI features, you may optionally choose to use the OpenAI API, which requires creating an API key. This is optional, and usage costs are typically very low. Free or local alternatives can also be used depending on your setup.

Note that this course is designed to be flexible, and you are NOT required to buy any hardware unless you want to.

You can choose any ONE of the following three approaches:

Three Ways to Build Your AI Robot (Important!)

(1) Standalone SAM Robot (No Extra Hardware Required)

• Build SAM using only an Android phone using MIT App Inventor block-base coding
• Focus on AI, speech, vision, and Large Language Models (LLMs)
• Ideal if you want to learn AI robot concepts without spending on electronics

(2) SAM + ESP32 Robot

• Use an ESP32 to control motors – program using Arduino IDE
• The Android phone remains the AI “brain” – built using MIT App Inventor block-based coding
• Great for beginners who want hands-on robotics with minimal complexity
• SAM (android phone) sends Bluetooth Classic commands to ESP32

(3) SAM + micro:bit Robot

• Use a micro:bit for more advanced control and expansion – program using Microsoft Makecode block-based coding
• Suitable for students who want greater flexibility and performance
• Includes the potential to use built-in sensors already available on the micro:bit kit for richer interactivity
• SAM (android phone), built using MIT App Inventor block-based coding, sends  Bluetooth BLE commands to the micro:bit

Buying ESP32 or micro:bit hardware is NOT compulsory.

You can fully follow and benefit from this course by building the standalone SAM robot using just your Android phone.

This course was inspired by the LOOI robot. LOOI demonstrates the powerful idea of using a smartphone as the central intelligence of a personal robot—leveraging the phone’s screen, camera, microphone, speakers, and AI capabilities to create natural interaction, expressive visuals, and conversational behavior. Its approachable form factor and emphasis on personality, presence, and human-like engagement show how compelling robots can be built around devices many people already own. SAM follows this same core philosophy, translating it into a beginner-friendly, educational project that you can fully build, customize, and understand using accessible tools and hardware.

This course will guide you in designing a robot that listens to your voice, responds using speech, and holds meaningful conversations powered by modern AI and LLM technologies. Your robot can tell jokes, offer companionship, answer questions, and behave like a friendly personal assistant. Through the phone’s sensors and camera, you’ll add user-tracking and object-tracking abilities so the robot can automatically turn to face the user, giving it lifelike presence.

You’ll also build expressive digital eyes that blink, track the user, and show emotions—plus the option to create your own custom faces or animations. Along the way, you’ll gain foundational skills in Bluetooth communication, AI and LLM integration, speech recognition, text-to-speech, facial recognition, chatbot behaviour, object tracking, and more.

By the end of this course, you’ll have a fully functional, AI-driven robot that blends hardware, software, creativity, and conversational intelligence—all built by you from the ground up!

Enroll today and join me inside to bring your AI robot—and your creativity—to life!