Arduino Nano 33 BLE Rev2 - Motion Trainer Airmouse

arduinoCristel jade Oribe
Arduino Nano 33 BLE Rev2 - Motion Trainer Airmouse
0 comments

Arduino Nano 33 BLE Rev2 - Motion Trainer Airmouse

Tiny Motion Mouse overview image
  1. 01 — Overview
  2. 02 — Hardware and Software Components
  3. 03 — Application Discussion
  4. 04 — Hardware Setup
  5. 05 — Software Setup
  6. 06 — Code
  7. 07 — Code Breakdown
  8. 08 — Testing and Calibration
  9. 09 — System Demonstration
  10. 10 — Conclusion
  11. 11 — References
  12. 12 — Project Authors

01 Overview

The Arduino Nano 33 BLE Rev2 with Tiny Motion Trainer project uses the board's onboard BMI270 IMU (accelerometer and gyroscope) to recognize hand gestures through Tiny Machine Learning (TinyML). Using Google Creative Lab's Tiny Motion Trainer, users can collect motion data, train a gesture recognition model in a web browser, and upload it directly to the Arduino without manually programming machine learning algorithms.

The system continuously detects trained gestures in real time and performs corresponding actions, such as controlling applications, counting movements, or providing user feedback. This project introduces students to the fundamentals of TinyML, gesture recognition, and embedded systems while allowing them to design and evaluate their own gesture-based applications.

Project Use Case

Gesture recognition applications that use the Arduino Nano 33 BLE Rev2 and Tiny Motion Trainer to detect custom hand movements and perform trained actions in real time.

02 Hardware and Software Components

Gather everything below before you start. This is your checklist to prepare for this device.

Hardware Components

Component Description
Arduino Nano 33 BLE Rev2 A compact microcontroller board based on the nRF52840 with built-in Bluetooth Low Energy (BLE), accelerometer, gyroscope, and magnetometer. It serves as the main controller of the project.
Host PC or phone A computer or smartphone used to receive BLE signals from the Arduino, program the board, or test the device.
Battery Holder/case plastic A plastic enclosure designed to securely hold lithium-ion batteries and provide convenient electrical connections.
Li-Ion Battery A rechargeable battery that supplies portable power to the system. Common examples include 18650 cells.
Micro-USB Cable Used for programming the Arduino, serial communication, and powering the board during development.
DC-DC Step Down Power Supply Module Also called a buck converter, it reduces a higher input voltage to a lower regulated voltage suitable for powering electronic components.

Software Tools

Software Version / Details
Arduino IDE 2.3.10
Tiny Motion Trainer By Google Creative Lab
File Description
TinyMotionTrainer-Example.ino Main Arduino program that will act as the intake for the data in the model.h to conduct gestures.
model.h The header file that contain the data for the trained gesture recognition model.
Safety Note: Do not use voltage higher than 3.3V to power the pins except the VIN input pin which is the minimum and maximum is 4.5v-21v.

03 Application Discussion

The Arduino Nano 33 BLE Rev2 serves as the central processing unit of the system. It features an nRF52840 microcontroller with integrated Bluetooth Low Energy (BLE) capability, making it suitable for wireless HID applications.

Inertial Measurement Unit

The onboard Inertial Measurement Unit contains an accelerometer, gyroscope and magnetometer to measure the device's acceleration, rotation, and direction.

LM2596 DC-DC Step Down Power Supply Module

The LM2596 is a versatile DC-DC step-down converter that reduces a higher input voltage to a lower regulated voltage, suitable for powering electronic components.

04 Hardware Setup

Wire the components to the board using the tables below.

Air Mouse Board

Air Mouse Board

Air Mouse Wiring

Air Mouse Board

Arduino GPIO

Pin / Signal Connects To
D2 Tactile Push Button Switch with Round Cap → GND

Components Pin

Component Pin Board Pin Description
LM2596 OUT+ VIN Input voltage to the power supply module
LM2596 OUT- GND Ground connection for the power supply module
Tactile Push Button Switch with Round Cap → GND D2 Conduct an input signal

Assembly Instructions

  1. Install Arduino Nano 33 BLE Rev2.
  2. Use micro-usb cable to upload firmware.
  3. Connect the Tactile Push Button Switch with Round Cap to the digital pin of the Arduino.
  4. Attach LM2596 with its power supply and connect the OUT+ to the VIN and OUT- to GND.
  5. Pair device.
Note: Make sure you know your input power requirements before connecting the device depending on the pin you connect if its 3.3v use the 3.3v pin or if its more that 4.5v but no more that 21v use the VIN pin.

05 Software Setup

Follow these steps in order. Do not skip any step.

Step 1 — Set Up Development Environment

  1. Run Arduino IDE
  2. Install the Arduino Mbed OS Nano Boards package
  3. Install the required libraries via the Library Manager

Step 2 — Board Settings

Use exactly these settings in your IDE. Wrong settings will cause upload failures.

Setting Value
Board Arduino Mbed OS Nano Boards: Arduino Nano 33 BLE
Port Arduino Nano 33 BLE Rev2

Step 3 — Install Libraries

Install the following libraries via the Library Manager:

  • Arduino_BMI270_BMM150 v1.2.3
  • Mbed BLE Mouse v1.3.0
  • Harvard_TinyMLx v1.2.4

Step 4 — Upload the Code

  1. Connect to the Arduino via micro-USB cable.
  2. Double press the reset button, if the orange LED is blinking it is in bootloader mode.
  3. Copy the code and paste to the Arduino IDE, Make sure it is in the correct port.
  4. Click Upload.

06 Code

Copy the code and prepare to upload to the device.

TinyMotionTrainer-Example.ino

Arduino / C++ 

#include <Arduino_BMI270_BMM150.h>
#include <TensorFlowLite.h>
#include <tensorflow/lite/micro/all_ops_resolver.h>
#include <tensorflow/lite/micro/micro_error_reporter.h>
#include <tensorflow/lite/micro/micro_interpreter.h>
#include <tensorflow/lite/schema/schema_generated.h>
#include <tensorflow/lite/version.h>
#include <HIDMouse.h>




#include "model.h"

#define MOTION_THRESHOLD 0.15

#define CAPTURE_DELAY 200

#define NUM_SAMPLES 20

HIDMouse bleMouse;

const float DEADZONE = 3.0;

const float SENSITIVITY = 0.20;

const int   MAX_SPEED = 35;

float

gX, gY, gZ,

aX, aY, aZ;

float gx_offset=0;

float gy_offset=0;

int Movx = 0;

int Movy = 0;

bool leftlock = false;

bool gestureStarted = false;

bool change = false;

const char *GESTURES[] = 
{
    "WAVE_UP", "WAVE_DOWN", "WAVE_RIGHT", "WAVE_LEFT"
};



#define NUM_GESTURES (sizeof(GESTURES) / sizeof(GESTURES[0]))

bool isCapturing = false;

int numSamplesRead = 0;

tflite::MicroErrorReporter tflErrorReporter;

tflite::AllOpsResolver tflOpsResolver;


const tflite::Model* tflModel = nullptr;

tflite::MicroInterpreter* tflInterpreter = nullptr;

TfLiteTensor* tflInputTensor = nullptr;

TfLiteTensor* tflOutputTensor = nullptr;

constexpr int tensorArenaSize = 8 * 1024;

byte tensorArena[tensorArenaSize];

void checkButton()
{

    bool button = digitalRead(2);

    if (button == LOW && !change)
    {

        gestureStarted = !gestureStarted;

        Serial.print("gestureStarted = ");

        Serial.println(gestureStarted);

        isCapturing = false;

        numSamplesRead = 0;

        Movx = 0;
        
        Movy = 0;

        bleMouse.move(0, 0);

        change = true;

    }

    if (button == HIGH)
    {

        change = false;

    }
}

void mouseMode()
{

    IMU.readGyroscope(gX, gY, gZ);

    gX -= gx_offset;

    gY -= gy_offset;

    Movx = 0;

    Movy = 0;

    if (fabsf(gX) > DEADZONE)
    {

      Movx = gX * SENSITIVITY;

    }

    if (fabsf(gY) > DEADZONE)
    {

      Movy = gY * SENSITIVITY;

    }
    
    Movx = constrain(Movx, -MAX_SPEED, MAX_SPEED);

    Movy = constrain(Movy, -MAX_SPEED, MAX_SPEED);

    if (Movx != 0 || Movy != 0)
    {

      bleMouse.move(Movx, Movy);
    
    }


    delay(5);
}

void gestureMode()
{

  while (!isCapturing) 
  {

    checkButton();

    if (!gestureStarted)
    {

        return;

    }
    if (IMU.accelerationAvailable() && IMU.gyroscopeAvailable()) {
     
      IMU.readAcceleration(aX, aY, aZ);

      IMU.readGyroscope(gX, gY, gZ);

 
      float average = fabs(aX / 4.0) + fabs(aY / 4.0) + fabs(aZ / 4.0) + fabs(gX / 2000.0) + fabs(gY / 2000.0) + fabs(gZ / 2000.0);
      average /= 6.;

      if (average >= MOTION_THRESHOLD) 
      {

        isCapturing = true;

        numSamplesRead = 0;

        break;

      }
    }
  }

  while (isCapturing) 
  {

    checkButton();

    if (!gestureStarted)
    {
        
        isCapturing = false;

        return;

    }

    if (IMU.accelerationAvailable() && IMU.gyroscopeAvailable()) {

      IMU.readAcceleration(aX, aY, aZ);

      IMU.readGyroscope(gX, gY, gZ);


      tflInputTensor->data.f[numSamplesRead * 6 + 0] = aX / 4.0;

      tflInputTensor->data.f[numSamplesRead * 6 + 1] = aY / 4.0;

      tflInputTensor->data.f[numSamplesRead * 6 + 2] = aZ / 4.0;

      tflInputTensor->data.f[numSamplesRead * 6 + 3] = gX / 2000.0;

      tflInputTensor->data.f[numSamplesRead * 6 + 4] = gY / 2000.0;

      tflInputTensor->data.f[numSamplesRead * 6 + 5] = gZ / 2000.0;

      numSamplesRead++;


      if (numSamplesRead == NUM_SAMPLES) {
        

        isCapturing = false;
        

        TfLiteStatus invokeStatus = tflInterpreter->Invoke();

        if (invokeStatus != kTfLiteOk) 
        {

          Serial.println("Error: Invoke failed!");

          while (1);

          return;

        }


        int maxIndex = 0;

        float maxValue = 0;

        for (int i = 0; i < NUM_GESTURES; i++) 
        {

          float _value = tflOutputTensor->data.f[i];

          if(_value > maxValue)
          {

            maxValue = _value;

            maxIndex = i;

          }

          Serial.print(GESTURES[i]);

          Serial.print(": ");

          Serial.println(tflOutputTensor->data.f[i], 6);

        }
        
        Serial.print("Winner: ");

        Serial.print(GESTURES[maxIndex]);

        if ("WAVE_UP" == GESTURES[maxIndex])
        {

          bleMouse.press(MOUSE_BUTTON_MIDDLE);
  
          Serial.print(GESTURES[maxIndex]);

          Serial.print("Clicked");

          delay(20);
  
          bleMouse.release(MOUSE_BUTTON_MIDDLE);

        }
        else if ("WAVE_DOWN" == GESTURES[maxIndex])
        {

          if  (leftlock)
          {

            bleMouse.press(MOUSE_BUTTON_LEFT);

            Serial.print(GESTURES[maxIndex]);

            Serial.print(" Left Locked");

          }
          else
          {

            bleMouse.release(MOUSE_BUTTON_LEFT);

            Serial.print(" Left Unlocked");

          }
        }
        else if ("WAVE_RIGHT" == GESTURES[maxIndex])
        {

          bleMouse.press(MOUSE_BUTTON_RIGHT);

          Serial.print(GESTURES[maxIndex]);

          Serial.print("Clicked");
  
          delay(20);
  
          bleMouse.release(MOUSE_BUTTON_RIGHT);

        }
        else if ("WAVE_LEFT" == GESTURES[maxIndex])
        {

          bleMouse.press(MOUSE_BUTTON_LEFT);

          Serial.print(GESTURES[maxIndex]);

          Serial.print("Clicked");
  
          delay(20);
  
          bleMouse.release(MOUSE_BUTTON_LEFT);

        }




        Serial.println();


        delay(CAPTURE_DELAY);

      }
    }
  }
}
  


void setup() {
  Serial.begin(9600);
while (!Serial)
{
  ;
}

  pinMode(LED_BUILTIN, OUTPUT);
        


  pinMode(2,  INPUT_PULLUP);


  if (!IMU.begin()) {
    Serial.println("Failed to initialize IMU!");
    while (1);
  }


  Serial.print("Accelerometer sample rate: ");
  Serial.print(IMU.accelerationSampleRate());
  Serial.println(" Hz");
  Serial.print("Gyroscope sample rate: ");
  Serial.print(IMU.gyroscopeSampleRate());
  Serial.println(" Hz");

  Serial.println();


  tflModel = tflite::GetModel(model);
  if (tflModel->version() != TFLITE_SCHEMA_VERSION) {
    Serial.println("Model schema mismatch!");
    while (1);
  }


  tflInterpreter = new tflite::MicroInterpreter(tflModel, tflOpsResolver, tensorArena, tensorArenaSize, &tflErrorReporter);


TfLiteStatus status = tflInterpreter->AllocateTensors();

if (status != kTfLiteOk)
{
    Serial.println("AllocateTensors FAILED");
    while (1);
}

Serial.println("AllocateTensors OK");


tflInputTensor = tflInterpreter->input(0);
tflOutputTensor = tflInterpreter->output(0);



  Serial.print("Input tensor bytes: ");
  Serial.println(tflInputTensor->bytes);

  Serial.print("Expected floats: ");
  Serial.println(tflInputTensor->bytes / sizeof(float));

  Serial.print("Model version: ");
  Serial.println(tflModel->version());

  Serial.print("Runtime version: ");
  Serial.println(TFLITE_SCHEMA_VERSION);

  bleMouse.setDeviceName("Tiny Motion Mouse");

  bleMouse.begin();

  for(int i=0;i<500;i++)
  {

    IMU.readGyroscope(gX,gY,gZ);

    gx_offset+=gX;

    gy_offset+=gY;

    delay(2);

  }

  gx_offset/=500;

  gy_offset/=500;

  pinMode(2,  INPUT_PULLUP);
}

void loop() {

  if (bleMouse.isConnected())
  {
    digitalWrite(LED_BUILTIN, HIGH);
  }
  else
  {
    digitalWrite(LED_BUILTIN, LOW);
  }   

  checkButton();

  if (!gestureStarted)
  {
    mouseMode();
  }
  else
  {
    gestureMode();
  }
}

07 Code Breakdown

Here is what each part of the code does. Read this after uploading.

Libraries

Library Purpose
Arduino_BMI270_BMM150 The Arduino_BMI270_BMM150 library is used to access the built-in gyroscope, accelerometer, and magnetometer of the Arduino Nano 33 BLE Rev2 for motion sensing, orientation tracking, and gesture recognition.
Mbed BLE Mouse The Mbed BLE Mouse library is used to simulate mouse movements and clicks over a BLE connection.
Harvard_TinyMLx The Harvard_TinyMLx library is used for machine learning inference on the Arduino Nano 33 BLE Rev2.

Key Functions

checkButton

Used to detect the button press to know when to activate gesture mode and exit to mouse mode.

mouseMode

It will conduct mouse movement using the gyroscope for smoother movement.

gestureMode

It will conduct mouse movement using the gyroscope for smoother movement.

setup

Checks the hardware, setting up the tensorflow lite and initializes the offset of the gyroscope, for smoother movement setting up.

loop

The main execution loop that continuously check for bluetooth connection and the switching of modes.

General Program Workflow

  1. default state: detecting Bluetooth connection
  2. Connection occurred LED turns on
  3. Check button for change in boolean
  4. Default straight to mouse movement
  5. Press button, check button starts
  6. Boolean change, gesture mode started cursor movement halts
  7. Conduct gesture accordingly
  8. Depending on gesture detected mouse action conducted
  9. Press Tactile button switch to exit gesture mode
  10. Return to cursor movement

08 Testing and Calibration

After uploading, verify each of the following to confirm the system is working correctly.

Cursor movement drifting

It tends to use the earliest detected gyroscope data, causing the cursor to drift over time. To fix go to your Bluetooth remove the Air Mouse, lay down the device on a flat surface, then press reset button on the Arduino then reconnect.

Bluetooth Connected but no response

The library Mbed BLE Mouse has issues on it's Bluetooth service, simply go to your Bluetooth then select remove device on the Air Mouse then reconnect.

Common Issue: Incorrect action activated due to its data, avoid shaking and inaccuracy is unavoidable.

09 System Demonstration

The images below show the working system. Use these to verify your output matches what is expected.

Air Mouse Serial

Air Mouse Serial

If you connect the Arduino to your computer via micro-usb please set the following.

  • Open serial monitor and set baud rate to 9600
  • Connect the device to the Bluetooth either the computer or other external device (Optional)
  • In the image are the expected output

Once you have received it expect of the following actions depending on whether press the tactile push button:

  • 21:51:30.708 -> gestureStarted = 1
  • 21:51:32.992 -> gestureStarted = 0
  • 21:51:39.258 -> gestureStarted = 1

These what will be expected the "1" value present the mode it is in, it means it is in gesture mode while "0" means it is in mouse mode.

These are what you expect depend on the motion:

  • 21:51:41.543 -> WAVE_UP: 0.001107
  • 21:51:41.544 -> WAVE_DOWN: 0.006683 (Hold left click)
  • 21:51:41.544 -> WAVE_LEFT: 0.991159 (Left click)
  • 21:51:41.544 -> WAVE_RIGHT: 0.001050 (Right click)
  • 21:51:41.544 -> Winner: WAVE_LEFTWAVE_LEFTClicked (The one with the highest accuracy will be chosen)

Motion Trainer Airmouse Device

Motion Trainer Airmouse Physical

5 voltage input is used, the pin taking it is VIN, avoid connecting to other pins if the voltage is higher than 3.3v, you may damage the arduino


Video Demonstration & Tutorial

10 Conclusion

The tiny motion trainer is very promising for providing an intuitive and portable alternative input device to gather data and give out proper feedback. It successfully integrates motion sensing, gesture recognition, and Bluetooth Low Energy (BLE) communication using the Arduino Nano 33 BLE Rev2.

Possible Improvements and Future Enhancements

  • Have a more robust gesture recognition system gather larger sample sizes.
  • Enhance the user interface for better usability.
  • Improve the handling of holding actions.

11 References

  • https://github.com/
  • https://www.arduino.cc/
  • https://experiments.withgoogle.com/tiny-motion-trainer
  • https://experiments.withgoogle.com/search?q=google%20creative%20lab

12 Project Authors

  • Russel Jon C. Recapente
  • Cristel Jade B. Oribe
Arduino Nano 33 BLE Rev2 - Motion Trainer Airmouse – CreateLabz
Tags: Arduino, Arduino ide, Arduino nano 33 ble rev2

Leave a comment

All comments are moderated before being published

Highlighted Posts

  1. Mini Weather Station (Rain Gauge, Wind Speed, Wind Direction) using Blynk and SIM800C
    Mini Weather Station (Rain Gauge, Wind Speed, Wind Direction) using Blynk and SIM800C
    arduinoJun 02, 2019
  2. Wireless Energy Monitoring System using ESP32 with Blynk Mobile App
    Wireless Energy Monitoring System using ESP32 with Blynk Mobile App
    AC Voltage SensorMay 31, 2019
  3. Heart Beat Counter on Raspberry Pi Zero Using Pulse sensor and LCD Display
    Heart Beat Counter on Raspberry Pi Zero Using Pulse sensor and LCD Display
    ADS1115May 30, 2019
  4. RFID and Biometric Door Lock system using Raspberry Pi ZeroW with MySQL database
    RFID and Biometric Door Lock system using Raspberry Pi ZeroW with MySQL database
    12V/2A Solenoid LockJun 07, 2019
  5. Controlling 4DOF Robotic Arm Claw Kit with 16 channel PWM/Servo Shield using analog input.
    Controlling 4DOF Robotic Arm Claw Kit with 16 channel PWM/Servo Shield using analog input.
    16 channel 12-bit PWM Servo ShieldApr 22, 2019