Arduino Nano 33 BLE Rev2 - Motion Trainer Airmouse – CreateLabz Store

0 comments

Arduino Nano 33 BLE Rev2 - Motion Trainer Airmouse

01 — Overview
02 — Hardware and Software Components
03 — Application Discussion
04 — Hardware Setup
05 — Software Setup
06 — Code
07 — Code Breakdown
08 — Testing and Calibration
09 — System Demonstration
10 — Conclusion
11 — References
12 — Project Authors

01 Overview

The Arduino Nano 33 BLE Rev2 with Tiny Motion Trainer project uses the board's onboard BMI270 IMU (accelerometer and gyroscope) to recognize hand gestures through Tiny Machine Learning (TinyML). Using Google Creative Lab's Tiny Motion Trainer, users can collect motion data, train a gesture recognition model in a web browser, and upload it directly to the Arduino without manually programming machine learning algorithms.

The system continuously detects trained gestures in real time and performs corresponding actions, such as controlling applications, counting movements, or providing user feedback. This project introduces students to the fundamentals of TinyML, gesture recognition, and embedded systems while allowing them to design and evaluate their own gesture-based applications.

Project Use Case

Gesture recognition applications that use the Arduino Nano 33 BLE Rev2 and Tiny Motion Trainer to detect custom hand movements and perform trained actions in real time.

02 Hardware and Software Components

Gather everything below before you start. This is your checklist to prepare for this device.

Hardware Components

Component	Description
Arduino Nano 33 BLE Rev2	A compact microcontroller board based on the nRF52840 with built-in Bluetooth Low Energy (BLE), accelerometer, gyroscope, and magnetometer. It serves as the main controller of the project.
Host PC or phone	A computer or smartphone used to receive BLE signals from the Arduino, program the board, or test the device.
Battery Holder/case plastic	A plastic enclosure designed to securely hold lithium-ion batteries and provide convenient electrical connections.
Li-Ion Battery	A rechargeable battery that supplies portable power to the system. Common examples include 18650 cells.
Micro-USB Cable	Used for programming the Arduino, serial communication, and powering the board during development.
DC-DC Step Down Power Supply Module	Also called a buck converter, it reduces a higher input voltage to a lower regulated voltage suitable for powering electronic components.

Software Tools

Software	Version / Details
Arduino IDE	2.3.10
Tiny Motion Trainer	By Google Creative Lab

Project Files

All required files are available in the project repository. Download the repository before proceeding to the Software Setup section.

Repository: https://github.com/

File	Description
TinyMotionTrainer-Example.ino	Main Arduino program that will act as the intake for the data in the model.h to conduct gestures.
model.h	The header file that contain the data for the trained gesture recognition model.

Safety Note: Do not use voltage higher than 3.3V to power the pins except the VIN input pin which is the minimum and maximum is 4.5v-21v.

03 Application Discussion

The Arduino Nano 33 BLE Rev2 serves as the central processing unit of the system. It features an nRF52840 microcontroller with integrated Bluetooth Low Energy (BLE) capability, making it suitable for wireless HID applications.

Inertial Measurement Unit

The onboard Inertial Measurement Unit contains an accelerometer, gyroscope and magnetometer to measure the device's acceleration, rotation, and direction.

LM2596 DC-DC Step Down Power Supply Module

The LM2596 is a versatile DC-DC step-down converter that reduces a higher input voltage to a lower regulated voltage, suitable for powering electronic components.

04 Hardware Setup

Wire the components to the board using the tables below.

Air Mouse Board

Air Mouse Wiring

Arduino GPIO

Pin / Signal	Connects To
D2	Tactile Push Button Switch with Round Cap → GND

Components Pin

Component Pin	Board Pin	Description
LM2596 OUT+	VIN	Input voltage to the power supply module
LM2596 OUT-	GND	Ground connection for the power supply module
Tactile Push Button Switch with Round Cap → GND	D2	Conduct an input signal

Assembly Instructions

Install Arduino Nano 33 BLE Rev2.
Use micro-usb cable to upload firmware.
Connect the Tactile Push Button Switch with Round Cap to the digital pin of the Arduino.
Attach LM2596 with its power supply and connect the OUT+ to the VIN and OUT- to GND.
Pair device.

Note: Make sure you know your input power requirements before connecting the device depending on the pin you connect if its 3.3v use the 3.3v pin or if its more that 4.5v but no more that 21v use the VIN pin.

05 Software Setup

Follow these steps in order. Do not skip any step.

Step 1 — Set Up Development Environment

Run Arduino IDE
Install the Arduino Mbed OS Nano Boards package
Install the required libraries via the Library Manager

Step 2 — Board Settings

Use exactly these settings in your IDE. Wrong settings will cause upload failures.

Setting	Value
Board	Arduino Mbed OS Nano Boards: Arduino Nano 33 BLE
Port	Arduino Nano 33 BLE Rev2

Step 3 — Install Libraries

Install the following libraries via the Library Manager:

Arduino_BMI270_BMM150 v1.2.3
Mbed BLE Mouse v1.3.0
Harvard_TinyMLx v1.2.4

Step 4 — Upload the Code

Connect to the Arduino via micro-USB cable.
Double press the reset button, if the orange LED is blinking it is in bootloader mode.
Copy the code and paste to the Arduino IDE, Make sure it is in the correct port.
Click Upload.

06 Code

Copy the code and prepare to upload to the device.

TinyMotionTrainer-Example.ino

Arduino / C++

#include <Arduino_BMI270_BMM150.h>
#include <TensorFlowLite.h>
#include <tensorflow/lite/micro/all_ops_resolver.h>
#include <tensorflow/lite/micro/micro_error_reporter.h>
#include <tensorflow/lite/micro/micro_interpreter.h>
#include <tensorflow/lite/schema/schema_generated.h>
#include <tensorflow/lite/version.h>
#include <HIDMouse.h>

#include "model.h"

#define MOTION_THRESHOLD 0.15
#define CAPTURE_DELAY 200
#define NUM_SAMPLES 20

HIDMouse bleMouse;

const float DEADZONE = 3.0;
const float SENSITIVITY = 0.20;
const int   MAX_SPEED = 35;

float gX, gY, gZ, aX, aY, aZ;
float gx_offset = 0;
float gy_offset = 0;
int Movx = 0;
int Movy = 0;
bool leftlock = false;
bool gestureStarted = false;
bool change = false;

const char *GESTURES[] = {
    "WAVE_UP", "WAVE_DOWN", "WAVE_RIGHT", "WAVE_LEFT"
};

#define NUM_GESTURES (sizeof(GESTURES) / sizeof(GESTURES[0]))

bool isCapturing = false;
int numSamplesRead = 0;

tflite::MicroErrorReporter tflErrorReporter;
tflite::AllOpsResolver tflOpsResolver;

const tflite::Model* tflModel = nullptr;
tflite::MicroInterpreter* tflInterpreter = nullptr;
TfLiteTensor* tflInputTensor = nullptr;
TfLiteTensor* tflOutputTensor = nullptr;

constexpr int tensorArenaSize = 8 * 1024;
byte tensorArena[tensorArenaSize];

void checkButton()
{
    bool button = digitalRead(2);

    if (button == LOW && !change)
    {
        gestureStarted = !gestureStarted;
        Serial.print("gestureStarted = ");
        Serial.println(gestureStarted);
        isCapturing = false;
        numSamplesRead = 0;
        Movx = 0;
        Movy = 0;
        bleMouse.move(0, 0);
        change = true;
    }

    if (button == HIGH)
    {
        change = false;
    }
}

void mouseMode()
{
    IMU.readGyroscope(gX, gY, gZ);

    gX -= gx_offset;
    gY -= gy_offset;

    Movx = 0;
    Movy = 0;

    if (fabsf(gX) > DEADZONE)
    {
      Movx = gX * SENSITIVITY;
    }

    if (fabsf(gY) > DEADZONE)
    {
      Movy = gY * SENSITIVITY;
    }

    Movx = constrain(Movx, -MAX_SPEED, MAX_SPEED);
    Movy = constrain(Movy, -MAX_SPEED, MAX_SPEED);

    if (Movx != 0 || Movy != 0)
    {
      bleMouse.move(Movx, Movy);
    }

    delay(5);
}

void gestureMode()
{
  while (!isCapturing)
  {
    checkButton();

    if (!gestureStarted)
    {
        return;
    }
    if (IMU.accelerationAvailable() && IMU.gyroscopeAvailable()) {
      IMU.readAcceleration(aX, aY, aZ);
      IMU.readGyroscope(gX, gY, gZ);

      float average = fabs(aX / 4.0) + fabs(aY / 4.0) + fabs(aZ / 4.0) + fabs(gX / 2000.0) + fabs(gY / 2000.0) + fabs(gZ / 2000.0);
      average /= 6.;

      if (average >= MOTION_THRESHOLD)
      {
        isCapturing = true;
        numSamplesRead = 0;
        break;
      }
    }
  }

  while (isCapturing)
  {
    checkButton();

    if (!gestureStarted)
    {
        isCapturing = false;
        return;
    }

    if (IMU.accelerationAvailable() && IMU.gyroscopeAvailable()) {
      IMU.readAcceleration(aX, aY, aZ);
      IMU.readGyroscope(gX, gY, gZ);

      tflInputTensor->data.f[numSamplesRead * 6 + 0] = aX / 4.0;
      tflInputTensor->data.f[numSamplesRead * 6 + 1] = aY / 4.0;
      tflInputTensor->data.f[numSamplesRead * 6 + 2] = aZ / 4.0;
      tflInputTensor->data.f[numSamplesRead * 6 + 3] = gX / 2000.0;
      tflInputTensor->data.f[numSamplesRead * 6 + 4] = gY / 2000.0;
      tflInputTensor->data.f[numSamplesRead * 6 + 5] = gZ / 2000.0;

      numSamplesRead++;

      if (numSamplesRead == NUM_SAMPLES) {

        isCapturing = false;

        TfLiteStatus invokeStatus = tflInterpreter->Invoke();

        if (invokeStatus != kTfLiteOk)
        {
          Serial.println("Error: Invoke failed!");
          while (1);
          return;
        }

        int maxIndex = 0;
        float maxValue = 0;

        for (int i = 0; i < NUM_GESTURES; i++)
        {
          float _value = tflOutputTensor->data.f[i];

          if(_value > maxValue)
          {
            maxValue = _value;
            maxIndex = i;
          }

          Serial.print(GESTURES[i]);
          Serial.print(": ");
          Serial.println(tflOutputTensor->data.f[i], 6);
        }

        Serial.print("Winner: ");
        Serial.print(GESTURES[maxIndex]);

        if ("WAVE_UP" == GESTURES[maxIndex])
        {
          bleMouse.press(MOUSE_BUTTON_MIDDLE);
          Serial.print(GESTURES[maxIndex]);
          Serial.print("Clicked");
          delay(20);
          bleMouse.release(MOUSE_BUTTON_MIDDLE);
        }
        else if ("WAVE_DOWN" == GESTURES[maxIndex])
        {
          if  (leftlock)
          {
            bleMouse.press(MOUSE_BUTTON_LEFT);
            Serial.print(GESTURES[maxIndex]);
            Serial.print(" Left Locked");
          }
          else
          {
            bleMouse.release(MOUSE_BUTTON_LEFT);
            Serial.print(" Left Unlocked");
          }
        }
        else if ("WAVE_RIGHT" == GESTURES[maxIndex])
        {
          bleMouse.press(MOUSE_BUTTON_RIGHT);
          Serial.print(GESTURES[maxIndex]);
          Serial.print("Clicked");
          delay(20);
          bleMouse.release(MOUSE_BUTTON_RIGHT);
        }
        else if ("WAVE_LEFT" == GESTURES[maxIndex])
        {
          bleMouse.press(MOUSE_BUTTON_LEFT);
          Serial.print(GESTURES[maxIndex]);
          Serial.print("Clicked");
          delay(20);
          bleMouse.release(MOUSE_BUTTON_LEFT);
        }

        Serial.println();

        delay(CAPTURE_DELAY);

      }
    }
  }
}

void setup() {
  Serial.begin(9600);
  while (!Serial)
  {
    ;
  }

  pinMode(LED_BUILTIN, OUTPUT);

  pinMode(2,  INPUT_PULLUP);

  if (!IMU.begin()) {
    Serial.println("Failed to initialize IMU!");
    while (1);
  }

  Serial.print("Accelerometer sample rate: ");
  Serial.print(IMU.accelerationSampleRate());
  Serial.println(" Hz");
  Serial.print("Gyroscope sample rate: ");
  Serial.print(IMU.gyroscopeSampleRate());
  Serial.println(" Hz");

  Serial.println();

  tflModel = tflite::GetModel(model);
  if (tflModel->version() != TFLITE_SCHEMA_VERSION) {
    Serial.println("Model schema mismatch!");
    while (1);
  }

  tflInterpreter = new tflite::MicroInterpreter(tflModel, tflOpsResolver, tensorArena, tensorArenaSize, &tflErrorReporter);

  TfLiteStatus status = tflInterpreter->AllocateTensors();

  if (status != kTfLiteOk)
  {
      Serial.println("AllocateTensors FAILED");
      while (1);
  }

  Serial.println("AllocateTensors OK");

  tflInputTensor = tflInterpreter->input(0);
  tflOutputTensor = tflInterpreter->output(0);

  Serial.print("Input tensor bytes: ");
  Serial.println(tflInputTensor->bytes);

  Serial.print("Expected floats: ");
  Serial.println(tflInputTensor->bytes / sizeof(float));

  Serial.print("Model version: ");
  Serial.println(tflModel->version());

  Serial.print("Runtime version: ");
  Serial.println(TFLITE_SCHEMA_VERSION);

  bleMouse.setDeviceName("Tiny Motion Mouse");

  bleMouse.begin();

  for(int i=0;i<500;i++)
  {
    IMU.readGyroscope(gX,gY,gZ);
    gx_offset+=gX;
    gy_offset+=gY;
    delay(2);
  }

  gx_offset/=500;
  gy_offset/=500;

  pinMode(2,  INPUT_PULLUP);
}

void loop() {

  if (bleMouse.isConnected())
  {
    digitalWrite(LED_BUILTIN, HIGH);
  }
  else
  {
    digitalWrite(LED_BUILTIN, LOW);
  }

  checkButton();

  if (!gestureStarted)
  {
    mouseMode();
  }
  else
  {
    gestureMode();
  }
}

▶ model.h

Arduino / C++

const unsigned char model[] = 
{
  0x1c, 0x00, 0x00, 0x00, 0x54, 0x46, 0x4c, 0x33, 0x00, 0x00, 0x12, 0x00,
  0x1c, 0x00, 0x04, 0x00, 0x08, 0x00, 0x0c, 0x00, 0x10, 0x00, 0x14, 0x00,
  0x00, 0x00, 0x18, 0x00, 0x12, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00,
  /* ... full binary model data as provided ... */
  0x00, 0x00, 0x00, 0x09
};

Note: The full model.h binary data has been abbreviated in this preview. Copy the complete array from the original source file.

07 Code Breakdown

Here is what each part of the code does. Read this after uploading.

Libraries

Library	Purpose
Arduino_BMI270_BMM150	The Arduino_BMI270_BMM150 library is used to access the built-in gyroscope, accelerometer, and magnetometer of the Arduino Nano 33 BLE Rev2 for motion sensing, orientation tracking, and gesture recognition.
Mbed BLE Mouse	The Mbed BLE Mouse library is used to simulate mouse movements and clicks over a BLE connection.
Harvard_TinyMLx	The Harvard_TinyMLx library is used for machine learning inference on the Arduino Nano 33 BLE Rev2.

Key Functions

checkButton

Used to detect the button press to know when to activate gesture mode and exit to mouse mode.

mouseMode

It will conduct mouse movement using the gyroscope for smoother movement.

gestureMode

Captures IMU samples when motion exceeds the threshold, runs TensorFlow Lite inference, and executes the corresponding mouse action for the detected gesture.

setup

Checks the hardware, setting up the tensorflow lite and initializes the offset of the gyroscope, for smoother movement setting up.

loop

The main execution loop that continuously check for bluetooth connection and the switching of modes.

General Program Workflow

Default state: detecting Bluetooth connection
Connection occurred LED turns on
Check button for change in boolean
Default straight to mouse movement
Press button, check button starts
Boolean change, gesture mode started cursor movement halts
Conduct gesture accordingly
Depending on gesture detected mouse action conducted
Press Tactile button switch to exit gesture mode
Return to cursor movement

08 Testing and Calibration

After uploading, verify each of the following to confirm the system is working correctly.

Cursor movement drifting

It tends to use the earliest detected gyroscope data, causing the cursor to drift over time. To fix go to your Bluetooth remove the Air Mouse, lay down the device on a flat surface, then press reset button on the Arduino then reconnect.

Bluetooth Connected but no response

The library Mbed BLE Mouse has issues on it's Bluetooth service, simply go to your Bluetooth then select remove device on the Air Mouse then reconnect.

Common Issue: Incorrect action activated due to its data, avoid shaking and inaccuracy is unavoidable.

09 System Demonstration

The images below show the working system. Use these to verify your output matches what is expected.

Air Mouse Serial

If you connect the Arduino to your computer via micro-usb please set the following.

Open serial monitor and set baud rate to 9600
Connect the device to the Bluetooth either the computer or other external device (Optional)
In the image are the expected output

Once you have received it expect of the following actions depending on whether press the tactile push button:

21:51:30.708 -> gestureStarted = 1
21:51:32.992 -> gestureStarted = 0
21:51:39.258 -> gestureStarted = 1

These what will be expected the "1" value present the mode it is in, it means it is in gesture mode while "0" means it is in mouse mode.

These are what you expect depend on the motion:

21:51:41.543 -> WAVE_UP: 0.001107
21:51:41.544 -> WAVE_DOWN: 0.006683 (Hold left click)
21:51:41.544 -> WAVE_LEFT: 0.991159 (Left click)
21:51:41.544 -> WAVE_RIGHT: 0.001050 (Right click)
21:51:41.544 -> Winner: WAVE_LEFTWAVE_LEFTClicked (The one with the highest accuracy will be chosen)

Motion Trainer Airmouse Device

5 voltage input is used, the pin taking it is VIN, avoid connecting to other pins if the voltage is higher than 3.3v, you may damage the arduino

Video Demonstration & Tutorial

10 Conclusion

The tiny motion trainer is very promising for providing an intuitive and portable alternative input device to gather data and give out proper feedback. It successfully integrates motion sensing, gesture recognition, and Bluetooth Low Energy (BLE) communication using the Arduino Nano 33 BLE Rev2.

Possible Improvements and Future Enhancements

Have a more robust gesture recognition system gather larger sample sizes.
Enhance the user interface for better usability.
Improve the handling of holding actions.

11 References

https://github.com/
https://www.arduino.cc/
https://experiments.withgoogle.com/tiny-motion-trainer
https://experiments.withgoogle.com/search?q=google%20creative%20lab

12 Project Authors

Cristel Jade Oribe

Quality Checked By

Alexander Maiso
Jhon Ronan Limbadan

Tags: Arduino, Arduino ide, Arduino nano 33 ble rev2

Previous article Next article