Sphero Edu Machine Learning Lab
The Machine Learning Lab introduces learners to the basics of machine learning through hands-on exploration of their robot's sensor data. Learners create classes, record examples, and adjust training settings called hyperparameters to build machine learning models they can use in block-based, JavaScript, or Python programs.
The Machine Learning Lab was introduced in Sphero Edu 8.4.0 for all platforms. Update the application if you don't have access to the feature.
Using the Machine Learning Lab
Below are the general steps to get started with the Machine Learning Lab.
1. Create a model for your robot type
Start by connecting to a robot and creating a model. When creating a model you will need to select:
- The robot sensors the model will use (gyroscope, robot orientation, ambient light sensor, etc.)
- The stabilization setting, if supported by your connected robot
- The model name
The sensor selections are part of the model. Once you create a model for a specific robot type (like BOLT+ or mini), you will then need to continue using the same robot type for that model.
Stabilization is disabled by default when creating a model. When stabilization is "on", round robots will spin inside their shell to maintain a heading. This makes it hard to capture data for certain types of classes, like tilting or twisting the robot. Disabling stabilization can help with these types of use cases.
2. Create classes and record data
Classes are the behaviors or actions the model should recognize. For example, a model might have classes named shake, still, and spin.The model then uses the recorded class data to determine which class is most likely happening at a given time. You will always need to create at least two classes and record data samples for each one before you can train.
- Create a class and follow the naming rules for classes
- Press the "Record" button to record a short sensor sequence of the action on the robot
- Repeat until every class has enough data samples of the actions
Each recording should contain at least a second or two of data. Use varied examples of the action so the model learns the behavior instead of memorizing one single movement.
Classes follow specific naming rules since the class names are used in the actual Python and JavaScript code in your programs. As a result, they must follow the respective coding conventions for variable and function naming as outline in the user interface.
3. Train the model
After every class has enough recordings, train the model. When training is selected:
- The machine-learning runtime reads the recorded sensor data
- It learns how those features correspond to the classes
- It returns a model when training succeeds
The training settings include the number of epochs and learning rate. These hyperparameters can be modified to adjust the behavior of the model. Training creates a new model every time. Changing classes or collecting different data samples will require re-training the model if you want the new changes to be reflected in your program.
The saved state contains the model topology, weights, class labels, sensor configuration, and training settings - everything needed for the model to work in a program.
4. Test the model
Testing checks how the trained model responds to live sensor data. When testing is enabled, the model runs while you stream sensor data from the robot. The test view shows a confidence value for each class so you can see which behaviors the model recognizes.
For reliable results:
- Use the same robot, sensors, and stabilization conditions used for training.
- Check whether the correct class consistently receives the highest confidence.
- Adjust detection thresholds or record better examples when results are unreliable.
If the model does not perform well, return to recording and training. More varied, balanced examples are usually more useful than simply increasing the number of training epochs. The better the data in your model classes, the better the detection results.
5. Integrate the model in your program
Once the model is trained and tested, add it to a Block, JavaScript, or Python program through the Machine Learning block and snippet categories or through Step 3 in the Machine Learning Lab.The program stores a link to the trained model. When the program starts, the app:
- Loads the saved model
- Starts the program and sends the robot sensor data to the model
- Makes the model's classes, confidence values, and machine-learning events available to the program
The program can then respond when the model recognizes a class—for example, changing the robot's lights when it detects a shake.
Machine learning is iterative. When testing reveals mistakes, improve the classes or recordings, retrain the model, retest it, and then run your program again.
Compatible Robots with the Machine Learning Lab
Legacy robots are not guaranteed to work well with data capture or training. The following robots are officially supported:
-
Sphero BOLT+
-
Sphero BOLT
-
Sphero Mini
-
Sphero RVR and RVR+
User Data and Machine Learning
The Machine Learning Lab does not capture any personally identifiable data. The only data that is collected are short samples of robot sensor data. These samples are collected from the robot's inertial measurement unit, ambient light sensor, or color sensor, and are only used for the training of the machine learning model.
Model training happens within the Sphero Edu app itself, locally on the user's device with a lightweight open source machine learning library. Models are never trained online. Robot sensor data is not shared to 3rd party services for processing, and is only included in saved programs on our platform as part of facilitating account services.