Process & Story
Our client, Maasloop, builds intelligent machines that automate the collection of recyclable bottles and cans, known as Reverse Vending Machines. To ensure the accuracy of returns and prevent fraud, they needed a solution that could verify an object’s shape, material, including distinguishing eligible packages from lookalikes and damaged containers.
We designed a lightweight machine learning system tailored to their hardware and performance constraints. The result? A fast and fraud-resistant packaging recognition system ready for nationwide deployment.
Problem
Maasloops’s machines operate in the context of a deposit return scheme. That means they don’t just need to accept recyclables — they must:
- Match the scanned EAN barcode to the actual physical package
- Verify the shape and dimensions of each item (no crushed or misshapen containers)
- Detect fraudulent attempts, such as scanning a valid EAN code while inserting a non-eligible or mismatched object
The system operates in a controlled, isolated environment: each machine accepts one object at a time inside an isolated chamber. Two images are captured per session — one of the empty chamber and one with the object inside — under identical, standardized conditions across all machines.
All of this runs in real time on a compact Raspberry Pi 5, within a strict 600ms total budget for all modules combined.
Solution
We designed and delivered a multi-module real-time recognition system to help our client verify recyclables and prevent fraud in deposit return machines. The solution was built in close collaboration with the client across five milestones, combining custom dataset development, lightweight model training, and edge deployment — all optimized to run on Raspberry Pi 5.
Module 1 — Segmentation & Dimension Estimation
We improved the previously delivered segmentation model for detecting masks of bottles and cans, with particular focus on difficult packaging types: black bottles and transparent containers. The primary goal of this module is to estimate object dimensions using the detected mask. Over the course of the project, the module received two updates, each delivering statistically significant improvements in IoU and MAPE-based evaluation metrics for estimated dimensions.
Module 2 — Color Distribution Analysis
We built a module to analyze the color distribution of each scanned object. For each EAN code, the historical database contains images with a known statistical color profile. This module checks whether the color distribution in a given scan falls within the historically observed spectrum for that EAN — flagging outliers that may indicate a mismatch or fraud attempt.
Module 3 — Logo & Visual Feature Detection
We developed a module to detect logos and other characteristic visual features that distinguish one product from another. It works similarly to the color distribution module but operates on the full feature representation of the image rather than color statistics alone, enabling finer-grained visual verification.
Module 4 — Scoring Function
We built a composite scoring function that aggregates data collected by the client’s systems for a given image — alongside the outputs of all the above modules — to predict whether the scanned object matches the EAN code presented at the machine. The model also returns a confidence estimate alongside its prediction, enabling downstream decision-making based on certainty thresholds.
Module 5 — Fraud Detection
We conducted a thorough statistical analysis of historical data to identify anomaly patterns associated with fraudulent returns — specifically cases where the scanned EAN code did not match the object’s visual characteristics. The findings were integrated into refinements of the scoring function, strengthening the system’s ability to flag and block fraudulent submissions.
Outcomes
- Custom dataset built from high-resolution machine-captured images, with balanced representation across packaging types including black and transparent containers
- Two iterative improvements to the segmentation model, each achieving statistically significant gains in in terms of IoU and MAPE
- Color distribution and visual feature modules that verify object identity against EAN-specific historical profiles
- Composite scoring function delivering per-scan predictions with confidence estimates
- Fraud detection insights embedded into the scoring pipeline, based on deep statistical analysis of historical anomalies
- 600ms inference time on Raspberry Pi 5 across all modules combined, enabling real-time validation at the point of return
- End-to-end ML pipeline delivered, including data ingestion, deduplication, labeling workflow, model training, optimization, and packaging
- Experiment tracking and performance benchmarking via OpenVINO, ensuring transparency and reproducibility
Custom dataset and labeling pipeline
To build models capable of verifying both object type and physical condition, we developed a tailored dataset from high-resolution images captured directly by the client’s machines. Since existing public datasets lacked the precision, coverage, or licensing required, we established a custom ingestion pipeline to organize raw image archives, eliminate duplicates, and preserve metadata such as object dimensions and material.
We deployed a self-hosted instance of Label Studio and onboarded a dedicated labeling team to efficiently annotate thousands of samples. Clear labeling guidelines and active feedback loops ensured accurate segmentation, including in edge cases such as open bottle caps, pull-tabs, black containers, and transparent packaging. This labeling infrastructure was designed for reusability — allowing the client to continuously expand the dataset and improve accuracy over time.
Iterative model development for segmentation and verification
We evaluated existing image segmentation models but found them too large or slow for on-device execution. Rather than using an off-the-shelf segmentation architecture, our team built on MobileNet — a model commonly used on edge devices, typically for classification — adding custom layers on top so it could function as a segmentation model. A data scientist built the initial version optimized for quality, which was then fine-tuned, ported, and benchmarked by an ML engineer for performance on the target hardware.
Each iteration was tracked with benchmark metrics, including per-image inference time and hardware compatibility validated via OpenVINO. The segmentation module received two major updates over the course of the engagement, each achieving statistically significant improvements in IoU and MAPE evaluation metrics for object dimension estimation. This iterative approach allowed us to gradually reduce model complexity while preserving accuracy, ultimately delivering a version that met both speed and reliability thresholds.
Lightweight, deployable architecture
To simplify installation and updates, we shipped the entire solution as a versioned Python package, including:
- The final trained model,
- Inference scripts optimized for the client’s setup,
- A requirements list to streamline dependency management.
This modular packaging strategy reduced update sizes and made it easy for the client to deploy the solution at scale across multiple machines, even in bandwidth-limited environments.
Ready for future expansion
The system architecture supports future needs such as:
- Adding classification for brands or deposit eligibility
- Including new object types (e.g. cardboard containers)
- Fine-tuning models as more labeled data becomes available
- Integrating fraud detection insights into dashboards and analytics tools
The final solution delivers accurate, real-time image analysis at the edge, enabling secure, fraud-resistant recycling flows. All recognition runs locally on-device, with internet connectivity used only for sending captured images for analysis and receiving model updates.
