Case study Krattenteller

Case study Krattenteller

Introduction

For a Dutch supermarket, we created an algorithm that recognizes the crates visible on an image. Then the system gives the user the total number of crates recognized by the algorithm back.

The problem

Every day, employees spend 1.5 hour counting crates and filling in forms. With 697 stores, 365 days a year, we calculated that the Jumbo can save 3.8 million euros a year on staff costs. Also getting rid of boring repetitive counting of the crates, workers at Jumbo can do more meaningful and fulfilling work, which makes the working environment a better place.

The process

First, we started to categorize the crates, some crates are of the same type, but have different dimensions. Others just don't look a like at all. We came up with a handy way of solving this; we started putting them in different categories. Here is a picture where you can see our algorithm classifying a picture full of crates.

Picture 1. Our algorithm classifying a cart full of crates.

 

We started gathering pictures of carts full of crates in the supermarket. We needed lots of samples to get good results, so we took a lot of pictures. The next step was starting the annotation process. We annotated a few hundred crates to see if our algorithm was able to distinguish the different types of crates, A1, A2 etc...

For example, this is a crate of type A3, which our custom algorithm is classifying with 91% accuracy:

Picture 2. Our algorithm classifying an A3 crate with 91% accuracy.

 

In the end, we had different types of crates ranging from A1 until D5. I think we can call ourselves real crate specialists by now;).

Why it matters

Employees of supermarkets spend hours a day counting crates by hand and filling out forms with the total amount of crates. These crates and forms need to be returned to the suppliers of the food that the supermarket receives. This is a labor-intensive and repetitive task that can easily be automated. Employees as well as the supermarket owners benefit from this automation step.

Although we didn't finish the project because of the Covid-19 pandemic, we showed in a PoC (Proof of Concept) that it's possible to save thousands of labor hours a year using smart machine vision automation. Also getting rid of boring repetitive counting of the crates, workers at Jumbo can do more meaningful and fulfilling work, which makes the working environment a better place.

Some further thoughts

The Krattenteller could be easily integrated in the supply chain between supermarket and suppliers, so the error-prone human counting and filling in of forms can be done by a smart bot. The employees can use the time saved to do more fulfilling and productive work. This is a great example of how smart Robotic Process Automation can help the supermarket industry to automate processes, enhance employee satisfaction and save costs.

MadeByHumans.ai created the whole Deep Learning production pipeline. From data extraction and labeling of hundreds of images, to model training and validation.