How accurate are Estimotes at measuring distance?

 

We have carried out a series of experiments in which we have positioned beacons around a room and measured the received signal strength and the distance measurement from Estimote’s SDK at specific points in the room. We have compared these measurements with the actual distances at each position.

We have taken a lot of care to ensure that the readings are taken in a consistent manner. We have positioned the iPhone on a tripod to ensure the phone is always at the same height and angle. Real distances have been measured with a laser rangefinder.

Each graph below plots the actual distance of the phone from the beacon along the horizontal axis and the difference with the measured distance received from the beacon along the vertical axis.

If the beacons were completely accurate we would see all the data points along the 0 horizontal, whatever the actual distance.

In practice the results are not encouraging. In some experiments and for some beacons we hit upon a ‘sweet spot’ of accuracy. This has tended to come at around the 3m mark. However, as can be seen below this isn’t consistent enough to be reliable. And in all experiments the variation of repeated measurements at the same distance was significant.

Experiment 1

The first experiment was carried out in our office. There was lots of interference – computer equipment, people, fluorescent lights, wifi. We had 3 Estimotes positioned on the walls at a height of around 2.2m. We recorded the distance indicated by each Estimote from four positions around the room. Each distance measurement is the average of 10 readings.

Estimote settings: power  -8dB,  advertising interval 320mS

If the Estimotes were completely accurate all readings would be around the centre line. We can see that the results are inconsistent. Beacon 1 was accurate when measuring a distance of around 2m, but increasingly inaccurate as the actual distance increased. Beacon 2 was accurate at 3.5m and 5m but significantly out at just under 3.5m. Beacon 3 was wildly out at each distance.

Experiment 2

In this experiment we moved down into the cellar beneath the office. This is a larger space, with much less interference.

Estimote settings: power  -8dB,  advertising interval 320mS

Experiment 3

We repeated the experiment, but switched the positions of the Estimotes to see if there were any consistent issues with the individual beacons.

It’s interesting to see that all beacons were very accurate at a distance of around 3m.

Experiment 4

The previous experiments were carried out with the 3 beacons from our initial Estimote Developers’ Kit. We had subsequently ordered three more packs so in this experiment we switched to three of the new beacons.

Again we see a sweet spot at a distance of around 3m, but at greater distances the errors are even bigger than before. The measured distances for Beacon 1 at 4m were ridiculous – errors of up to 17m!

Conclusion

It seems pretty clear to us that at the moment the distance measurement accuracy is a long way off being good enough to use for accurate positioning. We had hoped to be able to use the beacons for triangulation but with errors like we have seen above this doesn’t look possible at the moment.

We had hoped to establish some consistency to the errors, in which case we would have been able to implement some sort of correction algorithm. But the errors seem maddeningly inconsistent. A beacon that seems to giving an accurate reading at a certain distance then produces a wildly different reading at the same position.

For now we are not going to pursue using Estimotes for distance measurement but will instead looking at a more ‘zone-based’ solution. By determining the closest beacon we should be able to place the user in a zone within a room.