Images create Emotions…

…and emotions lead to interest. That is one of our thoughts behind Funky Retail. This video should explain what we mean. So please, see for yourself!

Just a short note on the analytics: The data we collect is merely an example to show that it is possible to connect the physical retail space. There may be other data that is more vital. The colourful, flashing lights also have the purpose to show that actions are recorded. We’re not necessarily saying future stores have to flash like crazy in all the different colours…(although we do think that would be quite cool).

Watch the Funky Interview here!

Moto Update: the smartphone is now our MQTT/BLE Gateway

It’s time for an update about ‘moto’ – sorry that this did not happen earlier but I’ve been busy with events like #cebit, #iotcon or #internetworld. We’ve now finalized the hardware design and our good friends at DerGrueneFisch are manufacturing a small series of moto prototypes (9 to be exact) in the coming weeks. This also means that I’ve moved on to more software-related challenges instead of hardware challenges.

Moto Architecture Diagram

If you remember the architecture diagram (find it again above), we connect the motos wirelessly via BLE. While I’ve been using some quick & dirty node.js based scripts on my mac for testing the communication over BLE, I’ve now written an Android app that acts as a MQTT/BLE gateway. Powered up, it will launch two services: the BLEService and the MQTTService. These services are started and then continue to run in the background. They are loosely coupled via Android Intents. Right now, we fire up the services when the Android Activity (that’s what is “shown on the screen when an Android app fires up) is shown. And we stop these services again, once the app becomes invisible. This is really convenient for testing, as we tear down/fire up the services a lot which is great for testing.

BLEService
This sticky service (meaning the system may restart it if it was removed due to resource constraints) is scanning for new, non-connected motos and will then try to connect. Once connected, we subscribe to notifications for one BLE characteristic which acts as  the event stream from the hardware. We also save a reference to another identified characteristic that we use to send our commands to. In order to react to commands and be able to forward events, we use Android intents. The BLEService registers listeners for all intents that the MQTTService is sending out, as they contain the moto commands that need to be forwarded to the moto’s. The BLEService also maps the incoming commands to the corresponding motos and – new – now is namespaced. That means the users of the Moto Android App will later be able to choose their namespace so the analytics data is kept separate from others.

MQTTService
For MQTT, we’re using the only Android/Java MQTT client we were able to get: Paho. Although there seems to be an existing Android Service wrapper around the Paho MQTT client, that one is little documented and it really was simpler to create our own service that does exactly what we want it to do. The MQTTService is again sticky and should be running all the time. It tries to keep a constant connection to the MQTT broker that we host on Amazon EC2. It is subscribed to all commands that fall into its namespace, e.g. moto/<namespace>/+/command – which is an MQTT topic with a +wildcard, meaning it will receive messages sent to moto/<namespace>/1/command for example.

Getting MQTT or BLE running on Android alone and for  a small demo is pretty easy. The complexity comes one you try to connect to multiple devices at once, because the Android BLE APIs are synchronous and firing too many BLE requests at once will simply override a few requests sent. So one has to work with a few delays and timers here and there to make sure it really works reliably. The idea is also, that sales agents with the app installed can roam freely and if one is close to the BLE devices, their phone/app will connect transparently. So far, this works realy nicely. After a few seconds outside of the coverage area, the BLEService starts to receive disconnect callbacks and we start removing the moto element from the list of connected ones. This will enable it to be added by another sales agent and his/her device that has the app installed.

The Protocol
At least for now, I’ve also frozen the “protocol”, e.g. which characteristics are used, what data is sent, how it is determined what is possible and what not. First of all, for sending and receiving data from/to the moto elements, I use two seperate BLE characteristics. This simply keeps everything a bit more organized and easier to understand. For sending from the BLE hardware to the smartphone, struct-based events like these are used (this is straight from the Arduino IDE):

Mainly due to issues with setting up multiple BLE notifications from Android at once, I decided to distinguish the two events that I send out via the first byte – see the “eventType” byte which is different for a PresenceData Event and MetaData event.  MetaData Events are sent our regularly to inform the smartphone and the server later that a device is live. We visualize the MetaEvents again via heartbeats. You can tell within 10 seconds if a device is connected or not. The PrenseceData Events are sent whenever the presence state (customer in front/customer lost) changes. Just like with tiles, we also calculate the duration of the presence directly on the device.

For incoming data, so-called moto commands, the protocol is slightly more complex. We distinguish between two broad categories of commands:

  • “standard” commands can change the current RGB colors and the motor state (this includes on/off, direction and speed level of the motor)
  • “special” commands are distinguished from normal commands by the value of the first byte. To be able to extend the command mechanism, they introduce a “subcommand” byte as the second byte. From the third byte on, the special command’s data is sent. Right now I’ve specified a “blink” command that will blink the RGB pixels for a certain duration in a certain color. Another command implemented is a rainbow chase, so the pixels will update according to a color wheel which looks like a rainbow in the end.

Some code example showing how I deal with incoming commands is below:

Android UI Adapter
One last element that got a lot of love from me is a special UI Adapter for the Android app. There’s nothing super special about this data/UI binding, it is just a lot of work. The UI will later try to come close to the action that the moto element is performing: if it blinks, the UI element in the android app will blink, colors will be reflected as well as possible and of course the spinning status will be represented. Once I have a few motos connected at once, I will shoot a few pics and show this to you in an update.

Up next
Now that I have the hardware specced out and a running gateway prototype via the Android App, the next thing that I’ll spend time on is the server side that collects all the data. This will also include a RESTful API to control each moto element, client/server communication via socket.io for the UI and early ideas for the skinning. I hope to receive the first elements of the produced series within 2-3 weeks and will try to update you on the progress made.

 

CeBIT presents the solar eclipse

CeBit 2015 – “Fresh Impulses for Digital Business”, innovations everywhere, and in the middle of it our Smart Wine Shelf. A prototype we developed about a year ago… Of course we had some new things with us as well! Funky Retail and Tiles completed our little IoT family. Everything started well on the Monday. Ian Kimbell, SAP’s chief presenter and self-appointed ‘Demo Guy’, showed our Wine Shelf (with physical web beacon) on SAP’s own stage.

IMG_20150316_160052-MOTION

But then on Friday, when Sven entered the stage, things looked like this…

share

…empty spaces, abandoned places… but the show had to go on. Bravely Sven tried to attract more attention by letting the Wine Shelf flash, showing videos of Funky Retail, and even setting up the colourful Tiles demo live on stage, always thinking “Did everyone realise that I don’t really prepare these kind of presentations…?”. To be fair on Sven though, it’s not easy when you’re contending with a solar eclipse… Slightly unlucky timing. So unlucky that even your colleagues turn their backs on you…

share copy

Sven took it with a big smile and now refers to the whole thing as #darkpresentation. And he did get his audience in the end…

IMG_20150316_1520221

Watch the video here!

Moto: exploring the smartphone as an IoT hub for retail

At #hybrislabs, we’ve explored IoT quite a bit now. We’ve begun with the smart wine shelf, our first IoT experience for the retail spaces that used a unique idea and combination of technologies to provide both customer and retailer value. Next up was funky retail, where we focused on the analytics in the retail space with both distance and pressure sensors. With tiles, we went wireless for the first time – but still used a central hub where all Bluetooth LE messages are collected and forwarded to the cloud.

Finally, with moto, we’re now filling a gap. We would like to explore one missing IoT topology in our portfolio: using the smartphone as a hub for the connected devices around you. Below is a pic how the current prototype looks. In the end, it will be a glass-protected, spinning disk that is lighted up from below. It will feature an IR distance sensor to detect customers, be able to change rotation speed and direction as well as the color. It will require a power cable, but communication will again be bluetooth low energy.  Here’s also a video of moto from a recent G+ post.

IMG_20150303_163718

What is more important and sadly almost invisible is *how* we connect these IoT elements. We’ll not use a central hub. Instead, the plan is to have iOS/Android Apps installed on the sales assistants phones that automatically connect to the retailers smart objects. These apps on the smartphones connect via BLE and forward the data to/from the cloud to/from the the things. The idea is that a sales assistant can freely move in the retail space. The app will scan and connect, might loose the connection from time to time and leave one “moto” disconnected, later move back in range and reconnect. If another sales assistant with the same app and configuration moves in range, he will take over.  Here’s the architecture:

Moto Architecture Diagram

At this time, we’ve successfully connected to the moto’s and defined the rough BLE-based protocol that we’ll use. We’ve got some node.js based code that works on a Mac for experimenting and testing. Next up will be the task to write a good Android app (iOS welcome, too), that launches, finds IoT elements, connects and then proxies the communication to the cloud. For the cloud communication, we’ll again use MQTT but still need to find a good and easy MQTT solution for Android/iOS. So if you have any good ideas and are able to point into the right direction, let us know! (@hansamann or comment – we actually do read them!)

To wrap this up, here’s the raw PCB of moto with the neopixel RGB ring and IR distance sensor connected to the PCB. The board again uses a LightBlue Bean for the BLE connectivity. As it is running on 9V for the stepper motor (which is not shown here), we need to step down the voltage twice from 9V – one time to 5V for the neopixel RGB LEDs, another time to 3.3V for the ligthblue bean. We’re also using a stepper motor driver, DRV8834 on a breakout,  that allows us to control the direction and speed of the stepper motor.

IMG_20150303_161139

A new Alliance

San Francisco, USA… and we’re scrubbing labels off of wine bottles in the bathtub again. That’s part of the hybris labs job description by now. Fly half way round the planet, check in at hotel, buy cheap wine, re-label bottles in hotel room.

IMG_1896

(Downtown San Francisco, Parc 55 Hotel, Room 1511, bathroom)

This time we did it to show the Smart Wine Shelf at the local SAP d-kom event in San Mateo, Silicon Valley. Furthermore we also presented Tiles. No scrubbing or bathing needed for that prototype, just in case you were wondering….

For this event we formed an alliance with our fellow compatriots from the SAP d-shop in Palo Alto. After we had picked up our badges at the SAP Office in Building 1, we took a walk through the park (yes park, not bad…) to Building 9 and were quite impressed. A proper lab full of people doing nerdy things! Lovely! They had organised a hackathon as a buildup to d-kom and the house was full. Very cool!

IMG_1911

(Hackathon at SAP d-shop in Palo Alto)

Next day it was time for business. It turns out there actually are still quite a few people who haven’t heard about our Smart Wine Shelf yet. SAP d-kom being an internal event for developers, we took the opportunity to show the insides. Wires, Arduino, Raspberry PI, boom. It’s really nice to experience the interest in our prototypes, and bathe in positive feedback sometimes.

IMG_1937

But  connecting with our friends from SAP d-shop was perhaps the nicest part of the trip. Hopefully we’ll be working on some projects together soon! Great minds think a like, they say (= they seem to be similarly mad to us).

IMG_1943

(from left to right: Nick Wood, Rocky Ongkowidjojo, Max Schrupp)

Tiles in color, plus finalized arch poster

Now that we have all major events (except the hybris xmas party) behind us, we can finally focus on getting a few projects really finalized. Tiles  made huge progress over the last weeks and I just got the fully-colored tiles in, plus I have a finalized architecture poster that I want to share with you. Big kudos go out to Elke and DerGrueneFish, our booth building partners for this and most other projects. The tiles (21 in total, for 3 complete demo sets) are colored in 4 fresh colors for a change (no boring white!). I absolutely love the way they look.  Over one day, I was soldering the first 7 which are currently connected to one hub.

IMG_20141208_114334

For the poster, Kathi at SNK did an awesome job. I already ordered our poster which we’ll then present at the hybris summit 15 in Munich at our booth. Having a descriptive poster will greatly help us to explain the IoT setup for this prototype. Right now we expect to have cans on top of the tiles, so we made that part of the poster.

tiles-90x60

 

 

Just to recap the architecture, have a read:

  • “Tiles” are the wirelessly connected platforms. We use Punchthrough’s LightBlue Bean and remove the battery holder to make the platforms 8mm high. We still use CR2032 batteries, which gives us about 1 week battery life right now. We would get more, but I send our a MetaEvent every 10sec which is hard on the battery.
  • The “Hub” collects all data. It scans for tiles, continuously, and connects. The hub runs on the raspberry pi, uses a BLE dongle (choice is key here) and uses node.js for all programming. It sends on data to the server with CoAP – a UDP-based IoT protocol.
  • The “Server” collects all data for all hubs (yep, there can be many) and provides the necessary APIs for managing the User/Tile association, authentication and authorization (Oauth2 used here), etc.

 

One change over the last days was that we can now associate products with the tiles. That means a store manager can just scan a tile (NFC or QR) and then add this tile to his private analytics page. The UI of these web pages is currently being worked on and will feature a few cool features such as a heartbeat every 10 seconds or the color of the scanned tile, that gets pulled via some static, factory-decided data. This system is all up and running now, currently with one live hub and 7 tiles connected.

14 - 2

What’s left is the callback mechanism plus the web ui. The callback mechanism will “call out” to external systems for each event reveived. So if a LiftEvent is received and a webhook is configured, we’ll send out a HTTP Post to the configured external service. I also plan to pull in the product details from YAAS, hybris’ on demand API offering.

14 - 1

 

Tiles Update – we've added blinky blinky

Our project Tiles, little BLE-connected platforms for customer interaction tracking, is entering a project phase which allows me to blog and inform you a bit more. Since yesterday night, the Raspberry PI and Arduino in the hub uses one power source. This makes the overall design easier. We also have been working on a Raspberry PI B+ hat, using Eagle, to further optimize our design.

One visible change is also that it now blinks 🙂 The hub rotates an LED light to signal the BLE scanning process. It flashes once you liftup the product, well, the apple in this case.

IMG_20141126_204216

 

We’ve now also locked down the architecture and below is a rough sketch that should help understand it. Again, a quick summary below.

tiles technical architecture

  • “Tiles” are the wirelessly connected platforms. We use Punchthrough’s LightBlue Bean and remove the battery holder to make the platforms 8mm high. We still use CR2032 batteries, which gives us about 1 week battery life right now. We would get more, but I send our a MetaEvent every 10sec which is hard on the battery.
  • The “Hub” collects all data. It scans for tiles, continuously, and connects. The hub runs on the raspberry pi, uses a BLE dongle (choice is key here) and uses node.js for all programming. It sends on data to the server with CoAP – a UDP-based IoT protocol.
  • The “Server” collects all data for all hubs (yep, there can be many) and provides the necessary APIs for managing the User/Tile association, authentication and authorization (Oauth2 used here), etc.

One more thing – I’ve connected the server to Xively, a data logging platform. We collect mainly the battery rundown to estimate battery life and also the temperature values from the lightblue beans. At this point I just want to share some nice graphs to show you how much sense it makes to track that data. It will definitely help us to optimize the design / battery consumption further. Right now we stay optimized for demo purposes, but we can later reduce the events sent for example to get a better battery life.

Screen Shot 2014-11-27 at 9.49.19 AM

Screen Shot 2014-11-27 at 9.49.13 AM