Consuming Spark Core SSE Events via node.js

I am attending ThingsCon right now and was in a workshop with Zach Supalla, the CEO of Spark. Besides hooking up a brand new spark core to the cloud and sending Server-Sent-Events down to a Web Page, I also finally figured out how to consume these events from a regular node.js program (and not *just* a web browser…). Here’s a quick tutorial to show you how easy it is. I am assuming you have a spark core ready and connected via WiFi.



First, the spark firmware. In this case I connected a very simple PIR motion sensor – you could change that and just send a publish event every few seconds.

You can see that a ping event is sent every 30 seconds, and a motion event only if there is motion detected. Very simple, right?

Let’s move on to the node.js code. I initially used the node.js EventSource library, but this did not work. While got event notifications from the library, the events themselves were empty objects. At #thingscon today I looked closer at the Spark Command Line library and finally ripped out the essential parts for subscribing to the SSE events from that lib. The result is this small script. It mainly connects to the Spark SSE source via the request library, then whenever some data is available, it parses the data according to the SSE spec (which is pretty much line by line, plus stripping off the event: starting strings).

I we assume the node.js program is saved to sse_request.js, you should be able to run this program on the command line via

I totally want to note that the code is really ripped out of the excellent Spark CLI – which I recommend to have installed for anyone doing stuff with the spark core. It is a great little helper.

Learning APIs with Zen Koans and test driven Learning

Together with hybris Training we’ve been working on a new way to train people for the ycommmercewebservices APIs. As you might know, the ycommercewebservices template extension is a hybris platform extension that provides RESTful APIs for commerce. The APIs are secured via OAuth2, follow a pragmatic RESTful pattern and support both JSON and XML representations. These APIs are the APIs used by our own mobile clients (Android, iOS) and will allow any third party client to be connected to hybris commerce functionality, too. While someone working regularly with RESTful APIs should feel at home very quickly, you still have to learn about the specific resources that we expose. You have to understand how to initiate a commerce session and put items into the cart, register a customer and later follow the checkout path. The goal of this training is to reduce the hurdles that you would typically find to a minimum.

All you need, is a Groovy installation on your system and a single Groovy script file.

This is actually an example of test driven learning. There is a single Groovy file that represents a collection of “Koans” which the trainee will have to fix and develop as he or she progresses. In Zen, a koan is a little story that demonstrates a principle of Zen philosophy. In software koans, each koan is a little method or use case, like adding items to a cart, registering a customer and checking the cart out. Technically, the Koan is a broken JUnit test case. Once the student fixes the test case, they have understood the principle. Instead of using Java, we developed these tests using Groovy, which is more expressive and simpler therefore easier to understand.

We’re at the point where we ruled out major roadblocks like a fully setup backend hybris instance, SSL connection issues and the overall setup of these tests. The next step will be to break the test in a meaningful way – in a way the trainee has a good chance to fix the test if he reads through the provided material. Once the trainee has successfully finished the Koans, he or she will have a good understanding for the hybris commerce APIs and also OAuth2.

This is kind of what the Koans look like. Please read from top to bottom as I added some comments.

Trainees have to put the correct stuff in each method or else the the test will fail. We provide links to the hybris documentation in the method description and further material if needed.

Once all methods are fixed, the trainee has come into contact with many of the API’s methods. From cart management to OAuth2 token management, from login to final checkout and verification that the order is in the list of available orders for this customer.

Again, all you need to get started is a Groovy installation and one single Groovy file. The hybris system used itself is provided via a management cloud instance of hybris. Each test run is self-contained, e.g. a required customer account is created on the fly and does not influence other accounts. Executing the same test twice, will create two fully isolated test runs.

We think this is an exciting and innovative way to teach developers about our products. Watch the hybris wiki for the upcoming releases.

Play, Collect, Redeem

This post is a follow up post to the physical endeavors we’re currently undertaking. We’re not only looking into “physical” because making is a huge trend, we also believe there are many interesting applications when it comes to in-store usage. Prototyping with electronics and other physical components gives us the required knowledge to build up prototypes around in-store ecommerce. One example is couponing. How would you use in-store couponing to drive sales to your online or offline store? How can a social profile be merged with coupons and used to gain insights or create viral coupon campaigns? What are the elements of coupons, the characteristics and how do users collect and redeem these coupons?

Our prototype explores exactly this.

The big picture – Play, Collect, Redeem


The somewhat  🙂 technical architecture below shows the key components and how the are linked together for form a couponing system. Customers can play physical games, then collect coupons that they can later redeem for products (or get discounts, etc.). Let’s take a look at each component, seperated via the swimlanes in the below graphic. For the better understanding, we begin with the physical gaming frontends.

hybris Coupons (1)

Physical Frontend

The physical frontends are easy-to-play games that customers will love to interact with and try out. These games are crossover projects between gaming, art and technology. The games include a physical keyboard (via PVC foil and capacitive sensors), a push-button game and a foosball table (kicker) which we put light barriers inside to get the scores.

We designed the system in a way, that only “authenticated” frontends can be used to obtain coupons from the backend. This authentication is done via a simple shared secret (more complex scenarios possible), but the shared secret is not in the web app or otherwise saved in resources that can be accessed from the web. The shared secret is part of the Arduino hardware and trasmitted via a set of keystrokes to the web frontend. That means, only if you are in the posession of the physcial game and the Arduino prototyping board, you can obtain the access token to the coupon backend.


Web Frontend

The web frontend is used to receive the game inputs from the physical games, receives the authentication key from time to time and is used to display the coupons. At a later stage, we will also use it to show the game rules to the users of the game (right now these frontends are very basic, ready to be skinned by an CSS guru!).

Via the web, or mobile web, users can also “sign-in” to a physical game frontend. The users can scan a QR code or scan a NFC tag to be redirected to a facebook login page. Once they’ve authorized our coupon application, their profiles and names appear on the web UI. Later on, these profiles will also be used to share the coupon activity to Facebook and to personalize the received coupon itself. We have all the stuff in place to help coupons generated go viral.
Coupon Backend

The coupon backend is the heart of the system and shared among all three frontends. With the required authentication token from the physcial game frontends, the web apps are able to create new coupons via AJAX calls after the users have played a game. Based on the score, a different coupon is created. Each coupon is unique and protected via a signed URL. This allows the coupon backend to trust incoming requests, as the secret for generating the URL signature is known to the backend only. If you get a coupon, tamper with the URL and try to view or redeem it, it won’t work.

The three main RESTful/JSON APIs that the coupon backend exposes are the APIs required to create coupons, view coupons and later redeem coupons. The viewing part has different representations – one is pure JSON, meant to be used for mobile clients or other systems accessing the coupon. The second is a web representation, e.g. a responsive web page which holds the coupon as a QR code. The third is an image-only representation, ready to be embedded into a HTML img tag. These three representation make it very easy to integrate coupons into any application.

As an example, we’ve already upgraded our hybris Wishlist Android application with the feature to collect coupons. As it is a native Android application, the QR code or NFC tag is scanned, which transmits the signed coupon URL to the app. The app then resolves the Coupon via the REST api and chooses the JSON representation to obtain access to all coupon data.
Mobile Frontend 

There are actually two mobile frontends: there are the shopping assistants, who scan the coupons and redeem the coupons for the users. They use a simple QR code scanner on any OS to view the coupon. They are the only ones who can mark  a coupon “used” – that means the coupon’s “use count” is increased by one. Once a coupon hits the max use count figure it is no longer valid.

The other frontend, used by normal users, is either a web representation of the coupon or a mobile app, such as our Android-based coupon app. The benefit of the app is that you can collect multiple coupons, store them, revisit them and later redeem them.


So far so good, what’s next…

We got a lot of essential systems in place, but there would be a lot of opportunity for further research. Think about data analysis, different coupon/product combinations, geofencing wishlist items and coupons or just coupon variations (one-off coupons, product segment coupons, single product coupons, shareable/viral coupons, etc.).

We’ll also go on a roadshow to present all this. First stop will be Derry in the UK for a technology and art festival. Next up might be a swiss conference which is not yet finally approved. All these conferences and presentations will help us to interact a lot with real users, potential customers of hybris systems and also end-users. It’s a great, playful and uncomplicated way to get real and honest feedback. It all begins with a game…


Physical Computing

If you’ve read Chris Anderson’s “The new industrial revolution” you’ll likely be infected with the virus of making. So are we. Chris writes about electronics, laser cutting, 3D printing and how this could lead to a new industrial revolution. It’s about connecting the physcal world to the internet. Expanding your reach to the physical world, which – as you know – is damn big! Among our hybris colleages, we discovered many that also “play” with robots, flying drones or automatic gardening systems. What previously looked like pure play got a serious touch as many of these things can be relevant even in an commerce setting. To explore physical frontends other than “just screens” in combination with ecommerce, we’re currently working on not just one, but already a few protoypes. This blog post will be the first one of (very likely) quite a few others that explore physical computing and “making”. If all works out, we’ll be presenting and talking about our experiences at the CultureTech Conference. Till then, a lot of making and learning has to happen!

Custom PCP board

Before we delve into our first prototype in more details, let me tell you one thing that we’ve learned already:  connecting physical things to the internet, to your computer or to other systems requires a bit of learning, but it has been an extremely rewarding process. Learning how to make a box yourself via a lasercut service is simply awesome and a great skill to learn. Armed with some basic knowledge of electronics, like the Arduino board you quickly realize that pretty much everything is possible. Connecting arcade buttons to a in-store display? Easy. Create a physical keyboard as a frontend for a coupon-generating backend in the cloud? Done. Flying into space? Well…

(very) first prototype

The very first prototype we were working on is a physcial keyboard. The idea is that users have to successfully complete a music challenge to be rewarded with a digital coupon via QR code or NFC. The idea was born at hybris’ customer and partner event 2013. We disdovered how much fun we all had interacting with technology in a physical world (in this case, it was hand gestures via Kinect). We used a simple poster printing service to print a keyboard on a large piece of PVC foil. Next, we had to think how we get the keypresses as input into a computer. Arduino, the low-cost board which can be easily programmed was a natural choice. The communication to the computer was prototyped with a Node.js server running on a Mac that opens a serial connection to the arduino. At the same time, that Node.js server is creating  local websocket to be connected to a local web application (served by Node, too). This allows us to keep the UI in the browser, the logic of the game in JavaScript and use lightning fast connections via a 115200 baud fast serial connection. It took a while to connect the dots, but it does work.

malte malte_sven

Figuring out how to create the sensors was a challenge in itself. I initially prototyped a keyboard with a few pressure-sensistive resistors before Malte had the idea to use simple and cheap copper foil for capacitive sensors (we ordered copper foil from eBay and finance was *not* impressed). He created our own, custom PCB board, send it to a PCB manufacturing service and finally programmed an ATMega chip you control a special capacitive sensing chip. We had to solder all these SMD (=SUPER SMALL) components onto the board, later connect the cables and fine-tune the sensing. The original goal to use the keyboard with your foot was dismissed due to too much jitter in the signals, we’ll now hang the keyboard at a wall and you’ll have to use your hands to operate it.


A physical frontend, like our physical keyboard is not created overnight. It requires all kind of disciplines to be combined. A diverse set of skills hase to come together to make this happen. I went to a local stitcher and paid her for stitching the PVC foil so the sensors cannot move freely between the top and bottom layer.  Malte and me soldered and I cannot tell you how many times I sat at home and tried out circuits and early prototypes connected via breadboard to the Arduino. For the software, I had to get into Node.js programming which I did via 2 hands-on books on my Kindle. I am still a total beginner when it comes to electronics but the basic knowledge is there and all the rest can be read on the internet. The diverse combination of disciplines is awesome, as it sparks creativity (diversity is a key principle of innovation).

In the end, we figured out that there is most likely and easier way to connect the keyboard to the web application 🙂 We’ve ordered two more keyboard foils and I am pretty confident that we’ll simplify the whole system.



Now that we have gained all this knowledge, we’re ready for a few more physical frontends. Likely, we’ll be creating gamification ideas using LED buttons, lasers and all kind of sensors. Stay tuned and go ask us questions. The hybris technology group on g+ is a great place to get in contact with us and discuss.

Exploring the physical world and connecting it to the internet was an awesome ride so far and the good news is that we just tapped into it!