IoT Workshop with Particle Photon

Just a quick update from an internal IoT event here at hybris. Today, we ran our IoT Workshop for the first time and had a hell of fun. We used our carefully crafted IoT Experimentation Kits which included a Particle Photon to educate 15 people about the internet of things. And yes, we’ve connected our first buttons to the YAAS PubSub Service, which was a lot of fun!

Below, you see the box that every participant received, plus a few other pics documenting the fun. At the beginning, we had quite some trouble to get some devices online. We bricked three devices, but in the end everybody was happy and got some YAAS buttons/LEDs connected.

IMG_20150922_092033

We’re now looking forward to seeing some cool creations over the next couple of weeks! Here are a few more impressions:

IMG_20150922_114951 IMG_20150922_114937 IMG_20150922_120348

Learning APIs with Zen Koans and test driven Learning

Together with hybris Training we’ve been working on a new way to train people for the ycommmercewebservices APIs. As you might know, the ycommercewebservices template extension is a hybris platform extension that provides RESTful APIs for commerce. The APIs are secured via OAuth2, follow a pragmatic RESTful pattern and support both JSON and XML representations. These APIs are the APIs used by our own mobile clients (Android, iOS) and will allow any third party client to be connected to hybris commerce functionality, too. While someone working regularly with RESTful APIs should feel at home very quickly, you still have to learn about the specific resources that we expose. You have to understand how to initiate a commerce session and put items into the cart, register a customer and later follow the checkout path. The goal of this training is to reduce the hurdles that you would typically find to a minimum.

All you need, is a Groovy installation on your system and a single Groovy script file.

This is actually an example of test driven learning. There is a single Groovy file that represents a collection of “Koans” which the trainee will have to fix and develop as he or she progresses. In Zen, a koan is a little story that demonstrates a principle of Zen philosophy. In software koans, each koan is a little method or use case, like adding items to a cart, registering a customer and checking the cart out. Technically, the Koan is a broken JUnit test case. Once the student fixes the test case, they have understood the principle. Instead of using Java, we developed these tests using Groovy, which is more expressive and simpler therefore easier to understand.

We’re at the point where we ruled out major roadblocks like a fully setup backend hybris instance, SSL connection issues and the overall setup of these tests. The next step will be to break the test in a meaningful way – in a way the trainee has a good chance to fix the test if he reads through the provided material. Once the trainee has successfully finished the Koans, he or she will have a good understanding for the hybris commerce APIs and also OAuth2.

This is kind of what the Koans look like. Please read from top to bottom as I added some comments.

Trainees have to put the correct stuff in each method or else the the test will fail. We provide links to the hybris documentation in the method description and further material if needed.

Once all methods are fixed, the trainee has come into contact with many of the API’s methods. From cart management to OAuth2 token management, from login to final checkout and verification that the order is in the list of available orders for this customer.

Again, all you need to get started is a Groovy installation and one single Groovy file. The hybris system used itself is provided via a management cloud instance of hybris. Each test run is self-contained, e.g. a required customer account is created on the fly and does not influence other accounts. Executing the same test twice, will create two fully isolated test runs.

We think this is an exciting and innovative way to teach developers about our products. Watch the hybris wiki for the upcoming releases.

Client-Side, Responsive, Commerce Web Apps :-)

Bullshit Bingo, anyone? Sorry for the extreme amount of geekery in the title, but our “Responsive” prototype that we’ve been working on in the labs is definitely not short of cool technologies. Our prototype, which is mainly (but not just) about exploring a commerce-driven responsive UI, is now live for you all to see. Please keep in mind that we are still working on this, so it might go down any time and we definitely do not make any promises that what we’ve put together here is ready for prime time. Also, for now, please use a recent browser such as Chrome, Firefox or IE10. We’ve also used Chrome on Mobile and Safari on the iPad for testing so far. Take a look, enjoy, resize the window, then read on.

So what are the technological elements that we explore here?

  • Responsive Web Design. Not exactly new, but still a lot of websites out there use plain old web design techniques. Not all websites need to be responsive, but for a lot of them, it could make sense. Respnsive Web Design means a single set of resources (HTML.CSS,JavaScript) are used on all kind of screens – desktop, tablet, mobile. CSS and sometimes JavaScript will make sure that the page looks “as good as possible” on all these screens. Also, responsive web designs are typically optimized for touch devices, too, so using them from an iPad, swiping pictures, easily pressing buttons and entering data is all possible. We have used Bootstrap, a widely known html framework and currently one of the “cool kids” out there. If you are not a tech guy, that means that Bootstrap means something to a lot of web designers and that should make it easier to get your project off the ground.

Screen Shot 2013-06-04 at 9.37.46 AM

  • Client-Side Web Applications. Our commerce website is hosted on responsiveui.appspot.com while all the commerce logic is exposed via the hybris OCC API and is available under a different host name. A JavaScript SDK was developed to talk to our backend OCC API. We’ve seperated the UI completely from the Logic. That is pretty interesting and has some advantages and disadvantages. One of the big advantages that we currently see is that the decoupled serves make the UI/Website development a breeze. Our working student, Simina, was able to rapidly prototype the website using the given API. Also, our designers from SNK were able to make quick changes and optimize the UI. This all became possible because we have very quick iterations. You can essentially run the frontend website on pretty much any web server and quickly change some resource, hit reload in the browser and see how it works. One downside is that for Cross-Origin AJAX requests, you need to use a technology called CORS. CORS is implemented well on most recent browsers, but things become a bit more problematic with IEs, even IE10 for example.

architecture

  • Login/Registration Variations. We support a lot of different ways to login and register. For registration, we support registration via Facebook and the “normal” form-based registration. For login, again we support Facebook, the client-side web application OAuth2 flow (e.g. redirect to hybris, then redirect back to the responsive web site) and a direct login form that underneath communicates with a front-end controller to exchange the Oauth2 tokens. This is a playground for login/registration, really, and it’s pretty cool what we explored. There are also issues in this area, for example you cannot sign-up via normal registration and then use Facebook login afterwards with our current system.

  • Smart ways to render the UI. Of course we render the UI completely on the client-side, within the browser. But we’ve also explored handlebars.js, a nice UI templating library written in JavaScript.

  • Clickable HMTL5 Videos. We’re currently preparing to add video on the homepage. This will be personalized based on gender and the data we get from a hybris login or facebook. The products shown in the video are clickable, so we can directly link to prodcut detail pages.

 

Next steps

We’ve almost completed this prototype. We’ve traded a full commerce flow (you cannot checkout, but you can add things to the cart and view the cart) for the width of technologies that we explored. We’ll try to influence our customers, partners and of course hybris itself to take on the techniques mentioned and use them in our products. If you want to get a demo and chat with us live, a great chance to meet some of the people behind this prototype is at our North American Customer Days in Chicago.

Thoughts about Voucher/Coupon URLs

In one of our hybris Innovation Lab tasks we currently deal with vouchers and I wanted to share my current thoughts on how to create and verify the uniqueness of vouchers (coupons, anything). The idea is that a trusted system (e.g. trusted as it is within a VPN, or accesses the voucher generation service via a authenticated session) can create vouchers that cannot be easily decoded and copied. Once a voucher is generated, trying to make changes to the format, which is in this case a URL, will make the voucher unusable. This means that a voucher can be identified as being generated by the trusted system and cannot be used twice.

What exact vouchers we generate does not matter at this point. The type of the voucher could be a general 20% off voucher or tied to a certain product or category. This is the next step, which we’ll be dealing with.

As we live in a web-based world, the vouchers we can generate have the form of a URL. This means the vouchers can be presented as a QR code or a NFC tag for example. The trick with the uniqueness is done via a URL signature. In our case we combine a few elements of the URL with a shared secret and create a SHA1 hash out of it. The the server encoded into the URLs can finally do the same processing to verify the voucher. Below is the code used for generating the signature:

The same shared secret is accessible from the generating system as well as the serving (verifying) system, the voucher server. The url to sign is combined with the shared secret and then a SHA1 hashi is generated and turned into hex String using some Groovy magic.

For generating vouchers, code similar to the code below is used:

The method getQRURL() will just turn the voucher into a QR image, using one of the many public QR code apis. This means the qr-URL, which holdes the signed voucher URL, can directly be used in a HTML img tag for example to allow the customer to scan it. In addition, I plan to hook this up with a dynamic NFC card reader to allow NFC-capable phones to touch the vouchers.

Now, let’s move to the verification part. A customer has stored the voucher and now invokes the URL:

For the verification, we again calculate the signature and compare them. If the signatures match, the voucher was generated by our trusted system. If not, something is smelly. On top of that, we also simply check that the voucher ID requested is actually present in the system. I guess in a production system one would hide this message under a general message to not give an intruder any kind of additional hint that the ID used was valid or not.

Once we now add voucher types, we have to include these additional parameters into the voucher signing process. Otherwise you would be able to change the type of a valid voucher by simply changing some parameters.

What is your experience with vouchers/coupons? Is this a viable solution?

Mobile Strategies

This is another blog post which got inspired by discussions at the hybris partner/customer event a few weeks ago. A lot of questions were around finding the right strategy for mobile. By now, it should be clear that mobile is an important touchpoint and not having any mobile experience, is clearly gonna hurt in terms of customer satisfaction and ultimately sales. So I am not gonna discuss whether mobile in general is useful. I guess most of us agree that mobile is a must. If it is 10%, 20%  or 30% of all transactions does not matter, it is already important  enough. From discussions with some of hybris’ customers I also know, that some even have up to 50% mobile usage and transactions.

Conquer the Desktop

If you are new to online, the first question will be: “Should we go after mobile first?” At this point, I believe that the majority of sales transactions is happening on desktop browsers, at least for most product categories. This means, number one is the desktop web. When it comes to hybris, this means you can use the hybris Accelerator products to get a decent and quick online web presence.

Go Mobile

After that, you should go for mobile. Last year, we introduced the Accelerator mobile template, which essentially gives you a smartphone-optimized web presence. The good thing here is that it is “mobile web”, meaning you create it once and whatever smartphone browser on whatever OS is browsing it, it looks fairly decent. This should be a first step towards mobile and I believe having mobile web optimized web pages is a really good one.

Go Mobile Native

With hybris 5 we are now introducing native mobile apps which use the OCC commerce APIs as a backend. For us, this is the next logical step for mobile. While the mobile web gives you a relatively quick way to be mobile, the mobile native apps will give you the chance to improve quality. Also, you have the possibility to be on your customer’s home screen, reminding them to use your app. The one thing that is critical here, is that you need to add services on top of the shopping experience. This will largely decide if customers download and install your app (and keep it installed). Ideas around these value-add services could be geofenced wishlists, reminders, CMS content, push notifications for deals and coupons, etc. This is really very specific and needs to make sense in the context of the customer. With the mobile native apps, you have the possibility to go to the next stage and create these experiences.

Sadly, you need it all

So that means you need both mobile web and mobile apps. Yes. That sounds like a whole lot of work and certainlyit will come with a few complexities. But there seems to be no other solution at this point, as you simply have to be where your customers are – and some happen to be on the mobile web and some prefer mobile apps.

An interesting outlook and topic for further discussion is responsive web design, e.g. creating a web presence that fits all screen sizes. This would basically merge step 1 (desktop) and step 2 (mobile web) into one step. It would give you a “one-fits-all” presence on the desktop, tablet and mobile (and whatever comes next and has a browser). This does not mean you’ll slash cost by 50%, but I data on the web suggests that with a certain overhead of cost for the desktop web development, you can get a decent responsive web presence. You’d then still need mobile native apps, which means a lot of specific, native app development (I told you I don’t believe in hybrid apps already, so that is not an option here…).

One way to further optimize all channels is by using the OCC commerce API for all channels. And yes, that means putting your API at the center of your strategy and serving all channels from the API, including the desktop web. Why is that a good idea? Because it will keep the cost low over time. As new channels emerge, new mobile apps have to be connected, etc. you already have a well maintained and capable API which will give you a lot of flexibility and save time and cost.

In hybris 5, we are adding important features to the OCC commerce API to make exactly that happen. We added OAuth2 with support for all defined flows, so diverse clients (server-side web, client-side only web, mobile apps) can choose the best OAuth2 flow to connect to the commerce functionality. We introduce CORS, so a web browser can connect to the API without security restrictions. And of course, we will evolve the API over time and continue to add more features.

Let me know what you think!

ETags… reducing bandwidth requirements of your API

In case you have never heard of ETags, I’ll try to provide a tiny introduction before we dive into a real-life example with the hybris Commerce Suite and its Commerce APIs. ETags – or entity tags – is part of the HTTP protocol spec and essentially a response header that a server sends back to the API user (client) to allow for conditional requests. The ETag is a hash value, in the end a simple string, sent as a response header. The server calculates this value based on the response headers and body, so it is essentially a hash value for the response.

The ETag value can be used by a supporting client to make conditional requests. This means the client must have cached a previous request and then include the formerly received ETag for upcoming requests. If a client wants to make a conditional request, it will use the If-None-Match request header with the ETag value. The server will receive the request and process it in the normal way. This also means, ETags are not really useful for saving processing resources (unless you implement additional logic). Once server wants to send the response down to the client, it will calculate the ETag and will check if it has received a If-None-Match header. If one is present, the values are compared. If they match, the server knows the client has the exact response in its local cache, so there is no need to send the response. It will respond with 304 Not Modified and an empty body. The client will use the 304 Not Modified response as a trigger to check its local cache and use the response therein.

The hybris Commerce Suite and especially the commerce APIs support ETags via the Spring ETag filter. Let’s see how ETags work in detail and start by making a request to the product search endpoint:

All right, we cropped the result quite a bit as especially the response body can be a pretty long piece of JSON. One thing that is a tiny bit confusing are thr quotes of the ETag value. These quotes are really part of the value, as you will see. What an excellent opportunity to cache this on the client, right? So let’s assume the client supports conditional requests and now the user makes the same request to the search endpoint again. This time, we’ll include the If-None-Match Header to signal to the server, that

  • we have a former response for this exact request inour cache already
  • we can deal with conditional requests,e.g. we know that we have to pull the response from our cache if we receive a 304 response

And that’s it. Look Ma, no Body 🙂 Notice the 304 response code. We also got the same ETag back in the response, so the client can treat each ETagged response the same way and even update the cache conditionally. If the If-None-Match and server-side calculated ETag values would not match, the server would send the full response as the content has likely changed. The client can then again cache the response under the ETag key and use it for subsequent requests.

Keep in mind, that Etags are an excellent way to conserve bandwidth, but they do not actually reduce the server workload, as the views (API responses) are still rendered but then conditionally returned. From a practical perspective, each API client will need to add additional logic to support the ETag, If-None-Match headers. Also, each client will need to check outgoing requests if they have been issued before and if an associated ETag is available. For RESTful APIs, only the GET request is idempotent, e.g. the request will not trigger processing that might result in different responses from the API. As all other requests (PUT, POST, DELETE) *should* lead to different responses, implementing ETag support only for GET requests is practical. The client can for example hash the full request URI and use an internal hash map to look up an associated ETag value and also the cached response once it received the 304 response from the server.

As always, please leave us comments if you wish!

HMAC-SHA1 in Groovy

Some APIs are nice and let you dance OAuth2 in order to play with them. Some others require you to calculate HMAC-SHA1 headers in order to play with them. HMAC-What? Yeah, so did I think. So here is a little snippet of Groovy code that I want to share. It allows you to calculate this hash-based message authentication code (HMAC) in Groovy, which is my favorite tool for quick experimentation with APIs.

The HTTP header I had to include in each call to a Mashape API was specified like this:

Public and private key are of course given by the API provider. The hard part is to create the HMAC_SHA1. It also turned out, that there seem to be a few variations, but this piece of code should get you at least started:

The entire response of the Hex-Encoded HMAC-SHA1 had to concatenated with the public key, seperated by colon and then again be Base64-Encoded (and converted from byte-array to String). Luckily, Groovy turned out to be a big helper with these steps:

This is obviously not a mind-blowing post but I hope that it helps a few of you looking to implement a HMAC-SHA1 in Java or Groovy.

 

Adding CORS support to Java Web Applications

CORS is a W3C effort (working draft status at the time of writing) to introduce a standard mechanism for enabling cross-domain requests in web browsers and participating servers (spec is here). I’ve blogged about this already, as it is pretty simple to add basic support for CORS to your HTTP Responses. As our entire stack here is based on Java, especially Java EE, I did a little web search for open source solutions for Java Web Applications… and found the CORS Filter by Vladimir Dzhuvinov.

While it is easy to add a few response headers to your HTTP responses, there are quite a few configuration options that you would have to implement for a configurable solution. Instead of re-inventing the wheel, I just integrated Vladimir’s excellent filter.

Installation is easy: download the zip (Maven artifact available, too), add the lib to your WEB-INF/lib folder and then add a filter and filter-mapping to your web.xml file:

For our RESTful eCommerce API we also use PUT requests, so I only had to change the default supported methods via the servlet initialization parameters to include PUT as well.

You might be wondering which browsers support CORS. The good news is that up to 87% as of July 2012 support CORS. The biggest trouble maker with this regard in the browser world in still IE, but when it comes to CORS, IE 10 will support CORS as well. Chrome, FireFox and Opera already have suppport for CORS since quite some time.

Using Spring RestTemplate with Object Deserialization from JSON

The hybris eCommerce APIs support the JSON and XML response formats. While many clients to the API will prefer to work directly with low level JSON and XML Objects (e.g. JSONObject from org.json), some developers will of course prefer to work with deserialized Objects such as SearchResult, Product or Facet. For the purpose of this blog post, I quickly evaluated the Spring RestTemplate which also supports Object deserialization from various formats – including JSON and XML. All examples below are for JSON, as it is my personal favorite out of the two we offer.

Let’s begin with a simple Groovy script that does not provide a lot of benefits over using the Groovy JsonSlurper, but get’s us started with RestTemplate. I encourage you to install Groovy to be able to try out a few examples.

Using the RestTemplate is pretty easy, once you’ve found the right Spring jar file 🙂 Luckily I was able to use the Grape dependency resolution which comes by default with Groovy. When the above script is launched, all required dependencies and the spring-web artifact will automatically be downloaded and added to the classpath.

Making a simple GET request is then pretty simple. a RestTemplate object is instantiated using the default constructor and the HTTP request is performed in one line of code:

Above line performs a GET HTTP Request against the specified URL (I removed our public URL). The second parameter specifies the Type that we expect back from the method call, the return type. Here, we use a very simple Type – String – which is a default type of the RestTemplate. The returned JSON String is then simply parsed via a Groovy JsonSlurper and we’re done.

Next, let’s not return a JSON String from the method call, but let’s return a SearchResult Object. Especially in statically typed programming languages, such as Java, this is a nice feature as it allows you to use code completion to figure out what the response is made of – instead of inspecting the low-level JSON response.

The first thing you should notice when you compare the two snippets is: the second one is way longer. This is mostly due to the fact that we now need to have the Model classes in our classpath – and although we ignore some elements of the JSON Response via

these are still quite a few: SearchResult, Product, StockLevelStatus, Price… Of course, if you have acces to the server-side classes used for serializing the objects into a JSON response, you might be able to directly use these classes (e.g. just jar them up and add them to the classpath). Things get a bit tricky though, when optimizations are applied, like not serializing some properties or changing the names. For the purpose of this example, I just created a few classes as Groovy makes this trivial, too.

Now that we are deserializing the JSON response into objects, we also need to add a converter to the RestTemplate, that is able to deal with JSON. The one used here is the MappingJacksonHttpMessageConverter. It turns JSON Strings into Java objects – if these are available on the classpath. Adding this converter to the default converters is simple:

The first line in the above code snipped creates a fresh RestTemplate. The second line accesses the list of message converters and adds the MappingJacksonHttpMessageConverter to it. If you wonder what converters are installed by default, here we go

  • ByteArrayHttpMessageConverter
  • StringHttpMessageConverter
  • ResourceHttpMessageConverter
  • SourceHttpMessageConverter
  • XmlAwareFormHttpMessageConverter
  • Jaxb2RootElementHttpMessageConverter

 

Once we have the JSON converter registered with the RestTemplate instance, we can now expect a SearchResult Object from the get method call:

Note, that we don’t get a String, we get a SearchResult object. And we then simply iterator over the products list and print the product names.