At Project A we decided to build our own e-commerce platform. In this post we will have a closer look at the architecture of Yves & Zed and resume some of the decisions we made during the process of development.
From a high level perspective, every serious shop system needs to have a very fast and simple front end, as well as a robust and highly extensible back end for processing orders and any other kind of business logic. From Magento we learned that it’s impossible to achieve these conflicting requirements in a single application, because a shop system can be either lightweight or heavily structured, but never both.
We knew that we wanted a front end that is separate from the back end. We could have done this by designing a service-oriented architecture (SOA) like Zalando successfully did when they replaced Magento some years ago, but we decided against it because we saw no obvious benefit from an SOA for our use case. Moreover, we wanted to avoid the overwhelming complexity and the overhead that usually comes with such an architecture. Our first draft of a system with two specialized applications looked like this:
In terms of the actual technology, we decided to use PHP for the front end. Of course, there exist more modern languages for web development, but from our experience PHP allows for quick changes, and it’s comparatively easy to hire developers for. We considered Java for the back end, but a language zoo always results in mixed teams and more difficult recruiting.
A comparison of available PHP frameworks resulted in the selection of the lightweight Yii Framework for the front end, and the more feature-rich Zend Framework for the back end. At the beginning of 2011, Yii was the fastest web framework on the market and we already had a lot of positive experience with Zend Framework. We then needed a good name, so we came up with Yves (Yii) and Zed (Zend). In 2012 we reimplemented Yves from scratch and switched to the Silex micro-framework, but kept the name.
When we started the implementation, we focused on an architecture that was optimized for future development speed instead of a big set of features (we will cover extensibility in a later blog post). Each of our ventures has a different set of requirements, so we only implemented the minimal intersection of all of them. This included a front end templating engine, a back end user interface, a configurable component for order-processing, plugins for different payment providers, a customer database, a CMS, a general product catalog for any kind of product type, and a business intelligence solution that includes a data warehouse.
The Architecture of Yves & Zed
This diagram illustrates how the main components of Yves & Zed are connected:
Our customers on the left side will only come in touch with Yves. There is no full page cache in Yves either because we need to be able to use personalized data such as product rankings, recommendations and tracking parameters on every page. The front end has no direct access to the Mysql application database and fetches all the necessary data from Couchbase and Solr, which are asynchronously filled by Zed.
Only when customer or order related data are needed, Yves calls a web service that is provided by Zed. For example, when a customer finishes the checkout, Yves sends the collected data to the “saveOrder()” service provided by Zed. As soon as the order is saved, Zed reduces the availability of the related products and marks them as dirty. There is a cron job running every minute that retrieves the current stocks for dirty products from Mysql and updates Couchbase and Solr. Compared to a caching system, the data in couchbase and solr is always up to date and must never be invalidated.
After an order has been saved, its life cycle (payment, shipment, return, refund and everything that can go wrong in between) is managed by payment method specific processes, which we implemented using state machines (we will cover order processing in another blog post). Under the hood, Zed communicates to external providers for payment, logistics, etc via several types of APIs. Finally, Zed has its own web based user interface for administrative tasks. In most of our ventures people from operations, customer care, business intelligence and management use Zed for their daily work.
On the bottom right corner of the diagram you find our data warehouse with its own Postgresql database. It integrates the transactional data from Zed with external data sources about user behavior, marketing cost, CRM activity, product histories and so on. On top of that, we put Mondrian engine for a consistent definition and processing of KPIs and dimensions. As a user interface, we use Saiku for ad hoc analysis and a custom reporting front end that is integrated in Zed.
Finally, there is an ActiveMQ attached to Zed. We use this queue for the asynchronous execution of time-consuming tasks, like email distribution or pdf creation for invoices. Also, when there are multiple databases for different countries, the queue acts as messaging system between the stores. For instance, a new page in our CMS can be broadcasted from the German shop to all other country stores via that queue.