Migros product-data API
Switzerland's largest retailer had its product data in thirty-something places. M-API turned it into one service that everything else could build on.
Eleven years of my time at Liip went mainly into one system: M-API, the product-data REST API of Migros. It ingested multiple millions of products — plus categories, brands and a long tail of smaller entities — and served them as a single, consistent interface to the rest of the company.
The ingestion side is where the real work was. Over thirty heterogeneous sources: CSV and XML imports, direct connections to MySQL, Oracle and MS SQL, FTP and SFTP drops, base64-encoded files arriving over RabbitMQ and later Apache Kafka, WebHDFS, Redis clusters, JSON and XML APIs. Everything landed in MySQL staging, was served from Elasticsearch, and sat behind Varnish — later Kong — as an API cache.
Two decisions mattered more than the rest. First, we deliberately embedded shared business logic inside the API instead of leaving each consumer to reimplement it; that is what turned a data feed into a crucial core service. Second, we built an admin backend that gave Migros staff live visibility into the pipeline — where a product sits in the queue, which sources are still pending — so the system stopped being a black box to the people who depended on it.
Liip published a detailed write-up of the architecture, including the diagram above.
Later I led a complete rewrite that enabled partial product mappings and markedly faster processing, added a message-oriented middleware layer with replayable change streams so the web shops could update only what changed, and supported the migration off Migros’ own hardware into Google Cloud. At the end of the engagement I embedded Migros’ own developers into the team and steered them through a full handover — knowledge, code and operations. Leaving a system in good hands is part of building it.