Data scientists engineering a Data Platform

A 0–1 journey of building a data platform at Pelago

Image by author

Introduction

Pelago is a travel experiences platform in South East Asia backed by Singapore Airlines. We strive to make our customers’ travel journey as seamless and worry-free as possible — from delivering personalised discovery feeds to ensuring a flexible and hassle-free booking and payments experience.

Problem

High-level plan

Architecture and component details

Architecture diagram of the Data Platform in Pelago; Image by author

Event streaming platform

Data Storage

Workflows manager

Daily triggered Airflow tasks to calculate product similarity; Image by author

Personalization service

Recommended similar products on the product details page on Pelago; Image from Pelago

Results

Learnings along the way

Future Work

Data scientist with experience working for technology, retail and FMCG companies.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store