Netflixing Pelago’s webpages

Dynamic personalized components to scale to hundreds of destinations

Background

Pelago is a travel experiences marketplace backed by Singapore Airlines. We strive to deliver a seamless discovery and booking experience to our customers with curated and personalized product catalogs reducing clutter and a quick checkout process to reduce drop-outs.

During the initial part of our short existence, Pelago was serving only a handful of destinations — Singapore, Kuala Lumpur, Hong Kong, and Phuket. As travel resumed, we have quickly expanded to 30+ destinations in 10 countries across Asia and Europe. This expansion meant that the number of destination pages also increased from a mere five to 30+.

While the common industry standard (see Fig. 1) is to maintain a generic page design and vary the content for each destination, we did not want to limit our business and marketing teams with a rigid design for all the destination pages. However, on the other hand, if we were going to give them that freedom of customization, the engineering team would need to carry the burden of translating design into code. We could not have 30 designs for the 30 destination pages maintained by the Front End. We needed a framework that would work for both parties. We pondered over how Netflix was able to provide a different homepage layout to each user. From that moment on, the seed was sowed and we got working.

Fig. 1 — Klook’s (a competitor) webpages for multiple destinations

Introduction

Fig. 2— Structure of a webpage

Before we go into the nitty-gritty of the framework, let’s understand the composition of a webpage.

Webpage

A webpage is a plain white canvas that can contain any User Interface (UI) elements ranging from text, images, buttons with hyperlinks to other web pages.

Component

Instead of dumping UI elements directly and disorderly on this webpage, we divide this page into multiple components or sections (an informal definition only for the sake of this article) to make it more structured and manageable. So, a webpage can contain any number of components.

Content

Now, each component can contain one or more UI elements within it. These will generally include component headers/titles and images of products along with product titles, price, category information, and hyperlinks to view more details of the product.

Pelago’s engineering team envisioned the creation of these components to act as building blocks for the construction of the webpage. The objective was to create a library of components such that they could be selected and ordered to construct the various destination pages on our platform.

Part 1 of the article focuses on the process and framework to build these components and further select and order them to design the web pages. Part 2 discusses how we leverage this framework to further personalize the webpage based on the user’s interest such that each one sees a different and personalized version of the page.

Part 1 — Component-Based Views

The first task was to split the destination page into multiple components. These components are created manually and stored in our database. They contain all the information (component metadata) required by the presentation layer to display including the order in which components are meant to be displayed and the layout/style for each component.

Fig. 3— Pelago’s Singapore destination page

Fig. 4 — Component representation of the destination page

Further, we also needed to identify which components contained static vs dynamic information. For example, the “Why Pelago” section contained text and images which would not change frequently or even across destinations. Whereas the product cards displayed within the “Steals under 50” section would change dynamically based on the destination. For dynamic components, an additional API call is required to retrieve dynamic information within the component — e.g. list of product cards.

Fig. 5— Static components vs Dynamic components

Moreover, these components are reusable across destinations and pages i.e. we can configure the “Steals under 50” component to show products priced under SGD 50 on Singapore’s destination page as well as to show products priced under AUD 50 on Australia’s New Year campaign landing page.

The last challenge was to enable business folks to customize each destination page to suit local demand and portray each destination uniquely. This was achieved by creating a list of components visible in each destination and their respective order on the page. We exposed this table to be controlled by the business teams along with the ability to create new components from a preset components catalog — our component library.

Architecture

The first two calls are always requesting Global parameters (such as user device and location information) and a list of components to be displayed on the page. These are standardized API calls from Front End — Back End for all pages. The next few API calls depend upon the type of components received in the response. Static components are immediately displayed, whereas each dynamic component requires an additional API call to load dynamically changing content. The Back End responds to dynamic component requests by making one or more API calls internally to internal systems such as the database, ElasticSearch, and/or data science for personalized products.

Fig. 6— Component-based view architecture

To summarize, we formalized the process of creating a webpage into three steps.

1. Creating a library of components and defining each component using via its metadata — stored in the DB
2. Selecting and ordering one or more components to construct the page — via an API call
3. Loading content within each component — via individual API calls for each component

The key difference between previous approach and the new one is the second step which is an additional API call introduced by this framework. The Front End requests for the list of components from the Back End (server-side or the data/business logic layer). It was previously an individual design for each page managed by the Front End.

This tradeoff gives us the ability to scale to hundreds of types of pages with a one-time effort of defining the components within each page.

Benefits

In addition to solving the challenge of customizing our destination pages, the benefits of component-based view architecture extend far beyond

1. Configuring all our web pages on the platform using these reusable components as building blocks
2. Scaling the eventing framework with ease and more granular control
3. A structured shift in mindset across the board to think towards building innovative components rather than building pages — a rather trivial exercise moving forward
4. Laying the groundwork for internationalization
5. Extension to personalized components which we’ll be discussing next

Part 2 — Personalized Components

Having achieved the feat of dividing our web page into multiple components and building a component library to design our pages from, we enabled the business teams to select any number of components, rank-order them, and configure them based on the requirements of the region, use-case, etc. While this gave the business teams fine-grained control over the content they wanted to show our users, it lacked personalization. For example, we were still not distinguishing between “solo travelers” and “families with kids” — in terms of product sections.

Destination pages displaying products were divided into multiple components. The content (see Fig. 2) within these components was curated by the business along with a list of applicable products. Post that, the data science team would personalize the list of products to suit the user's profile. With personalized components, we could now go one step further to determine which components were to be displayed to the user and which were not — see Fig 7 below.

Fig. 7 — Personalized destination page components for a solo traveler

Personalized components provide us the ability to customize and design all our web pages in real-time for the end-user which would otherwise be humanely impossible to do.

Architecture

Now that we’ve settled the “What”, it is rather more interesting to look at the “How”. Recollect from the dynamic components architecture above, the Front End needs to make an extra API request to the backend for a list of components to be displayed on the destination page. The Back End, in turn, retrieves a list of relevant components and their metadata for the page. Finally, this list is passed on to the data science system/ personalization system to select and rank components to be displayed to the user.

In the background, a scheduled task is run daily or weekly to create a profile for each user visiting the platform. Using this profile, available components can be mapped to the user. For example, a profile for a “solo traveler in the mid-30s” may be created looking at product tags and categories interacted with (using Events data collected on the platform), and demographic information if available. This profile is then mapped to components — “Bar hopping at Clarke Quay” and “Adventure for the solo traveler” by calculating the similarity between the user and component attributes. This offline workflow is facilitated in Apache Airflow.

Fig. 8 — Architecture + Data Flow Diagram for Personalized Components

Finally, by stitching all the above pieces together we are able to achieve at least 90% (if not 100) of what Netflix does with their homepage movie/TV show carousels.

Fig. 9— Different personalized versions of the same page for two users

Key Results

1. Creating a new destination page takes a few minutes and can be achieved by a non-engineering person as opposed to at least a couple of days taken by (usually more costly) engineering persons earlier
2. We’ve advanced from showing the same page layout across all countries and destinations to a different layout (list of components) for each user
3. We expect a minimum of 10% increase in baseline CTR for returning users on the destination page

Learnings

A true North Star

We wanted to achieve dynamic and customized product carousels just like the experience Netflix is able to provide for movie and TV show recommendations on its homepage. This ensured we were all working towards a common goal.

Long term vs immediate term gains

While features immediately visible on the webpage are necessary for immediate user engagement and tangible results, building platform-level features with no immediate impact is equally important

1. Keeps the platform abreast with the latest technology
2. Future proofs your platform's capability to build newer features

Data science without engineering support is useless

We tried an engineering first approach for personalized components as we realized that this was going to be the more complex part. We wanted to ensure our API calls and handshaking protocols were built correctly before we got to building algorithms for the user—component mapping.

Thanks

Pawan — the mastermind and biggest sponsor of the project

Shivani, Chris, Ripal, and Zaid— the gal and guys who put their blood and sweat into making this happen

Sai and Kush to help review this article

To the entire team at Pelago that has put their faith in us and continues to support us.

We are always on the lookout for great talent. If you are someone who loves to code, play with data, break existing systems before creating new masterpieces, we will love you! Please do check out our careers page.