Piyush Sinha

Background
Agoda is the leading online accommodation booking service in South East Asia. A part of Fortune 100 company Booking Holdings Group (formerly The Priceline Group), It is the sister company of Booking.com, Kayak, Priceline.com, OpenTable and RentalCars.com

The Challenge
While Agoda had been in business for a long time, they didn’t have any kind of human centered design process in place. The existing “design process” was PMs literally instructing the visual designers on exactly what the interface should look and function like. 

I was hired by the VP of Product to join Agoda to help pioneer and build up the UX practice at Agoda from the ground up.

Stakeholders

• VP of Product
• All Product Owners / Managers
• All Designers
• Product Engineering Teams

My role

• Strategy
• UX Research
• UX Design

Measure of Success
All customer facing product teams engaging in a human centered design process

The Process

Understanding the existing product design lifecycle

Before introducing any changes to the product design process, I wanted to understand the challenges that we might face and need to overcome. To accomplish this, I first went about observing the various teams in action to understand how each team and their PM approached their product.

In doing so I came away with some key observations –

• Each team had very different dynamics.
  • In some teams the PMs were very strongly prescriptive about the product feature, in other teams they encouraged collaboration amongst the product team during the ideation / design stages.
  • In some teams, the engineering teams were very disengaged, only referring to their JIRA tickets for details, while in other teams the engineering teams were very engaged from the start of the project providing inputs during the ideation and design stages.
• None of the teams were doing any user research (formative, generative or evaluative). They would just build on an idea that they PM brought to the team.
• In all of the teams, the “design process” was PMs literally instructing the visual designers on exactly what the interface should look and function like.
• There were no design reviews with other stakeholders in the company as a result a lot of inconsistent flows and experiences were shipped.
• While the organization was heavily data driven with PMs conducting lots of A/B tests, the experiments had no clear hypothesis and the experimental setup used varied from one PM to the next, making them an untrustworthy source of data, yet it was used by PMs to push features forward
• None of the teams however ever closed the loop on the product features that they released, by doing a retrospective on user adoption of the features they had just released to identify improvements or unmet needs that could be opportunities for new product features.
• Most of the teams weren’t familiar with a human centered design process.
• Some PMs weren’t keen to change anything in their process as they felt it would take away some of their control of the product and/or increase project timeline without any additional benefits.

Formulating a plan and executing it

Having made these observations, I knew it wasn’t going to be easy to introduce UX to Agoda. It would require multiple phases, some time and support from senior management before we could get the organization to begin adopting it as part of their regular process. I knew I would have to start really lean, show significant results and then scale out via organic adoption by teams (and eventually top down enforcement if there were still hold-outs).

With that in mind, I formulated a strategy involving 3 phases to introduce UX to the teams –

Phase 1: Foundations, Design Reviews and UX Education
Phase 2: Lead by Design
Phase 3: Inform with Research

Phase 1: Foundations, Design Reviews and UX Education

The goals of this phase were to:

1. Lay down the basic foundations for good design to occur
2. Implement design reviews with all involved stakeholders to improve the quality of shipped designs
3. To create a shared understanding in the entire organization of how the UX Research and Design process fit into the product process.

To achieve this, I knew that we would need to have all the product teams learn through hands on application. However from my conversations with PMs I knew they wouldn’t be comfortable investing significant amount of time into this just yet.

I took inspiration from Google Venture’s Design Sprint, and modified it to fit within the span of 1 business day.

Firstly, I planned a series of training design sprints with every team and ensured that all stakeholders on the product teams were involved in these. This helped with laying down the foundations for good design to occur by increasing stakeholder participation and familiarizing everyone with the UX process.

Next, with the support of the VP of product, we mandated cross team design reviews before any feature could be moved into design sprints to ensure that the quality of shipped designs improved by taking into account impact to the user experience across product features.

Lastly, I began working with PMs to identify features in their roadmaps and began proactively scheduling lean design sprints with them, design reviews and a final design approval review with the VP of product. This helped embed the design sprint process into the regular product design lifecycle.

Phase 2: Creating an appetite for Design

In phase 2, I took a step back from proactively scheduling design sprints with PMs and letting the PM’s reach out to initiate it when they were ready to build something new. A majority of the PMs were already starting to reach out by themselves as they had seen huge benefits to their product roadmaps (quicker buy-in from senior management, higher rate of success in A/B testing). Yet there were still some PMs who were resistant to the new process. By identifying the PMs who weren’t reaching out, I then tried different approaches to encourage them to implement the UX process in their products. Eventually the fact that the UX process was producing better results than their old approach, won them over.

Also in phase 2, I started to fix some of the tradeoffs I had made in phase 1 with the lean design sprint – namely conducting evaluative research with actual customers instead of guerrilla testing with non product department staff. To facilitate this, I partnered with a research vendor to recruit users for us on a weekly basis to conduct evaluative testing with. This enabled teams to simply sign up for a research time slot to test their product features rather than have to delay product timelines due to participant recruiting timelines.

Phase 3: Creating a thirst for Research

In phase 2, as the mini design sprints gained traction amongst the various teams, the teams started to understand that there were knowledge gaps that existed despite the evaluative research that we conducted at the end of design sprints. At this point, I started to educate teams on the various stages at which research could be engaged – namely formative, generative and evaluative research. After discussions with the teams, I found that while they had an appetite for research, it wasn’t yet strong enough to begin projects with it, but they were comfortable with doing generative research. To cater to this, I started augmenting the evaluative research in design sprints with participatory generative research methods during the design sprints.

Over the next few months, as generative research started to gain traction, the teams had gained enough confidence and comfort with the UX design and research process, that PMs started reaching out well in advance of any concrete ideas they had for product features, requesting formative research to help further develop their ideas through user research.

Finally at this stage I had successfully achieved complete success with  all customer facing product teams engaging in a human centered design process. 

Outcomes

Over the course of 4 months, I was able to achieve success with the goals we had set out to achieve. Following this success, we had access to more head count and budget, so we started to evaluate gaps in the organization and started hiring accordingly to staff up the UX team as required.