Testing UX hypothesis for Digital Campaigns

How do you prove that ‘the way it’s always been done’ isn’t always the best way? Hypothesise, create, test and report!

Background

Every month, Australia’s biggest travel agency, Flight Centre Australia (FCAU), holds a primary promotional campaign, which would last 2-4 weeks and attract the most marketing attention through above and below the line marketing channels. These campaigns followed a templated approach, with products segmented by type (flights, hotels, etc) and were created using a temporary promotions page that would be archived after once the campaign ended.

In early 2015, I was planning my workflow to deliver the online portion of the Asia destinations focussed ‘Flavours of Asia’ campaign, coming up in March. I wanted to do something more with the primary campaign this time around and I had grown to learn that ‘the way it has always been done’ was not necessarily the most effective way.

I knew that creating the same, one-time-use campaign page was creating design and tech debt for the digital and marketing teams. For example, any customised designs were produced as ‘work around’ code hacks that would disappear as soon as the campaign was over. I also had a hunch that reporting metrics of the campaign, especially comparing against similar promotions, would be near impossible if the page did not exist beyond the campaign time.

At the same time, the Digital Experience Leader had made a discovery – the Flight Centre “Asia Holidays” landing page ranked second in Google searches for “Asia Holidays.”

We decided to combine forces and come up with a few hypotheses to test the effectiveness of producing an online campaign differently.


Hypothesis

Our hypothesis revolved around leveraging Flight Centre’s exisiting, high performing Asia Holidays destination page for the campaign instead of the usual workflow of creating a disposable campaign page.

We hypothesised that by using existing Flight Centre destination pages for the Flavours of Asia campaign there would be:

  • an increase number of products sold;
  • an improved customer experience of the campaign;
  • tracking and recording campaign statistics (traffic, products sold, number of enquiries) would be easier and more accurate to report.

Strategy

We decided to not only leverage the Asia Holidays page, but also the holiday landing pages of fifteen featured Asian destinations. These included primary campaign partners Singapore, Japan and Malaysia as well as popular secondary destinations such as Indonesia and Vietnam. From a visibility standpoint alone, fifteen extra opportunities were created on the Flight Centre website to reach our audience as opposed to the usual one.

Campaign-Compare-2


Execution

The key difference in the execution of the Flavours of Asia campaign, compared to the previous production method, was to leverage the existing holidays pages. We consulted with our developers to create a second version of the campaign related pages (Asia Holidays and fifteen related destination pages) – using our CMS. These campaign themed pages would take over for the duration of March and would featured the campaign creative, selected hot products and unique content.

The featured Flavours of Asia campaign pages were also given a styling facelift in the form of:

  • a striking full-width hero banner/slideshow which put the campaign creative and select hot products centre stage;
  • a streamlined search bar to make it easier for customers to search for a particular product or destination;
  • a reorganisation of tabbed product content to better highlight partner products (for example, on the Asia Holiday page, products were sorted by destination, rather than product type);
  • faux facet “pills” (a secondary tab UI) so users could easily differentiate between air (flights) and land products (holidays, accommodation and tours) within the destination tabs;
  • unique, updated SEO content, including the Social hashtag #FlavoursOfAsia user-generated gallery to increase customer engagement and brand awareness;
  • relevant, up-to-date blog content and videos created by the in house creative studio.

The Campaign also had a heavier internal advertising presence throughout the FCAU website. Advertising featured a mix of generic Flavours of Asia campaign messaging and attractive lead-in products on the homepage (hero banner and sidebar advertising) as well as the regular Holidays section.

Newly released updates to the FCAU website helped enhance the experience for our customers. These include the newly responsive homepage hero banner and the simple
yet efficient re-styling of tabs (looks matter)!

 

FOA-Campaign
The Flavours of Asia campaign wasn’t just for the holidays/asia page. It also took over the partner destination pages such as Singapore, Japan and Kuala Lumpur.

 

FOA_Homepage
Flavours of Asia taking over the Flight Centre homepage. The responsive carousel banners were all campaign products, so no matter when a customer clicked, they would be immersed into the campaign.

Results

The first result was immediately noticeable – the Campaign pages looked more professional, better organised and confident in their product offering.

But, the proof was in the reporting and with some fantastic results. The following highlights are taken from the official Campaign Results and Insights report generated by the marketing and SEO teams.

  • 1218% increase in enquiry
    Month on Month February 2015 v March 2015
    The holidays/asia page in campaign mode increased enquiry by 1218%, with all support countries except Vietnam significantly increasing enquiry percentage.
    Note: this statistic was significant in that FCAU was still measuring customer enquiry as a success metric for non-flight products.
  • 150k+ page views = a dramatic increases in traffic for Campaign v Campaign
    Flavours of Asia (Mar 2015) v Asia – Just a Hop, Skip and Jump (Jul 2014)
    Incorporating campaigns into the website’s permanent pages has resulted in signifficant traffic increases. This has ensured that customers organically landing on the website are fully aware that Flight Centre is running an Asian destinations campaign.
  • Organic Traffic on Par with Paid Digital & Paid Social (±19%)
    This is particularly noteworthy as past Asia campaign pages were not indexed and organic did not feature at all. Also, the fact Organic featured on par with Paid Digital and Paid Social (around 19% each) is a testament to the success of incorporating the website’s permanent pages into campaign mode.
  • 9 out of 107 products generated 90% of online enquiry
    I found this statistic incredibly interesting as it posed questions for further hypothesis testing about choice, quantity, price and product/brand recall.
  • Better customer engagement with the new Holidays Asia page in campaign mode
    The holidays/asia and supporting pages in campaign mode resulted in better engagement with a higher average time on page, lower bounce rate and lower % exit.

Key Learnings

This campaign took a lot more time, energy, resources and stakeholder management than I had experienced before or anticipated. However, it was the most challenging and enjoyable campaign I had worked on as I could see the potential that it had and
was truly proud of the efforts of myself and my team.

Two key points

  • For a larger campaign such as this one to succeed, a lot more organisation and communication was required with internal stakeholders, such as marketing, creative and product teams. Without the support of my design leader to help champion the experiment, it’s unlikely the campaign would have reached the same positive outcomes.
  • Support from developers needs to be communicated as soon as possible! While the blue pill faux filters were a great success and were delivered on time in terms of production, there were other functionalities that couldn’t be built in time, including:
    • Switching the page from normal to campaign mode could not be automated and became a manual task
    • A full width Google Map of destinations and products was not able to be included in the campaign.

Going Forward

This experiment proved to management, product teams, marketing campaign teams that leveraging existing, high performing pages for relevant promotional activity can yield great more profits and more positive customer engagement.

This case study also awarded me a Flight Centre Global Winner (2015), as recognition as for my achievements in the Digital Experience space.