AT&T's FanLab was a first of its kind co-creation beta program that allowed participants to test unique concepts and products in their own natural way and directly influence AT&T's future products. I wore many hats in this challenging project, but my primary role was at the intersection of UX research and design. 

Client: AT&T + DIRECTV   |   Year: 2015 - 2016   |   Platform: Web + Mobile-web

How FanLab Was Conceived

Our team over at AT&T's (formally DIRECTV’s) Digital Innovation Lab clearly understood that innovation is at its best when collaborating with the people who are actually intended to use it. It cannot be confined to the team who develops it or to the time traditional user testing allows. These notions developed a vision of unleashing the innovation process to the real world, having it live there for a while and seeing how it naturally evolves.


  • Foster a direct relationship with fans and learn more about them as individuals and as a group.

  • Turn our early innovative concepts into mature products as they hit the market.

  • Design products that offer an elevated and familiar experience that deliver the unexpected.

  • Capture the real behavior hidden behind an intent that drives users to action.

Beta Testing Feels Like Work

Looking for 3rd party solutions to help us accomplish our grand vision, we found that traditional beta testing platforms provided a very poor and cumbersome experience, starting from recruitment, engagement, to data collection. Each area discouraged participation and desire to provide meaningful feedback. 

  1. Inefficient recruiting of "trained" testers who might provide feedback you want to hear/see instead of observing their natural intuition and understanding.

  2. Users wasted time following lengthy and detailed written testing instructions.

  3. Task-oriented scripts set a linear testing flow that narrowed users' natural usage behavior.

  4. Users had difficulty accessing and installing the beta version on devices they didn't normally use.

  5. Closed feedback loop created a perception that the participants' feedback was not truly valued or fully incorporated in the product.

  6. No relationship development with a strong community of fans that were interested in contributing more of their experience.

So We Built A Better Way to Beta

Our solution was to build a unique co-creation environment that made it fun and easy for participants to test future ideas and products in their own natural way.

  1. Constantly refreshing the pool of testers and concepts, eliminated the "trained tester" syndrome and created a fresh and invigorating perspective that left cognitive bias behind and reduced tester fatigue.

    • Introducing a simplified device provisioning process and providing short setup instructions before the testing began, eased participants into the testing process and provided quick access to the beta version builds.

  2. Brief and fun intro videos provided basic testing direction that left room for participants to explore the product "their way." The tone of the video also reminded participants to get out of “test mode” and just be themselves.

  3. Participants were now free from following scripted tasks and could choose how, when, and where they wanted to test and on which of their own devices. Giving users a choice that could adapt to their lifestyle, made the testing feel natural and less "test-like."

  4. A viewing experience should be relaxing, so we removed any strict timeframes. Now participants could set their own pace to get acquainted with the product, which enabled them to mature with the product usage.

  5. Creating an open communication channel between participants and product teams created a feeling of transparency and empowerment in that their feedback was instrumental in the development process. This led to high engagement and helped foster a relationship of collaboration, setting the tone that participants were not just a “means to an end,” but an influential part of the innovation process.

  6. It’s not just about the money. We rewarded participants at each stage by “paying” them with gifts that were relevant to their progress and interest level, which was essential to keep them engaged throughout the process and reminded them that they matter every step of the way.

  7. Maintaining a personalized and gamified approach throughout the process, assured that participants stayed engaged and valued.

Wait a Minute, Why Focus on Fans? 

TV Fans are a special breed, they share a strong and dynamic connection with their content that changes intensity levels depending on mood, content, viewing mode, social setting, location, medium, genres, device and other factors. Our research around understanding the Fandom Spectrum (separate case study to follow) has clearly led us to see how fans have the power to influence consumer behavior as individuals and as a group:

  • Fans will adopt new product experiences that deepen their relationships with their shows and characters.

  • Fans rally and converse about their favorite shows and connect others around their enthusiasm, which helps drive viewership and evangelize "non-fans."

  • Fan choices yield strong indications of emerging mainstream consumer behavior.

  • Fans stay loyal to the platforms that serve them an experience that empowers their fandom.

“Fanlab isn't just another project, this is a fully-scaled company that reinvented the way we test projects.”


Discovering Your Fan Powers 

Fans don't know they are fans or what type of fan they are and how powerful they can be, so before we ask them to help us, we wanted to show them how their influence affects the way content and products are created and shaped based on their fan personas. To do so, we used our extensive Fandom research to lead us to define five Fan Attributes that drive their content choices, six dynamic Fan Personas they identify with, and a single spectrum scale that displayed how dynamic fan intensity levels could be. 

We used this data to generate a personalized Fan Profile for each user who went through the FanLab onboarding flow. Now, fans could discover what type of fan they were, what attributes defined their actions, what TV/movie characters matched their profile, what they should watch next, and most importantly, what products could empower their fandom.

As a result, fans could clearly see that they had the power to help us shape things from the inside out instead of the opposite.


Reinventing Beta Research

Besides the fun and play, Fanlab offered a new way to administer beta tests and evaluate the experience value of our products.

Our research approach included a mix of qualitative and quantitative methods, such as Unmoderated UX studies, True Intent Studies, Concept Testing, and Participatory Design, but at its core, was a series of smart surveys designed to capture the overall- to-specific reaction and perception users experienced while using our products.




Our surveys were visual driven and minimalistic, written in a humorous and slightly subversive tone as if we were having a conversation with our participants. The surveys were built using question mapping, logic jumps (branching) and had a relationship between each of our 10 surveys that dynamically changed based on users' responses, progress, and relevance.


We successfully met our goal of designing a fun and conversational survey experience that dramatically reduced fatigue and abandonment throughout the process, while still providing us with the data collection funnels and tools needed for analysis.


What Should We Test First?

 It was a natural choice to have DIRECTV Now as our first product to test with the Fanlab platform. After all, DIRECTV Now set a new bar for future viewing behavior by being the first streaming service that combined the best of live TV and on-demand viewing.

The study was meant to gain insights into users’ mental models and behaviors as they related to the overall product experience and the perception of value and reaction to different aspects of the product across all available platforms/devices.

Over the course of 4 weeks, we asked 4,872 participants to use DIRECTV Now as they would naturally, and share their experience. Our job was to provide the participants with minimal guidance as to what "task" they should perform, gather feedback in different methods and discover insights that would inform product design decisions and roadmap priorities.


Here’s a snapshot of the participation metrics; the full DIRECTV Now case study will soon be published. Want me to let you know when it is? 








Let's Beta Test

Now that the tested product is chosen, the goals are set and the questions are written, it’s time to let FanLab do its magic.


© 2017 by Moshe Shmuel.