Evolving Protect
Overview
After a long bout of trying to grow an MVP with lean techniques, there was a need to research into the problem area deeply to better serve the user and meet business goals. This project dives into an active research period following through to ideation workshop days, which ended with a new set of planned user centred deliverables and a roadmap to prove that an identity monitoring based proposition could work as a subscription model.
Role
Product Designer
Timeline
5 Weeks
Team
2 Designers
1 Researcher
1 Project Manager
1 Product Owner
Project Summary
Problem
Product not performing for business and user
User’s not getting ongoing value for a subscription based product
Product still MVP esc but has untapped growth opportunities
Solution
Several week long discovery period which lead us to gain insights which we fed into Ideation workshops
A collation of ideas and insights led myself and the senior designer to reprioritise our roadmap
Process
Discovery - Set research goals and dove deeper into the fraud space and subscriptions
Define - Narrow our focus to distil what areas of safety would people pay for
Ideation - Insights where played back to the team and lots of exercise in a 3 day ideation workshop
Redefine & Prioritise - Reassessed everything we had discovered and thought of and prioritised next steps based off of impact
Impact
Higher Team Moral - After working on a product that was underperforming the squads moral was higher when contributing to the what next
Stakeholder Approval - C suite loved the work we had shown (Less the blue sky thinking but more at what was next)
Higher Design Maturity - Team was working in a higher design maturity way post discovery, Engineering where more keen to contribute ideas and iterate with us
Roadmap (A bit messy but the workings and details of it)
Part 1
Context
Background context
ClearScore launched a new feature in April 2020 with the proposition of helping users stay safer from identity fraud by giving them safety information and showing any user data that was involved in data breaches. The tool which showed stolen passwords and leaked information had great reception and was a big step for the company which normally only operated in the credit space. For the first 6 months of launch most of the product work was around growth design and optimisation but this approach failed to gain the traction we wanted and influence user behaviour in the desired way.
The Challenge
The driver in which we started this massive discovery project was that after 5 months of growth work (Upsells, funnel metrics, copy changes, A/B Tests etc) we were finding that the paid version of the product wasn’t getting the targeted sales numbers it needed. We needed to understand why and create a new strategy to achieve a better performing subscription based product that helped users stay safer.
Responsibilities
I was in the Protect squad and at this stage had been working on the product for around 6 months on a range of different projects. I was on this discovery journey with the team, around 80% of my time was dedicated to the project.
The Strategy of Protect so far
The strategy of the product post launch was to grow it with lean techniques. This included lots of A / B tests with visuals and copy to test different hypothesis. We didn’t have the opportunity to do any in depth user research or deep dives to understand user behaviours.
1.7 Million Activated Users
20K Paid Users
100K Safety Actions Confirmed
Performance of the product
Early framing
Our head of product did some framing exercises before the rest of the team officially started the discovery process. Most of what he did was fed back to us and we had the chance to challenge it but there was alignment across everyone within the workshop. We thought through having different business models but we stuck with Free + Paid within the product because of two main reasons. Firstly, some data suggested that Protect staying in app had a lot of benefit and uplift in metrics to other parts of the product if a user engaged with us. Secondly, the other options either held massive risk or a paid only approach felt off-brand.
The Business Strategy helped frame some of the discovery and gave clearer constraints, these where mostly around the type of solution (In app with a paid model with low marketing spend)
Subscription first vs Marketplace first
The model of subscription we went for was the marketplace first business which leant on a free product which gave away limited value but higher value for paid. Free users would still get a good product which was why the brand did this in the first place and it mapped to the core product well. For the 2.0 of the product it was decided to keep this strategy as pivoting away was too risky for a range of reasons
Research
From the primary scooping and overlooking some key performance metrics we decided that we needed to understand the ClearScore user base’s opinion on certain aspects of identity safety and subscription based services
Research Goals
What subscription based products hold value for users
How does their stage in life influence their behaviour of safety
What are user’s current understanding of identity fraud and what safety related behaviours do they have
Part 2
Discovery
The open ended discovery to better understand user needs and product value gaps
The Initial Plan
Overview on chosen research methods
This was the rough discovery plan that was on the agenda for a group of 4 of us to go through, I will explain why certain research tools were chosen.
We wanted to undergo user interviews because we needed a way to discover what people used as subscription services, what they thought of identity monitoring and get deeper insight into what they thought of Protect 1.0.
Expert interviews for products that are freemium based were great because it would help if further position on what aspects of Protect’s value could be shuffled to better suit the paid service.
Expert interviews with people that are knowledgable in the identity space would be a large leg up as the current value of Protect was around changing passwords and seeing some stolen information. There are other aspects to the space that we hadn’t fully thought about before so we wanted more insight here.
Research on other freemium design work was needed. By looking online as well as downloading some apps it would help us to further adapt what we had in the product at the time and see if we could leverage more established industry patterns.
Research on other identity / safety products was also needed for us to look at what other services are available and what aspect of the field they have decided to target. This would help inform which areas we should double down on or shift to.
Competitor Analysis, at this stage there where other services offering exposed password features. We needed to look at other examples of this and see how we stacked against them
Using A/B clickthrough tests we could easily test engagement for new propositions and run pricing tests. This would help inform what aspects of the ideation to go through with when it came to prioritisation of the roadmap
Product Analytic Data. We could get some more quantitive insight into who was using the current product, track upsell metrics and more.
Past Research playback. Good practice was to run over past research done into Protect. Most of the research in question was done before the launch of Protect or immediately after but would be great to not miss any gems.
Primary Research
Discussion Guide
User Interviews
To get started, we defined our research objectives and the approach for collecting our insights. The researcher led the discussion and the creation of the discussion guide. We involved the whole team in participating in interviews, so everyone was highly engaged with our findings.
It was agreed that we would interview 20 people interview as we wanted to get a broad range of user’s perceptions on different topics. As life stages affected the problem area quite a lot, getting a section of users who were a mixture of different demographics and property status would highlight different behaviours.
The topics in the structured discussion would be around:
Peace of mind
Life Goals
Credit management
What subscriptions they currently have
Fraud / digital crime
Their usage of ClearScore / Protect.
Secondary Research
Light breakdowns of different security products
Competitor Analysis
I lead this aspect of the discovery. I spent a range of time looking at different types of products that offered some aspect of fraud / cyber crime prevention. It was important to look at what the value proposition was, what subscription model it had (if it had any), how well the product was received and what sort of solution type it was.
I looked at products that acted as information hubs, apps that would delete data for you, B2B services that highlighted at risk information, VPNs, portals etc. This was a great exercise but it really highlight two main things to me while doing it, there are a lot of problems in this space as well as some very different solution types (even massively differing solutions for the same problem).
We found a broad range of models to support these propositions, when I then fed this back to the team it led to some great discussions about how products would work in different countries, with different marketing budgets and what it would mean for a brand like ClearScore to do it.
Data showing the upsell rate by age of current paid product
Product Analytical Data
We had a wealth of data from the existing product that we collated and skimmed over. It wasn’t important in the sense that we had already used it to frame the discovery but it was useful to go over it to know that the current product performed well in older demographics for the paid version
Idea exploration
Early Product Ideas
We had a space to brain dump some ideas when we were researching as a way of keeping them before ideation properly began. Some of these ideas came from a selection of previous working on the product, working on other parts of CS from the outside looking in and looking at what technologies we could leverage from B2B companies we researched.
Collection of articles
Freemium Models & The Identity Space
We had collated a range of articles that were related to great examples of products that had implemented freemium models well and industry insights into the fraud space. There was a range of insight that were known to us from the first iteration of Protect but new areas to the space that were new.
How I approached structure of the onboarding
Freemium design patterns
The other designer I was working with looked at different ways we could integrate and move forward to better utilise industry patterns. Visuals were taken to be studied further of how they look and behave in a flow
Part 2
Definition
Biggest problems and insights we learnt
Secondary Research Findings
Themes for discussion
Expert interview findings - Freemium Design
We were unlucky in that we didn’t get to speak with any design or product experts that had launched successful freemium model businesses but we were successful on finding areas that we wanted to further research if we got the opportunity . Doing this exercise as a design pair helped us further research the right things and refine our thinking on tests that we would want to run in the future
Expert interview findings - ID Experts
Product led some of the talks here and managed to speak to 2 experts in the field, 1 being very famous and owning a ID service themselves. We had gotten some insight on what data sources to look out for as well as some broad insights that where fed back to marketing team
Primary Research Findings
Some quotes that resonated with me during interviews
User Interviews findings
We got a lot a great insight from the interviews that we underwent. Overall we interviewed around 20 ClearScore users that were in high to medium income brackets with a mix of different family or property stages. We went with older / career users in general as this was informed by the addressable market which was highlighted in the discovery framing.
Some key take aways from the interviews I led that resonated with me was the general apathy to staying safe. Even when folks had a lot to lose, they had some online behaviour which was very dangerous that they either didn’t realise or knew about. What is most worrying is that they didn’t seem to think the danger would happen to them. Most users had a false sense of security in general, the only driver that seemed to change perceptions was if they happened to know or where a victim of identity fraud at some stage of their life.
Insight theming from user interviews
Analysing findings from interviews
We worked as a team with the researcher leading the workshop to distill all our findings from the user interviews that we did. We handled this in stages which then helped us to define who we should target and what drivers were likely to effect change. This exercise was to nail down the key opportunity areas and pain points.
• The first phase of the workshop was clustering on how do high income users feel about their finances overall and what their main priories where
• The second phase was around what users associated with peace of mind
• The third phase was use looking how a users life stage impacts their perception of fraud
• The fourth phase was why users don’t care about online identity and fraud protection
• The fifth phase was starting to define what the jobs to be done were for users with identity protection and finance management
• The sixth phase was nailing down key user groups
Personas created for the ideation workshops
Personas
The designer I was working with created these personas for visual representation of the research we did and who we would be trying to aim for. Personas were useful for ideation workshops and were helpful for communicating to stakeholders and members of the team who were not involved in the research.
Part 3
Product IDEATION
Lots of workshops
Context
After we finalised our discovery and defined core problems, we got ready to have a 3-days workshop with our entire squad plus some folks who had worked on Protect adhoc over the months. This was over 15 people in total.
Ideation Board
Day 1 - Explore and Ideate
Main activities in the day
Digital Footprint exercise (What data was found | where was it found | How could it be used against you)
How might we.. (Help users understand | assess where they stand | etc)
How another company might solve the problem
Ideation Board
Day 2 - Developing Concepts
Main activities in the day
HMW Exercise for Evaluating / Actioning / Monitoring
Creating “Press Release” exercise to explain how
Design Concepts / Engineering Whiteboarding / Business Model Concept
Stakeholder Meeting
Ideation Board
Day 3 - Iteration / Early Roadmap
Main activities in the day
Re-Evaluated Last 2 days based off meeting
1 last HMW
Theming
Voting
Prioritisation based off of value to user / business vs effort
Early Road Map
Part 4
Product Strategy
How we decided to pivot the product
Shift in Key Deliverables
After a roadmap had been agreed by the team, the other designer and I started to get to work. I realised however that in the ideation session workshops we had gone a little off track towards the end and some of what we had prioritised either in terms of timeframe or what would like happen because of research findings meant that the set of deliveries needed a bit of adjusting to be effective. To fix this I spent some time going over all the discovery and ideation work we had done and tried to theme / categorise and map everything together to build a stronger set of deliverables.
Mappings and Prioritisation
2 Different types of behaviours / attitudes
Going back to the primary research that we did, there are 2 main behaviours that we observed in users as well as previous insights from the old product. As there was people who were very apathetic and only really actioned safety risks in a reactive way, we needed to make the solution more tailored for them as they made up most of our addressable market. We needed to look for the behaviours that were present across both groups to target
Business metrics to changes in the product
I looked at the type of business metrics that we were expected to impact. These were important because if we didn’t meet these in the next few months it would mean that the product we worked on wouldn’t pass day 365. I mapped the business metrics that would measure success to 2 core values in which we could change
Mapping User needs to to changes of behaviour to increased business value
After making sense of some of the user problems, we discovered we needed a way to map them to each other. I looked over a range of the solutions that we had from the ideation session and boiled them down to 3 main areas that I felt would accurately be measurable and buildable for a user experience. I then mapped the user needs to these core solution types and from that map those to the business goals. This helped give us more of a structure for tackling the roadmap
Solution themes to user impact
One of the biggest things that started to go off track in the ideation session was that there was lots of ways to solve the same problem but some of these great ideas were lost in the themes as some ideas combined multiple ideas. I had started to go through an exercise where I recategorised them to their core themes. I then put some of the unique ideas under the themes which were related but solved or framed the solution in a different enough way to be good enough to warrant being separate. Once that was done, we mapped which of these individual solutions would affect user behaviour and how strongly. From that we picked which type of solution we should further explore.
New roadmap with restructure and planning in mind
From all of these mapping exercises we created a set of planned deliverables with a fully integrated roadmap which got stakeholder approval and instant squad delivery resource towards moving us into the direction of the 2.0
Part 5
Outcome
A higher design maturity team
The process of this project and opening to the whole team made everyone in the squad more design driven as well as helping everyone have a clear understanding of the user problems we where trying to solve
Stakeholder Approval
The roadmap and restructure of it was approved by the CEO and head of product when presenting it back to them
Higher Team Moral
While this is harder to track, after seeing the Protect product slow down in terms of business performance for the last few months there was more much energy and productive after the ideation session within the squad
Key Learnings / Reflection
The problem we were trying to solve for would be tricky with even without a subscription model so it really challenged us to think very hard when we got to the ideation stage what to do that would be affective and be feasible to build. This aspect wasn’t an easy one which made us rethink a few deliverables after the ideation workshop to have more of a footing business wise.
This was my first 10 weeks full time discovery project as a designer so I learnt quite a bit about process and how what time can go into purely finding insight .
The Identity space is quite shocking, a lot of people seem to be really unaware about. We heard some stories that have happened in the UK and I personally would be terrified having to suffer some of those damages.
Thank you for reading