A Getting Started Guide to Establish an Internal Product Culture
“If we have data, let’s look at the data. If all we have are opinions, let’s go with mine.”
- Jim Barksdale
“Fundamentally, the root cause of many of the problems and disappointments that organizations experience with all platforms is the failure to properly treat them as products.”
- Thoughtworks TechRadar, Volume 27
While there is growing information on “what” a product culture should be, there aren’t many resources to show “how” to get there. This article aims to provide a simple “getting started” guide to answer the question: “How do we start and grow a product-driven culture in our Platform team?”
The Playbook
There are three main stages:
Collection
Synthesis
Socialization
Collection
The goal of the collection stage is to collect data. At the end of this stage, the organization has a single, populated, centralized place to aggregate all feedback.
Feedback as Fundamental Data
In my discussion with Matthew Clark, Senior Product Director of Foundations at Tinder, he said, “Everybody wants the perfect metric.” KPI practices are challenging to set up and train, and in a desire to show data that perfectly explains behavior and impact, false precision and a lack of data get the product process stuck.
As we get started, we want data that is accurate and easy enough to collect routinely. From multiple companies, collecting feedback from your users is the fundamental data that meets these criteria and sets up the base for a data-driven product workflow. We’ll get into validating data from feedback in a later step.
Interviews > Surveys
In the beginning, collect feedback with 1:1 interviews over surveys. Of course, surveys have their place, but there is a good chance they introduce incorrect data and get the process stuck through problems like low response rates or inaccurate interpretations of the question.
The main point is to set up a user-centric feedback collection process with 1:1 and group discussions.
Start with Uniform Questions
Several teams asked a partner UX or user research team to help them establish typical interview best practices like recording the interviews and leading discussions with common questions.
In “Build Stuff People Will Actually Use,” Michael Galloway suggests these predefined questions, which is an excellent place to get started:
What are they working on?
What are they mostly focused on during the week?
What tools do they like to use?
What do they like or not like?
What tasks are tedious or repetitive?
Talk to more than Engineers
PMs, EMs, and business stakeholders have valuable feedback to provide. It’s common for these roles to experience pain due to developers needing the right Platform toolsets, and these unique perspectives aid in understanding the connection between engineer frustration and holistic business impact.
“Go where you are wanted”
Internal PMs from Stripe to Indeed to Microsoft have all said the same thing: go where you are wanted. Consider these two commonly seen approaches:
From the company level: Go to the teams incentivized to work with you. Often they are identified via the company’s goal-setting process or have goals directly linked to the Platform organization.
From the team or IC level: While at Doma, Michael Galloway “got [interviewee] names by pressuring their management to identify willing participants.” to set up 1:1 interviews. You’ll find that teams have more willing members to champion Platform tools and become partners.
Interview new Developers
PMs should attend each new engineer onboarding and get honest critiques of the tools. Netflix found that “Engineers have a surprisingly high tolerance for dealing with toil.” The longer an engineer has been there, the more used to the toil they become or developed minor workarounds to where they didn’t provide the most helpful feedback. New users, however, provided some of the most impactful feedback.
Action Items
Build a list of people to interview. Do this by identifying the teams incentivized to work with you via the company’s goal-setting processes and highlighting the teams asking to work with you. Pay attention to the non-engineers in those groups and new engineer onboarding.
Create interview standards all PMs will follow, such as common questions and recording interviews. In some cases, user research or UX teams help provide guidance. See if your company’s resources can help advise this process.
Create a single place all feedback flows. I’ve seen examples using spreadsheets, Airtable, Canny, and ProductBoard. Airtable has a great guide with examples that inspired our approach at Twilio. Think of this as the start of your internal customer management database, or ICRM.
Putting the feedback into a shared place enables the next stage: “synthesis.”
Synthesis
The synthesis stage aims to identify opportunities from collected data and develop proposals. At the end of this stage, you should have a vision/mission, a strategy, and a roadmap for prioritized product work.
In “Build Stuff People Will Actually Use,” Michael Galloway talks about two primary problems that come with collecting lots of feedback:
Volume. How do we process so much feedback? Feedback volume will easily exceed the team’s capacity, and the initial rush of feedback often shocks a team into reactive behavior.
Accuracy. Is what the feedback asks for really what’s needed or best for the organization? There are two common pitfalls: we need to interpret the feedback correctly, or the input asks for something that wouldn’t provide the best solution for the company.
So there’s a backlog problem: how do we understand the feedback, and how do we make it worthwhile?
Triaging: Quick Fixes
Several Platform and Dev Tool groups found that keeping “low-hanging fruit” as a part of their regular product prioritization work addresses pains that most teams would overlook.
Not only do the engineers feel heard on a “no issue too small” viewpoint, but the lead time to process this feedback is short, establishing a quick feedback loop that reinforces the customer-product relationship. Customers appreciate seeing action, and this type of work directly improves their confidence and satisfaction with the tools.
The pitfall is doing too many quick fixes or treating each piece of feedback like it could be addressed as a quick fix. That’s where strategic triaging comes into play.
Triaging: Strategic
Strategic triaging addresses the second main problem the collection process presents: is the customer asking for the right thing to solve? And if we did solve that problem, would it apply to the broader enterprise?
During the strategic triaging step, try to ask and answer questions to estimate how much leverage the organization would gain by solving the problem. Common examples are:
How many groups may be impacted by this problem? Which teams specifically?
What difference would the business see if we solved this problem?
Does this relate to any current goals? Or would it support a future objective?
Answering some of these questions gets the feedback into a place where the team can understand if it aligns with a current objective, needs further root cause analysis, or isn’t something to address now.
Start Manually
There is a desire to have a perfect triaging mechanism from the beginning— still, every team I’ve researched and worked on started with manual triaging and feedback evaluation.
User Journeys/Maps
Many Platform teams referenced user journeys or maps as valuable in identifying root causes and determining solutions to provide impact. They are visual and workflow-oriented and can reference feedback at different stages.
That context provides value on two fronts: to discuss solutions with engineering and impact with executives.
Apply an Outcome Framework
Companies like DoorDash and Stripe use the SPACE framework to understand how their platform benefits the engineers. Higher-level outcomes add context for understanding the benefit of addressing the feedback and facilitate creating an outcome-oriented product roadmap.
Don’t expect this to be a perfect solution, however. Those companies report that it helps them understand alignment and impact, but the approach is still improving.
Action Items
Set up a regular schedule to triage feedback as either: a quick fix or strategic. The point of triaging is quickly getting an owner and a decision on the feedback our customers provide. Get the whole product team involved since platform products are more powerful when providing interconnected features. See also: the local optimization problem.
Identify quick fixes. Addressing quick fixes provides immediate benefits by showing the engineers that changes are happening. Remember that delivery builds credibility and trust.
Grow the team’s ability to provide context by establishing categories for outcomes and build user journeys to see how addressing the problem provides strategic impact.
Work with architecture or engineering to negotiate roadmaps for the problems.
Now the team is ready for the final stage: socialization.
Socialization
“Maintaining goodwill is key to adoption.”
This stage aims to validate that the problem has been identified correctly and reinforce partnership with your customers.
Your team has the feedback and has done the work to build an outcome-oriented roadmap. The final step is to socialize. Socialization is essential for two reasons:
Validation: There is a risk in the synthesis stage that the problem definition and the solution planned to approach it needs to be corrected. Your customers need the ability to provide buy-in on the approach.
Adoption: This is the first step in the go-to-internal-market plan – the beginning of adoption. Inviting customers early for feedback makes them feel like partners in the process and aware of what’s coming. Doing this sets the stage for faster adoption later. Michael Galloway said it best: “The more you invite them in, the more they will become a partner and not a customer.”
Make your issues and roadmap easy to discover and easy to critique
PMs at organizations like Microsoft said making their roadmap easy to discover and critique significantly impacted adoption and building better things for their engineers. They also allowed customers to link their ideas and goals to provide more context.
A single, central roadmap or “portal” to which customers can add context reinforces the previous stages. Here is an example from Equinix that is quite nice using a product called Canny.
Plan and Execute an Internal Marketing Campaign
“There is an important caveat: internal marketing shouldn’t be delivered as a top-down edict. Instead, it needs to be something that facilitates and enables dialogue between those driving a platform and those using it. Marketing should initiate dialogue, encouraging users to ask questions, request new features and raise issues.”
Using your public, data-connected roadmap as the foundation, you need to get into discussions and more than you may like. The key is repetition.
Remember to stick to the vision: to enable partnership and dialogue. During these sessions, talk about what matters to them in their own words (you have the feedback), down to the team and individual levels.
Let Customers Vote
During roadshows or reviews with teams, voting provides a sense of collective buy-in and a path for engagement. Michael Galloway notes how it worked well to identify when plans had to be tuned and as a constructive outlet for critique:
“[allowing customers to vote] is a way to establish a sense of collective buy-in. If most folks gave a thumbs down, it’d be a great opportunity to take that back and tune your plans (better to find out now than later when they complain to each other or their boss about their development experience!).”
You MUST Deliver
If you never deliver on what you say you will, confidence in your ability as a partner decreases. Fewer people will want to participate, leading to less engagement, lower quality solutions, and lower adoption of tools. Delivering expectations closes the loop, reinforcing the relationship with every delivery.
As you proceed, track how often you complete what you say you will. Lack of delivery on the roadmap undermines the relationship.
Action Items
Create a single, public roadmap where the community can critique and link to it. Make sure getting access and discovering it is easy to do, as well as the collection and synthesis work done to arrive at this roadmap.
Plan and execute the internal marketing campaign with roadshows, town halls, and meetings with team members – whatever you have to do to get this in front of customers for feedback, awareness, and buy-in. During these sessions, make it easy for them to vote thumbs-up or thumbs-down on what’s proposed. Confirm that all teams you served could learn and critique the plans.
Measure your completion of expectations and deliveries.
Conclusion: All Together
From start to finish, here’s the homework:
Build a list of people to interview.
Create interview standards all PMs will follow, such as common questions and recording interviews.
Start your internal customer management database or ICRM by creating a single place all feedback flows.
Set up a regular schedule to triage feedback. Don’t underestimate the effort required to keep up with feedback volumes.
Assign the appropriate owner and action per triaged feedback. Release quick fixes to reinforce the customer relationship. Start building context for strategic work.
Work with architecture or engineering to negotiate roadmaps for the problems and results of strategic research.
Create a single public roadmap to which the community can critique and link.
Plan and execute the internal marketing campaign with roadshows, town halls, and meetings with team members. During these sessions, make it easy for them to vote thumbs-up or thumbs-down on what’s proposed.
Measure your completion of expectations and deliveries. You must deliver on the customer agreement, or the whole process is forfeited.
What’s Next
As you read through this post, there were possible ways you considered to improve it; as you implement this, you will find ways that work better for your organization. So that’s what comes next.
Continuing to evolve this process has enormous potential to provide better value to your organization. Here are some things organizations do that are options as you develop:
introducing surveys
instrumenting tools developers use for more fine-grained analytics
expanding the ICRM to include metrics
evolving the use of outcomes
introducing automation in the categorization and triaging of feedback
introducing inner/open-sourced practices for closer collaboration, prioritization, buy-in, and delivery