Guidelines for a Successful Stemma Rollout

Every Stemma rollout is different, depending on your organization’s particular needs, your data storage and management applications, and the people who use them in various ways and with varying priorities. So the following are guidelines, not rules, but they are based on the accumulated experience of Stemma’s founders and engineers, and are a good place for you to start.

Criteria for Success

The two criteria that matter in measuring the success of a data catalog are

• adoption; and
• customer satisfaction

If your rollout achieves both of these in a high degree, you can declare victory in the knowledge that you have improved the lives– and in particular the productivity, efficiency, and awareness– of your users and are contributing significantly to your organization’s success.

The following broad steps are designed to help you get there.

Steps for Rollout and Adoption

Step 1. Start with a persona and its use-cases

There are many user personas and use cases for a data catalog. Here’s a simplified view of the most common ones. It’s less important which persona you start with than that you choose a specific group of alpha users.

Step 2. Launch in phases

Phase 1: Identify a small set of tables to get alpha user feedback on.

This set can be your organization’s most commonly used tables (often referred to as “core” tables) or one domain within the company, such as marketing, growth or finance, etc. Core tables are often the best choice, because they have the most impact and there’s often a central data team which is responsible for maintaining them.

Phase 2: Populate MVP metadata on these tables.

• Stemma can ingest the majority of this metadata automatically (see What Stemma Enables You To Do) leaving you and your users to curate only certain important items such as ownership, table descriptions and table status
  .* Where you must, to glean tribal knowledge, it may also help to do a “docs jam session” with a group of data producers and consumers. (You might even offer a reward such as a gift card for those who enter the most documentation!)

Phase 3: Alpha launch to 5-20 alpha users.

1. From the persona you decided on earlier, choose highly vocal users and those with the most tribal knowledge. These users will become Stemma’s avid supporters when you launch to a broader audience. (Note:Make sure these alpha users have the access they need to do work in Stemma (see SSO integration under What We Need From You) and that Stemma user names and authorities map properly to users of integrated applications such as Snowflake
2. Incorporate feedback and iterate. Look particularly for feedback that indicates opportunities for productive metadata propagation and cross-referencing (for example, someone might say something like, “Oh, we already have this metadata in this spreadsheet — we should pull that in here, too.” )

Tasks your alpha users should become comfortable doing include:

• Search – Search for tables and dashboards using a Page-Rank style search
  • Descriptions, etc. from DBT
  • Tables from Snowflake:

• View table details:
  • Description and status
  • Issues (JIRAs)
  • Last updated, table size, etc
  • Data owner(s)
  • Column descriptions & column lineage

• Perform table actions:
  • Edit table status, description, issues, owners
  • Add/edit/remove tags

• View table usage and history

• View table lineage

Phase 4: Beta launch to all users of the prioritized persona.

1. Focus your beta launch on your chosen persona (data consumers, for example). Don’t blur the focus of your launch by opening it up to all personas. Those other personas will come on board at GA, after your alpha and beta users have thoroughly learned and exercised the product and can serve as its advocates and in-house experts.
2. Graduate to GA if you can meet success targets.

Step 3. Achieve widespread adoption

Best practices for achieving the greatest adoption include:

• Update Slack channel headers where people ask each other questions. Stemma’s Slack integration links these conversations to the catalog automatically.
• Embed Stemma training in new hire training. Tagging data sets by domain (marketing, growth, etc.) can help new hires quickly onboard to their domains. Showcase Sremma as an entry point into your existing training, for example by having all technical new hires instrument a metric during onboarding.
• Create linkages with other products. Create links between various data tools. For example, auto-populate a link between an Airflow DAG that populates a table and the table page in Stemma (and vice-versa). (See Sharing additional metadata). Another productive linkage is between the table page in the data catalog and a link to the code that is used to generate the table.
• Showcase the catalog at a group or company meeting. Deliver a short five-minute demo at a meeting that includes your target users. Educate, answer questions, and thank your alpha users — this creates more awareness and provides an opportunity for everyone to learn.

Step 4. Measure success

1. Adoption

• Track WAUs (Weekly Active Users).
  • Start with Weekly Active Users (WAUs) rather than Daily Active Users or Monthly Active Users. Common usage frequency is weekly, not daily or monthly.‍
    • Target Penetration rate: 80%. A great penetration rate is 80% of WAUs within your target persona.

2. Customer Satisfaction (CSAT)

• Measure CSAT periodically.
  • Send out a survey every three to six months asking users to rate their experience with Stemma.

Other metrics that organizations often consider include documentation quality, search quality, etc. But it’s best to stick to the above core metrics at the outset. As your Stemma implementation matures and ingests more metadata over time, you can begin to broaden your criteria.