DataPowered

Using messy data is like trying to bake a cake with mislabeled ingredients—what you think is sugar might be salt, and instead of a delicious result, you get an eyebrow-raising result. This dashboard is your recipe book and taste tester, ensuring your data mix is just right before you serve up insights.

The dashboard helps you maintain data trust by identifying and solving functionality-breaking issues in Amplitude. It does not focus on tracking plan quality (there is plenty of literature on this); instead, the dashboard will help you identify difficult and often hidden issues that may block your usage of Amplitude.

If you’d like to see what the final dashboard would look like, here’s a preview.

Before we begin

Let’s ensure you’re sending data to Amplitude reliably and minimizing data loss:

• Go to Data > Sources > Ingestion Debugger and switch the time range to “Last 30 Days.”
• Look below at the “Error Requests” chart and compare this number to your total daily events over the same period.
• If your error totals make up more than 0.1% of your total events, I recommend investigating further depending on the type of errors you’re getting.

Part I: User Identity & Journey Accuracy

1. Does your User ID/Device ID ratio look healthy?

A healthy ratio means you are tracking users correctly, especially for the user authentication step of your product. What a healthy ratio looks like is different from product to product and from industry to industry – a social media platform may require all users to register and will have registration rates as high as 90%, while others might have that ratio reversed. So hopefully you have a baseline, otherwise, it’s good to go through this check regardless to ensure you’re following user tracking best practices.

• If you are not tracking user registration or login (i.e., only tracking anonymous users), you can skip this chart if the ratio shows 100% Device IDs. The same applies if you’re only tracking registered users.
• If you suspect you have too many Device ID users compared to User ID users, that means you may have an inflated number of anonymous users or the user authentication is not tracked correctly.

🛠️ How to build the chart:

• Chart Type: Segmentation
• Events:
  • Any Event where User ID ≠ (none)
  • Any Event where User ID = (none)
• Measured as Uniques

💡 How to fix:

• If you have more users with Device IDs than expected, ask your engineering team to:
  • Ensure that User IDs are assigned correctly to your users, usually at the time of login/authentication.
  • Ensure the Device ID is not accidentally reset after the User ID is set (usually after the user authenticates). When you set the User ID, the Device ID should stay the same as it was until that point in the user journey.
  • You might be getting a lot of one-off anonymous users, i.e. users who only perform a few actions or page views and then bounce. You can build a chart like this one to identify one-off users.
• If you have more users with a User ID than expected:
  • Ensure that User IDs are assigned correctly to your users, usually at the time of login/authentication and not earlier.

2. Do you have bandit User IDs?

Sometimes, the same User ID is accidentally allocated to many different users (or Device IDs). Look at the chart below and check if one or more User IDs show up coupled with many device IDs (usually >30).

🛠️ How to build the chart:

• Chart Type: Segmentation
• Events: Any Event
• Measured as Uniques
• Group Segment By: Device ID
• Visualization: Bar chart
• Order from Highest to Lowest Value

💡 How to fix:

• If the chart looks balanced and the numbers are low, you’re probably not affected. The same applies if, by the nature of your product, it is expected that many different users use the same device or account or if you set your Amplitude cookies to expire quickly.
• If you do have outlying User IDs or some User IDs are coupled with more than 30-40 Device IDs, check the following with your engineering team:
  • Ensure the Device ID is not accidentally reset after the User ID is set (usually after the user authenticates). When you set the User ID, the Device ID should stay the same as it was until that point in the user journey.
  • Ensure there’s no “fallback” or default User ID set for exceptional situations (i.e., when the real User ID isn’t found). If that is the case, the correct approach is to avoid setting a User ID altogether.

3. Are your user journeys accurately reflected?

Ideally, user events are reflected in Amplitude in the same order as they would happen in real life. If you’re going to a landing page, logging in, and then managing your account, for example, all those actions should be reflected in Amplitude in the same exact order. One key risk here is your funnel/dropoff analysis being unusable.

🛠️ How to build the chart:

• Chart Type: Journeys
• Paths Starting With: Start Session (or any specific starting path you’d like to check)
• Measured as Uniques
• Expand chart further via “More”

💡 How to fix:

• If you see events in the wrong order or missing events, notify your engineering team. Likely causes include incorrect use of Amplitude SDK, incorrect event timestamps, or incorrect SDK/API setup.
• If you’ve observed disrupted journeys (e.g., path starting where it should not), you may have a user identity issue:
  • Ensure the Device ID is not accidentally reset after the User ID is set (usually after the user authenticates). When you set the User ID, the Device ID should stay the same as it was until that point in the user journey.
  • Ensure there’s no “fallback” or default User ID set for exceptional situations (i.e., when the real User ID isn’t found). If that is the case, the correct approach is to not set any User ID.

4. Are your User IDs unique?

Any authenticated user needs to have a unique identifier set as their User ID. We’re going to check for:

• Non-unique User ID values like usernames or emails (which can change and disrupt user journeys)
• Odd User ID values that bypass Amplitude requirements or are not proper unique identifiers, i.e. “undefined”, “0”, “1”, “Error” and anything else under 5 char length.

🛠️ How to build the chart:

• Chart Type: Segmentation
• Events:
  • Any Event where User ID ∋ @, error, undefined, unknown, null, and none (but not the default Amplitude (none) since we’re not looking for blank values)
  • If your implementation is mature (pre-2023), you could also create another segment with Any Event where User ID = 0, 1. 0 and 1 are sometimes used as fallback values, but this is less likely if you implemented in the last few years, since Amplitude now blocks using User IDs with a length of less than 5 characters.
• Measured as Uniques

💡 How to fix:

• If you’re seeing a small number of users in the chart (less than 10% of your user base), it may mean you’re in one of these scenarios:
  • If you’re seeing results for values like “error” or “undefined”, your code is probably trying to assign a value because no User ID was found. If this is the case, the correct approach is to not set a User ID.
  • If you’re seeing results for values like “0” or “1” or a small integer, then no action may be needed. In the future, avoid setting consecutive integers as User ID and instead leverage a UUID.
• If you’re seeing a larger number of users in the chart (at least 10% of your user base) it might mean you need to rethink your User ID logic altogether. Connect with Amplitude Support to understand what options you have for changing your User ID logic.

5. Is there an overwhelming number of bounced users?

This is important for a few key reasons:

• A large number of bounced users (meaning one-off users that only perform a few events before leaving and never returning) can inflate your MTU count and event volume. Your Amplitude subscription is based on one of these two and can either cost you overages or platform access (if you’re on the free plan).
• An inaccurate number of users can mess with your analysis: it can inflate user counts in DAU-based charts and conversely (pun intended) it can artificially lower your conversion rates.

The chart below will give you the percentage of users who perform very few events. Ideally, the share of users who only perform 2 events should not exceed 80%. There are, however, exceptions depending on the type of product you have.

💡 How to fix:

• Avoid tracking irrelevant events if you can, like weak entry points to your product (especially if you’re not interested in marketing attribution), one-off landing pages that aren’t well connected to your product, and other high-volume events that aren’t relevant for your use cases/KPIs.
• Create a cohort with the following definition and then use it to exclude bounced users from important charts:
  • Users who did perform Any Event
  • with count < 2 times
  • any time during the last 90 days (or whatever range you feel comfortable with)

Part II: Session Accuracy

1. Are your sessions tracked correctly?

The first step is to ensure your data includes session information. This can enable additional functionality in Amplitude, most notably a more accurate Pathfinder chart. If you’re already using time-based session tracking (Settings > Projects > Your Project > Session Definition) skip this check.

🛠️ How to build the chart:

• Chart Type: Segmentation
• Events:
  • Any Event where Session ID = -1

💡 How to fix:

• If your chart is showing 0 results, it means you’re not missing the default session information tracking.
• If you’re observing numbers over zero, that means some of your events are missing session information, and you’re missing out on session data. To resolve this:
  • Check with your engineering team to see if this omission is intentional. In some situations, sessions cannot be tracked. If that is the case, you can change the session definition to time-based session tracking (Settings > Projects > Your Project > Session Definition). In this scenario, Amplitude will compute sessions purely based on event timestamps and will disregard the session_id parameter.

2. Is there an unusual number of 0-second sessions?

While some users spending very little time in your app is not unusual, a lot of zero-second sessions usually means your tracking can be optimized.

🛠️ How to build the chart:

• Chart Type: User Sessions
• Measured as Distribution

💡 How to fix:

First, if you’re tracking the Start Session and End Session events, I’d suggest considering whether you really need them. Unless you care about very precise session length metrics or you’re using them in advanced analysis (i.e., formulas or metrics), they mostly occupy an aesthetic role. Amplitude charts (i.e., Sessions, Pathfinder) will function without them.

If your chart is showing an unusually high number of sessions lasting between 0 and 1 second, the potential causes are:

• You’re not tracking enough event types. Your users might open the app and stay active for a while, but if you’re only tracking a handful of events, your session length will be misrepresented.
• Your app is brought briefly to the foreground but is not used. This may happen due to various types of app notifications, multitasking, widgets, or interactions with other apps (i.e., phone camera). If that is the case, test initializing the Amplitude SDK somewhere else in the app lifecycle.
• Bots. You can enable additional block filters (Data > Filters) to mitigate this.

Part III: Properties and Revenue Tracking

1. Do you have missing property values?

Missing property values will limit your analysis because instead of valuable data, you’ll see a lot of empty information. Replace the event property or user property in the chart below with your most important properties to check your data for (none) values:

🛠️ How to build the chart:

• Chart Type: Segmentation
• Events:
  • Any Event where [Your Property] = (none)
  • Any Event where [Your Property] ≠ (none)

💡 How to fix:

Having (none) values means no property information was attached at the time of the event. Sometimes, this is expected, especially for events performed during the anonymous part of the journey. But if you’re seeing this happen at points in the user journey where a non-null value is expected, for example, you should investigate further:

• You should check whether a certain property is expected to populate on a user’s events. UTM parameters, for example, should be present from the beginning of the (web) journey – they will show up as “EMPTY” even if there are no UTMs set. Other properties, known only after the user logs in, for example, will show up as (none) until authentication or later.
• The next thing to check is whether you’re accidentally blocking that property in Data > Filters or Data > Properties > Any Status > Blocked/Deleted. You can unblock/undelete a given property if that’s the case.
• Next, if your event property is missing values, there’s likely an error in your code or that data is simply not available at the time the event is performed by the user.
• If your user property or group property is missing values, there might be a synchronization issue. Double-check that these properties are set at the correct time in your code or data source setup.
• If another type of property (e.g., lookup or derived) is missing values, check your configuration in Data > Sources and Data > Properties, respectively.

2. Is your revenue data instrumented correctly?

Amplitude has a reserved set of event properties for revenue tracking. That means if you’re sending revenue data differently, you won’t be able to use features like the Revenue Lifetime Value chart and revenue-based formulas.

🛠️ How to build the chart:

• Chart Type: Revenue LTV

💡 How to fix:

• If the Revenue LTV chart in Amplitude is not displaying any results it means you’re not using the reserved revenue properties. Instrument revenue tracking depending on your main data source(s).

Part IV: Other Checks and Best Practices

1. Ingestion errors. Make sure you’re minimizing any data loss or ingestion issues by checking for ingestion errors. Go to Data > Sources > Ingestion Debugger and switch the date range to the last 7 days. Scroll down to the Error Requests chart and note any error types that make up a large share of your event volume. Use Amplitude’s response guide to troubleshoot further and any ingestion logs you may have for your data source(s).
2. PII (Personal Identifiable Information). Avoid collecting PII and connect with your legal advisors to define what you should and shouldn’t track (with and without user consent). Check if you’re tracking any properties like real user names, emails, phone numbers, real addresses, etc., in Data > Properties and make the necessary changes. Keep in mind 1) deleting data in the Amplitude UI doesn’t mean the historical data is deleted and 2) removing historical data filtering by properties is currently a paid service at Amplitude.
3. Setting early event volume/MTU alerts. Amplitude currently alerts you for being at risk of exceeding event/MTU limits at 70% of your monthly/yearly consumption. However, you can customize chart alerts for early signs of volume increase. If your subscription with Amplitude is based on event volume, build a chart like this one; otherwise, build an MAU chart like this one. Then, save the chart and set a custom alert with your subscription threshold minus the volume you allocated to your other projects. Keep in mind this is only meant to increase awareness and is not a 100% reliable solution as it doesn’t account for data ingested with a past timestamp, and it only accounts for one project at a time. Event volume and MAU count always scan data ingested within a given month across all your projects.
4. Data de-duplication. If you have a server-side data source (i.e., sending data directly via HTTP API or Batch API or importing data through cloud storage or a data warehouse), then it’s highly recommended to send an insert_id with your data to avoid duplicate data.

Would you like to discuss fixing your data issues and restoring data trust? Get in touch, and let’s chat!