Data Catalog

The Data Catalog allows you to view the data points (either an event, attribute, user identifier, or user attribute) ingested into your mParticle workspace.

You can use the Data Catalog to view specific details and statistics about each data point. You can also use the Data Catalog to add your own tags and descriptions to data points.

To find the Data Catalog, go to Data Platform > Data Catalog using the left hand navigation.

By default, the Data Catalog lists all data points, sorted by name, with the Date range set to All Time Available.

You can use this list to:

• Spot data points which are duplicated or inconsistently named
• Identify and eliminate any unnecessary or redundant data points

To view the details of a specific data point, select that data point from the list. You can also add a description or tag to a data point directly from the list view.

Data point categories

The Data Catalog organizes data points into categories:

• Custom Events
• Screen Views
• Commerce Events
• User Attributes
• Identity
• Application Lifecycle
• Push Messaging
• Consent

To view all data points in a specific category, select that category from the list shown in the left of the Data Catalog.

Date Range, Search, and Filter

You can filter the list view to display specific data points:

• Date Range: show data points that have been seen within a selected date range.
• Search: show data points with a matching name or description.

• Filter icon: show data points that match the criteria you specify:

  • Partner Feed/App Version: show data points that have been seen for the selected partner feed/app versions.
  • Environment: show data points that have been seen in the dev or prod environments.

  • Channel: show data points that have been seen for the selected channel. Channel is distinct from input and describes how a data point is ingested. For example, a data point may arrive from the client side, server side, or from a partner feed. Valid channels include:

    • SDK
    • Feed
    • Server to Server
    • Pixel

Combine date ranges and filters with your search terms to browse and explore data points.

What information is available in the Data Catalog?

The Data Catalog provides descriptive metadata about the types of data points processed in your mParticle account, but it does not provide any actual values of individual data points. For example, you can use the Data Catalog to see the name, category, and data type of the $firstname user attribute. You can even see which inputs recently ingested values for this attribute, but you can’t use the Data Catalog to see that mParticle recently ingested a $firstname user attribute with a value of “John”. For this kind of information, you can use the Live Stream to see what actual values are ingested in your development environment.

The metadata for your data points is contained in a data store, of which there are two versions: a less detailed data store that provides information to the Data Catalog faster (which helps the page to quickly display high level info) and a more detailed data store that takes the mParticle app longer to access and which only contains information back to December 2018.

The slower (but more detailed) store was added in December 2018. The added detail is what enables you to use filters in the Data Catalog. This explains why when you apply filters, the Data Catalog can only display information back to 2018 and why the Data Catalog may display a different total count of data points (even when the Date Range is set to “All Time Available”).

Details view

The details view gives you detailed information on an individual data point, including environments the event has been captured for, and when an event was last seen for each platform.

Users with admin access can annotate data points in the following ways:

• Tags: a freeform list of labels you’d like to associate to the data point.
• External Link: a link to your wiki or any other resource containing documentation about the data point
• Description: a custom text field where you can describe the data point, expected attributes, how it’s used, and any other relevant information.
• Additional Names: a list of alternate names the data point is known by. For example, legacy names or names from a partner feed.

Non-editable details displayed for data points are:

• Name: the name of the data point as it appears in the mParticle data schema.
• Category: the category of the data point.
• Event Tier: the event tier determines how the data point is processed by mParticle. Learn more in Tiered Events.
• Data Type: the data type of the value of the data point. For example: String or Number.
• Environment: indicates which data environment the data point was ingested in. Either DEV and/or PROD.
• Inputs: indicates each input that ingested the data point. The date listed indicates the most recent date the data point was ingested.

Data Point Attributes

Your event data points may include attributes. The details view shows all attribute names that have been seen for the given data point. You can see the total volume received in the last 30 days, when the attribute was last seen, and the detected data type of the attribute. The supported detection types are:

• String
• String-list
• Boolean
• Number
• Date-time

Stats view

For data points, the stats view shows two important groups of statistics for a selected date:

• Input stats show the number of times an event has been ingested through each input, by environment and channel.
• Output stats show the number of times an event has been forwarded to each output, as well as the difference between the number forwarded and the number ingested. This difference can be useful when troubleshooting connections, but remember that a difference between how many events are ingested vs. forwarded does not necessarily indicate a problem. For example, the expansion of ecommerce events can cause multiple messages to be sent to an output for a single event. Filtering or an output partners minimum requirements can also result in fewer forwarded events.