All About Matia’s dbt Integration

Matia integrates seamlessly with both dbt Cloud and dbt Core, enabling comprehensive metadata management, data testing visibility, and light orchestration capabilities. This integration allows you to maintain a complete overview of your data transformation processes while ensuring proper synchronization throughout your Ingestion (ETL), transformation workflows, and Reverse ETL.

Overview

The Matia dbt integration provides:

• Complete visibility into your dbt models and jobs and their metadata
• Access to dbt data test results, including a historical view
• Bi-directional orchestration between Matia and dbt workflows
• Comprehensive data lineage tracking

Prerequisites

• A Matia account with administrative access
• Either:some text
  • A dbt Cloud account (Team, or Enterprise tier)
  • A dbt Core installation with your repository

Features

Metadata Management

Matia provides comprehensive metadata management capabilities for your dbt environment:

Model Discovery

• Complete catalog of all dbt models and jobs
• Detailed model and column metadata
• Automated documentation synchronization

Data Lineage

• Visual representation of model dependencies
• End-to-end data flow tracking
• Impact analysis for upstream and downstream changes

‍

‍

Testing and Validation

Monitor the health of your dbt environment.

‍

Test Results

• Real-time visibility into test execution
• Historical data test results tracking
• Detailed failure analysis

‍

Run History

• Complete job execution history
• Performance metrics and timing analysis
• Error logging and troubleshooting information

Orchestration

Matia supports bi-directional orchestration with dbt.

Ingestion (ETL) to dbt

• Trigger dbt jobs after Matia ingestion sync is completed
• Configure conditional job execution
• Set up dependency chains

dbt to Reverse ETL

• Initiate Matia RETL jobs once dbt jobs finish running
• Schedule dependent workflows
• Manage complex data pipelines

Setup Instructions

dbt Cloud Setup

1. Account Identifier

• Navigate to your dbt Cloud Account Settings
• Located in the URL: https://cloud.getdbt.com/accounts/ <account-identifier>/pages/account
• Once at the page you can grab the account identifier from the URL itself or from the Account ID field as seen in the screenshot below.
  

2. Access URL

• The access URL varies depending on your dbt account region.
• While logged into your dbt Cloud account, click on the bottom left icon (right above your profile image) and copy the URL shown there (see screenshot below)

3. API Token

• Go to your dbt Cloud Account Settings
• Select "Service Tokens"

• Select Create a new token
  • If you are on the Team plan then set the token permission to "Member"
  • If you are on the Enterprise plan then set the token permission to "Developer"

• Once the new token is created copy it into Matia account

dbt Core Setup

1. GitHub Repository Details

• Repository Name: The name of your dbt project repository
• Owner Name: Your GitHub username or organization name
• GitHub Token: A Personal Access Token with repo scope

2. Repository Requirements

• Must contain a valid dbt_project.yml
• Should include your models directory
• Must be accessible with the provided token

What’s next?

As our customers constantly share how much they love dbt, the team here at Matia is inspired to keep building exciting features for our dbt integration.

The next big thing on our roadmap? A more powerful integration that ties together observability events (think anomaly detection), transformation jobs, ETLs, and reverse ETLs in a seamless way.

Take this for example: many customers have asked for the ability to pause reverse ETL jobs whenever anomalies pop up in upstream transformation models. Because let’s face it, for that one-in-a-million (sure, let’s go with that!) scenario when your dbt data tests fail, you’d probably prefer not to send bad data into your operational tools.

This is the power of Matia's unified platform in action.

Troubleshooting

Common issues and their solutions:

Connection Issues

• Verify service token permissions
• Check network connectivity
• Ensure proper account configuration

Orchestration Failures

• Validate job dependencies
• Check execution permissions
• Review error logs

Best Practices

1. Implement comprehensive testing for all critical models
2. Set up appropriate alerting thresholds
3. Regularly review job performance metrics
4. Maintain clear documentation for custom transformations
5. Use version control for all dbt code changes

‍

Ready to transform the way you manage your data workflows?
Try Matia today and experience the seamless integration of dbt with a unified platform built for modern data operations.

For more information or assistance, feel free to reach out to us at support@matia.io. We’re here to help you get started.

Start your trial now and take your dbt workflows to the next level with Matia.

‍