Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dagster-dbt: DagsterInvariantViolationError in dbt_assets with valid manifest.json #27192

Open
traanhn opened this issue Jan 17, 2025 · 1 comment
Labels
type: bug Something isn't working

Comments

@traanhn
Copy link

traanhn commented Jan 17, 2025

What's the issue?

Summary

After upgrading from Dagster 1.9.6 to 1.9.7, the dbt_assets function fails with the error:

dagster._core.errors.DagsterInvariantViolationError: Unexpected 'None' output value. If a 'None' value is intentional, set the output type to None by adding return type annotation '-> None'.

This issue does not occur with Dagster 1.9.6 or smaller version .

Dagster Version

  • Dagster Version: 1.9.7 (also tested 1.9.8, same issue)
  • dagster-dbt Version: 0.25.7
  • DBT Version: 1.8.7

Observed Behavior

The job fails immediately after yielding the first output from dbt.cli(...).stream(). The STEP_FAILURE logs show that Dagster encounters a None value unexpectedly.

Workaround

Reverting to Dagster 1.9.6 resolves the issue.

Environment Information

  • OS: macOS 15.2 ARM (M1)
  • Python Version: 3.12.0
  • Dagster Version: 1.9.7 and 1.9.8
  • Dagster-DBT Version: 0.25.7 and 0.25.8
  • DBT Version: 1.8.7

What did you expect to happen?

Expected Behavior

The job should run successfully without errors, as it does with Dagster 1.9.6.

How to reproduce?

Steps to Reproduce

  1. Create a dbt_assets function with a valid manifest.json:
@dbt_assets(
    manifest=env_config.dbt_manifest_path,
    dagster_dbt_translator=CustomDagsterDbtTranslator(),
    io_manager_key="snowflake_analytics_io_manager"
)
def all_dbt_assets(context: AssetExecutionContext, dbt: DbtCliResource, config: DbtConfig):
    dbt_build_args = ["build"]
    if config.full_refresh:
        dbt_build_args += ["--full-refresh"]

    dbt_build_invocation = dbt.cli(dbt_build_args, context=context)
    yield from dbt_build_invocation.stream()
  1. Run the job:
    dagster job execute -f my_job.py
  2. Observe the error:
    dagster._core.errors.DagsterInvariantViolationError: Unexpected 'None' output value. If a 'None' value is intentional, set the output type to None by adding return type annotation '-> None'.
    

Dagster version

1.9.7

Deployment type

Local

Deployment details

No response

Additional information

Testing with GKE staging environment generates the same issue with version 1.9.7 for dagster-dbt= 0.25.7
Revert to 0.25.6 then it works in both local and GKE.

It seems there are some changes in dbt_assets decorator in version 1.9.7 which has caused this unexpectedly

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

@traanhn traanhn added the type: bug Something isn't working label Jan 17, 2025
@cdchan
Copy link
Contributor

cdchan commented Jan 18, 2025

We've been seeing this too after upgrading to 1.9.8. It looks like it might be related to #26614 which removed the explicit AssetOut with type Nothing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants