Skip to main content

Contributing

We love to see our community members get involved! If you are planning to contribute to Dagster, you will first need to set up a local development environment.

Environment setup​

You can develop for Dagster using macOS, Linux, or Windows. If using Windows, you will need to install and use WSL to follow the steps in this guide.

  1. Clone the Dagster repository to the destination of your choice:

    git clone git@github.com:dagster-io/dagster.git
    cd dagster

  2. Install uv. You can use curl to download the script and execute it with sh:

    curl -LsSf https://astral.sh/uv/install.sh | sh

  3. Create and activate a virtual environment using uv with a Python version that Dagster supports:

    uv venv --python 3.12
    source .venv/bin/activate

    Dagster supports Python 3.10 through 3.12.

  4. Install Node.js (20.X and above). We recommend using nvm (Node Version Manager) to manage Node.js versions. It allows you to switch between different Node versions:

    # Install and use Node 20.x
    nvm install 20
    nvm use 20

    # Verify Node installation
    node -v

    Once you have Node installed, install yarn

    npm install --global yarn

  5. Run make dev_install at the root of the repository. This sets up a full Dagster developer environment with all modules and runs tests that do not require heavy external dependencies such as docker.

    make dev_install

    This will take a few minutes.

Note for Macs with an Apple silicon chip

Some users have reported installation problems due to missing wheels for arm64 Macs when installing the grpcio package. To install the dagster development environment using our pre-built wheel of the grpcio package for M1, M2, and M3 machines, run make dev_install_m1_grpcio_wheel instead of make dev_install.

  1. Verify your local setup is correct

    $ dg --version
    dg, version 1!0+dev

    $ dagster-webserver --version
    dagster-webserver, version 1!0+dev

    As long as you see version 1!0+dev in the output, you are all set up and ready to start making code changes to Dagster! 🎉

  2. (optional)

    You can further verify your environment is set up by running Dagster's test suite, however this can take an hour or more to complete.

    python -m pytest python_modules/dagster/dagster_tests

    To run only some of the tests, see this page of the pytest documentation.

Developing Dagster​

Some notes on developing in Dagster:

  • Ruff/Pyright: We use ruff for formatting, linting and import sorting, and pyright for static type-checking. We test these in our CI/CD pipeline.
    • Run make ruff from the repo root to format, sort imports, and autofix some lint errors. It will also print out errors that need to be manually fixed.
    • Run make pyright from the repo root to analyze the whole repo for type-correctness. Note that the first time you run this, it will take several minutes because a new virtualenv will be constructed.
  • Line Width: We use a line width of 100.
  • IDE: We recommend setting up your IDE to format and check with ruff on save, but you can always run make ruff in the root Dagster directory before submitting a pull request. If you're also using VS Code, you can see what we're using for our settings.json here.
  • Docker: Some tests require Docker Desktop to be able to run them locally.

Developing the Dagster webserver/UI​

For development, run an instance of the webserver providing GraphQL service on a different port than the webapp, with any pipeline. For example:

cd examples/docs_snippets/docs_snippets/intro_tutorial/basics/connecting_ops/
dagster-webserver -p 3333 -f complex_job.py

Keep this running. Then, in another terminal, run the local development (autoreloading, etc.) version of the webapp:

tip

Don't forget to activate the virtual environment (source .venv/bin/activate) when you open another terminal!

cd js_modules/dagster-ui
make dev_webapp

During development, you might find these commands useful. Run them from js_modules/dagster-ui:

  • make ts: Typescript typechecking
  • make lint: Linting with autofix
  • make jest: Test runner that runs the full suite of tests

Developing documentation​

To run the Dagster documentation website locally, run the following commands:

cd docs
yarn install

API documentation is built separately using Sphinx. Before starting the dev server for the first time, you need to build the API docs. If you change any .rst files, be sure to re-run the following command in the docs directory:

yarn build-api-docs

Then start the local development server:

yarn start

For the full guidelines for writing and debugging documentation, please refer to the docs/README.md and docs/CONTRIBUTING.md documents.

Picking a GitHub Issue​

We encourage you to start with an issue labeled with the tag good first issue on the Github issue board, to get familiar with our codebase as a first-time contributor.

When you are ready for more of a challenge, you can tackle issues with the most 👍 reactions. We factor engagement into prioritization of the issues. You can also explore other labels and pick any issue based on your interest.

Submit Your Code​

To submit your code, fork the Dagster repository, create a new branch on your fork, and open a Pull Request (PR) once your work is ready for review.

In the PR template, please describe the change, including the motivation/context, test coverage, and any other relevant information. Please note if the PR is a breaking change or if it is related to an open GitHub issue.

A Core reviewer will review your PR in around one business day and provide feedback on any changes it requires to be approved. Once approved and all the tests (including Buildkite!) pass, the reviewer will click the Squash and merge button in GitHub 🥳.

Your PR is now merged into Dagster! We’ll shout out your contribution in the weekly release notes.