dstack 0.1: Introducing local and remote backends  February 7th 2023

Local and remote backends

Previously, dstack required a cloud account for use, even when running workflows locally, as the artifacts would automatically be pushed to cloud storage. The new update resolves this by introducing two distinct backends: local and remote.

By default, all workflows now run locally. If you prefer to run a workflow in the cloud, simply add the --remote flag:

dstack run train-mnist --remote

RUN        WORKFLOW    SUBMITTED  STATUS      TAG BACKENDS
chicken-0  mnist-data  now         Submitted      aws

Provisioning... It may take up to a minute. ✓

To interrupt, press Ctrl+C.

When a workflow is run locally, the resulting artifacts are also stored locally.

A cloud account is no longer mandatory to use dstack locally.

Pushing and pulling artifacts

Reusing output artifacts across workflows is a core feature of dstack.

The latest update allows for local workflows to reuse the output artifacts of other local workflows, and for remote workflows to reuse the output artifacts of other remote workflows.

To reuse the output artifacts of a remote workflow locally, use the dstack pull command to retrieve the remote run locally.

dstack pull --help     
usage: dstack pull [-h] (RUN | :TAG)

positional arguments:
  (RUN | :TAG)  A name of a run or a tag

To reuse the output artifacts of local workflows in the cloud, use the dstack push command to upload the local run to the configured cloud.

Other changes

Below is a list of other notable changes:

• The dstack artifacts list command was renamed to dstack ls
• The dstack artifacts download command was discontinued and replaced with dstack pull
• The before_run property in YAML was renamed to setup

The changes have been incorporated into the updated documentation and examples.

Try it now

To install the latest version of dstack, simply run the following command

pip install dstack --upgrade

Should you have any feedback, whether it be regarding an issue with the functionality or a question, please share it in our Slack chat or report it as a GitHub issue.

Meanwhile, stay tuned, everyone! We're hard at work on some really cool things that we'll be sharing with you soon.