Skip to content

Configuring Storage

The Storage window is accessible from the Profile > Settings menu and displays the list of storage locations that you have configured for the current project.

Step 2

You can delete existing storage locations and add new ones.

Adding storage

  1. Click Add Storage.

    Step 2

  2. Choose a cloud provider or Local from the list. Type in a name for the container. (In the case of Azure, type in the storage account name, too.)

    Click Add.

    Step 1

    The new bucket appears in the list of storage buckets.

  3. If you selected a cloud service, validate that your machine has access to it:

    • For Google Cloud: If your private bucket resides on Google Cloud, make sure that the machine that you run the commands from has access to Google Cloud storage. Run the following command:

       gsutil ls
      

      Make sure that you see the bucket name where the data volume resides.

      Note

      Each machine that is going to access the data volume using the CLI needs to access the Google Cloud storage bucket.

    • For AWS S3: If your private bucket resides on AWS S3, make sure that the machine that you run the commands from has access to S3. Run the following command:

       aws s3 ls
      

      Make sure that you see the bucket name where the data volume resides.

      Note

      Each machine that is going to access the data volume from MissingLink CLI needs to access the S3 bucket.

    • For Azure: If your private bucket resides on Azure, make sure to perform the following:

      • Ensure that the machine that you run the commands from has access to the storage container. Run the following command:

        az storage container list [--account-key] [--account-name]

        Make sure that you see the bucket name where the data volume resides.

      • Automate the Azure storage process by performing the following steps:

        1. Copy the keys from an authenticated client by running:

          az storage account keys list -n sabcdatasetstore --output table

        2. Open the ~/.azure/config configuration file.

        3. Paste the keys into the following section:

          [storage] key=KEY_FROM_COMMAND

      Note

      Each machine that is going to access the data volume from MissingLink CLI needs to access the Azure storage.

  4. (This step is required only if you are using MissingLink's Resource Management.)

    When you add a new bucket on Azure, machines created with Resource Management are not authorized to access it. The required permissions cannot be granted from the backend; instead you, the user, must perform this authorization with your set of permissions. The easier way is to simply rerun: ml resources azure/aws init after the bucket has been added.

    This rerun is totally safe and will not change anything to the already existing entities.