Skip to main content
When you restart a stopped Bnode, you might see a message telling you that there are “Zero GPU Bnodes.” This is because there are no GPUs available on the machine where your Bnode was running.

Why does this happen?

When you deploy a Bnode, it’s assigned to a GPU on a specific physical machine. This creates a link between your Bnode and that particular piece of hardware. As long as your Bnode is running, that GPU is exclusively reserved for you. When you stop your Bnode, you release that specific GPU, allowing other users to rent it. Your Bnode’s volume storage remains on the physical machine, but the GPU slot becomes available. If another user rents that GPU while your Bnode is stopped, the GPU will be occupied when you try to restart. Because your Bnode is still tied to that original machine, it cannot start with a GPU. When this happens, Brightnode gives you the option to start the Bnode with zero GPUs. This is primarily a data recovery feature, allowing you to access your Bnode’s volume disk without access to the GPU.

What are my options?

If you encounter this situation, you have three choices:
  1. Start with zero GPUs for data access: Start the Bnode without a GPU to access its local storage. This is useful for retrieving files, but the Bnode will have limited CPU resources and is not suitable for compute tasks. You should use this option to back up or transfer your data before terminating the Bnode.
  2. Wait and retry: You can wait and try to restart the Bnode again later. The GPU may become available if the other user stops their Bnode, but there is no guarantee of when that will happen.
  3. Terminate and redeploy: If you need a GPU immediately, terminate the current Bnode and deploy a new one with the same configuration. The new Bnode will be scheduled on any machine in the Brightnode network with an available GPU of your chosen type.

How do I prevent this?

The most effective way to avoid this issue is to use network volumes. Network volumes decouple your data from a specific physical machine. Your /workspace data is stored on a separate, persistent volume that can be attached to any Bnode. If you need to terminate a Bnode, you can simply deploy a new one and attach the same network volume, giving you immediate access to your data on a new machine with an available GPU.