Skip to content

Hydra -- Autoscaler

Our own autoscaler Hydra does all interaction with compute providers (internal and external). These are currently two different functionalities:

  1. Create instances automatically according to execlet requirements
  2. Create instances on request to deliver bare VMs to customers

The creation of instances goes through some Hydra-internal statuses, which are different from the statuses of the external provider; these status transitions involve going through our own provisioning

Autoscale

Hydra regularly polls the job queue of our scheduler (streamer). If a job is waiting in the queue, we assume that no machine is ready to pick up a job and create a new one with an execlet service running on there. This flow can be observed in the following architecture diagram.

uml diagram

  • The number of automatically created instances of a specific type is capped in the supported_gpus.yml file of our inventory.
  • For now, the instances are not automatically terminated; the termination still happens manually by humans through the provider's web interface.

Instances on Request

Once someone asks for the creation of a bare VM instance (through our web dashboard or the CLI), the request goes through the gateway to reach Hydra. Hydra then creates the instance with a compute provider, if available, and provisions it. Our provisioning involves removing our SSH key for access and giving SSH access only to the requester. This means that we cannot reprovision such a VM instance after a successful provisioning attempt.

uml diagram

Note

Pay attention to Atlas which functions as our internal provider.