elasticluster.providers.gce¶
Cloud provider for the Google Compute Engine.
See <https://code.google.com/p/google-cloud-platform-samples/source/browse/python-client-library-example/gce.py?repo=compute> for reference.
-
elasticluster.providers.gce.
GCE_SCOPE
= 'https://www.googleapis.com/auth/compute'¶ the OAuth scope for the GCE web API
-
class
elasticluster.providers.gce.
GoogleCloudProvider
(gce_project_id, gce_client_id='', gce_client_secret='', email='default', network='default', noauth_local_webserver=False, zone='us-central1-a', storage_path=None)[source]¶ Cloud provider for the Google Compute Engine.
Parameters: - gce_client_id (str) – Client ID to use in OAuth authentication.
- gce_client_secret (str) – Client secret (password) to use in OAuth authentication.
- gce_project_id (str) – Project name to log in to GCE.
- noauth_local_webserver – do not pop up a webserver for auth
- zone (str) – gce zone, default is us-central1-a
- network (str) – network to use, default is default
- email (str) – service email to use, default is default
- storage_path (str) – path to store authentication data (oauth.dat file). If no path is specified, the login data has to be entered after every request.
-
IMAGE_NAME_SHORTCUTS
= {'backports-debian': 'debian-cloud', 'centos': 'centos-cloud', 'container-vm': 'google-containers', 'coreos': 'coreos-cloud', 'cos': 'cos-cloud', 'debian': 'debian-cloud', 'rhel': 'rhel-cloud', 'sles': 'suse-cloud', 'ubuntu': 'ubuntu-os-cloud'}¶ Map image names to projects, based on prefix.
The image names and full resource URLs for several Google- provided images (debian, centos, etc.) follow a consistent pattern, and so ElastiCluster supports a short-hand of just an image name, such as:
"debian-7-wheezy-v20150526
The cloud project in this case is then
debian-cloud
.
-
get_ips
(instance_id)[source]¶ Retrieves the ip addresses (public) from the cloud provider by the given instance id.
Parameters: instance_id (str) – id of the instance Returns: list (ips) Raises: InstanceError if the ip could not be retrieved.
-
is_instance_running
(instance_id)[source]¶ Check whether the instance is up and running.
Parameters: instance_id (str) – instance identifier Reutrn: True if instance is running, False otherwise
-
list_instances
(filter=None)[source]¶ List instances on GCE, optionally filtering the results.
Parameters: filter (str) – Filter specification; see https://developers.google.com/compute/docs/reference/latest/instances/list for details. Returns: list of instances
-
pause_instance
(instance_id)[source]¶ Pauses the instance, retaining disk and config.
Parameters: instance_id (str) – instance identifier Raises: InstanceError if instance cannot be paused Returns: dict - information needed to restart instance.
-
resume_instance
(paused_info)[source]¶ Restarts a paused instance, retaining disk and config.
Parameters: instance_id (str) – instance identifier Raises: InstanceError if instance cannot be resumed. Returns: dict - information needed to restart instance.
-
start_instance
(key_name, public_key_path, private_key_path, security_group, flavor, image_id, image_userdata, cluster_name, username=None, node_name=None, boot_disk_type='pd-standard', boot_disk_size=10, tags=None, scheduling=None, accelerator_count=0, accelerator_type='default', allow_project_ssh_keys=True, local_ssd_count=0, local_ssd_interface='SCSI', min_cpu_platform=None, **kwargs)[source]¶ Starts a new instance with the given properties and returns the instance id.
Parameters: - key_name (str) – name of the ssh key to connect
- public_key_path (str) – path to ssh public key
- private_key_path (str) – path to ssh private key
- security_group (str) – firewall rule definition to apply on the instance
- flavor (str) – machine type to use for the instance
- image_id (str) – image type (os) to use for the instance
- image_userdata (str) – command to execute after startup
- username (str) – username for the given ssh key, default None
- node_name (str) – name of the instance
- tags (str|Sequence) – “Tags” to label the instance. Can be either a single string (individual tags are comma-separated), or a sequence of strings (each string being a single tag).
- scheduling (str) – scheduling option to use for the instance (“preemptible”)
- accelerator_count (int) – Number of accelerators (e.g., GPUs) to make available in instance
- accelerator_type (str) –
Type of accelerator to request. Can be one of:
- Full URL specifying an accelerator type valid for the zone and project VMs are being created in. For example,
https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/acceleratorTypes/[ACCELERATOR_TYPE]
- An accelerator type name (any string which is not a valid URL). This is internally prefixed with the string
https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/acceleratorTypes/
to form a full URL.
- Full URL specifying an accelerator type valid for the zone and project VMs are being created in. For example,
- allow_project_ssh_keys (bool) – When
True
(default), SSH login is allowed to a node using any of the project-wide SSH keys (if they are defined). WhenFalse
, only the SSH key specified by ElastiCluster config’s[login/*]
section will be allowed to log in (instance-level key). - local_ssd_count (int) – Number of local SSD disks (each 375GB size) to make available in instance
- local_ssd_interface (int) – Attachment interface for local SSD disks; either
'SCSI'
(default) or'NVME'
. - min_cpu_platform (str) –
require CPUs of this type or better (e.g., “Intel Skylake”)
Only used if
accelerator_count
is > 0.
Returns: str - instance id of the started instance