elasticluster.utils

class elasticluster.utils.Struct(initializer=None, **extra)[source]

A dict-like object, whose keys can be accessed with the usual ‘[…]’ lookup syntax, or with the ‘.’ get attribute syntax.

Examples::
>>> a = Struct()
>>> a['x'] = 1
>>> a.x
1
>>> a.y = 2
>>> a['y']
2

Values can also be initially set by specifying them as keyword arguments to the constructor:

>>> a = Struct(z=3)
>>> a['z']
3
>>> a.z
3

Like dict instances, Struct`s have a `copy method to get a shallow copy of the instance:

>>> b = a.copy()
>>> b.z
3

Note

This class is a clone of the gc3libs.utils.Struct class from the GC3Pie package sources

copy()[source]

Return a (shallow) copy of this Struct instance.

keys() → a set-like object providing a view on D's keys[source]
elasticluster.utils.confirm_or_abort(prompt, exitcode=75, msg=None, **extra_args)[source]

Prompt user for confirmation and exit on negative reply.

Arguments prompt and extra_args will be passed unchanged to click.confirm() (which is used for actual prompting).

Parameters:
  • prompt (str) – Prompt string to display.
  • exitcode (int) – Program exit code if negative reply given.
  • msg (str) – Message to display before exiting.
elasticluster.utils.environment(**kv)[source]

Context manager to run Python code with a modified UNIX process environment.

All key/value pairs in the keyword arguments are added (or changed, if the key names an existing environmental variable) in the process environment upon entrance into the context. Changes are undone upon exit: added environmental variables are removed from the environment, and those whose value was changed are reset to their pristine value.

elasticluster.utils.expand_ssh_proxy_command(command, user, addr, port=22)[source]

Expand spacial digraphs %h, %p, and %r.

Return a copy of command with the following string substitutions applied:

  • %h is replaced by addr
  • %p is replaced by port
  • %r is replaced by user
  • %% is replaced by %.

See also: man page ssh_config, section “TOKENS”.

elasticluster.utils.fingerprint_str(key)[source]

Return a printable string representation of an SSH key fingerprint.

elasticluster.utils.format_warning_oneline(message, category, filename, lineno, file=None, line=None)[source]

Format a warning for logging.

The returned value should be a single-line string, for better logging style (although this is not enforced by the code).

This methods’ arguments have the same meaning of the like-named arguments from warnings.formatwarning.

elasticluster.utils.get_num_processors()[source]

Return number of online processor cores.

elasticluster.utils.has_nested_keys(mapping, k1, *more)[source]

Return True if mapping[k1][k2]…[kN] is valid.

Example:

>>> D = {
...   'a': {
...     'x':0,
...     'y':{
...       'z': 1,
...     },
...   },
...   'b': 3
... }
>>> has_nested_keys(D, 'a', 'x')
True
>>> has_nested_keys(D, 'a', 'y', 'z')
True
>>> has_nested_keys(D, 'a', 'q')
False

When a single key is passed, this is just another way of writing k1 in mapping:

>>> has_nested_keys(D, 'b')
True
class elasticluster.utils.memoize(ttl)[source]

Cache a function’s return value each time it is called within a TTL.

If called within the TTL and the same arguments, the cached value is returned, If called outside the TTL or a different value, a fresh value is returned (and cached for future occurrences).

Warning

Only works on functions that take no keyword arguments.

Originally taken from: http://jonebird.com/2012/02/07/python-memoize-decorator-with-ttl-argument/

elasticluster.utils.parse_ip_address_and_port(addr, default_port=22)[source]

Return a pair (IP address, port) extracted from string addr.

Different formats are accepted for the address/port string:

  • IPv6 literals in square brackets, with or without an optional port specification, as used in URLs:

    >>> parse_ip_address_and_port('[fe80::dead:beef]:1234')
    (IPAddress('fe80::dead:beef'), 1234)
    
    >>> parse_ip_address_and_port('[fe80::dead:beef]')
    (IPAddress('fe80::dead:beef'), 22)
    
  • IPv6 literals with a “local interface” specification:

    >>> parse_ip_address_and_port('[fe80::dead:beef%eth0]')
    (IPAddress('fe80::dead:beef'), 22)
    
    >>> parse_ip_address_and_port('fe80::dead:beef%eth0')
    (IPAddress('fe80::dead:beef'), 22)
    
  • bare IPv6 addresses:

    >>> parse_ip_address_and_port('fe80::dead:beef')
    (IPAddress('fe80::dead:beef'), 22)
    
    >>> parse_ip_address_and_port('2001:db8:5ca1:1f0:f816:3eff:fe05:f40f')
    (IPAddress('2001:db8:5ca1:1f0:f816:3eff:fe05:f40f'), 22)
    
  • IPv4 addresses, with or without an additional port specification:

    >>> parse_ip_address_and_port('192.0.2.123')
    (IPAddress('192.0.2.123'), 22)
    
    >>> parse_ip_address_and_port('192.0.2.123:999')
    (IPAddress('192.0.2.123'), 999)
    

Note that the default port can be changed by passing an additional parameter:

>>> parse_ip_address_and_port('192.0.2.123', 987)
(IPAddress('192.0.2.123'), 987)

>>> parse_ip_address_and_port('fe80::dead:beef', 987)
(IPAddress('fe80::dead:beef'), 987)
Raises:netaddr.AddrFormatError – Upon parse failure, e.g., syntactically incorrect IP address.
elasticluster.utils.redirect_warnings(capture=True, logger='py.warnings')[source]

If capture is true, redirect all warnings to the logging package. If capture is False, ensure that warnings are not redirected to logging but to their original destinations.

elasticluster.utils.setitem_nested(mapping, path, value, cls=None)[source]

Set a leaf key to a value in a nested mapping.

Argument path is a list of keys: setitem_nested will recursively descend mapping and set the last key in path to the given value.

Example:

>>> D = { 'a': 1, 'b': {'c': {'d': 2}}}
>>> setitem_nested(D, ['b', 'c', 'd'], 3)

{‘a’: 1, ‘b’: {‘c’: {‘d’: 3}}} >>> assert D[‘b’][‘c’][‘d’] == 3 >>> assert D[‘a’] == 1

Argument value can be any valid Python value:

>>> setitem_nested(D, ['b', 'c', 'd'], [1, 2])

{‘a’: 1, ‘b’: {‘c’: {‘d’: [1, 2]}}} >>> assert D[‘b’][‘c’][‘d’] == [1, 2]

Argument path need not lead to a leaf key; any nested key can be overwritten:

>>> setitem_nested(D, ['b', 'c'], {'e': 4})

{‘a’: 1, ‘b’: {‘c’: {‘e’: 4}}} >>> assert D[‘b’][‘c’][‘e’] == 4

As a degenerate case, if path is a list of one element, setitem_nested works exactly like dictionary key assignment:

>>> setitem_nested(D, ['a'], {'f': 5})

{‘a’: {‘f’: 5}, ‘b’: {‘c’: {‘e’: 4}}} >>> assert D[‘a’][‘f’] == 5

If, at any point during the descent (except at the end of path), a non-existing key is found, it will be bound to a new mapping of type cls (default: the same class as first argument mapping itself):

>>> setitem_nested(D, ['a', 'g', 'h'], 6)

{‘a’: {‘g’: {‘h’: 6}, ‘f’: 5}, ‘b’: {‘c’: {‘e’: 4}}} >>> assert D[‘a’][‘g’] == {‘h’: 6}

The type of such constructed mappings can be set with optional last argument cls.

elasticluster.utils.sighandler(signum, handler)[source]

Context manager to run code with UNIX signal signum bound to handler.

The existing handler is saved upon entering the context and restored upon exit.

The handler argument may be anything that can be passed to Python’s signal.signal standard library call.

elasticluster.utils.string_to_boolean(word)[source]

Convert word to a Python boolean value and return it. The strings true, yes, on, 1 (with any capitalization and any amount of leading and trailing spaces) are recognized as meaning Python True:

>>> string_to_boolean('yes')
True
>>> string_to_boolean('Yes')
True
>>> string_to_boolean('YES')
True
>>> string_to_boolean(' 1 ')
True
>>> string_to_boolean('True')
True
>>> string_to_boolean('on')
True

Any other word is considered as boolean False:

>>> string_to_boolean('no')
False
>>> string_to_boolean('No')
False
>>> string_to_boolean('Nay!')
False
>>> string_to_boolean('woo-hoo')
False

This includes also the empty string and whitespace-only:

>>> string_to_boolean('')
False
>>> string_to_boolean('  ')
False
elasticluster.utils.temporary_dir(delete=True, dir=None, prefix='elasticluster.', suffix='.d')[source]

Make a temporary directory and make it current for the code in this context.

Delete temporary directory upon exit from the context, unless delete=False is passed in the arguments.

Arguments suffix, prefix and dir are exactly as in tempfile.mkdtemp() (but have different defaults).

elasticluster.utils.timeout(delay, handler=None)[source]

Context manager to run code and deliver a SIGALRM signal after delay seconds.

Note that delay must be a whole number; otherwise it is converted to an integer by Python’s int() built-in function. For floating-point numbers, that means rounding off to the nearest integer from below.

If the optional argument handler is supplied, it must be a callable that is invoked if the alarm triggers while the code is still running. If no handler is provided (default), then a RuntimeError with message Timeout is raised.