Porting Modules to Python 3¶

Which version of Python-3.x and which version of Python-2.x are our minimums?¶

The short answer is Python-3.4 and Python-2.4 but please read on for more information.

For Python-3 we are currently using Python-3.4 as a minimum. However, no long term supported Linux distributions currently ship with Python-3. When that occurs, we will probably take that as our minimum Python-3 version rather than Python-3.4. Thus far, Python-3 has been adding small changes that make it more compatible with Python-2 in its newer versions (For instance, Python-3.5 added the ability to use percent-formatted byte strings.) so it should be more pleasant to use a newer version of Python-3 if it’s available. At some point this will change but we’ll just have to cross that bridge when we get to it.

For Python-2 the default is for modules to run on Python-2.4. This allows users with older distributions that are stuck on Python-2.4 to manage their machines. Modules are allowed to drop support for Python-2.4 when one of their dependent libraries require a higher version of python. This is not an invitation to add unnecessary dependent libraries in order to force your module to be usable only with a newer version of Python. Instead it is an acknowledgment that some libraries (for instance, boto3 and docker-py) will only function with newer Python.

Supporting only Python-2 or only Python-3¶

Sometimes a module’s dependent libraries only run on Python-2 or only run on Python-3. We do not yet have a strategy for these modules but we’ll need to come up with one. I see three possibilities:

1. We treat these libraries like any other libraries that may not be installed on the system. When we import them we check if the import was successful. If so, then we continue. If not we return an error about the library being missing. Users will have to find out that the library is unavailable on their version of Python either by searching for the library on their own or reading the requirements section in ansible-doc.
2. The shebang line is the only metadata that Ansible extracts from a module so we may end up using that to specify what we mean. Something like #!/usr/bin/python means the module will run on both Python-2 and Python-3, #!/usr/bin/python2 means the module will only run on Python-2, and #!/usr/bin/python3 means the module will only run on Python-3. Ansible’s code will need to be modified to accommodate this. For python2, if ansible_python2_interpreter is not set, it will have to fallback to `` ansible_python_interpreter`` and if that’s not set, fallback to /usr/bin/python. For python3, Ansible will have to first try ansible_python3_interpreter and then fallback to /usr/bin/python3 as normal.
3. We add a way for Ansible to retrieve metadata about modules. The metadata will include the version of Python that is required.

Methods 2 and 3 will both require that we modify modules or otherwise add this additional information somewhere. 2 needs only a little code changes in executor/module_common.py to parse. 3 will require a lot of work. This is probably not worthwhile if this is the only change but could be worthwhile if there’s other things as well. 1 requires that we port all modules to work with python3 syntax but only the code path to get to the library import being attempted and then a fail_json() being called because the libraries are unavailable needs to actually work.

Tips, tricks, and idioms to adopt¶

try:
    a = 2/0
except ValueError as e:
    module.fail_json(msg="Tried to divide by zero!")

from ansible.module_utils.pycompat24 import get_exception
[...]

try:
    a = 2/0
except ValueError:
    e = get_exception()
    module.fail_json(msg="Tried to divide by zero!")

# Can't use 0755 on Python-3 and can't use 0o755 on Python-2.4
EXECUTABLE_PERMS = int('0755', 8)

from ansible.module_utils import six

env:
  global:
    - PY3_EXCLUDE_LIST="cloud/amazon/cloudformation.py
      cloud/amazon/ec2_ami.py
      [...]
      utilities/logic/wait_for.py"