Authenticating to Google Cloud as oneself within Ansible

In my day job at Cottage Labs, we're developing a Google Cloud deployment infrastructure for a client, e using Ansible for configuration management.

Ansible has a whole suite of modules for managing Google Cloud resources. Each of these takes the same parameters for authentication, but they don't have the best of explanations:

  • auth_kind — The type of credential used. (choices: application, machineaccount, serviceaccount)
  • service_account_contents — The contents of a Service Account JSON file, either in a dictionary or as a JSON string that represents it.
  • service_account_email — An optional service account email address if machineaccount is selected and the user does not wish to use the default email.
  • service_account_file — The path of a Service Account JSON file if serviceaccount is selected as type.
— Google Cloud Ansible collection documentation

Until now I'd been setting the GOOGLE_APPLICATION_CREDENTIALS environment variable within my playbooks, pointing at a fixed path containing the JSON-formatted key for a service account. This works fine when you have the Owner role within the Google Cloud project, but when you're an Editor you don't have the permissions to assign your Editor role to your service account. It also feels messy —when it's you deploying as yourself, in person, you should be authenticated as yourself. The service account is intended to be used by an automated process, and is not linked to any user.


So how do we authenticate as ourselves? The answer lies slightly hidden in the google-api-core auth documentation and more fully in the google.auth.default docstring.

First off, we ensure that GOOGLE_APPLICATION_CREDENTIALS is not set.

Next, we run:

$ gcloud auth application-default login
Setting application default credentials

This will pop up a browser window asking which of your Google accounts you want to authenticate as. On your terminal you'll then see that the gcloud tool has created a default credentials file:

[alex@misha your-project]$ gcloud auth application-default login
Your browser has been opened to visit:[redacted]&redirect_uri=http%3A%2F%2Flocalhost%3A8085%2F&[redacted]&access_type=offline&code_challenge=[redacted]&code_challenge_method=S256

Credentials saved to file: [/home/alex/.config/gcloud/application_default_credentials.json]

These credentials will be used by any library that requests Application Default Credentials (ADC).
Cannot find a quota project to add to ADC. You might receive a "quota exceeded" or "API not enabled" error. Run $ gcloud auth application-default set-quota-project to add a quota project.
Result of setting application default credentials

Finally, we choose the application auth_kind in our Ansible playbook tasks:

- hosts: all
  connection: local
  gather_facts: no

    - name: enable all the Google Cloud APIs we use
        name: '{{ item }}'
        project: '{{ project_id }}'
        auth_kind: application
        state: present

      - vars/main.yml
      - 'vars/environment/{{ environment_name }}.yml'

By mapping Google Cloud projects to Ansible hosts, …

    ansible_python_interpreter: /usr/bin/python3

      project_id: unflappable-penguin-123456
      environment_name: production

      project_id: screeching-owl-123456
      environment_name: qa

      project_id: mythical-unicorn-123456
      environment_name: dev
Ansible hosts file

… we can then deploy to multiple projects with ease:

# Select our Google account
$ gcloud auth application-default login
# Deploy to QA
$ ansible-playbook enable-services-playbook.yml --limit qa