Debugging might not be one of your favorite things to do, but when your automation fails it is good to know where to look to find information and troubleshoot. In this blog post, we investigate how to make sure Ansible Automation is correctly configured inside CloudForms, and how to troubleshoot issues that might occur when running Ansible Automation. Content for this blog post is based on the knowledge base article published on Red Hat Customer Portal.
Before you start, make sure Ansible Automation is correctly configured and running
First of all, before you start performing a deep debugging, make sure these steps are in place:
- Embedded Ansible Role is enabled. Note that only one appliance per Region running Ansible Role is supported today.
- Internet connectivity is available for the appliance with the Embedded Ansible Role before the role is enabled.
- Ansible Worker is properly running (you can check its status under Configuration > Diagnostics).
- You can restart the Embedded Ansible process by executing ansible-tower-service restart from the appliance command line.
If the role is not properly configured and running, you will not be able to add playbooks or credentials.
Ok, Ansible Automation is up and running, but our playbook launches are not successful
Well, in that case, you need to dig a bit more to figure out what is wrong:
- Verify, under Services > Request that the playbook is executed. Every playbook is executed as a service and status can be tracked here.
- If the playbook does not execute as expected, check that the version of the playbook is appropriate and that you do not have further dependencies (like ansible roles or modules requirements).
- Re-sync the repository from the CloudForms UI under Automation > Ansible > Repositories. On the appliance, playbooks can be found under /var/lib/awx/projects/.
- Check the standard output of the service to verify the execution of the playbook itself. This can be found under Services > My Services > Selected Services > Provisioning > Standard Output.
- Make sure all required playbook variables are provided to the playbook for execution. CloudForms can automatically create service dialogs while defining a new service to provide these variables. The dialog element name must match the playbook variable.
- The execution of the playbook in real time can be found in a file for each new run in the /var/lib/awx/job_status directory. Note that the name of the file is a hash and has nothing to do with the playbook name. This is currently the only way to track the status of a run when executing the playbook as a policy or alert action.
Where else can I find additional information?
If all the above steps are validated and you still have trouble, a good place to continue troubleshooting is to look at the CloudForms standard log files under /var/www/miq/vmdb on the appliance:
- Production.log – Operations UI & Service UI
- Automation.log – Service Ordering, Automation
- Policy.log – Events, Policy
- Evm.log – Everything else