azure-packer-alternative

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

azure-packer-alternative

(★ 5)

Azure RM template that creates VM template and Azure DevOps pipeline that does deployment

azure-packer-alternative Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

| Build status  | License |
| ------------- | ------------- |
| [![Build Status](https://dev.azure.com/kagarlickij/azure-packer-alternative/_apis/build/status/azure-packer-alternative-ci?branchName=master)](https://dev.azure.com/kagarlickij/azure-packer-alternative/_build/latest?definitionId=34&branchName=master) | [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://raw.githubusercontent.com/kagarlickij/azure-packer-alternative/master/LICENSE.md)  |

# Task
Task is to create 3 Azure VM images for legacy application installed and configured by Ansible

## Temporary virtual machine (used to capture image from):
1. Must be placed in pre-defined Azure VNet's Subnet (instead of temporary VNet and Subnet created by Packer by default)
2. Must have Private IP only (by default Packer creates Public IPs)
3. Must be set by Ansible (for demo purposes Ansible creates `C:\build_config.txt`) using variables from:
  * `var1` from Azure DevOps Variable group `ansible-build-common` as regular variable
  * `var2` from Azure DevOps Variable group `ansible-build-common` as secret variable
  * `var3` from Azure DevOps Variable group `ansible-build-vm*` as regular variable
  * `var4` from Azure DevOps Variable group `ansible-build-vm*` as secret variable

## Production (resulting) virtual machine:
1. Must be placed in pre-defined Azure VNet's Subnet
2. Must have Private IP only
3. Must be set by Ansible (for demo purposes Ansible creates `C:\deploy_config.txt`) using variables from:
  * `var1` from Azure DevOps Variable group `ansible-deploy-common` as regular variable
  * `var2` from Azure DevOps Variable group `ansible-deploy-common` as secret variable
  * `var3` from Azure DevOps Variable group `ansible-deploy-vm*` as regular variable
  * `var4` from Azure DevOps Variable group `ansible-deploy-vm*` as secret variable
  * `var5` **for VM2 and VM3 only:** IP address of VM1 from Azure DevOps pipeline
  * `var6` **for VM3 only:** IP address of VM2 from Azure DevOps pipeline

## Azure Resources
![azure-resources](https://raw.githubusercontent.com/kagarlickij/azure-packer-alternative/master/azure-resources.png)

# Challenges
## Ansible variables
As you might noticed above `ansible-build-common` and `ansible-deploy-common` have the same variable names as required by legacy app  
..and all VMs have the same variable names in `ansible-deploy-vm1`, `ansible-deploy-vm2` and `ansible-deploy-vm3`  
This issue is solved by linking Azure DevOps Variable groups to specific stages and jobs in pipeline instead of the whole pipeline's level

## Issues with Packer
## 1. Azure DevOps Service Connection for Azure doesn't work with Packer
Solution is to use "Azure Active Directory Service Principal" which is less safe and convenient than Azure DevOps Service connection  
## 2. Packer's Ansible connection plugin fails for Windows
https://stackoverflow.com/questions/59599834/packers-ansible-connection-plugin-fails-for-windows  
## 3. Packer's {{ .WinRMPassword }} var is empty for azure-arm builder
https://stackoverflow.com/questions/59624603/packers-winrmpassword-var-is-empty-for-azure-arm-builder  
## 4. Multiple builders in Packer template don't work
https://stackoverflow.com/questions/59864732/packer-dedicated-provisioners-for-builders  
## 5. Build the same template on multiple agents (VMs or Docker containers) in parallel always fails
https://stackoverflow.com/questions/59864317/packer-azure-arm-fails-when-running-in-parallel  
## 6. My favorite one: Packer is too Flaky
https://stackoverflow.com/questions/59990155/packer-vs-vm-on-azure-timeout-waiting-for-winrm

# Packer replacement
Solution is to use Azure CLI to replace what Packer does, e.g. CReate VM, Setup WinRM, Run Ansible, Capture image, Terminate tmp resources  
![ci-cd-job-templates](https://raw.githubusercontent.com/kagarlickij/azure-packer-alternative/master/ci-cd-job-templates.png)

# Azure DevOps pipeline
![ci-cd-stages](https://raw.githubusercontent.com/kagarlickij/azure-packer-alternative/master/ci-cd-stages.png)  
## Production vs Pull request verification
Pipeline supports Pull requests verification (on Pull request to `master` branch) and VM image build and deploy (on commit to `master` branch)  
Pull request verification (PRV) builds and deploys image to temporary environment:  
If PRV verification is successful, code snapshot is marked as valid for merging and tmp environment is deleted  
If PRV verification failed code snapshot is marked as invalid and tmp environment is not deleted for manual investigation  
## Agent and network
Azure DevOps agent must have access to the virtual network that is used for VM image build  
Azure DevOps agents are executed in Docker [kagarlickij/packer-ansible-azure-docker-runtime:3.1.0](https://hub.docker.com/repository/registry-1.docker.io/kagarlickij/packer-ansible-azure-docker-runtime/builds/31d492d9-4d3b-4366-9add-d837c1d757d6) runtime with preinstalled Packer, Ansible, Azure CLI and necessary Python packages  
Azure DevOps agents can be started on VM on system startup: `@reboot /root/ados-agents-start.sh` in `crontab -e`  
`ados-agents-start.sh` script:
```
#!/bin/bash

for run in {1..5}
do
  docker run -d -e VSTS_ACCOUNT='kagarlickij' -e VSTS_POOL='Self-Hosted-Containers' -e VSTS_TOKEN='a***q' kagarlickij/packer-ansible-azure-docker-runtime:3.1.0 > /dev/null 2>&1
done
```

## Templates
Pipeline has some repeating steps which are moved to `./templates` to avoid code duplication  
Templates are not plugins so some pieces (e.g network settings) are "hardcoded"  
Template types are Job (for deploy) and Step (for all other) not because of the most suitable kind but because [templates can not be used as dependency for other jobs/stages/etc](https://stackoverflow.com/questions/59937679/azure-devops-template-as-a-dependency-for-a-job?noredirect=1#comment105997940_59937679)

## Deploy approval
Environment in ADOS Pipelines will be created automatically by Pipeline  
[To Set Deploy approval open `Approvals and checks` params of Environment and add Approval](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/approvals?view=azure-devops&tabs=check-pass)  
[Deploy notifications don't work in ADOS yet](https://stackoverflow.com/questions/59702813/azure-devops-doesnt-send-deployment-approval/)

## ansible-build-common
| Name | Value | Type | Comment |
|--|--|--|--|
| var1 | `sample-value-of-ansible-build-common-var1` | Regular | Just a sandbox example |
| var2 | `********` | Secret | Just a sandbox example |

## ansible-build-vm1
| Name | Value | Type | Comment |
|--|--|--|--|
| var3 | `sample-value-of-ansible-build-vm1-var3` | Regular | Just a sandbox example |
| var4 | `********` | Secret | Just a sandbox example |

## ansible-build-vm2
| Name | Value | Type | Comment |
|--|--|--|--|
| var3 | `sample-value-of-ansible-build-vm2-var3` | Regular | Just a sandbox example |
| var4 | `********` | Secret | Just a sandbox example |

## ansible-build-vm3
| Name | Value | Type | Comment |
|--|--|--|--|
| var3 | `sample-value-of-ansible-build-vm3-var3` | Regular | Just a sandbox example |
| var4 | `********` | Secret | Just a sandbox example |

## ansible-deploy-common
| Name | Value | Type | Comment |
|--|--|--|--|
| var1 | `sample-value-of-ansible-deploy-common-var1` | Regular | Just a sandbox example |
| var2 | `********` | Secret | Just a sandbox example |

## ansible-deploy-vm1
| Name | Value | Type | Comment |
|--|--|--|--|
| var3 | `sample-value-of-ansible-deploy-vm1-var3` | Regular | Just a sandbox example |
| var4 | `********` | Secret | Just a sandbox example |

## ansible-deploy-vm2
| Name | Value | Type | Comment |
|--|--|--|--|
| var3 | `sample-value-of-ansible-deploy-vm2-var3` | Regular | Just a sandbox example |
| var4 | `********` | Secret | Just a sandbox example |

## ansible-deploy-vm3
| Name | Value | Type | Comment |
|--|--|--|--|
| var3 | `sample-value-of-ansible-deploy-vm3-var3` | Regular | Just a sandbox example |
| var4 | `********` | Secret | Just a sandbox example |

azure-packer-alternative

azure-packer-alternative Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

azure-packer-alternative

azure-packer-alternative Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README