This README is mainly used a log / wiki of results and challenges encounted during the POC
Things to clean up: Did not had time to sort how to completly migrate from operator-sdh to kubebuilder. Most of the scaffolding is done using the operator-sdk but once the default files are created, the build process mainly relies on kubebuilder
Contains the main.go for the armada operator
Contains the golang definition of the CRDs. make generate will recreate the yaml definitions
of the CRDs that have to be provided to kubectl in order to deploy the new CRDs.
This current version of the operator uses "act" for shortname of ArmadaChart,
"acg" as shortname for ArmadaChartGroup and "amf" as shortname for ArmadaManifest.
The first version of the golang code has generated using tool such as "schema-generate" from the schema definition provided with airship-armada project.
Contains the bulk of the interfaces used by the armada controller.
To get the process doing, some of the code for ArmadaChart handling is coming from the operator-sdk helm-operator. That code is relying on tiller component which is gone for Helm3. Hence the three directory helm (Interface and Common code), helmv2 (tiller) and helmv3.
The golang package structure is different between helmv2 and helmv3. The Armada Operator will most likely ultimatly have support two branches. In order to delay that milestone, the golang code has been instrumentated with "v2" and "v3" tags which allows to compile either the helm v3 version of the operator or the helm v3 version.
Mainly contain the code for ArmadaChartGroup, ArmadaManifest as will ArmadaBackupLocation handling
Contains the controller and the "Reconcile" functions for ArmadaChart, ArmadaChartGroup and ArmadaManifest. There are currently three controllers (act-controller, acg-controller and amf-controller).
Upon change of the CRD golang definition, the yaml files have to be regenerated
Note 1: Don't understand yet how to build using operator-sdk operator with the same level of detailes than controller-gen. Big hack that have to be included in Makefile.
Note 2: The generation tool seems to comply with some of OpenAPI specs. The "validation" schema added to in the CRD yaml definition does not contain fields using underscore. Most of those fields containing underscore where defined such a way in the original airship-armada.
make generateTo keep the directory tree ligthweight, the vendor directory is not checked in in the current repo. TODO: Since the operator is only using one git branch, the developer has to comment out the helmv2 and add the helmv3 in Gopkg.toml if he wants to build the helmv3 version of the operator. This is still WIP.
dep ensureTo build the v2 version
make docker-build-v2To build the v3 version
make docker-build-v3If you installed kubebuilder on your system, you will have access to a standalone apiserver, etcd server and kubectl.
Because of a lack of time, the current makefile test statement, will attempt to stop your kubelet and associated container in your local kubernetes cluster, before starting apiserver, etcdserver. TODO: We still need to figure out if it necessary
In order to run the unit tests and the e2e integration tests:
make unittestNote the current deployment of the operator relies itself on helm.
To install the helm v2 version
make install-v2To install the helm v3 version
make install-v3For testing purpose the current Docker file includes a dummy chart deliverd under armada-charts. This removes the needs to access external chart repository which is also an aspect of helm changing from 2 to 3.
In that directory, the ArmadaChart are enabled by default and the charts are installed as soon as the ArmadaChart CRD are created.
Upon creation of the custom resource, the controller will
- Deploy the Armada Manifest described in the CRD
- Update status of the custom resources.
- Add events to the custom resources.
kubectl create -f examples/armada
kubectl describe amf/simple-armada
kubectl get amf
kubectl get acg
kubectl get actUpon deletion of its depending resources, the controller will recreate it,
kubectl delete deployment.apps/blog-2-testchart
kubectl get all
kubectl describe act blog-2When deleting the CRD, the corresponding Armada Manifest should be uninstalled.
kubectl delete -f simple/armadaThis directory contains invidual act,acg and amf files which allow the "step" by "step" testing of the deployment.
In that directory, the ArmadaChart (act) are disabled by default and the charts not installed automatically when the act CR are created. This example assumes that the argo controller has been installed. When the "argo worflow" CR is created, the "argo controller" is waked up and it orchestrates the enablement of the ArmadaChart according to the worflow.
Note: You need to have argo installed in your cluster.
kubectl apply -f example/argo
armadachart.armada.airshipit.org/blog-1 created
armadachart.armada.airshipit.org/blog-2 created
workflow.argoproj.io/wf-blog-group createdThe first ArmadaChart is installed:
kubectl get all
pod/armada-operator-cbbc7d7f7-zxj5n 1/1 Running 0 60s
pod/blog-1-testchart-5dd8c474f4-26574 0/1 ContainerCreating 0 6s
pod/wf-blog-group-1193326311 0/1 Completed 0 8s
pod/wf-blog-group-2026876860 0/1 ContainerCreating 0 1s
pod/wf-blog-group-2432013970 0/1 Completed 0 4s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/armada-operator ClusterIP 10.98.2.253 <none> 8383/TCP 57s
service/blog-1-testchart ClusterIP 10.104.240.7 <none> 80/TCP 6s
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7m43s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/armada-operator 1/1 1 1 60s
deployment.apps/blog-1-testchart 0/1 1 0 6s
NAME DESIRED CURRENT READY AGE
replicaset.apps/armada-operator-cbbc7d7f7 1 1 1 60s
replicaset.apps/blog-1-testchart-5dd8c474f4 1 1 0 6sLater both charts have been installed
NAME READY STATUS RESTARTS AGE
pod/armada-operator-cbbc7d7f7-zxj5n 1/1 Running 0 44m
pod/blog-1-testchart-5dd8c474f4-26574 1/1 Running 0 43m
pod/blog-2-testchart-57f86dd9c5-xmhd7 1/1 Running 0 43m
pod/wf-blog-group-1193326311 0/1 Completed 0 43m
pod/wf-blog-group-2026876860 0/1 Completed 0 43m
pod/wf-blog-group-2234690393 0/1 Completed 0 43m
pod/wf-blog-group-2432013970 0/1 Completed 0 43m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/armada-operator ClusterIP 10.98.2.253 <none> 8383/TCP 44m
service/blog-1-testchart ClusterIP 10.104.240.7 <none> 80/TCP 43m
service/blog-2-testchart ClusterIP 10.110.160.242 <none> 80/TCP 43m
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 51m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/armada-operator 1/1 1 1 44m
deployment.apps/blog-1-testchart 1/1 1 1 43m
deployment.apps/blog-2-testchart 1/1 1 1 43m
NAME DESIRED CURRENT READY AGE
replicaset.apps/armada-operator-cbbc7d7f7 1 1 1 44m
replicaset.apps/blog-1-testchart-5dd8c474f4 1 1 1 43m
replicaset.apps/blog-2-testchart-57f86dd9c5 1 1 1 43margo is tracing the steps take to sequence the deployment of the charts:
argo get wf-blog-group
Name: wf-blog-group
Namespace: default
ServiceAccount: armada-argo-sa
Status: Succeeded
Created: Fri Mar 22 10:27:28 -0500 (26 seconds ago)
Started: Fri Mar 22 10:27:28 -0500 (26 seconds ago)
Finished: Fri Mar 22 10:27:42 -0500 (12 seconds ago)
Duration: 14 seconds
STEP PODNAME DURATION MESSAGE
✔ wf-blog-group
├---✔ enable-blog-1 wf-blog-group-1193326311 2s
├---✔ blog-1-ready wf-blog-group-2432013970 3s
├---✔ enable-blog-2 wf-blog-group-2026876860 3s
└---✔ blog-2-ready wf-blog-group-2234690393 3sWe can check the state of the ArmadaChart
kubectl get act
NAME STATE TARGET STATE SATISFIED
blog-1 deployed deployed true
blog-2 deployed deployed trueRun the cleanup
kubectl delete -f examples/argoIn that directory, the ArmadaChart (act) are disabled by default and the charts not installed automatically when the act CR are created. When the "ArmadaChartGroup" CR is created, the "chartgroup controller" receives an event and it orchestrate the order of deployment/enablement of the ArmadaChart. The ArmadaChartGroup also becomes owner of the ArmadaChart.
This is basically the same sequencing that above except that it is implemented using an ArmadaChartGroup and an ArmadaManifest
This directory contains the CR definitions involved during an ArmadaBackup procedure.
- WIP
This directory contains the CR definitions involved during an ArmadaRestore procedure.
- WIP
POCs contains additional notes regarding successful and failed attempts.