Skip to content

Commit 59d3376

Browse files
ccremerccremer
authored
Merge pull request #3 from projectsyn/tutorial-test
Add tutorial for writing Component tests
2 parents d398771 + 5c6ed94 commit 59d3376

4 files changed

Lines changed: 222 additions & 0 deletions

File tree

‎Makefile‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,16 +6,20 @@ docker_opts ?= --rm --tty --user "$$(id -u)"
66
asciidoctor_pdf_cmd ?= $(docker_cmd) run $(docker_opts) --volume "$${PWD}":/documents/ vshn/asciidoctor-pdf:1.4
77
asciidoctor_opts ?= --destination-dir=$(out_dir)
88

9+
.PHONY: all
910
all: pdf
1011

12+
.PHONY: pdf
1113
pdf: build/tutorial.pdf
1214

1315
build/tutorial.pdf: tutorial.adoc
1416
$(asciidoctor_pdf_cmd) $(asciidoctor_opts) $<
1517

18+
.PHONY: clean
1619
clean:
1720
rm -rf build
1821

22+
.PHONY: setup
1923
setup:
2024
./0_requirements.sh; \
2125
./1_lieutenant_on_minikube.sh; \

‎docs/modules/ROOT/nav.adoc‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
.Tutorial
22
* xref:index.adoc[Home]
3+
* xref:testing.adoc[Tests]

‎docs/modules/ROOT/pages/testing.adoc‎

Lines changed: 216 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,216 @@
1+
= Tutorial: Writing Commodore Component Tests
2+
3+
This tutorial covers the topic of writing tests for your new or existing Commodore Component.
4+
It assumes that you are familiar with writing Commodore Components.
5+
If not, see link:index.adoc[Writing your First Commodore Component].
6+
7+
Currently, we can test components with two approaches:
8+
9+
. Unit tests with Go.
10+
Easy to understand and write if you are already a Go developer.
11+
. Policy tests with Conftest.
12+
Uses the https://www.openpolicyagent.org/docs/latest/policy-language/[Rego] syntax from https://www.openpolicyagent.org/[OpenPolicyAgent].
13+
14+
It is up to you to decide which test framework you want to use.
15+
Some tests are simpler to do in Go, some are simpler in Rego.
16+
A combination of both will combine their advantages.
17+
18+
NOTE: The policy tests run with the Conftest tool, but for the purpose of this tutorial we will refer to the Rego language, as the policies are written in that syntax.
19+
20+
== Requirements
21+
22+
NOTE: This tutorial *was written on a Linux system*.
23+
24+
. `Go` version 1.15, developer environment with Go modules enabled.
25+
. `docker` version 19
26+
27+
== Setting up test infrastructure with Go
28+
29+
We'll start with Go.
30+
Create the following directory structure:
31+
[source,console]
32+
----
33+
.
34+
├── tests
35+
│   ├── test.yml
36+
│   └── unit
37+
│   ├── defaults_test.go
38+
│   ├── go.mod
39+
│   └── go.sum
40+
----
41+
The `go.mod` and `go.sum` files are created when executing `go mod init` inside `test/unit/`.
42+
Since we are only creating test code and not an actual Go binary, all Go test files have to end with `_test.go`.
43+
`tests/test.yml` is sometimes used by components to override values that would only be needed by Commodore when compiling whole catalogs, you can leave it empty for now.
44+
We will now start writing the first tests in `defaults_test.go`.
45+
46+
== Writing unit tests with Go
47+
48+
If you are already a Go developer, these should look fairly familiar to you.
49+
We will showcase the tests with the Espejo component.
50+
If you have `component-somename`, then leave out `component-`.
51+
52+
[source,go]
53+
----
54+
package main
55+
56+
import (
57+
"github.com/stretchr/testify/assert"
58+
"github.com/stretchr/testify/require"
59+
"testing"
60+
)
61+
62+
var (
63+
testPath = "../../compiled/espejo/espejo"
64+
)
65+
66+
func Test_Deployment_DefaultParameters(t *testing.T) {
67+
68+
subject := DecodeDeployment(t, testPath+"/10_deployment.yaml")
69+
require.NotEmpty(t, subject.Spec.Template.Spec.Containers)
70+
container := subject.Spec.Template.Spec.Containers[0]
71+
72+
assert.Equal(t, "espejo", container.Name)
73+
assert.Contains(t, container.Args, "--verbose=false")
74+
assert.Contains(t, container.Args, "--reconcile-interval=10m")
75+
assert.Contains(t, container.Args, "--metrics-addr=:8080")
76+
assert.Contains(t, container.Args, "--enable-leader-election=true")
77+
78+
require.NotEmpty(t, container.Env)
79+
env := container.Env[0]
80+
assert.Equal(t, "WATCH_NAMESPACE", env.Name)
81+
assert.Equal(t, "metadata.namespace", env.ValueFrom.FieldRef.FieldPath)
82+
}
83+
84+
func Test_Namespace(t *testing.T) {
85+
86+
subject := DecodeNamespace(t, testPath+"/01_namespace.yaml")
87+
88+
assert.Equal(t, "syn-espejo", subject.Name)
89+
assert.Contains(t, subject.Labels, "name")
90+
}
91+
92+
----
93+
94+
CAUTION: We have not yet built a library to host the boilerplate code and common functions.
95+
96+
As you can see, it's pretty straight forward:
97+
98+
. First, load the pre-compiled YAML file into a Go K8s struct that we all know and love
99+
. Then, we verify if the values were parsed correctly, using any assertion library of your choice.
100+
101+
To actually run our unit test case, we need to run a Commodore Component compilation first:
102+
[source,bash]
103+
----
104+
COMPONENT_NAME=$(basename ${PWD} | sed s/component-//)
105+
DOCKER_CMD() {docker run --rm --user "$(id -u)" -v "${PWD}:/${COMPONENT_NAME}" --workdir /${COMPONENT_NAME} $*}
106+
DOCKER_CMD --entrypoint /usr/local/bin/jb projectsyn/commodore:latest install
107+
DOCKER_CMD projectsyn/commodore:latest component compile . -f tests/test.yml
108+
----
109+
110+
Running the tests could look like this:
111+
[source,bash]
112+
----
113+
$ pushd tests/unit > /dev/null && go test -v ./... && popd > /dev/null
114+
=== RUN Test_Deployment_DefaultParameters
115+
--- PASS: Test_Deployment_DefaultParameters (0.01s)
116+
=== RUN Test_Namespace
117+
--- PASS: Test_Namespace (0.00s)
118+
PASS
119+
ok github.com/projectsyn/component-espejo
120+
----
121+
122+
== Writing policy tests with Rego
123+
124+
Some tests are easier to write in Rego than Go unit tests.
125+
Consider the following use case:
126+
We want to ensure that all generated manifests have a certain label.
127+
128+
With Go unit tests, we would have to
129+
130+
. Recursively parse all YAML files
131+
. Decode the YAML files into generic objects, so that we can access `.metadata.labels`
132+
. Assert that the desired label is there.
133+
134+
With Rego, this particular test is relatively easy:
135+
[source,rego]
136+
----
137+
recommended_labels {
138+
input.metadata.labels["app.kubernetes.io/managed-by"]
139+
}
140+
141+
warn_labels[msg] {
142+
input.kind != "CustomResourceDefinition"
143+
not recommended_labels
144+
145+
msg = sprintf("%s/%s has not recommended labels", [input.kind, name])
146+
}
147+
----
148+
149+
Let's break down the structure:
150+
151+
. `recommended_labels` is an object that verifies that `.metadata.labels` contain the desired label keys.
152+
. `warn_labels[msg]` is a Rule.
153+
If all expressions in the brackets match (including `msg`), this Rule is considered `true`.
154+
. Since the prefix of the rule is `warn_`, it will only print a Warning message if there is an object that matches the rule.
155+
With `deny_`, it would fail the test.
156+
157+
IMPORTANT: Rego (like Datalog and its ancestor Prolog) is declarative.
158+
The lines within a rule are not evaluated imperatively.
159+
It is important to keep that in mind when writing rules, as it can cause many headaches.
160+
161+
Let's translate the example to English:
162+
163+
. In `recommended_labels`, we will test whether the Kubernetes object (named `input`) contains "app.kubernetes.io/managed-by" in the `.metadata.labels` dictionary.
164+
We ignore the actual value here.
165+
Since `recommended_labels` is not a rule, it's not yet used.
166+
. When conftest matches an Object against the rule `warn_labels`, all expressions in the rule have to evaluate `True`.
167+
. If we pass a CRD, the result of the rule is `False` because of `input.kind != "CustomResourceDefinition"`, thus the rule does not match, and the test passes.
168+
. If we pass a `Deployment`, we have at least `input.kind != "CustomResourceDefinition"` that equals to `True`, but remember, all expressions have to be evaluated.
169+
. The other expression, `not recommended_labels` checks if the object is missing the desired labels.
170+
If the given Deployment has the labels, it will fail the rule and pass the test.
171+
A Deployment that doesn't have the labels would match the rule, and thus fail the test.
172+
. By now the rule would already match with a Deployment without the labels, and thus fail the test, but we want to give a reason why.
173+
As the final expression, we will assign the `msg` variable a human readable message why the rule matches.
174+
Remember, this line can also be the first one since the execution order is determined by Rego and not line by line.
175+
176+
If we now also pass a `Namespace` or `Service` objects, the same rules can be applied, since all these objects share the common property `.metadata.labels`.
177+
178+
179+
If we want to check whether a Namespace has the correct name, this could look like this:
180+
[source, rego]
181+
----
182+
deny_namespace[msg] {
183+
input.kind = "Namespace"
184+
ns := "syn-espejo"
185+
not input.metadata.name = ns
186+
187+
msg = sprintf("Namespace is not %s", [ns])
188+
}
189+
----
190+
In this example, we are using the variable `ns` to not repeat ourselves.
191+
The expression `not input.metadata.name = "syn-espejo"` is equivalent, but we want to reduce code duplication in the `msg` expression.
192+
193+
Running the policies could look like this:
194+
[source,bash]
195+
----
196+
$ DOCKER_CMD --volume "${PWD}/tests/policies:/policy" openpolicyagent/conftest:latest test --policy /policy $(find . -type f -wholename "./compiled/${COMPONENT_NAME}/*.yaml")
197+
WARN - ./compiled/espejo/espejo/05_rbac.yaml - ClusterRole/syn-espejo has not recommended labels
198+
WARN - ./compiled/espejo/espejo/05_rbac.yaml - ServiceAccount/espejo has not recommended labels
199+
WARN - ./compiled/espejo/espejo/05_rbac.yaml - ClusterRoleBinding/syn-espejo has not recommended labels
200+
WARN - ./compiled/espejo/espejo/01_namespace.yaml - Namespace/syn-espejo has not recommended labels
201+
202+
14 tests, 10 passed, 4 warnings, 0 failures, 0 exceptions
203+
----
204+
205+
== Run all tests
206+
207+
You could declare all the test commands in the `Makefile`.
208+
Have a look at https://github.com/projectsyn/component-espejo/blob/master/Makefile[Component-Espejo] for an example.
209+
This should also help running tests in any CI/CD pipelines, such as GitHub Actions.
210+
211+
== Conclusion
212+
213+
I hope this guide has shown how we can test our component without having to compile a whole catalog and applying it to a cluster.
214+
215+
At the moment, we are limited to only have tests against a single compilation (e.g. the default parameters).
216+
Later on, we want to enable testing different parameter sets.

‎tutorial.adoc‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,3 +13,4 @@
1313
:examplesdir: /documents/docs/modules/ROOT/examples/
1414

1515
include::docs/modules/ROOT/pages/index.adoc[]
16+
include::docs/modules/ROOT/pages/testing.adoc[]

0 commit comments

Comments
 (0)