Agent Instructions for provider-keycloak

This page collects context and instructions for AI coding agents (GitHub Copilot, Cursor, Claude, etc.) working on the provider-keycloak repository.

What This Repository Is

provider-keycloak is a Crossplane provider that lets you manage Keycloak resources as Kubernetes custom resources. It is generated with Upjet on top of the Keycloak Terraform Provider.

One-line flow:

Keycloak Terraform Provider  →  Upjet (code generator)  →  Crossplane provider  →  Kubernetes CRDs

Users declare Keycloak resources as YAML (spec.forProvider maps to Terraform arguments), and the provider reconciles them continuously against a live Keycloak instance.

Repository Layout

apis/               Crossplane API types (generated + hand-authored)
cmd/                provider and generator entry points
config/             Upjet resource configuration (external names, references, cross-resource refs)
docs/               Hugo (hextra) documentation site
examples/           Hand-authored example manifests for each managed resource
examples-generated/ Auto-generated example manifests (do not edit by hand)
package/crds/       Generated CRD YAML files (source of truth for field schemas)
internal/           Internal controller and reconciler logic
generate/           Generation scripts
cluster/            Uptest end-to-end test manifests and setup
dev/                Local development environment scripts
scripts/            Utility scripts

Core Concepts

• ProviderConfig – holds connection details for a Keycloak instance (URL, client ID, credentials secret reference).
• Managed Resources – Kubernetes CRDs that map 1:1 to Keycloak objects. spec.forProvider maps to Terraform resource arguments.
• Reconciliation – the provider controller continuously ensures Keycloak matches the desired state expressed in spec.forProvider.
• External Name – the Keycloak-side identifier wired in config/external_name.go. This is the ID or name that Keycloak assigns to the resource.
• References – cross-resource references (e.g., realmIdRef) are configured in config/<group>/config.go. They wire one managed resource’s external name into another resource’s field.

Key Files for Common Tasks

Code Generation

Always run make generate after changing config/ to regenerate CRDs and Go types. Never edit files in apis/ or package/crds/ by hand — they are generated outputs.

make generate is the single entry point for all generation — code, DAG and docs. There is no need to run the individual targets by hand.

The generation pipeline:

1. generate/main.go calls Upjet with the Terraform provider schema.
2. Upjet writes Go type definitions into apis/<group>/<version>/.
3. make generate runs go generate ./... which invokes controller-gen to write CRDs into package/crds/.
4. make generate then runs e2e-index (DAG: cluster/test/e2e-index.json), generated-lst (config/generated.lst) and docs-gen (docs/static/llms.txt, docs/static/llms-full.txt).

Testing

• Unit tests: make test
• E2E tests: make e2e (requires a running Keycloak and Crossplane cluster)
• E2E coverage is limited to resources listed in cluster/test/cases.txt

The E2E suite uses uptest. Only resources explicitly listed in cluster/test/cases.txt receive e2e coverage.

Adding a New Resource

Use make schema-diff OLD_PROVIDER_VERSION=<prev> (or compare config/generated.lst against config/schema.json) to find Terraform resources that aren’t yet exposed as managed resources. The schema-diff-issues GitHub Actions workflow automates this and files one issue per missing resource that isn’t already tracked.

1. Add an entry to config/external_name.go. Choose config.IdentifierFromProvider when the Terraform provider already returns a stable ID (including composite {realm}/... IDs), or a <group>.<Resource>IdentifierFromIdentifyingProperties helper (see config/openidclient/ or config/group/) when the ID must be derived from identifying attributes such as name + realm.
2. Create or update config/<group>/config.go to configure references and any custom behaviors. If an attribute can reference more than one resource type, use config/multitypes (see Multi-Type References) rather than leaving it as a raw ID field.
3. Run make generate to regenerate CRDs and Go types.
4. Add a hand-authored example to examples/<group>/<resource>.yaml.
5. Optionally add a docs page to docs/content/docs/using/resources/<resource>.md.
6. Optionally add the resource to cluster/test/cases.txt plus a chainsaw/uptest manifest under cluster/test/ or dev/demos/ for e2e coverage.

To allow a resource to be imported/observed by its properties (avoiding 409 on create), wire its external_name.go entry to a lookup.BuildIdentifyingPropertiesLookup config in the config/<group> package (see config/openidclient/config.go for an example).

Automated Schema Diff Issues

.github/workflows/schema-diff-issues.yml runs scripts/schema_diff_issues.py, which diffs config/schema.json against config/generated.lst and files one GitHub issue per missing resource that isn’t already tracked by an existing open issue (matched by exact resource name in the issue title/body).

config/generated.lst is itself generated from config.ExternalNameConfigs (make generated-lst, run as part of make generate), and the workflow refreshes it before diffing, so an already implemented resource is never reported as missing. make generated-lst-check fails CI when the committed file is stale.

• On pull_request, it only runs in dry-run mode (reports only, creates nothing) when the automation itself changes (the script or the workflow file) — not on every PR.
• On a weekly schedule, on push to main that touches the Makefile (a Terraform provider version bump), and on manual workflow_dispatch, it creates real issues.

Cross-Resource References

References are wired in config/<group>/config.go using r.References on the Upjet resource configuration. The reference resolver fills in the referenced resource’s external name at reconciliation time. Example pattern:

r.References["realm_id"] = config.Reference{
    TerraformName: "keycloak_realm",
}

Multi-Type References

r.References accepts a single TerraformName, so it cannot describe a Terraform attribute whose value may be the ID of several different resource types. For those attributes, use the config/multitypes package instead of exposing a raw ID field or hand-rolling synthetic fields. It creates one synthetic, strongly-typed field per referenceable type and consolidates the resolved values back into the original Terraform field before the request is sent to Terraform.

Scalar field — a role’s client_id may point at an OpenID or a SAML client:

multitypes.ApplyToWithOptions(r, "client_id",
    &multitypes.Options{KeepOriginalField: true}, // keep client_id settable
    multitypes.Instance{
        Name: "client_id",
        Reference: config.Reference{
            TerraformName: "keycloak_openid_client",
            Extractor:     common.PathUUIDExtractor,
        },
    },
    multitypes.Instance{
        Name: "saml_client_id",
        Reference: config.Reference{
            TerraformName: "keycloak_saml_client",
            Extractor:     common.PathUUIDExtractor,
        },
    },
)

List/set field — keycloak_openid_client_aggregate_policy.policies holds IDs of any authorization policy type:

multitypes.ApplyToAsList(r, "policies",
    multitypes.Instance{
        Name: "time_policies",
        Reference: config.Reference{
            TerraformName: "keycloak_openid_client_time_policy",
            Extractor:     common.PathUUIDExtractor,
        },
    },
    multitypes.Instance{
        Name: "role_policies",
        Reference: config.Reference{
            TerraformName: "keycloak_openid_client_role_policy",
            Extractor:     common.PathUUIDExtractor,
        },
    },
    // ... one Instance per referenceable policy type
)

Rules and gotchas:

• Options.KeepOriginalField: true is required (and only allowed) when one Instance reuses the original field name; use it to keep the existing field settable for backward compatibility. The helper panics on a mismatch.
• An Instance that reuses the original field name may omit its Reference entirely. Such an “untyped” instance gets no Ref/Selector fields; the original field simply stays settable for raw IDs of types that have no managed resource yet, and its value still takes part in consolidation. Omitting the Reference on a synthetic instance panics.
• If no Instance reuses the original name, the original field becomes computed-only (status.atProvider). As a side effect, a required Terraform field no longer emits a required-parameter CEL rule, so no CRD post-processing is needed.
• For scalar fields only one synthetic field may be set at a time; the consolidation injector errors otherwise. For list fields all synthetic lists are unioned.
• Every referenced TerraformName must have an entry in config/external_name.go, otherwise make generate panics with cannot find configuration for Terraform resource.
• Examples: config/role/config.go and config/mapper/config.go (client_id/saml_client_id), config/authentication/config.go (parent_flow_alias/parent_subflow_alias), config/openidclient/config.go (clients/saml_clients and aggregate-policy policies), config/identityprovider/config.go (provider_alias/identity_provider_alias wired to every identity provider type).

Documentation Site

The docs use Hugo with the Hextra theme.

cd docs && hugo server --buildDrafts   # local preview
make generate                          # regenerates code, DAG and llms.txt
make docs-gen                          # regenerate llms.txt and llms-full.txt only
make docs-freshness-check             # CI: verify llms.txt is current

Every page is available as clean Markdown at the same URL with .md appended (e.g., /docs/using/resources/realms/index.md). This is useful for AI agents consuming individual pages.

LLM Files

• /llms.txt — brief categorized index for AI assistants
• /llms-full.txt — all doc pages concatenated for full-context ingestion

Known Constraints and Pitfalls

• Never edit examples-generated/ by hand. These are auto-generated.
• Never edit generated files in apis/ or package/crds/ by hand.
• Never edit config/generated.lst by hand. It is generated from config.ExternalNameConfigs by make generated-lst (part of make generate).
• github.com/keycloak/terraform-provider-keycloak updates are grouped; major bumps require manual review. The go.mod pseudo-version and its pinned Makefile version are grouped into a single weekly Renovate PR. Minor/patch/digest updates auto-merge once tests pass; major version bumps are not auto-merged since upgrading requires deliberate schema migration.
• E2E tests only cover resources in cluster/test/cases.txt. New resources are not automatically e2e tested.
• Upjet does not support +nullable markers. Do not add nullable annotations to generated types; the kubebuilder Options struct only supports Required, Minimum, Maximum, Default.
• Membership ownership: Avoid managing the same group’s membership with both a Memberships resource and a Groups resource with exhaustive=true at the same time; this can cause reconciliation loops.
• E2E Crossplane startup: When waiting for Crossplane to be ready in CI/dev scripts, wait on the deployment availability rather than pods by selector — pods may not exist yet when the wait command runs.
• E2E CI versioning: Jobs that build or deploy local xpkgs must fetch git tags (git fetch --tags) so that build/makelib/common.mk derives the correct VERSION that matches the pre-cached xpkg.

Troubleshooting Common Issues