Armory Agent Service Configuration

Learn how to configure the Armory Scale Agent for Spinnaker and Kubernetes based on installation mode and environment restrictions. This guide contains a detailed list of configuration options for the Agent service.

Where to configure the Armory Agent

Configure Kubernetes accounts

You can configure Kubernetes accounts in Armory CD in multiple places:

  • Clouddriver configuration files: clouddriver.yml, clouddriver-local.yml, spinnaker.yml, spinnaker-local.yml
  • Spring Cloud Config Server reading accounts from Git, Vault, or another supported backend
  • Plugins

Behavior when reading Kubernetes account configuration from multiple sources:

  • When you configure different accounts in Agent and Armory CD, Agent merges both sources.
  • If you configure an account with the same name in Agent as well as Armory CD (Clouddriver or Spring Cloud Config Server backends), Agent account configuration always overrides the Armory CD configuration.
  • If you configure an account with the same name in Agent as well as in the Clouddriver plugin, the account that is used depends on the order of precedence defined in the plugin’s CredentialsDefinitionSource interface. The Agent has an order precedence of 1000. Check with your plugin provider for the plugin’s order of precedence.
    • If the plugin has a higher precedence than the Armory Agent, the plugin’s account is used. For example, if the plugin’s precedence is 500, the plugin’s account is used.
    • If the plugin has no precedence defined or a lower precedence than the Armory Agent, the account defined in the Armory Agent is used.

Migrate accounts from Clouddriver to Agent

Copy the account definition from its original source, such as Clouddriver, to Agent’s configuration. Depending on how you installed Agent, this configuration could be in the armory-agent.yml data section of a ConfigMap or in the armory-agent.yml file in the Armory Agent pod.

Agent may not use all the properties you copy from the original source definition. Unused properties are ignored.

Agent mode

In Agent mode, your configuration should look similar to this:

kubernetes:
  accounts:
    - name: account-01
      serviceAccount: true
      ...

Spinnaker Service and Infrastructure modes

You set up multiple accounts per Agent in these modes. Your configuration should look similar to this:

kubernetes:
  accounts:
    - name: account-name-01
      kubeconfigFile: /kubeconfigfiles/kubecfg-account01.yaml
    - name: account-name-02
      kubeconfigFile: /kubeconfigfiles/kubecfg-account02.yaml
    - ...  

Migrate accounts from Agent to Clouddriver

Follow these steps to migrate accounts from Agent to Clouddriver:

  • Delete the account definition from your Agent configuration. Depending on how you installed Agent, this configuration could be in the armory-agent.yml data section of a ConfigMap or in the armory-agent.yml file in the Armory Agent pod.
  • Add the account definition to the source that Clouddriver uses.

Permissions format

Permissions for the Armory Agent use a format that is slightly different than the format that Clouddriver uses for permissions. Define your permissions like this:

kubernetes:
  accounts:
    - name: my-k8s-account
      permissions:
        - READ: ['role1', 'role2']
        - WRITE: ['role3', 'role4']

Make sure that there are no whitespaces in the role configurations under the READ or WRITE tags. The permissions are not applied if there are whitespaces in the configuration. This means that all users will have access to the account.

Restricted environments

Network access

The Agent needs access to its control plane (Spinnaker) as well to the various Kubernetes clusters it is configured to monitor. You can control which traffic should go through an HTTP proxy via the usual HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables.

A common case is to force the connection back to the control plane via a proxy but bypass it for Kubernetes clusters. In that case, define the environment variable HTTPS_PROXY=https://my.corporate.proxy and use the kubernetes.noProxy: true setting to not have to maintain the list of Kubernetes hosts in NO_PROXY.

Kubernetes authorization

The Agent should be configured to access each Kubernetes cluster it monitors with a service account. You can limit what Spinnaker can do via the role you assign to that service account. For example, if you’d like Spinnaker to see NetworkPolicies but not deploy them:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: agent-role
rules:
- apiGroups: ["networking.k8s.io"]
  resources: ["networkpolicies"]
  verbs: [ "get", "list", "watch"]
...

Namespace restrictions

You can limit the Armory Agent to monitoring specific namespaces by listing them under namespaces. If you need to prevent the Armory Agent from accessing cluster-wide (non-namespaced) resources, use the onlyNamespacedResources setting.

A side effect of disabling cluster-wide resources is that CustomResourceDefinitions won’t be known (and therefore deployable by Spinnaker). CustomResourceDefinitions are cluster-wide resources, but the custom resources themselves may be namespaced. To workaround the limitation, you can define customResourceDefinitions. Both namespaces and CRDs are sent to Spinnaker as “synthetic” resources. They won’t be queried or watched, but Spinnaker knows about their existence.

kubernetes:
    accounts:
        - name: production
          ...
          # Restricts the agent to namespaces `ns1` and `ns2`
          namespaces:
            - ns1
            - ns2
          # Prevents the Armory Agent from querying non-namespaced resources
          onlyNamespacedResources: true
          # Whitelist CRDs so Spinnaker
          customResourceDefinitions:
            - kind: ServiceMonitor.monitoring.coreos.com
            - kind: SpinnakerService.spinnaker.armory.io

Configuration options

Settings Type Default Description
clouddriver.auth.token string none 0.3.0+ Optional bearer token added to each request back to the endpoint.
clouddriver.auth.tokenCommand.command
clouddriver.auth.tokenCommand.args
clouddriver.auth.tokenCommand.format
clouddriver.auth.tokenCommand.refreshIntervalSeconds
string
[]string
string
integer
none
none
[]
0
0.3.0+ Allows to invoke a command every refreshIntervalSeconds seconds that outputs either the token (format is raw) or a JSON object with an attribute of token if format is json or left empty. args is the optional list of parameters to the command.
clouddriver.grpc string (hostname) spin-clouddriver-grpc:9091 Hostname of the Clouddriver or gRPC proxy endpoint.
clouddriver.keepAliveHeartbeatSeconds integer none 0.6.9+Optional. How often the gPRC keep alive message is sent.
- Not set: (default) sent every 60 seconds.
- 0: not sent.
- Any n value greater than 0: sent every n seconds.
clouddriver.keepAliveTimeOutSeconds integer none Timeout before closing the grpc connection.
clouddriver.insecure boolean false If true, you are connecting to a non TLS server.
clouddriver.noProxy boolean false 0.3.1+ Ignore the HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables when connecting back to the control plane (Armory Enterprise).
clouddriver.responseRetries.enabled
clouddriver.responseRetries.maxRetries
clouddriver.responseRetries.backOffMs
boolean
integer
integer
true
1200
3000
0.6.7+Enables or disables retries.
0.6.7+How many times to retry sending the response to Clouddriver.
0.6.7+How much time to wait between retries in milliseconds.
Note that Agent Plugin config option kubesvc.cache.operationWaitMs should be set so that it does not time out before the retries are complete.
clouddriver.tls.serverName string none Server name on the remote certificate (override from the hostname).
clouddriver.tls.insecureSkipVerify boolean false Do not verify the endpoint's certificate.
clouddriver.tls.clientCertFile
clouddriver.tls.clientKeyFile
clouddriver.tls.clientKeyFilePassword
string
string
string
none
none
none
Client certificate file for mTLS.
Client key file if not included in the certificate.
Password the key file if needed.
clouddriver.tls.cacertFile string none If provided, verify endpoint certificate with the trust store. Otherwise, the system trust store is used.
kubernetes.noProxy boolean false 0.3.1+ Ignore the HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables when connecting to any Kubernetes cluster.
kubernetes.reconnectTimeoutMs integer 5000 How long to wait before reconnecting to Armory Enterprise.
kubernetes.accounts[].context string empty If provided, use the given context of the configured kubeconfig.
kubernetes.accounts[].customResourceDefinitions []{kind: } empty 0.4.0+ List of CustomResourceDefinition to expose to Armory Enterprise. This is not needed if onlyNamespacedResources is left off. The format of kind is Kind.group.
kubernetes.accounts[].customResourceDefinitions.scope string Namespaced 1.0.9+ Possible values are `Cluster` and `Namespaced`. The default value if left unconfigured is `Namespaced`.
kubernetes.accounts[].kinds []string empty Defines which Kubernetes kinds to cache for the Spinnaker UI. If not empty, only kinds in the list are cached. Use only the kind name in singular form and without group name (e.g. deployment, not deployment.apps). This also applies to CRDs.
There’s no benefit to caching kinds that don’t display in the Spinnaker UI. The recommended set of kinds to include (per account) is: replicaSet, service, ingress, daemonSet, deployment, pod, statefulSet, job, cronJob.
Note: Operations are still possible on every kind regardless of the config, and statuses for those operations are not affected. For example, if you need to deploy another kind such as HorizontalPodAutoscaler, and this kind is not defined in this list, the deployment still succeeds.
kubernetes.accounts[].kubeConfigFile string none Path to the kubeconfig file if not using serviceAccount.
kubernetes.accounts[].insecure boolean false Do not verify the TLS certificate of the Kubernetes API server
Don’t use without a good reason.
kubernetes.accounts[].metrics boolean false When true, sends pod metrics back to Armory Enterprise every 20s.
kubernetes.accounts[].name string none, required Name of the Kubernetes cluster in Armory Enterprise.
kubernetes.accounts[].namespaces []string empty 0.4.0+ Whitelist of namespaces to monitor.
This comes at a greater cost of multiplying the resources by the number of namespaces.
kubernetes.accounts[].noProxy boolean false 0.3.1+ Ignore the HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables when connecting to that Kubernetes cluster.
kubernetes.accounts[].oAuthScopes []string empty List of OAuth scope when authenticating with gcp provider
Cluster access for kubectl.
kubernetes.accounts[].omitKinds []string empty List of kinds not to cache.
kubernetes.accounts[].omitNamespaces []string empty Blacklist of namespaces
This comes at a greater cost of multiplying the resources by the number of namespaces.
NOT CURRENTLY IMPLEMENTED
kubernetes.accounts[].onlyNamespacedResources boolean false 0.4.0+ If true, the Agent ignores non-namespaced resources; namespaces must be whitelisted with namespaces setting and CRDs with customResourceDefinitons.
kubernetes.accounts[].onlySpinnakerManaged boolean false Only return Armory Enterprise managed resources
NOT IMPLEMENTED in the Agent but added to the plugin see kubesvc.runtime.defaults.onlySpinnakerManaged.
kubernetes.accounts[].permissions list empty List of permissions (currently READ or WRITE) with a list of authorized roles. For more information, see Permissions format.
kubernetes.accounts[].maxResumableResourceAgeMs integer 300000 (5m) When connecting to Armory Enterprise, the Agent asks Clouddriver for the latest resource version known per resource that is not older than that setting.

The resource version is used to resume the watch without first doing a list - saving memory and time. There’s no guarantee that the resource version is still known. If not “remembered” by the Kubernetes API server, a list call is used. Kubernetes API Concepts
kubernetes.accounts[].serviceAccount boolean false If true and the Agent runs in Kubernetes - use the current service account to call to the current API server. In that mode, you don’t need to provide a kubeconfig file.
kubernetes.retries.enabled boolean true Optional. Enable or disable retries when the Agent makes a failed request to the Kubernetes API server.
kubernetes.retries.maxRetries integer 3 Optional. The number of times that the Agent will try the same request if it fails.
kubernetes.retries.backOffMs integer 3000 Optional. How much time (in milliseconds) to wait between retry attempts.
kubernetes.retries.retryAnyError boolean false Optional. If true, Agent will retry when encountering any error from the Kubernetes API server. If false, Agent will only retry if the error contains any item from retryableErrors.
kubernetes.retries.retryableErrors string - timeout
- deadline exceeded
Optional. If the error from the Kubernetes API server contains any item from this list, the request will be retried. Requires retryAnyError to be false.
logging.file string stdout if not defined File to save logs to.
logging.format string text Format for the Agent logs. Can be text or json
logging.level string INFO Log level. Can be any of (case insensitive):
panic, fatal , error, warn (or warning), info, debug, trace.
pprof.enabled boolean false Enable pprof endpoint. Useful for troubleshooting, slowness, memory leaks, and more!
pprof README.
pprof.port integer 6060 Port on which to respond to pprof requests.
prometheus.enabled boolean false Enable Prometheus handler.
prometheus.port integer 8008 Port to expose Prometheus metrics on. Responds to both /metrics (standard) and /prometheus_metrics (Armory Enterprise default).
server.host string localhost Hostname of the server health check.
server.port integer 8082 Port of the server health check.
server.ssl.enabled, server.ssl.certFile, server.ssl.keyFile, server.ssl.keyPassword, server.ssl.caCertFile, server.ssl.keyFilePassword, server.ssl.clientAuth Various options to control TLS config. Don’t bother, it’s just for the health endpoint.
secrets.vault.* object none Vault configuration.
tasks.totalBudget integer 0 If > 0, limits the number of tasks that can be started. Tasks have different cost. Watches are considered free because they are part of the normal operations of the Agent.
tasks.budgetPerAccount integer 0 Same as totalBudget but per account. If both settings are provided, they’re both checked.
tasks.queueCheckFrequencyMs integer 2000 Frequency at which the Agent checks for new tasks to launch. Once launched a task is not stopped until explicitly requested (account unregistered or connection to Armory Enterprise lost).
timeoutSeconds integer 0 0.5.12+ The maximum length of time to wait before giving up on a server request. A value of zero means no timeout.

Last modified August 12, 2022: (a62dd8ed)