Armory Agent Service Configuration
Where to configure the Armory Agent
- If you install the Armory Agent service using
kubectl
, configure the Armory Agent using aConfigMap
. See thekubectl
installation instruction’s Configure the Armory Agent section for an example. - If you install the Armory Agent service using Helm, see the Helm chart installation instructions
Configure Kubernetes accounts
You can configure Kubernetes accounts in Armory CD in multiple places:
- Clouddriver configuration files:
clouddriver.yml
, clouddriver-local.yml
,spinnaker.yml
,spinnaker-local.yml
- Spring Cloud Config Server reading accounts from Git, Vault, or another supported backend
- Plugins
Behavior when reading Kubernetes account configuration from multiple sources:
- When you configure different accounts in Agent and Armory CD, Agent merges both sources.
- If you configure an account with the same name in Agent as well as Armory CD (Clouddriver or Spring Cloud Config Server backends), Agent account configuration always overrides the Armory CD configuration.
- If you configure an account with the same name in Agent as well as in the Clouddriver plugin, the account that is used depends on the order of precedence defined in the plugin’s
CredentialsDefinitionSource
interface. The Agent has an order precedence of 1000. Check with your plugin provider for the plugin’s order of precedence.- If the plugin has a higher precedence than the Armory Agent, the plugin’s account is used. For example, if the plugin’s precedence is 500, the plugin’s account is used.
- If the plugin has no precedence defined or a lower precedence than the Armory Agent, the account defined in the Armory Agent is used.
Migrate accounts from Clouddriver to Agent
Copy the account definition from its original source, such as Clouddriver, to Agent’s configuration. Depending on how you installed Agent, this configuration could be in the armory-agent.yml
data section of a ConfigMap
or in the armory-agent.yml
file in the Armory Agent pod.
Agent may not use all the properties you copy from the original source definition. Unused properties are ignored.
Agent mode
In Agent mode, your configuration should look similar to this:
kubernetes:
accounts:
- name: account-01
serviceAccount: true
...
Spinnaker Service and Infrastructure modes
You set up multiple accounts per Agent in these modes. Your configuration should look similar to this:
kubernetes:
accounts:
- name: account-name-01
kubeconfigFile: /kubeconfigfiles/kubecfg-account01.yaml
- name: account-name-02
kubeconfigFile: /kubeconfigfiles/kubecfg-account02.yaml
- ...
Migrate accounts from Agent to Clouddriver
Follow these steps to migrate accounts from Agent to Clouddriver:
- Delete the account definition from your Agent configuration. Depending on how you installed Agent, this configuration could be in the
armory-agent.yml
data section of aConfigMap
or in thearmory-agent.yml
file in the Armory Agent pod. - Add the account definition to the source that Clouddriver uses.
Permissions format
Permissions for the Armory Agent use a format that is slightly different than the format that Clouddriver uses for permissions. Define your permissions like this:
kubernetes:
accounts:
- name: my-k8s-account
permissions:
- READ: ['role1', 'role2']
- WRITE: ['role3', 'role4']
Make sure that there are no whitespaces in the role configurations under the
READ
orWRITE
tags. The permissions are not applied if there are whitespaces in the configuration. This means that all users will have access to the account.
Restricted environments
Network access
The Agent needs access to its control plane (Spinnaker) as well to the various Kubernetes clusters it is configured to monitor. You can control which traffic should go through an HTTP proxy via the usual HTTP_PROXY
, HTTPS_PROXY
, and NO_PROXY
environment variables.
A common case is to force the connection back to the control plane via a proxy but bypass it for Kubernetes clusters. In that case, define the environment variable HTTPS_PROXY=https://my.corporate.proxy
and use the kubernetes.noProxy: true
setting to not have to maintain the list of Kubernetes hosts in NO_PROXY
.
Kubernetes authorization
The Agent should be configured to access each Kubernetes cluster it monitors with a service account. You can limit what Spinnaker can do via the role you assign to that service account. For example, if you’d like Spinnaker to see NetworkPolicies
but not deploy them:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: agent-role
rules:
- apiGroups: ["networking.k8s.io"]
resources: ["networkpolicies"]
verbs: [ "get", "list", "watch"]
...
Namespace restrictions
You can limit the Armory Agent to monitoring specific namespaces by listing them under namespaces
. If you need to prevent the Armory Agent from accessing cluster-wide (non-namespaced) resources, use the onlyNamespacedResources
setting.
A side effect of disabling cluster-wide resources is that CustomResourceDefinitions won’t be known (and therefore deployable by Spinnaker). CustomResourceDefinitions
are cluster-wide resources, but the custom resources themselves may be namespaced. To workaround the limitation, you can define customResourceDefinitions
. Both namespaces and CRDs are sent to Spinnaker as “synthetic” resources. They won’t be queried or watched, but Spinnaker knows about their existence.
kubernetes:
accounts:
- name: production
...
# Restricts the agent to namespaces `ns1` and `ns2`
namespaces:
- ns1
- ns2
# Prevents the Armory Agent from querying non-namespaced resources
onlyNamespacedResources: true
# Whitelist CRDs so Spinnaker
customResourceDefinitions:
- kind: ServiceMonitor.monitoring.coreos.com
- kind: SpinnakerService.spinnaker.armory.io
Configuration options
Settings | Type | Default | Description |
---|---|---|---|
clouddriver.auth.token |
string | none | 0.3.0+ Optional bearer token added to each request back to the endpoint. |
clouddriver.auth.tokenCommand.command clouddriver.auth.tokenCommand.args clouddriver.auth.tokenCommand.format clouddriver.auth.tokenCommand.refreshIntervalSeconds |
string []string string integer |
none none [] 0 |
0.3.0+ Allows to invoke a command every refreshIntervalSeconds seconds that outputs either the token (format is raw ) or a JSON object with an attribute of token if format is json or left empty. args is the optional list of parameters to the command. |
clouddriver.grpc |
string (hostname) | spin-clouddriver-grpc:9091 |
Hostname of the Clouddriver or gRPC proxy endpoint. |
clouddriver.keepAliveHeartbeatSeconds |
integer | none | 0.6.9+Optional. How often the gPRC keep alive message is sent. - Not set: (default) sent every 60 seconds. - 0: not sent. - Any n value greater than 0: sent every n seconds. |
clouddriver.keepAliveTimeOutSeconds |
integer | none | Timeout before closing the grpc connection. |
clouddriver.insecure |
boolean | false | If true, you are connecting to a non TLS server. |
clouddriver.noProxy |
boolean | false | 0.3.1+ Ignore the HTTP_PROXY , HTTPS_PROXY , and NO_PROXY environment variables when connecting back to the control plane (Armory Enterprise). |
clouddriver.responseRetries.enabled clouddriver.responseRetries.maxRetries clouddriver.responseRetries.backOffMs |
boolean integer integer |
true 1200 3000 |
0.6.7+Enables or disables retries. 0.6.7+How many times to retry sending the response to Clouddriver. 0.6.7+How much time to wait between retries in milliseconds. Note that Agent Plugin config option kubesvc.cache.operationWaitMs should be set so that it does not time out before the retries are complete. |
clouddriver.tls.serverName |
string | none | Server name on the remote certificate (override from the hostname). |
clouddriver.tls.insecureSkipVerify |
boolean | false | Do not verify the endpoint's certificate. |
clouddriver.tls.clientCertFile clouddriver.tls.clientKeyFile clouddriver.tls.clientKeyFilePassword |
string string string |
none none none |
Client certificate file for mTLS. Client key file if not included in the certificate. Password the key file if needed. |
clouddriver.tls.cacertFile |
string | none | If provided, verify endpoint certificate with the trust store. Otherwise, the system trust store is used. |
kubernetes.noProxy |
boolean | false | 0.3.1+ Ignore the HTTP_PROXY , HTTPS_PROXY , and NO_PROXY environment variables when connecting to any Kubernetes cluster. |
kubernetes.reconnectTimeoutMs |
integer | 5000 | How long to wait before reconnecting to Armory Enterprise. |
kubernetes.accounts[].context |
string | empty | If provided, use the given context of the configured kubeconfig. |
kubernetes.accounts[].customResourceDefinitions |
[]{kind: |
empty | 0.4.0+ List of CustomResourceDefinition to expose to Armory Enterprise. This is not needed if onlyNamespacedResources is left off. The format of kind is Kind.group . |
kubernetes.accounts[].customResourceDefinitions.scope |
string | Namespaced | 1.0.9+ Possible values are `Cluster` and `Namespaced`. The default value if left unconfigured is `Namespaced`. |
kubernetes.accounts[].kinds |
[]string | empty | Defines which Kubernetes kinds to cache for the Spinnaker UI. If not empty, only kinds in the list are cached. Use only the kind name in singular form and without group name (e.g. deployment , not deployment.apps ). This also applies to CRDs.There’s no benefit to caching kinds that don’t display in the Spinnaker UI. The recommended set of kinds to include (per account) is: replicaSet , service , ingress , daemonSet , deployment , pod , statefulSet , job , cronJob .Note: Operations are still possible on every kind regardless of the config, and statuses for those operations are not affected. For example, if you need to deploy another kind such as HorizontalPodAutoscaler , and this kind is not defined in this list, the deployment still succeeds. |
kubernetes.accounts[].kubeConfigFile |
string | none | Path to the kubeconfig file if not using serviceAccount . |
kubernetes.accounts[].insecure |
boolean | false | Do not verify the TLS certificate of the Kubernetes API server Don’t use without a good reason. |
kubernetes.accounts[].metrics |
boolean | false | When true, sends pod metrics back to Armory Enterprise every 20s. |
kubernetes.accounts[].name |
string | none, required | Name of the Kubernetes cluster in Armory Enterprise. |
kubernetes.accounts[].namespaces |
[]string | empty | 0.4.0+ Whitelist of namespaces to monitor. This comes at a greater cost of multiplying the resources by the number of namespaces. |
kubernetes.accounts[].noProxy |
boolean | false | 0.3.1+ Ignore the HTTP_PROXY , HTTPS_PROXY , and NO_PROXY environment variables when connecting to that Kubernetes cluster. |
kubernetes.accounts[].oAuthScopes |
[]string | empty | List of OAuth scope when authenticating with gcp provider Cluster access for kubectl. |
kubernetes.accounts[].omitKinds |
[]string | empty | List of kinds not to cache. |
kubernetes.accounts[].omitNamespaces |
[]string | empty | Blacklist of namespaces This comes at a greater cost of multiplying the resources by the number of namespaces. NOT CURRENTLY IMPLEMENTED |
kubernetes.accounts[].onlyNamespacedResources |
boolean | false | 0.4.0+ If true, the Agent ignores non-namespaced resources; namespaces must be whitelisted with namespaces setting and CRDs with customResourceDefinitons . |
kubernetes.accounts[].onlySpinnakerManaged |
boolean | false | Only return Armory Enterprise managed resources NOT IMPLEMENTED in the Agent but added to the plugin see kubesvc.runtime.defaults.onlySpinnakerManaged . |
kubernetes.accounts[].permissions |
list | empty | List of permissions (currently READ or WRITE ) with a list of authorized roles. For more information, see Permissions format. |
kubernetes.accounts[].maxResumableResourceAgeMs |
integer | 300000 (5m) | When connecting to Armory Enterprise, the Agent asks Clouddriver for the latest resource version known per resource that is not older than that setting. The resource version is used to resume the watch without first doing a list - saving memory and time. There’s no guarantee that the resource version is still known. If not “remembered” by the Kubernetes API server, a list call is used. Kubernetes API Concepts |
kubernetes.accounts[].serviceAccount |
boolean | false | If true and the Agent runs in Kubernetes - use the current service account to call to the current API server. In that mode, you don’t need to provide a kubeconfig file. |
kubernetes.retries.enabled |
boolean | true | Optional. Enable or disable retries when the Agent makes a failed request to the Kubernetes API server. |
kubernetes.retries.maxRetries |
integer | 3 | Optional. The number of times that the Agent will try the same request if it fails. |
kubernetes.retries.backOffMs |
integer | 3000 | Optional. How much time (in milliseconds) to wait between retry attempts. |
kubernetes.retries.retryAnyError |
boolean | false | Optional. If true, Agent will retry when encountering any error from the Kubernetes API server. If false, Agent will only retry if the error contains any item from retryableErrors . |
kubernetes.retries.retryableErrors |
string | - timeout - deadline exceeded |
Optional. If the error from the Kubernetes API server contains any item from this list, the request will be retried. Requires retryAnyError to be false. |
logging.file |
string | stdout if not defined | File to save logs to. |
logging.format |
string | text | Format for the Agent logs. Can be text or json |
logging.level |
string | INFO |
Log level. Can be any of (case insensitive):panic , fatal , error , warn (or warning ), info , debug , trace . |
pprof.enabled |
boolean | false | Enable pprof endpoint. Useful for troubleshooting, slowness, memory leaks, and more! pprof README. |
pprof.port |
integer | 6060 | Port on which to respond to pprof requests. |
prometheus.enabled |
boolean | false | Enable Prometheus handler. |
prometheus.port |
integer | 8008 | Port to expose Prometheus metrics on. Responds to both /metrics (standard) and /prometheus_metrics (Armory Enterprise default). |
server.host |
string | localhost | Hostname of the server health check. |
server.port |
integer | 8082 | Port of the server health check. |
server.ssl.enabled , server.ssl.certFile , server.ssl.keyFile , server.ssl.keyPassword , server.ssl.caCertFile , server.ssl.keyFilePassword , server.ssl.clientAuth |
Various options to control TLS config. Don’t bother, it’s just for the health endpoint. | ||
secrets.vault.* |
object | none | Vault configuration. |
tasks.totalBudget |
integer | 0 | If > 0, limits the number of tasks that can be started. Tasks have different cost. Watches are considered free because they are part of the normal operations of the Agent. |
tasks.budgetPerAccount |
integer | 0 | Same as totalBudget but per account. If both settings are provided, they’re both checked. |
tasks.queueCheckFrequencyMs |
integer | 2000 | Frequency at which the Agent checks for new tasks to launch. Once launched a task is not stopped until explicitly requested (account unregistered or connection to Armory Enterprise lost). |
timeoutSeconds |
integer | 0 | 0.5.12+ The maximum length of time to wait before giving up on a server request. A value of zero means no timeout. |
Feedback
Was this page helpful?
Thank you for letting us know!
Sorry to hear that. Please tell us how we can improve.
Last modified August 12, 2022: (a62dd8ed)