Armory Scale Agent for Spinnaker and Kubernetes

What is the Armory Agent?

The Armory Scale Agent for Spinnaker and Kubernetes consists of a lightweight Agent service that you deploy on Kubernetes and a plugin that you install into Clouddriver.

The Armory Agent is compatible with Armory CD and open source Spinnaker.

Advantages of using the Armory Agent

• Scalability

  • Caching and deployment scales to thousands of Kubernetes clusters for your largest applications.
  • By leveraging the Kubernetes watch mechanism, the Armory Agent detects changes to Kubernetes and streams them in real time over a single TCP connection per cluster to Spinnaker^TM.
  • The Agent optimizes how infrastructure information is cached, making Force Cache Refresh almost instantaneous. This means optimal performance for your end users and your pipeline executions.

• Security

  • Keep your Kubernetes API servers private from Spinnaker.
  • Only the information Armory CD needs leaves the cluster.
  • Decentralize your account management. Using Kubernetes Service Accounts, teams control what Spinnaker can do. Add or remove accounts in real time. and then use them without restarting Spinnaker.
  • Use Kubernetes Service Accounts or store your kubeconfig files in one of the supported secret engines, or provision them via the method of your choice as Kubernetes secrets.

• Usability

  • Use the Armory Agent alongside Armory CD and benefit from performance improvements.
  • Use the Armory Agent in a target cluster and get Kubernetes accounts automatically registered.
  • Use YAML, a HELM chart, or a Kustomize template to inject the Armory Agent into newly provisioned Kubernetes clusters, and immediately make those clusters software deployment targets.
  • Use the Armory Agent with little operational overhead or changes in the way you currently manage Armory CD.

Check out the installation guide for how to deploy the Armory Agent components in Armory CD and in your Kubernetes infrastructure.

Deployment modes

Agent mode

In this mode, the Armory Agent service acts as a piece of infrastructure. It authenticates using a service account token. You use RBAC service account permissions to configure what the Armory Agent service is authorized to do.

If the Clouddriver plugin is unable to communicate with the Armory Agent service, the plugin attempts to reconnect during a defined grace period. If the plugin still can’t communicate with the Armory Agent service after the grace period has expired, the cluster associated with the Armory Agent service is removed from Armory CD.

Keep the following pros and cons in mind when deciding if Agent mode fits your use case:

Pros

• Agent mode is scalable because each agent only manages one Kubernetes account.
• Initial setup is easier because there’s no need to create kubeconfig files.
• Target clusters can remain private or in separate VPCs from the Armory CD (Spinnaker) cluster because agents initiate the connection to Armory CD (Spinnaker).

Cons

• It is difficult to get agent logs, upgrade Agent, or check configurations because agents (often) run in third-party clusters that the DevOps team operating Armory CD (Spinnaker) doesn’t have access.
• There is no authentication/authorization, so any team can start an Agent and register itself with Armory CD (Spinnaker). mTLS encryption can be used so that only agents with the right certificate can register. For information about how to configure mTLS, see Configure Mutual TLS Authentication.
• You need to expose gRPC port for Clouddriver through an external load balancer capable of handling HTTP/2 for gRPC communication.

Infrastructure mode

In infrastructure mode, multiple Agent service deployments handle different groups of Kubernetes clusters. Each service deployment is configured separately.

Keep the following pros and cons in mind when deciding if Infrastructure mode fits your use case:

Pro

• Agent can be managed centrally, so it is easy to get logs or configs and upgrade.

Cons

• You need to create kubeconfig file for each account.
• The API servers of the target clusters need to be accessible from the Armory Agent cluster.
• You need to expose gRPC port for Clouddriver through an external load balancer capable of handling HTTP/2 for gRPC communication.

Account name must still be unique across all your infrastructure. Clouddriver will reject new accounts with a name that matches a different cluster.

Spinnaker service mode

In this mode, the Armory Agent is installed as a new Spinnaker service (spin-armory-agent) and can be configured like other services.

If you provision clusters automatically, the Armory Agent service can dynamically reload accounts when armory-agent.yaml changes. You could, for example, configure accounts in a ConfigMap mounting to /opt/armory/config/armory-agent-local.yaml. The Agent service reflects ConfigMap changes within seconds after etcd sync.

Pros

• You do not have to configure external Network/Application Load Balancers to expose the Clouddriver gRPC port.
• Agent can be managed centrally, so it is easy to get logs or configs and upgrade.

Cons

• You need to create kubeconfig file for each account.
• The API servers of the target clusters need to be accessible from the Armory CD (Spinnaker) cluster.

Communication with Clouddriver

The Agent service does outbound calls only, except for a local health check, over a single gPRC connection to the Clouddriver plugin. The connection can be over TLS or mTLS. You can terminate TLS:

1. On Clouddriver: in the case of running the Armory Agent in Spinnaker Service mode or if declaring spin-clouddriver-grpc as a network load balancer.
2. On a gRPC proxy that directs request to the spin-clouddriver-grpc service.

The Clouddriver plugin uses the bidirectional communication channel to receive changes from Kubernetes accounts as well as send operations to the Armory Agent service.

Information sent by the Armory Agent service

The Agent service sends the following information about the cluster it is watching to the Clouddriver plugin:

• Account properties as configured in kubernetes.accounts[].
• Kubernetes API server host, certificate fingerprint, version.
• All the Kubernetes objects it is configured to watch and has permissions to access. You can ignore certain Kubernetes kinds (kubernetes.accounts[].omitKinds) or configure specific kinds to watch (kubernetes.accounts[].kinds).

The Agent service always scrubs data from Secret in memory before it is sent and even before that data makes it onto the Armory Agent’s memory heap.

Security

Since the Armory Agent service does outbound calls only, you can have Agent services running on-premises or in public clouds such as AWS, GCP, Azure, Oracle, or Alibaba.

What Armory CD can do in the target cluster is limited by what it is running as:

• a serviceAccount in Agent mode
• a kubeconfig setup for infrastructure or Spinnaker service mode

Communications are secured with TLS and optionally mTLS.

Furthermore, in Agent mode, Armory CD never gets credentials, and account registration is dynamic.

Scalability

Each Agent can scale to hundreds of Kubernetes clusters. The more types of Kubernetes objects the Armory Agent has to watch, the more memory it uses. Memory usage is bursty. You can control burst with budget. See Agent options) for configuration information.

Scaling the Armory Agent can mean:

• Scaling the Kubernetes Deployment the Armory Agent service is part of.
• Sharding Kubernetes clusters into groups that can in turn be scaled as described in Infrastructure mode above.

You can also mix deployment strategies if you have complex Kubernetes infrastructure and permissions:

• Run as a Spinnaker service
• Run in target clusters
• Run next to traditional Spinnaker Kubernetes accounts

Supported Spinnaker and Armory CD versions

Your Clouddriver service must use a MySQL-compatible database. See the Configure Clouddriver to use a SQL Database guide for instructions.

Database compatibility:

You can find a full list of previous releases in the artifactory.

Docs organization

If you’re familiar with the Kubernetes docs, you may notice that the Armory Scale Agent docs are organized in a similar fashion:

• Installation: This section contains guides for installing and uninstalling the Armory Scale Agent.
• Concepts: These pages explain aspects of the Armory Scale Agent. The content is objective, containing architecture, definitions, rules, and guidelines. Rather than containing a sequence of steps, these pages link to related tasks and tutorials.
• Tasks: Task pages show you how to do a single procedure by following a short series of steps that produce an intended outcome. Task content expects a minimum level of background knowledge, and each page links to conceptual content that you should be familiar with before you begin the task.
• Reference: This section contains both manually maintained and autogenerated reference material for configuring the plugin and service and using the APIs.
• Release Notes