Guides

Service Accounts

Machine-to-machine authentication for Kafka

Overview

Service accounts provide machine-to-machine authentication for applications that need to interact with Kafka clusters. Unlike user accounts (which are tied to individuals), service accounts are designed for automated systems, CI/CD pipelines, and application workloads.

Each service account is created with a specific type (producer or consumer) that determines its role bindings — the set of permissions it receives on cluster resources like topics, consumer groups, and Schema Registry subjects.

Naming Convention

Service account names follow the same naming convention as other resources. The display name is automatically generated based on your cluster type:

Dedicated Clusters

{business_unit}-{stage}-{id}

Example: sales-dev-order-processor

Shared Clusters

{landing_zone}-{business_unit}-{stage}-{id}

Example: edh-shared-scada-dev-order-processor

landing_zone: Shared cluster landing zone identifier (e.g., "edh-shared")

business_unit: Your organization's identifier (e.g., "scada", "sales")

stage: Environment stage (dev, qas, or run)

id: Descriptive name for your service account (e.g., "order-processor", "analytics-sink")

Note

When creating a service account, you only need to provide the identifier. The system automatically normalizes it (lowercase, hyphens instead of underscores/dots/spaces) and adds the appropriate prefix based on your cluster configuration.

Account Types

The type you choose determines which role bindings are created for the service account. Role bindings define the permissions the account has on cluster resources. These are system-defined templates — additional types will be added in future releases.

Producer

Read/write access to topics and transactional IDs, read access to consumer groups and Schema Registry subjects, plus Operator role to view other consumers and producers.

Role Resource
Operator Cluster
DeveloperRead + DeveloperWrite Topic
DeveloperRead + DeveloperWrite TransactionalId
DeveloperRead Group
DeveloperRead Subject

Consumer

Read access to topics, consumer groups, and Schema Registry subjects, plus Operator role to view other consumers and producers.

Role Resource
Operator Cluster
DeveloperRead Topic
DeveloperRead Group
DeveloperRead Subject

Creating Service Accounts

When you create a service account, the following happens automatically:

  1. Name normalization: Your identifier is normalized and prefixed based on the naming convention
  2. Account creation: The service account is registered in Confluent Cloud
  3. Role bindings: Permissions are assigned based on the selected type (producer/consumer)
  4. API key generation: A Cluster API key is created and both key and secret are stored in your Azure Key Vault

Important

Both the API key and secret are automatically stored in your business unit's Azure Key Vault. You can view the Key Vault reference on the Info page.

Bulk Creation

For scenarios where you need to create multiple service accounts at once, the portal supports bulk creation. This is useful when onboarding a new team or setting up a microservices architecture.

How it works

  1. Define multiple service accounts with their names and types
  2. All items are validated upfront — if any item fails validation, nothing is created
  3. A background job is submitted to process the batch asynchronously
  4. Track progress via the job status endpoint or the UI

Note

Bulk creation jobs run asynchronously as Kubernetes jobs. API key secrets for each created account are stored in Azure Key Vault automatically.

API Keys

Each service account receives a Cluster API key upon creation. This key pair (key + secret) is used to authenticate the application against the Kafka cluster.

Key Properties

  • Resource-scoped: The API key is scoped to the specific Kafka cluster
  • Key Vault storage: Both the API key and secret are automatically stored in your business unit's Azure Key Vault

Using API Keys

Configure your Kafka client with the API key and secret:

bootstrap.servers=<bootstrap_server>
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
  username="<API_KEY>" \
  password="<API_SECRET>";

Best Practices

  • Create one service account per application or microservice
  • Use descriptive names that identify the application (e.g., "order-processor", "analytics-sink")
  • Choose the minimum required type — use "consumer" when the application only reads data
  • Store API key secrets in a secure vault; never commit them to source control
  • Rotate API keys regularly as part of your security practices
  • Test service accounts in dev/qas before creating them in production
  • Use bulk creation for consistent onboarding of multiple services
Esc