engineering article image

Baked-in data security with ACLs

Securing Redpanda data access with ACLs

By David Castillo & Ben Mansheim on May 13, 2021

Time to read icon12 min read

Baked-in data security with ACLs

Security should be at the heart of the design of any software project, and Redpanda is no exception. Here we’ll dive into authorization, one of the baked-in mechanisms that helps you make your Redpanda cluster more secure. If want to know about authentication with mTLS and SASL, check out our docs on security. But, if you want a demonstration ACLs, follow along.

Note: System security requires attention to many domains, such as network configuration and organization-wide roles management. Here we only address security of client access to data. Make sure to review all of your other security requirements on a regular basis.

Access control lists (ACLs) make sure your data is protected by granular, least-privilege policies. By following this guide, you will get to know how to set up Redpanda with ACLs, both on a conceptual level and on a technical level, through hands-on examples.

While authentication is about making sure that whatever client connects to your cluster is verified as a trusted client, authorization is about making sure that each client has access to exactly the data it should. ACLs are the main mechanism supported by Redpanda to manage permissions. Our implementation of ACLs is Kafka-compatible, too!

Access Control Lists (ACLs)

At a high level, ACLs specify what authenticated users can or cannot do.

ACLs are all about who from where can or cannot do what action to what object.

In ACL terminology, the entities accessing the resources are called principals. Principals can be either users (user) or consumer groups (groups). An ACL allows or denies the specified principals access to resources. You can also specify from which hosts the principals are allowed or denied.

For more granular ACLs, you can allow or deny principals access to specific resources, such as specific topics, and for specific operations, such as “read”, “write”, or “describe”. These ACLs really lock down access to your data so you can rest assured that no one is getting unauthorized access.

The fields and values you can use are:

Principal (who)--allow-principal,
User name, consumer group name
Host (from where)--allow-hosts,
Host IP address
Operation (action)--operationany, all, read, write, create, delete, alter,
describe, clusteraction, describeconfigs,
alterconfigs, idempotentwrite
Resource type (objects)--resource*, cluster, group, topic, transactionalid
Resource name (object)--resource-nameName of a specific resource

In addition to other Kafka-compatible tools, ACLs are managed with the Redpanda rpk utility with the commands:

  • Creating ACLs - rpk acl create
  • Listing ACLs - rpk acl list
  • Deleting ACLs - rpk acl delete

Securing your data with ACLs

Now we are going to create some ACLs so you get the feel of how it works. Make sure you have the latest version of Redpanda, or check out our Getting Started guides to set up an environment.

Creating granular ACLs with a single command

  • Create an ACL allowing a user to perform all operations from all hosts to a topic named “pings”:
rpk acl create \
--allow-principal 'User:Charlie' \
--allow-hosts '*' \
--operation all \
--resource topic \
--resource-name pings

rpk confirms that you created the ACL:

Created ACL for principal 'User:Charlie' with host '*', operation 'All' and permission 'Allow'

You can also use a single command to create multiple ACLs, including ACLs that deny certain principals access to a resource and allow other principals access to the same resource:

rpk acl create \
--resource cluster \
--allow-host \
--deny-host \
--deny-principal 'User:david' \
--allow-principal 'User:Alex' \
--allow-principal 'User:Ben'

rpk confirms that you created multiple ACLs:

Created ACL for principal 'User:david' with host '', operation 'All' and permission 'Deny'
Created ACL for principal 'User:Alex' with host '', operation 'All' and permission 'Allow'
Created ACL for principal 'User:Ben' with host '', operation 'All' and permission 'Allow'

What ACLs do I have now?

Now let’s see what the ACLs look like for a specific user:

rpk acl list --principal 'User:Ben'

This list of ACLs is shown with all of their parameters:

User:Ben All Allow Cluster kafka-cluster

Of course, you can list ACLs with other filters, like host, permission and resource, or just see all of the ACLs with rpk acl list.

Cleaning up ACLs

rpk acl delete allows you to delete ACLs. It’s important to note, however, that wildcard values such as any for operations and permissions, as well as * for hosts and resources may result in the deletion filters matching more than one ACL.

  • Delete all ACLs for a specific user targeting a specific resource:

    rpk acl delete --deny-principal 'User:david' --resource cluster

    All of the related ACLs are listed as deleted:

    yes User:david All Deny Cluster kafka-cluster None
  • Delete all ACLs granting a principal permissions to all operations from a host:

    rpk acl delete \
    --allow-principal 'User:Ben' \
    --allow-host \
    --operation all

We are moving fast to keep up with the demand for Redpanda, the intelligent data API, and security with ACLs is only one of the many features that we’re working on. Join our Slack community to stay up on the latest developments.

Related articles