engineering article image
engineering

Baked-in data security with ACLs

Securing Redpanda data access with ACLs

By David Castillo & Ben Mansheim on May 13, 2021

Time to read icon11 min read

Baked-in data security with ACLs

Security should be at the heart of the design of any software project, and Redpanda is no exception. Here we’ll dive into authorization, one of the baked-in mechanisms that helps you make your Redpanda cluster more secure. If you want to know about authentication with mTLS and SASL, check out our docs on security. But if you want a demonstration of ACLs, follow along.

Note: System security requires attention to many domains, such as network configuration and organization-wide roles management. Here we only address the security of client access to data. Make sure to review all of your other security requirements on a regular basis.

Access control lists (ACLs) make sure your data is protected by granular, least-privilege policies. By following this guide, you will get to know how to set up Redpanda with ACLs, both on a conceptual level and on a technical level, through hands-on examples.

While authentication is about making sure that whatever client connects to your cluster is verified as a trusted client, authorization is about making sure that each client has access to exactly the data it should. ACLs are the main mechanism supported by Redpanda to manage permissions. Our implementation of ACLs is Kafka-compatible, too!

Access Control Lists (ACLs)

At a high level, ACLs specify what authenticated users can or cannot do.

ACLs are all about who from where can or cannot do what action to what object.

In ACL terminology, the entities accessing the resources are called principals. Principals are users (User:) that are allowed or denied access to resources. You can also specify from which hosts the principals are allowed or denied.

For more granular ACLs, you can allow or deny principals access to specific resources, such as specific topics, and for specific operations, such as “read”, “write”, or “describe”. These ACLs really lock down access to your data so you can rest assured that no one is getting unauthorized access.

The fields and values you can use are:

Field FlagDescriptionValues
Principal (who) --allow-principal
--deny-principal
Principal for which access will be granted or denied. (repeatable)User name, consumer group name
Host (from where)--allow-host
--deny-host
Host to which permissions will be granted or denied. (repeatable)Host IP address
Cluster (resource type)--clusterWhether to grant ACLs to the cluster.Cluster name
Group (resource type)--groupGroup to grant ACLs for. (repeatable)Group name
Operation (action) --operationOperation that the principal will be allowed or denied. Can be passed many times.

all, read , write, create, delete, alter, describe, describe_configs, alter_configs

Resource Pattern Type Matching--resource-pattern-typePattern to use when matching resource names (literal or prefixed. Default "literal"). If literal, it matches literal foo. If prefix matches foobar, for example.

any, match, prefix, literal

Topic (resource type)--topicTopic to grant ACLs for. (repeatable)Topic name
Transactional id (resource type) --transactional-idTransactional IDs to grant ACLs for. (repeatable)Transactional id used

In addition to other Kafka-compatible tools, ACLs are managed with the Redpanda rpk utility with the commands:

  • Creating ACLs - rpk acl create
  • Listing ACLs - rpk acl list
  • Deleting ACLs - rpk acl delete

Securing your data with ACLs

Now we are going to create some ACLs so you get the feel of how it works. Make sure you have the latest version of Redpanda, or check out our Getting Started guides to set up an environment.

Creating granular ACLs with a single command

  • Create an ACL allowing a user to perform all operations from all hosts to a topic named “pings”:
rpk acl create \
--allow-principal 'User:Charlie' \
--allow-host '*' \
--operation all \
--topic pings

rpk confirms that you created the ACL:

PRINCIPAL HOST RESOURCE-TYPE RESOURCE-NAME RESOURCE-PATTERN-TYPE OPERATION PERMISSION ERROR
User:Charlie * TOPIC pings LITERAL ALL ALLOW

You can also use a single command to create multiple ACLs, including ACLs that deny certain principals access to a resource and allow other principals access to the same resource:

rpk acl create \
--cluster \
--allow-host 168.72.98.52 \
--deny-host 169.78.31.9 \
--deny-principal 'User:david' \
--allow-principal 'User:Alex' \
--allow-principal 'User:Ben' \
--operation 'all'

rpk confirms that you created multiple ACLs:

PRINCIPAL HOST RESOURCE-TYPE RESOURCE-NAME RESOURCE-PATTERN-TYPE OPERATION PERMISSION ERROR
User:Ben 168.72.98.52 CLUSTER kafka-cluster LITERAL ALL ALLOW
User:david 168.72.98.52 CLUSTER kafka-cluster LITERAL ALL ALLOW
User:david 169.78.31.9 CLUSTER kafka-cluster LITERAL ALL DENY

What ACLs do I have now?

Now let’s see what the ACLs look like for a specific user:

rpk acl list --allow-principal 'User:david' --deny-principal 'User:david'

This list of ACLs is shown with all of their parameters:

PRINCIPAL HOST RESOURCE-TYPE RESOURCE-NAME RESOURCE-PATTERN-TYPE OPERATION PERMISSION ERROR
User:david 169.78.31.9 CLUSTER kafka-cluster LITERAL ALL DENY
User:david 168.72.98.52 CLUSTER kafka-cluster LITERAL ALL ALLOW

Notice that you can choose which flags to use in specific situations.

Of course, you can list ACLs with other filters, like host, permission and resource.

You can check a list of possible flags with rpk acl list -h

If you want to see them all execute rpk acl list.

Cleaning up ACLs

rpk acl delete allows you to delete ACLs. It’s important to note, however, that wildcard values such as any for operations and permissions, as well as * for hosts and resources may result in the deletion filters matching more than one ACL.

  • Delete all ACLs for a specific user targeting a specific resource:

    rpk acl delete --allow-principal 'User:david' --deny-principal 'User:david' --cluster

    Redpanda will ask you for confirmation

    ? Confirm deletion of the above matching ACLs? (Y/n)

    All of the related ACLs are listed as deleted:

    DELETIONS
    =========
    PRINCIPAL HOST RESOURCE-TYPE RESOURCE-NAME RESOURCE-PATTERN-TYPE OPERATION PERMISSION ERROR
    User:david 169.78.31.9 CLUSTER kafka-cluster LITERAL ALL DENY
  • Delete all ACLs granting a principal permissions to all operations from a host:

    rpk acl delete \
    --allow-principal 'User:Ben' \
    --allow-host 168.72.98.52 \
    --operation all

This is just a small example of what we can do with ACLs.

If you want to dive deeper, check our in-depth guide about ACLs.

We are moving fast to keep up with the demand for Redpanda, the intelligent data API, and security with ACLs is only one of the many features that we’re working on.

Join our Slack community to stay up on the latest developments.

Related articles