If your machine data does not contain sensitive information, you don’t really need to read this – you got it all figured out. Just stop here and go back to surfing the interwebs or…maybe you want to check again?! 🙂
If you’re still reading, you know that while your machine data is vital for your operations/security analytics, depending on the source, you may have sensitive information coming in. This could be names, locations, addresses, IDs etc., and if you don’t guard it properly, your organization may be exposed to major risk. Most customers that we talk to do one or both of the following to minimize that risk:
- Omit the use of potentially sensitive sources: They intentionally do not send data that is known to contain sensitive information to their analytics systems (e.g. Splunk, Elasticsearch etc.). I.e. they’re willing tradeoff better and more complete analysis for security. This is suboptimal as they’re not getting all the insights possible from that data.
- Duplicate and Redact: They send a redacted copy and a clear-text copy to their analytics system and control access to each via strict ACL/RBAC mechanisms. In some cases copies would go to entirely different systems. I.e. they’re willing to tradeoff space/complexity for security.
In the latest version of Cribl LogStream 1.3 we added support for another fundamental option in dealing with sensitive data; real-time encryption. If your machine data analytics system is Splunk, you can now do field-level encryption in real-time and decrypt on retrieval using Roles and Capabilities.
Let’s take a look and see how you can do it.
Encrypting in Cribl and Decrypting in Splunk
In Cribl, fields or patterns within events can be encrypted with
C.Crypto.encrypt()using a Mask function. Patterns are defined via regular expressions and one or more of them can be encrypted simultaneously. Encryption Keys can be configured through the UI or CLI. They can be organized into Key Classes which can be used to implement multiple levels of access control. Users or groups/roles with access to data with encrypted fields/patterns can be associated with Key Classes for even more granular, field-level compartmentalized access.
Assuming you have data flowing through Cribl to Splunk, let’s walk through it:
1. Define one or more Keys and Key Classes on Cribl.
On the Cribl instance that is doing the encryption, go to Settings > Encryption Keys and generate a handful of keys. Add as many keys across as many Key Classes as necessary. Chose an appropriate expiration time.
auth folder with decryption side (Splunk)
On your Search Head (SH) install Cribl using the Splunk app package and convert it to
$SPLUNK_HOME/bin/splunk cmd node --harmony $CRIBL_HOME/bin/cribl.bundle.js mode-searchhead
In order for the SH to decrypt encrypted patterns you must make the encrypting keys available. Using a secure mechanism (e.g., Deployment Server, Config Management) sync
$CRIBL_HOME/local/cribl/auth/(cribl.secret|keys.json) from the encrypting Cribl instance to the Search Head.
Encrypting Cribl Instance:
Next, assign permissions to the
decrypt command per your requirements. This could be one or several Roles.
Finally, assign finer grained capabilities to your Roles per your requirements. Capability names should follow the format
N is the Cribl Key Class number. For example, a role with capability
cribl_keyclass_1 has access to all key ids associated with key class
1. You can use more capabilities as long as they follow this naming convention. The Cribl app ships with 5 Key Class Capabilities out of the box.
3. Apply the Mask function with
C.Crypto.encrypt() to fields of interest
Assume we have sample events (unencrypted) as below. Notice values of
In Cribl, create a simple Pipeline with a Mask function that targets the fields in question. In this case, we’ll encrypt:
fieldA values using Key Class
fieldB values using Key Class
Notice in the Replace Expression, the
C.Crypto.encrypt()first parameter is the payload to be encrypted and the second is Key Class.
On the Search Head, notice how the values of
fieldB look after encryption has been applied.
4. Decrypt on Search Head using RBAC on
Logged in user has been assigned capability
cribl_keyclass_2 but not
fieldB can be decrypted but NOT
With Cribl real-time field-level encryption you can:
- Safely bring in all your data and encrypt sensitive parts as necessary.
- Decrypt based on Role and Capabilities assignment
- No longer need to duplicate data
If this is of interest to you, please check us out at Cribl.io and get started with your deployment. If you’d like more details on installation or configuration, see our documentation or join us in Slack #cribl, tweet at us @cribl_io, or contact us via email@example.com. We’d love to help you!
Enjoy it! — The Cribl Team