Apache Knox Parcel & CSD for Cloudera Manager

written by Lars Francke on 2018-11-07

This might be interesting for any Cloudera CDH users out there who wish to use Apache Knox.

Knox is supported in Hortonworks HDP distribution of Hadoop but not with CDH. If you wanted to use it you had to install it manually.

Cloudera Manager does support custom extensions called Parcels & CSDs (Custom Service Descriptor) documented on Github here.

What we did is to build a Parcel and an accompanying CSD for Knox. It can be found on our Github: knox-cloudera.

The Parcel

This is pretty straightforward. A Parcel contains little to no logic at all.

This is currently based on Knox 1.1.0 and only requires changes to a few shell scripts for reasons documented in this mailing list thread.

We have included an Ansible playbook that creates the Parcel. We are not currently hosting a prebuilt one ourselves but if there's interest we might do so.

The CSD

This is more complex and contains the actual logic of starting, stopping and configuring Knox from within Cloudera Manager.

Here we also had to take a few shortcuts and the user experience is not what you're used to from built-in services.

But it can be used to bring up a Knox Gateway, configure it, create Kerberos keytabs etc. For changes to topologies you currently need to change the files on the Gateway server itself.

Request for Feedback & Sponsors

This is a basic version to get started. It's missing features as well as better documentation. It hasn't seen production deployment or usage so I'm sure there are things that are not working or missing. If you have any feedback, please let us know!

If you or your company are willing to sponsor some more work on this, please also get in touch. There are some improvements that could be brought back to the upstream Knox project.

We are also working on or planning to work on:

If you or your company are interested in sponsoring our work please get in touch!

Thanks

This work was possible due to a corporate customer who wishes to remain unnamed but who allowed us to open source this project. Thank you very much!

If you enjoy working on new technologies, traveling, consulting clients, writing software or documentation please reach out to us. We're always looking for new colleagues!