Welcome to the OpenCore blog! We aim to publish articles regularly, highlighting some of the things we learned in the field that might be of value to our customers. The blog will give a small taste into the kind of content customers have access to in our extensive knowledge base.
This first week's installment will give a quick introduction to who and what we are. Starting next week we'll begin publishing the meaty stuff. Not interested in the boring business details but want to learn how to access a secure HBase cluster from within your Spark job? Look here and subscribe to our blog or Twitter account and you'll get notified when the next post is up.
If you are interested in the details keep on reading. We're going to do this in the form of an FAQ.
This is going to be slightly long so feel free to skip whatever is not of interest to you and if you want to talk to an actual person just contact us and we'll get in touch.
OpenCore is a German company founded in January 2016 by three colleagues Lars George, Sönke Liebau and Lars Francke. We've surely met some of you. Those who we've not had the pleasure to meet yet can read up on our bios here and we'd love to chat so please give us a shout!
Our background is Hadoop, Big Data, Search and all the related technologies. Lars George is an HBase committer, author of HBase: The Definitive Guide and all around Hadoop expert, Lars Francke is a Hive committer, HBase contributor, long-time Big Data consultant and Sönke has been focusing on Elasticsearch, Kibana and all the related technologies like Kafka.
Having said that it's only natural that we're going to focus on these technologies in our new company. Since we couldn’t find a proper name to cover all of the relevant technologies we went with calling it emerging data technologies to distinguish them from other older established technologies.
In addition to the technologies mentioned above a few more that we're working with are Spark, Kafka, Oozie, NiFi, Impala, Kudu and related technologies. This is an ever evolving list.
Glad you asked. We're starting with three things:
All of these offerings are a direct result of (unmet) demand we've had from previous customers and I'll go into some detail now - hopefully without boring you.
Consulting
You're probably familiar with what this is, right? This is where you need us on-site or remote to help you with specific issues you might have around any of the technologies we support. Now, while we'd love to, we can't be on-site for six months (that wouldn't be fair to other customers). So we're there to help for critical parts of projects:
Need to transform your existing ETL workloads to take advantage of what Hadoop has to offer? We know how that is done. Want to build an interactive serving layer backed by a NoSQL database such as HBase? Well, you can talk to someone who has written the book about it.
This is either through shorter on-site visits (a week to a month), remote support, or workshops to get everyone on the same page. This is how we've worked in the past and it worked great but we're open to any model that suits you.
Should that not be enough we've got partners with the resources to help you out in longer-term projects. We see ourselves as the experts you hopefully don't need that often but we're here when you do (“Oops, where did my HDFS Metadata go?”). So whatever you need: Contact us, we can either help you directly or refer you to a trusted partner.
Training
This is something our partners already excel at. They provide good training for all the basics you'd need as well as specialized trainings. We do not aim to replace those, in fact we're training partners for some ourselves so we're happy to deliver those to you. But we want to build on them where it makes sense.
One example that we're seeing quite frequently is that of advanced Hadoop operations trainings. This is where you'd actually get to play with a cluster and see how to deal with it when something fails (“Uh oh...I've just lost a hard drive, what do I do?”). By also covering a broad range of topics and not being a vendor ourselves we can also provide custom tailored trainings for you (for example a bit of Elasticsearch, a bit of Kafka and a bit of Hadoop mixed together). This way you can get by with just one training instead of multiple separate ones.
Sounds good? Excellent! There's even more though.
Let's say you've done a training a year ago and really really don't want to listen to us go over all the basics again (Yes, by now you know how HDFS splits files into blocks). So what we offer is a long-term relationship tailored to your needs: We can come by every so often (how does every six months sound?) and just give you the overview on what's changed in the past months in all the projects that are of interest to you.
This way you don't need to send your employees to a three-day training course every year where only half a day is relevant. Instead you get (and pay for) only what you really need.
As usual: Contact us if that sounds interesting.
Insight
Last, but definitely not least, is something we like to call OpenCore Insight. This is a brand new service that does not yet exist on the market. It is once again borne out of past experiences with customers.
This is a subscription service (that ties into the two other services in multiple places but I'm getting ahead of myself) where you get access to two things:
Access to help when you need it via phone and Email (well we're also happy to jump on a screen share session or SSH if that's possible). This does not replace the support you get from your vendors but is in addition to that and for different kinds of question. One example would be something that's coming up fairly often in the last few months: "You know our architecture, right? Can you give us an overview of the current state of the art of backup solutions for Hadoop? We're going into production soon and we need a backup solution but are not sure what to do." This is something where you don't need to pay someone to come on-site (but if you want to we'll obviously also come to your offices) and is a perfect example of something where a phone conversation with a follow-up mail can help tremendously.
Access to web-based services to help you with your day-to-day business. The first and obvious example is a knowledge base where we collect everything we learn at our clients so you can benefit from it. Other examples are an improved way of searching for JIRAs ("In which CDH version was this JIRA actually integrated?"), Mailing list posts, Tweets, Blog posts and other things. You'll also get regular updates on what's happening in the bug trackers and source code repositories as it's not always easy to translate the technical language into something that makes sense to your business. We're also working on more tools and are always open to ideas.
So you can think of this as a retainer consulting service. You can get the help you need when you need it without the lead time attached to typical consulting gigs.
We have been working with open-source software for the last decade and have invested (and are investing) lots of time and money into keeping up-to-date and maintaining good relationships with the people from the community. Of course you can do this yourselves but is that really your core business? Probably not. We're happy to invest the time so you don't have to.
So that's why you should choose us. Why now? Because these new technologies become more and more important for your business and it does not look like this trend will stop any time soon. And there's no central project management, no single contact person you can talk to like you used to with your usual vendors. We won't be able to replace that but we'll try our very best to help you navigate those projects and get what you need while respecting and working with the open-source community.
That's what the whole name refers to: At our very core we're working with open-source projects and giving back to them as well (just search the Apache JIRA for our names and you'll see that we mean it).
You're correct! Almost forgot to mention. We offer all three of our products in German and English. We haven't quite gotten around to setting up this blog multilingual (yet), but it is on our list and we aim to also publish all future posts in both languages.
Let us know how we can help you succeed with your Big Data and Hadoop ideas! Contact us at info@opencore.com today.
In fact we are! Head over to our Careers page or just send us a mail and we'll get back to you shortly.