Unwelcome…
… are platform instability, downtime, {hardware} failure, poor efficiency, cluster useful resource competition, repeated course of failures, runaway stay queries, crucial providers alarms, invisibility into alarm cacophony… the checklist goes on. If these are illnesses you want to treatment …
Welcome!
To this six-part sequence, the place we’ll have a look at the best way to get management of the well being of your Cloudera Data platform (CDP) atmosphere. Out of the field, CDP performs fantastically, however over time, if information structure, information engineering, and DevOps greatest practices are usually not maintained, the Knowledge Metropolis you’ve erected atop a stable CDP bedrock can change into the wild, wild, west. Maybe it’s time for some regulation and order to forestall additional crimes towards the tech.
Greater than a case research, we’ve interwoven greatest practices gleaned from a number of configurations and shopper websites right into a complete, straightforward to grasp set of directions to diagnose and resolve most of the points that adversely influence CDP environmental well being.
With every weblog we’ll define the signs and root causes of widespread environmental well being challenges and prescribe options. The place we are able to, we’ll embrace useful hyperlinks to step-by-step directions to information you thru profitable implementation. Once we conclude the sequence, we’ll share a homegrown software, an environmental well being scorecard, to observe and handle the well being of your atmosphere.
There are lots of, many causes that an atmosphere could carry out poorly, and positively some resolutions take effort and time, however there may be fairly succulent low hanging fruit. Our nice hope is that you simply discover impactful fast wins that encourage you to pursue a number of avenues of well being enchancment. You might also determine to accomplice with our Cloudera Professional Services crew who greater than doubled a buyer’s well being rating in two quick quarters.
Classes of CDP Environmental Well being
We’ve categorized elements of environmental well being for this sequence.
Visibility and Transparency
Into the cluster, platform, providers, and processes. We gained’t be capable of make a lot progress if we shouldn’t have correct visibility into the issues. That’s observability. On this weblog we offer directions and instruments on the best way to achieve visibility, suppress alarm noise, discover and analyze the basis causes of probably the most important alternatives, and proactively notify your customers when incidents happen
Knowledge Asset Standardization
Of widespread datasets, pipelines, processes, and studies. Admittedly, information asset standardization is a multiyear journey; however, addressing solely your most problematic and resource-intensive processes and belongings could yield extra environmental well being enchancment than every other class. We’ll share greatest practices on the best way to find and capitalize on these alternatives.
Platform Well being
Contains {hardware} and providers settings and configurations. Cloudera Knowledge Platform (CDP) have to be configured correctly to operate nicely with excessive efficiency. Moreover, as enterprise wants regularly change, so will your use of the platform, and that may necessitate re-tuning. That can assist you on that journey, we’ll checklist some widespread signs, hyperlink them to root trigger evaluation steps, present correct configuration tips, and description the steps to correctly tune your atmosphere.
The Proper Software for the Job
Contains the correct use of Impala, CDSW, Airflow, Nifi, and CM. You is perhaps shocked on the adversarial environmental influence of utilizing CDSW as an ETL pipeline software or utilizing Impala to jot down unwieldy queries with an embarrassing variety of joins. We’ve carried out it too. We confess. We’ll spotlight some great benefits of utilizing Airflow to handle complicated information pipelines with its facility to divide workflow into small impartial duties. We’ll checklist different do’s and don’ts.
Environmental Well being Scoring
Brings all of it collectively by demonstrating the best way to measure, rating, monitor, and management environmental well being via dashboards that we offer for you together with directions to hook them as much as your logs.
For those who’ve bought the signs, the medical doctors are in. Let the therapeutic start!
