Data Mesh is completely changing the perspective on how we look at data inside a company. Read what is Data Mesh and how it works.
Hi! If you got here is probably because:
A) you know what Data Mesh is;
B) you are about to start (or already started) a journey towards Data Mesh;
C) you and your managers are sure that Data Mesh is the way to sky-rocket the data-driven value for your business/company; but
D) you are among the few who have bought the thing in and around you there are skepticism and worries.
The good news is: you’re DEFINITELY not alone!
At Agile Lab we’re driving Data Mesh journeys for several customers of ours and, every time a new project kicks off, this roadblock is always one of the first to tackle.
Hoping to help out anyone just approaching the problem and/or looking for some tips, here are 10 ways we found out being somehow successful in real Data Mesh projects. We’re gonna group them by “area of worry”.
Photo by Karolina Grabowska from Pexels
Well, you know, in the IT world every decision is usually based on the price VS benefit trade-off.
Domains are expected to start developing Data Products as soon as possible and we need them aboard, but usually “there’s no budget for new developments”, or “there’s no budget to enlarge the cluster size”. There you go:
This is a general architectural decision rather than a tip: you should design your output port (and their consumption model) so to provide shared governed access to potential consumers: with simple ACLs over your resources, you can allow secure read-only access to your Data Products’ output ports so that no compute resources are necessary at the (source) domain/Data Product level in order to consume that data, all the power is required (AND BILLED) at the consumers’ sides.
Ok, so let’s say we figured out a way to avoid (reduce?) TCO for our Data Products. But I’m what about the budget to develop them?
In complex organizations there are usually several budget streams flowing for operational systems maintenance: leverage them to create Source Aligned Data Products (which are flawlessly part of the domain so nobody will argue)!
Very well, we’re probably now getting some more traction, but why should we maintain long-term ownership over this stuff?
I know, this is bold and, so far, still in the utopia sphere. But there’s a key point to catch: if you figure out a way to “give something back” to Data Products’ creators/owners/maintainers, being that extra budget for every 5 consumers of an output port, or participation by the consumers to the maintenance budget … anything can pull up the spirit and diffuse the “it’s not a give-only thing this one” mood. If you get it, you win.
But let’s say I’m the coordinator of the Data Mesh journey and I received a budget just for a PoC… well, then:
No big bang, don’t make too much loud (yet), pick the right PoC use case and make it your success story to promote the adoption company-wise, without risking too much.
This will probably make the C-level managers happy. The same ones who are repeatedly asking for the budget to migrate (at least) the analytical workloads to the Cloud, without receiving it “because we’re still amortizing the expense for that on-prem system”. Well, here’s the catch: if they want you to build a Data Mesh, you’re gonna need an underline stack that allows you to develop the “Self-Serve Infrastructure-as-a-Platform”. That’s it:
Your domain teams will probably be happy to fast forward to the present days technology-wise and work on Cloud-based solutions.
This last tip brings to light the other “area of worry”:
Photo by Jens Johnsson from Pexels
Data Mesh is an organizational shift that involves processes and, most of all, people.
People are usually scared of what they don’t know, especially in the work environment where they might be “THE reference” to everybody’s eyes for system XY, while they’d feel losing power and control by “simply becoming part of a decentralized domain’s Data Product Team”.
First of all, you’re gonna need to:
Training and mentoring of key (influencer?) people across the domains is a successful-proven way of making people UNDERSTAND WHY the company decided to embrace this journey. Data Mesh Experts should mentor the focal point so that they can, in their turn, spread the positive word within their teams.
But teams are probably scared too of being forced to learn new stuff, so:
Do they really wanna miss the train? Changes like this one might come once every 10 or 20 years in big organizations: don’t waste the opportunity and get out of that comfort zone!
Nice story, so are you telling me we have organizational inefficiencies and we don’t make smart use of our data to drive or automate business decisions? Well, my dear, it’s not true: MY DOMAIN publishes data that I’ve been told (by a business request) should be used somewhere.
I see you, but understand that your view might be limited. There are plenty of others complaining about change management issues, dependencies issues, slowness in pushing evolutions and innovations, lack of ownership and quality, etc.. etc.. etc …
You will be surprised by the amount of “hidden technical debt” that will come out, and people will start empathizing with each other: this will start transferring the “My data can bring REAL VALUE” (because real people are complaining in front of them) to the organization.
Does this end once the Data Products have been released? Absolutely not! We talked about «long-term ownership» so we need to provide «long-term motivation»:
This can start as simple as «Number of consumers of a Data Product» (what if it’s zero?) and evolve in more complex metrics like the time-to-market (it’s expected to be reduced, it’s the time to bring into production a new Data Product), the network effect (the more the interconnections across Domains, the more value is probably added afterwards), and other metrics more specific and valuable to your corporation.
You convinced me. I will develop Data Products. Where do I start?
Following the DDD (Domain Driven Design) practice the first pillar of Data Mesh is based on, Domain-oriented Decentralized Ownership, and adding it up to the Data-as-a-Product one, we get a “business decisions”-driven design model to identify data opportunities (i.e. Data Products to be created in order to support/automate data-driven business decisions).
On this topic, you might be interested in the Data Product Flow we designed, or you can learn more about how Witboost can get your Data mesh implementation started quickly.
Speaking of implementation, read our white paper on Successful Data Mesh Implementations.
Also, you can talk to our experts by clicking the button below.
Data Mesh is completely changing the perspective on how we look at data inside a company. Read about what Data Mesh and how it works.
Data Mesh is completely changing the perspective on how we look at data inside a company. Read what is Data Mesh and how it works.