The temptation of the core services team

One of the things I see a lot of companies doing once they start trying to get to grips with their development process is to look at building common services and components. Things like user authentication, customer notifications or file uploading that up until that point have been built multiple times with slightly different features in each project that needed them.

This thought process is a really good thing. It's a sign of maturity in the development process, of starting to think about the future and how to more efficiently deliver projects plural rather than throwing whatever's on the table currently out of the door as quickly as possible.

The next question is how to deliver those services. What tends to come up is the idea of a core services team - who analyse the requirements, construct best-in-breed solutions, evangelise them, and support their usage. This is... not such a good thing. It's a beguiling idea, for sure; take your best and brightest developers, and have them work on the really important stuff. Except for the small detail that it's not necessary, and in places actively harmful.

Core Services Team: Considered Unnecessary

I'll let you into a secret. When I worked at SDWS, we built, extended and re-used shared services constantly. By the time I left, the average product would consist of more reusable components than it did bespoke code. Do you know how many core services teams we had supporting that?

0.

What supported it was not a team, but an attitude: anybody can build something useful. Even the maintenance team were allowed to put together a reusable library if they thought it would help reduce future firefighting, and a lot of the feature development on core APIs and services came from maintenance deciding simply that things could be better.

Sometimes it took an architect to nudge a project team into realising that what they were about to build had wider applicability, but most of the time I was impressed how often even the most junior of developers would say, "hang on, isn't this something we can make reusable?" and their team would rally round the idea.

That's not to say we didn't encounter the occasional bout of Not Invented Here syndrome or fight over whose nascent library was the best of breed, of course. But that leads me on to the next problem...

Core Services Team: Considered Harmful

So you have your core services team. They wander round your development teams, poking at architecture and picking at problems. They find a few common problems, do a bit of gap analysis, and come up with the gold standard implementation. They disappear into the bunker for months and emerge blinking in the sunlight with the service to rule all services.

Unfortunately, during this time the following has happened:

• Teams who couldn't wait for the masterpiece reference implementation have gone off and created their own services with wildly incompatible interfaces.
• Nobody really wants to spend a ton of development effort integrating with a different version of a thing they already have.
• Being developed in isolation inevitably results in the gold standard implementation being divorced from the reality of what the applications actually need.

The typical next phase is that the regular development teams are beaten with a stick until they agree to use the shared service. Teams being forced to do work they don't see any point in always ends well:

• Missing features become a pitched battle between, "this service is useless" on the one side and "you shouldn't even need that feature" on the other.
• Everything that goes wrong is the fault of the other side. Debugging only starts in earnest once every finger-pointing opportunity has been exhausted.
• Two or three services in, the core services team is so mired in firefighting and support that they don't have the time or energy to effectively generate anything new.

Why does this happen? Well - I made a point many months ago about the idea of ownership being critical to how a team approaches its responsibilities. When you make a core services team, you're effectively creating two streams of abdicated responsibility:

1. "I'm never going to have to write a shared library. I don't have to care about problems like interface compatibility or supporting multiple subscribers. As long as my application works, everything's fine."
2. "I'm never going to have to write a customer-facing application. I don't have to care about complex requirements or holistic performance. As long as my service maintains its purist architectural vision, everything's fine."

Now, let's abandon the core services team. Flip this around to a situation where you have two teams - but this time, each has one customer-facing application and one shared service. Here, both teams understand the difficulties of being given requirements which your services don't support. But they also know the pain of grafting arbitrary and unrelated pieces of functionality onto a previously coherent API. (And for that matter, the pain of trying to stay integrated with a constantly moving target). They understand each other's problems, and that gives them a much better chance of collaborating rather than fighting.

As an illustration, one of the biggest steps we took toward shared services at SDWS was when I started challenging developers to stop seeing our original (and at the time only) core service as the domain of solely the old hands: "it's your code too, make the change you need and we can review it together". People started to understand the difficulties of API development, but they also realised that providing they navigated those difficulties carefully, they could fix their frustrations as consumers. By the time I left, a team was rewriting the entire package search capability while considering versioning and migration - not as a core services group, but because their project needed the functionality.

But why?

There are two reasons for the formation of a core services team that crop up more often than not:

• Organisations don't feel their developers are "good enough" to work on services that will underpin multiple applications.
• The organisation doesn't feel the projects they have coming through support writing reusable components.

If you find yourself thinking this, you need to be asking some searching questions. How are those developers ever going to get good enough? If your projects don't support generating reusable components as an obvious dependency, how is creating such components completely divorced from any project requirement going to help?

I keep using SDWS as the example, but mainly because it's a good one: we had those problems. We probably weren't "good enough" developers at the start; our early shared services were Byzantine complexities with illogical interfaces and abstractions that leaked almost every underlying implementational detail of the systems below. But we learnt - and it turns out that if you have your collaborative working in place, only one person on each team needs to be "good enough" for the whole team to be "good enough".

Project flow was the other problem. At first, we had what we pejoratively called an "agency mindset"; throw a project out the door as quickly and cheaply as possible, nuke the site from orbit and start again with no time for reflection. Because we weren't reflecting, we weren't even seeing the opportunities for re-use: teams were building three slightly different variations on a results table for three projects and refusing to believe there could be any commonality to take advantage of.

I can't claim more than partial credit for this solution, but I can tell you what we did: we changed our culture. As we grew, we started viewing ourselves less as a retail firm - "get it out the door, time spent programming is time spent not selling" - and more as a technology firm: "What are we going to build of value today?" We started to realise that projects could generate worth beyond simple immediate cash flow. Once we started looking for opportunities to build reusable components as a part of our projects, they became obvious.

(Although again, the start was a little shaky. We started out with a fair number of supposedly world-conquering ideas that never made it beyond the application which spawned them... but that was a vital part of learning the difference between an opportunity to make a shared service and an opportunity to wildly over-engineer something simple of limited applicability.)

How to solve the problem

I'm still looking for the solutions. It's likely there's no silver bullet, and what you do will be as much informed by your organisation's culture and where you want to head as it will be by advice from the likes of me.

That said, I find myself leaning more and more toward the idea of "internal consultants" these days; a collection of architects, tech evangelists and just straight-up good developers who can embed within a team for a few sprints, help them get over the initial hurdles, and steer them away from the dead ends.

I've used this model for everything from Scrum process to building core services (and even things as basic as simple mechanical code quality) and it turns out that having someone who knows their shit in the middle of a team supercharges the learning process to a point where what takes months to figure out alone is internalised within weeks. You also have the advantage over a dedicated core services team that all of your developers end up at the same high standard, not just one tiny subset.

The key is not to give in and take what looks like the easy, obvious route. It's not effective, and you simply don't need to do it to get to a point where you're regularly building shared components by instinct.