Inside changes to IT and dev culture.
NAB’s adoption of microservices under a $1.5 billion transformation is driving fundamental changes to internal team structure as well as to the bank’s IT architecture.
Engineering manager Kevin Littlejohn told AWS Dev Day Australia that he had seen the impact of microservices on team structure in a past role, and this experience was largely being repeated at NAB, albeit on a much larger scale.
Broadly, NAB’s transformation touches on three main areas: the adoption of microservices and APIs, deeper use of cloud, and data delivered as a service on the cloud.
The bank has discussed in detail the latter two aspects, but this is the first time it has publicly talked about its microservices journey and the flow-on effects.
“We talk a lot about microservices and team structure,” Littlejohn said.
“Where I am at the moment is we’re seeing a lot of shift from monoliths to API-driven systems, whether they’re microservices or something slightly larger or smaller.
“It definitely has an impact on the team structure.”
Littlejohn said that, while at a former employer, his team “took a large .NET monolith, broke it down into event-sourced microservices and an event-driven architecture – the whole bit.”
“We started with the technical change and got to see very clearly in a fairly small organisation what that does to the [team] structure,” he said. “It just naturally flowed out to small teams.”
In some ways, this simply reflected good practice.
“I’m a huge fan in most areas of reducing problems down to small batch sizes – the old ‘how do you eat an elephant? One bite at a time’,” Littlejohn said.
“It applies to almost everything in life.”
Adopting a microservices architecture meant having to “think carefully about the APIs between your components because those are the bits that are really critical”.
“What’s inside the components becomes less and less critical,” he said.
“You can apply the same logic … to your organisational structures.
“I haven’t seen this done much but think about your company and the organisational units in there – not just the tech teams, but the wider organisation – what’s the ‘API’ for each of those org units?
“What are their SLAs? What’s your service discovery mechanism through your organisation? How do you apply what we think of as good software engineering to the organisation?
“I think there’s a new wave of this coming as the microservices stuff rolls out. We’re going to start thinking about our organisational structures in those terms.”
Crawl-walk-run
NAB has repeatedly used a ‘crawl-walk-run’ analogy this year to describe its journey into the cloud.
Very briefly, the bank started in 2012 by putting its public-facing content and some non-production workloads into AWS.
It began to ‘walk’ in 2016 when it put in its first transactional workloads, but these are still being lifted into AWS one at a time.
“We’re moving into what we call a ‘run’ phase now where we’re addressing how we continue to do this at scale,” Littlejohn said.
“So it’s not just take a workload and move it up, take another workload and move it up – it’s how do we provide the foundations for the entire bank to allow developers to move up at scale and to develop rapidly in AWS.”
To standardise or not
There’s a large amount of background work currently underway to regear systems, processes, organisational structure and culture to be able to develop for the cloud at scale.
“Because we have a lot of legacy, we have change management, risk management, production readiness processes that are all geared towards a very old way of doing things,” Littlejohn said.
“We’re facing into those and trying to come up with new ways to demonstrate that we can get the same sets of results in a superior and consistent way across the bank through things like automation and cloud technologies.”
Achieving consistency in an environment with so many people and teams is challenging, Littlejohn conceded. The bank’s toughest problems are cultural.
“As we scale, a lot of our challenge right now is around getting the consistency across all the teams but in a way that’s not necessarily mandated,” he said.
“We don’t want to go out to the teams and say ‘you must do it this way’ because we’ll lose all of our ability to innovate.
“So the challenge for us at the moment is how do we enable the teams, how do we share architecture/good design and engineering across a lot of silos, and how do we enable the developers without locking them down but still make sure everything they do is within guardrails and is secure.”
Littlejohn noted the tension that exists between standardisation and freedom of choice – of tools or programming languages, for example.
Standardisation is “absolutely helpful” in a large organisation “if you want people to be able to move from team to team and problem to problem”, he said.
“It’s really valuable to be able to move from one team to another and know your CI processes look the same, the build servers are the same, and you understand how all this is put together.
“You don’t want every team having to pay the price of reinventing those things every time.”
However, he saw almost a conditional type of standardisation as being the right approach.
“I think there’s a role for that standardisation as an optional thing to say ‘here’s the out-of-the-box tools, these will give you an easy path to production and make your life easy, go with it. But if you want to do something different, here’s all the things you’ll need to tick off’.”
Littlejohn said NAB is also working to embed learning deeply within its culture.
“A lot of the work we’re putting in at the moment is to try and build that knowledge share, so that every lesson learned is available to everyone else in the organisation, and how you do that – whether you build Spotify ‘Guilds’ or run brown bag sessions or communities of practice.”
A ‘serverless’ future
Littlejohn also used the Dev Day session to explore where NAB is likely to land in the containers versus serverless debate.
Two years ago, most major organisations were going down the Docker path, containerising applications before pushing them to cloud.
However, the advent of serverless was widely seen as a way to bypass containerisation altogether, and perhaps the only thing that stopped that occurring in the early days was a lack of confidence over which would win out.
As it has turned out, both containers and serverless have landed significant enterprise followings. However, NAB sees its future in one camp more so than the other.
“We see a lot of move to containers as the first step. I won’t say a shift to containers is necessarily an easy step but it’s the easier step [of the two],” Littlejohn said.
“It’s often simpler to pick up an existing system, put it in a container, deploy the container on a cluster and then start picking it apart from there.
“Personally, I think serverless is going to win out eventually over containers.”
Littlejohn’s prediction is, in part, based on a much broader view of serverless than, say, AWS Lambda.
“One thing I’d call out – and I think it’s one that’s not often called out – is that serverless is more than just Lambda,” he said.
“I’ve got a little pet project that I’m working on at home at the moment that’s a Minecraft cluster on [AWS] Fargate with a little React frontend on it. I’m not going to stand up a server for anything [to do with this]. It’s all going to be completely serverless.
“It makes you realise that the value in serverless is not just ‘hey I can write a Lambda and run it on demand’; the value is in that entire ecosystem being at your fingertips and being able to say I need user authentication – there’s Cognito.
“I sound like I’m selling AWS, but I used to say a few years back that we’ve shifted to the point where your server is now your AWS account, your processes that you were running are now your resources inside.
“It is a slightly different way of thinking about things but it’s so much richer, and it allows you to move a lot faster.
“So I think that serverless – the wider serverless – will win out. I think containers is just one little implementation detail inside of all of that.”
