Build it Modular!!!
Those are the words I will always remember. About 5 years ago I was talking to a retired Engineer in South Florida about an idea I had for an app at the time. His response was “Build it modular!”. At the time it was already too late I had built the app and launched it and watched it flounder and fail. Those words haunted me and lead me to ultimately research modular software architecture in depth. I was determined to never again build a Monolithic Big ball of mud.
One of the main reasons the app failed was because every time I would fix a bug another would popup. I had built a monolithic Big ball of mud , It was the way I had learned to build programs in college and books etc. I had even gone as far as creating UML ( unified modeling language ) diagrams and EER ( enhanced entity relation model ). The problem with those is you change your mind while coding, so you have to also change your UML and EER or it gets out of sync with your code and database. It was thought to be helpful to have the UML and EER’s match the actual code and database as basic documentation of the system for lead engineers. I now prefer agile modelling methods.
We don’t learn modular programming.
Many college degrees, online courses etc. don’t cover modular software design. Most of the focus is on programming and making things work. While learning to program, most of the code we learn from as examples is fairly small. Inevitably as developers at some point we start building side projects. These side projects, especially when we are just beginning, start out as simple ideas and slowly grow. Eventually these projects turn into “Big balls of mud” where the code is tightly coupled and nearly impossible to add to or change.
How to find the modules of your program?
One subject that any developer aspiring to reach the top should invest time in is Domain Driven Design (DDD) . Domain driven design sounds complicated at first with all kinds of buzzwords like “Value Objects” and “Bounded Contexts”, but it helps you break down large applications and create a language everyone from the business leaders to the developers can understand. This breaking down of the domain ( aka your project) into smaller pieces, naturally leads to a modular design, but it is how you wire it all together that makes the difference.
By using the concepts of DDD you can more easily discover your domains bounded contexts ( aka modules). This is not always easy and you won’t always get it right the first time, it is an iterative process. Within each bounded context ( module ) you have aggregates, these are interacted with by calling api methods that work with each other to perform the tasks of the module. These modules are what we consider microservices when going the microservice route. Finding your aggregates can be sort of hard. Here is a link to more information about aggregates. Here is another helpful link about aggregates.
How do modules work?
Each module should have one and only one responsibility, it should perform one single function/operation. The login module shouldn’t contain logic to also register a new user for example. Each module should be responsible for and have 100% control over it’s data. This means that one module should not access the data of another module. You may have a user_authentication table for example that stores the users password and email. Your system should have a UserAuthentication module and the login module should use it.
How to communicate between modules?
Modules can communicate via (application programming interface ) API interfaces. These interfaces simply consist of commands you can send to the module to perform tasks. Basically the API interface is a file, a list of functions, a class( depending on the language/framework or your style ) a main method, it is the router of the module routing requests ( calling code ). The API Interface allows you to contact the aggregates of modules to get work done. If you have ever created a class with public methods then you have created an API. The public methods of that class are the Application programmers interface for that class.
In Domain Driven Design these API Interfaces are known as aggregate roots and the root is made up of aggregates. Aggregates do things, like the methods of an API do things. These aggregate roots are how you interact with modules/microservices to complete tasks.
There are many ways to integrate your modules ( make them communicate, glue them together ) into a larger system. There are message queues, RPC (remote procedure calls) REST ( representational state transfer ), actor frameworks and more. Which system you choose depends on the circumstances of your app. The main thing is finding your bounded contexts within your domain and splitting your app into modules.
For more information about how microservices/modules can communicate see Chapter 11 of Patterns, Principles and Practices of Domain-Driven Design by Scott Millett and Nick Tune Another good source on how microservices can communicate is Chapter 4 of Building Microservices by Sam Newman. In a future series of blog posts I will cover how to use AKKA/Lagom to build an example app and explain the concepts in detail.
What is the goal of modular software design?
The main goal is to not create a Big ball of mud monolith. The idea is to divide and conquer, to be able to chunk our larger monolith app into smaller more manageable pieces, that can be split among teams and developed autonomously then combined into one working system. If we can properly find our bounded contexts and make those our modules, we can turn those modules into microservices.
Modular is Microservices
Basically building modular is building microservices. Microservices are hard, they are essentially a distributed system. With microservices you have to deal with many things like Service Discovery, Load Balancing and much more. With a monolith your system is tightly coupled and hard to scale. What is the happy medium? Surely someone has thought about all of this and figured it all out.
Over a series of posts I will cover how to find a domains bounded contexts and how to translate that into Scala and AKKA/Play/Lagom Framework code. The problem I’ve had most when learning all of these concepts is, how does all of this work together? How do I break the system down then write the code in Scala and have it all not fail later? In the series I will break down an entire Social Platform and describe the why I make the design decisions I do. I will show what structures in Scala relate to the concepts of Domain Driven Design. Sometime next year I will complete the Social Platform and offer a free partially built Scala app that interacts with the Social Platforms API to demonstrate all of the concepts.
Reactive Microservices Architecture: Design Principles For Distributed Systems This is an ebook that covers the concepts mentioned in this article.