Resources for web development with Scala
Scala tags. This library is for creating HTML from Scala. Thanks lihaoyi
ScalaCSS a library for creating CSS from Scala
Stream processing is useful for many things, including real-time suggestions to users about content that may interest them based on their actions. You can feed streams of events to an event processor to have it search text. Designing Data Intensive Applications by Martin Kleppmann has great information about Streaming and Stream Processing in Chapter 11 Stream processing, this chapter covers the subject in great depth. If you don’t own that book you absolutely should if you are a software engineer. This book doesn’t cover Microservices, however it covers all of the software you use to create microservices and explains each in depth. If you are engineering any kind of modern app you either have that book or you are lost as to what you are doing and haphazardly choose software because it is all hypie and cool sounding. This book explains the internals of most Databases, NoSQL, RDBMS etc., it really gives you the info you need to make sound decisions. This book will save you hundreds of hours of raw research.
There are many stream processors, below I list a few.
Event storming is a process of modelling software that works well with domain driven design principles. During an event storming session all involved parties get together and help model a domain, this helps everyone from the developer to the business decision makers understand the system and from a ubiquitous language ( common language). Event storming is just another modelling tool to help with quick design and development.
Event Storming 101: Accelerating Your Software Development in Domain-Driven Design This article covers the absolute basics of Event Storming what it is and how it is used.
Here is another short article about event sourcing. This article includes many useful links at the bottom.
This video is from the man who created the concept of Event Storming. In this video Alberto discusses how he invented this technique of Event Storming with PostIt notes. He describes and demonstrates how this sort of Brain Storming session is a great way to get all involved parties together. This technique will help you more quickly and completely model your software. I’ll be writing about this technique in my articles about microservices with Scala.
Before you get too excited and go all functional in Scala you should know that 100% Functional Scala can be serious Memory hog because you are making copies of everything everywhere because functional means immutable data. Be careful with things such as map and flatmap they create lots of extra objects.
Avoiding loops for the sake of Functional programming strictness is a smooth brained action. When programming avoid hype, understand what you are doing, what your goals are and figure out how to get there. Never blindly ride the hype train.
This article lihaoyi points out some of the inefficiencies of Scala collections for example.
Scala Closures – article about closures in Scala
Pure functions in Scala – Article about what pure functions are in Scala
Basic Functional Programming in Scala – a short article covering some of the basics of Function programming with Scala
GOTO 2018 • Functional Programming in 40 Minutes • Russ Olsen
When learning about Domain Driven Design one of the harder things is wrapping your head around what aggregates are. In this post I list some facts, some may be redundant, but redundancy helps reinforce concepts.
Here is my personal list of facts and notes about domain driven design aggregates in no general order. This list will be updated as time goes on.
Developing microservices with aggregates – Chris Richardson
I found this to be one of the best explanations of aggregates and how they relate in microservice based systems. Here is a link to the site mentioned in the video microservices.io where you can find a lot more useful information about microservices.
Domain Driven Design and it’s related concepts can take some time to understand. Below are links to articles, books and videos I have found helpful.
Here is a free guide to Domain Driven Design written by microsoft. This guide covers CQRS and Event Sourcing complete with an app that can be downloaded and inspected while reading the guide. This covers bounded contexts and suggestions for finding them and much more. This is an excellent free source.
Effective aggregate design series vaughn vernon.
An introduction to event storming: The easy way to achieve domain-driven design
Modelling Reactive Systems with Event Storming and Domain-Driven Design
Domain driven design isn’t always the best design choice for all systems. This article details the many reasons why.
Why I don’t do Domain Driven Design.
Tactical domain driven design. More information on domain driven design.
Domain Driven Design Quickly – a free eBook summing up DDD the Eric Evans book.
Finding your domains bounded contexts is an iterative process. In this video Nick covers some of the ideas behind this iterative process of mapping out your domains bounded contexts.
Nick tune is funny. This is a really good talk about DDD Domain driven Design.
Dependency Injection is evil. The downsides of Dependency injection.
Semantic versioning explained – how to version your software
Apache Archiva™ is an extensible repository management software that helps taking care of your own personal or enterprise-wide build artifact repository. It is the perfect companion for build tools such as Maven, Continuum, and ANT.
Archiva offers several capabilities, amongst which remote repository proxying, security access management, build artifact storage, delivery, browsing, indexing and usage reporting, extensible scanning functionality… and many more! Get the latest updates, follow us on twitter @archiva.
As I find useful resources to AKKA I will list them here.
Akka Documentation. The documentation is a great place to start.
Slideshare Akka Microservices. This slide series shows the related DDD concepts and how they relate to AKKA and Scala.
Here is a link about why you shouldn’t create too many akka actor systems in your application.
Akka anti-patterns: too many actor systems
Learning Akka by Jason Goodwin. I like this book because it is small and to the point. At about 250 pages this book jumps right into AKKA, it doesn’t tell any stories and the examples are concise and to the point. This is also a great starting book for Java developers looking to get started with Akka because the author shows how to do everything with Java then with Scala. What immediately grabs your attention as you are reading this book is… how much more code it takes to do everything with Java. <– And that is why I don’t Java anymore. Java is like writing a novel to describe a page. This book mostly covers the basics of AKKA it doesn’t go into depth about all of the AKKA features and abilities. This book is just a nice, clean, overall introduction to AKKA with Java and Scala. If you are new to AKKA this should be your first book.
AKKA concurrency by Derek Wyatt. This is an older book, however it is still relevant. This book is kind of long at over 500 pages, but it digs into how AKKA actually works. What is nice about this book is it starts with an example and follows through with iterative changes as the book proceeds. The example is of a plane and flight crew and how all of the parts ( actors ) of the plane and crew interact to make the plane work. This book is more advanced and in depth than Jason Goodwins book, but I feel it explains the reasons behind what you are doing with AKKA. I’d read this book after reading Goodwins book, as it digs a little deeper into things giving more info on how AKKA works.
Those are the words I will always remember. About 5 years ago I was talking to a retired Engineer in South Florida about an idea I had for an app at the time. His response was “Build it modular!”. At the time it was already too late I had built the app and launched it and watched it flounder and fail. Those words haunted me and lead me to ultimately research modular software architecture in depth. I was determined to never again build a Monolithic Big ball of mud.
One of the main reasons the app failed was because every time I would fix a bug another would popup. I had built a monolithic Big ball of mud , It was the way I had learned to build programs in college and books etc. I had even gone as far as creating UML ( unified modeling language ) diagrams and EER ( enhanced entity relation model ). The problem with those is you change your mind while coding, so you have to also change your UML and EER or it gets out of sync with your code and database. It was thought to be helpful to have the UML and EER’s match the actual code and database as basic documentation of the system for lead engineers. I now prefer agile modelling methods.
Many college degrees, online courses etc. don’t cover modular software design. Most of the focus is on programming and making things work. While learning to program, most of the code we learn from as examples is fairly small. Inevitably as developers at some point we start building side projects. These side projects, especially when we are just beginning, start out as simple ideas and slowly grow. Eventually these projects turn into “Big balls of mud” where the code is tightly coupled and nearly impossible to add to or change.
One subject that any developer aspiring to reach the top should invest time in is Domain Driven Design (DDD) . Domain driven design sounds complicated at first with all kinds of buzzwords like “Value Objects” and “Bounded Contexts”, but it helps you break down large applications and create a language everyone from the business leaders to the developers can understand. This breaking down of the domain ( aka your project) into smaller pieces, naturally leads to a modular design, but it is how you wire it all together that makes the difference.
By using the concepts of DDD you can more easily discover your domains bounded contexts ( aka modules). This is not always easy and you won’t always get it right the first time, it is an iterative process. Within each bounded context ( module ) you have aggregates, these are interacted with by calling api methods that work with each other to perform the tasks of the module. These modules are what we consider microservices when going the microservice route. Finding your aggregates can be sort of hard. Here is a link to more information about aggregates. Here is another helpful link about aggregates.
Each module should have one and only one responsibility, it should perform one single function/operation. The login module shouldn’t contain logic to also register a new user for example. Each module should be responsible for and have 100% control over it’s data. This means that one module should not access the data of another module. You may have a user_authentication table for example that stores the users password and email. Your system should have a UserAuthentication module and the login module should use it.
Modules can communicate via (application programming interface ) API interfaces. These interfaces simply consist of commands you can send to the module to perform tasks. Basically the API interface is a file, a list of functions, a class( depending on the language/framework or your style ) a main method, it is the router of the module routing requests ( calling code ). The API Interface allows you to contact the aggregates of modules to get work done. If you have ever created a class with public methods then you have created an API. The public methods of that class are the Application programmers interface for that class.
In Domain Driven Design these API Interfaces are known as aggregate roots and the root is made up of aggregates. Aggregates do things, like the methods of an API do things. These aggregate roots are how you interact with modules/microservices to complete tasks.
There are many ways to integrate your modules ( make them communicate, glue them together ) into a larger system. There are message queues, RPC (remote procedure calls) REST ( representational state transfer ), actor frameworks and more. Which system you choose depends on the circumstances of your app. The main thing is finding your bounded contexts within your domain and splitting your app into modules.
For more information about how microservices/modules can communicate see Chapter 11 of Patterns, Principles and Practices of Domain-Driven Design by Scott Millett and Nick Tune Another good source on how microservices can communicate is Chapter 4 of Building Microservices by Sam Newman. In a future series of blog posts I will cover how to use AKKA/Lagom to build an example app and explain the concepts in detail.
The main goal is to not create a Big ball of mud monolith. The idea is to divide and conquer, to be able to chunk our larger monolith app into smaller more manageable pieces, that can be split among teams and developed autonomously then combined into one working system. If we can properly find our bounded contexts and make those our modules, we can turn those modules into microservices.
Basically building modular is building microservices. Microservices are hard, they are essentially a distributed system. With microservices you have to deal with many things like Service Discovery, Load Balancing and much more. With a monolith your system is tightly coupled and hard to scale. What is the happy medium? Surely someone has thought about all of this and figured it all out.
Over a series of posts I will cover how to find a domains bounded contexts and how to translate that into Scala and AKKA/Play/Lagom Framework code. The problem I’ve had most when learning all of these concepts is, how does all of this work together? How do I break the system down then write the code in Scala and have it all not fail later? In the series I will break down an entire Social Platform and describe the why I make the design decisions I do. I will show what structures in Scala relate to the concepts of Domain Driven Design. Sometime next year I will complete the Social Platform and offer a free partially built Scala app that interacts with the Social Platforms API to demonstrate all of the concepts.
Reactive Microservices Architecture: Design Principles For Distributed Systems This is an ebook that covers the concepts mentioned in this article.