Microservice Resources
Most of us older people have been building monoliths for most of our careers. Now BAM! a new buzzyword is born Microservices. Microservices are all about tearing apart the dreaded monolith into smaller services that all function together to perform the actions of the monolith. Microservices have many benefits over Monoliths, however they have many disadvantages as well. Microservice systems are inherently distributed. Technology has changed so much recently. In the past we had to rent entire servers or virtual private servers etc.
Now we have the cloud, we can buy server power for super cheap, my favorite host Vultr.com has hosting packages for less than $5 USD. Vultr has the most awesome hosting interface for us Developers too. No intrusive demands for a credit card, you can pay with Paypal, Bitcoin and more.
This cheap hosting combined with containerization technology such as Docker/Kubernetes, makes getting into microservices easier than ever. The problem may be deciding if you need microservices or not.
I’ve spent about two years reading and researching Microservices below I list some of the Articles, Books and Videos that helped me along the way. I list these because there is a ton of old and bad information floating around the internettens and it took some effort to get to the bottom of what is currently the best accepted practices. This also gives me the ability to look back for information I can’t quite remember where I saw it since all I do is read, day and night.
Articles
Microservices – Not A Free Lunch! — This is a good article to start with. This highlights the fact that microservices are not always appropriate for every situation and you should need them more than you want them.
Microservices guide by Martin Fowler this is an excellent first start.
This link has links to many useful articles about microservices, this is a great place to start.
Microservices communications. Why you should switch to message queues.
This article covers how message queues work in a microservices setting. How to integrate microservices with message queues and why it is better than HTTP endpoints with microservices discovery and load balancing tools.
Implementing event-based communication between microservices (integration events) This article covers using message queues for event-based microservice communication.
Service Mesh VS API Gateway VS Message Queue – when to use what?
This is a small article that covers some of the pros and cons of each communication architecture style.
Microservice pattern explained. This article covers the idea of the microservice pattern. It covers the uses of the microservice pattern and compares it to the traditional Microservice pattern. This article glosses over communication between services. This article also covers the pros and cons of the microservice pattern. This article has good information about how to split your application into microservices, it has quite a few good links too.
REST vs Messaging for Microservices – Which One is Best? This article has some information about microservice communication and how it has changed over time.
Message Broker or Bus – what’s the difference? This is a good article that explains the minute differences between a Message Bus and a Message Broker. One of the worst things about the Software Industry is all the gimmicky terms and buzzwords, it is enough to make you want to move on to Physics where bosons, quarks and leptons are the buzzwords.
The Saga Pattern This article describes a useful microservice communication pattern called the Saga pattern. This pattern has it’s uses but I would not use it everywhere. This pattern allows a series of events to be reversed if a condition is not met or something along the line of communication fails.
Principles for Microservices Integration
This article has some good information about transactions between microservices. This article also covers choreography vs orchestration. Orchestration leads to a highly coordinated system which can create more tight coupling between microservices because some central authority orchestrates everything it must know about everything. A choreographed system decreases coupling by using an event sourced approach allowing microservices to publish events and decide what to do when an even they are listening to occurs. See the section about Choreographed vs Orchestrated in the article.
How Integration Patterns Impact Your Microservices Architecture This is a good article that quickly covers the most basic ways of integrating Microservices. This article also has some excellent Resource links at the bottom of the page.
Apache Kafka vs Enterprise services bus
While researching Kafka as a message broker I wanted to know what the difference between Kafka and an ESB was, this excellent article covers just that topic. It covers event sourced/based systems and how Kafka is used in this type of setup.
Positioning APIs and Services – Let’s End the Confusion
This article talks about SOA Software Oriented Architecture which was the prelude to Microservices. This article is from 2015, this is about the time the word Microservice was born. This shift from SOA to microservices has made researching microservices a pain in the ass since during the shift many things were copied over or renamed etc. It has been hard to find any sort of consensus on many of the patterns and software used to build microservices.
The article mentions how SOA uses Enterprise Service Buses (ESB) as a way to integrate services. It mentions that before ESB the services connected to each other directly. This article is actually an interesting read for those of us who have not been in the industry for 20 years, it is a great recent history lesson. Before SOA it was the monolithic architecture, the mega pains of which is what spawned the SOA culture. Once you are bitten by a monolithic pile of crap you never ever want to see or create one ever again, it is the worst horror filled nightmare one can endure.
From Microliths to Microsystems: Jonas Bonér at QCon London
This article quickly mentions some of the bad habits of building Microservices as mini monoliths which are hard to scale. It quickly covers the basics of what makes a good microservice. This article contains many handy links worth reading. The article is a summary of a Talk from QCon Here is a link to the slides From Monoliths to Microsystems. You an find the video and summary below under the same name.
Videos
Implementing Microservices with Scala and Akka
GOTO 2014 • Microservices • Martin Fowler
Martin begins this talk by asking “What is a microservice” and compares it to a monolith giving a nice definition of Microservices. This video gives a good overview of microservices. Here is Martin Fowlers website link to microservices, lots of good links here, the same as the top link of the page. This video contains a lot of good information, Martin talks about the past and how we ended up with Microservice architecture. In this video he explains the move away from Enterprise Service Bus in the idea of dumb pipelines and smart end points. This is a very good very short explanation of Microservices and the rules of Microservices. Martin also talks about how we should design our microservices to fail and use monitoring for self healing. He also covers Microservices vs Service Oriented Architecture (SOA). At about the 18:00 mark Martin Talks about Microservices vs Monoliths and when you should choose which architecture, this topic is covered fairly thoroughly. Personally I would say Always avoid the monolith Always unless you like pain and misery. Also this video is from 2014, things have changed especially with Container Technology like Kubernetes and Docker, which make microservices much easier now.
Greg Young – The Long Sad History of MicroServices
The talk starts at about the 9:00 minute mark. Greg makes a great point about microservices – everyone has their own definition it seems. This video makes the point that the software industry repeats everything over and over, and likes to rebuild the wheel. This video mentions smart endpoints and dumb pipes as well if you are wondering what that is.
This video talks about Software Oriented Architecture (SOA) and how we ended up with Microservices which are basically SOA done correctly, or applying best concepts to SOA. Turns out SOA was a buzzword too, just like Microservices. Microservices is basically just a new buzzwordy term for SOA.
The concepts of Microservices are very old, if you are an older developer trying to wrap your head around these microservice concepts then this video will help clarify a lot of things for you. Greg talks about many of the misunderstood concepts of Microservices like how many lines of code and microservice cohesion. This video also covers the Database per service idea vs a single database per company.
|Greg also talks about how Microservices should be asynchronous and event driven and why. At 46:00 greg talks about securing events, very interesting concept. He makes the point that microservices are not a cure for everything and it is sort of a fad.
GOTO 2016 • The Entity Microservice Trap You May Be Doing It Wrong • Fred George
I like this video because Fred takes the time to explain the history of software design and development and slowly leads up to Microservices. This video will help you avoid some pitfalls of Microservice design and help clarify what Microservices really are and how they work.
GOTO 2017 • DDD Today – Modeling Uncertainty • Vaughn Vernon
In this video Vaughn talks about some of the misconceptions we have when we first start learning about Microservices. Through a series of funny stories he covers how most people think of microservices and DDD and how we should think of the two. Microservices are distributed systems and they require modeling uncertainty. This video will also help clarify what is going on with microservices, how do they communicate with commands and events etc. I like the coverage of how the commands, events and microservices should work in this video. This video covers a lot and will help clarify some concepts of DDD and how they are used in Microservices.
Eric Evans DDD and Microservices: At Last, Some Boundaries!
This video covers Domain Driven Design and boundaries between microservices. Eric also covers some of the decisions that need to be made on how microservices communicate with each other.
Developing microservices with aggregates – Chris Richardson
Chris breaks down how microservices and aggregates relate in this video. This video answers lots of questions about how Microservices are built and communicate. It also covers aggregates and how to pick what goes into an aggregate. This video does a good job explaining how Event sourcing works as well. This is actually one of the most useful videos because it covers Microservices and DDD concepts at a very high view level, giving you an overall idea of what everything means. This video really helps tie many of the Microservice and DDD concepts together and will give you aha moments. microservices.io mentioned in the video half way through.
The Enterprise Architects Intro To Microservices Part 1 Kevin Webber
This video starts by covering some of the history of software design and how the industry arrived at Microservices. Kevin then covers how users expectations have changed since the internet was born and how these days people expect things nearly immediately. This video is an excellent explanation of why the microservice architecture is so important in the Enterprise world.
Kevin soon covers the history of SOA, how and why it was a failure because ESB was a single point of failure. This gives good coverage of Enterprise service bus failings. This video gives a great explanation of what a microservice is with a nice diagram. This is an absolutely excellent explanation of the Microservices Architecture or MSA.
The Enterprise Architects Intro To Microservices Part 2 – Jonas Boner
This is the second video in the Lightbend Microservice series. The talk begins at about the 3:00 mark, before is an introduction. This video starts with the differences between hardware in the past compared to today. Jonas talks about how microservices allow for better isolation between services and why this is good for preventing entire cascading system failure when one service fails. He then compares Microservices to the Titanic and how microservices allow natural bulkheading because of their isolation by nature. This isolation allows for self healing microservices.
Why Asynchronous communication?(15:00 mark)
Jonas then talks about asynchronous communication, why synchronous is bad and asynchronous is better. If you are wondering why everyone is saying you should use asynchronous then this video gives an excellent explanation with a nice diagram. You shouldn’t always use asynchronous communication, you can also use a Message Broker and pass messages.
What is microservice state? (~26:00 mark)
Jonas talks about microservices should own their state, they should own their data or database that is. This is one of the most important points of Microservices. One microservice should not reach into another microservices database either to write or read. He talks about the fact that there is no such thing as stateless architectures (~28:00 mark) scammy buzzword.
Bounded Contexts (~29:00)Jonas talks about bounded contexts in relation to microservices. Event Logging and Event Sourcing explained(~31:00 mark)
The Enterprise Architects Intro To Microservices Part 3 – Jonas Boner
This is the third video in the Enterprise Architects series. This video starts out by giving a good description of microservices and how they should work. Jonas talks about how microservices are distributed systems by nature and the sort of things that need to be considered when designing your architecture. Jonas makes the point that no matter what in a microservices system you will be dealing with stale data and inconsistencies and that is just the way it is. Inside data vs Outside data (~10:00 mark) this is a good concept to understand. One microservice communicates to another it wants something done via a command. Commands should have a side effect. The challenges of microservices (~11:30 mark ) Service discovery (~13:00 mark) is covered. Anti-corruption layer discussed (~16:15 mark) Api gateway discussed (~17:00 mark) Communication challenges discussed (~19:00 mark) Integrating with other systems discussed (~21:00 mark ) Circuit breakers explained (~22:00 mark) Back pressure explained (~24:32 mark ) Securing microservices discussed quickly (~26:00 mark ) Saga Pattern discussed (~35:45 mark) More about Saga pattern here.
Books listed in video
Reactive Microservices Architecture: free O’Reilly report by Lightbend CTO Jonas Bonér
Developing Reactive Microservices: free O’Reilly mini-book by Java Champion Markus Eisele
GOTO 2019 • 3 Common Pitfalls in Microservice Integration & How to Avoid Them • Bernd Rücker
Bernd is an entertaining person and he is quite funny in this video at times. Here is a link to the article mentioned in the video. He mentions in this video one of the biggest problems with microservices is they are distributed systems and many developers don’t know how distributed systems work. Developers building distributed systems should plan for failure (3:00). At about 5:30 Bernd mentions using REST for service communication and the issues involved. At about 8:30 he talks about circuit breakers with REST communication. Handling service state ~15:30. Workflow engines ~18:00 . Idempotency ~24:30 and why it is important. Asynchronous communication ~32:30. Message Bus 34:30 and monitoring dead messages. Distributed transactions ~38:30 Saga pattern mentioned ~41:00
Lost in Transaction – Strategies to deal with consistency in distributed systems (QCon London 2019)Bernd Rücker
Bernd talks about transactions, as in when you have two or more things that depend on each other. This is the video mentioned in the video above 3 Common Pitfalls… At ~3:30 he mentions Two Phase Commit (2pc) and why it doesn’t work with distributed transactions. Idempotentcy and why it is important ~10:00 and it’s basic implementation. Stateful retry strategy ~15:45 describes retry strategies with a state machine. An example of using a message queue pattern ~22:30 Saga pattern described ~27:30 and how it got it’s name, I hate the name. Choreography vs Orchestration event-driven pattern ~29:00 He goes on to talk about why fully 100% event driven systems are not the best way to build microservices. At ~35:00 he starts to talk about alternatives involving orchestration using commands. Orchestration with Saga workflows and BPMN(business process model and notation) pattern ~36:30
Distributed Sagas: A Protocol for Coordinating Microservices – Caitie McCaffrey – JOTB17
Sagas are a way to orchestrate transactions between microservices that need to either all succeed or all fail. This video was mentioned in the video above “Lost in Transaction”. Caitie speaks a little too fast, you may need to adjust the video speed down some. This is a great video that does a good job of explaining Sagas and what a Distributed Saga is . She covers what 2pc ( two phase commits) are and why they don’t work because they don’t scale well. She then goes on to talk about distributed Sagas. A saga is a way to reverse transactions when one fails in a series. The talk about Distributed Sagas starts at ~15:00, this is the definition. How to define a distributed saga ~ 21:30. Distributed Saga log ~23:00, basically using a log to move a multi-transaction process forward or reverse it in error. Failures in Distributed Sagas ~27:00, explains how to perform a Saga Rollback. Each of your microservices need a way to reverse transactions.
Avoiding Microservice Megadisasters – Jimmy Bogard
There are some things to avoid when designing a Microservice Architecture (MSA). In this excellent video Jimmy highlights some of the “do not do’s” of Microservice design. He does this by telling a story of history to help us understand how we got to where we are and the mistakes we made along the way. At ~10:00 he starts to cover API hell, and how it is created by microservices calling microservices calling microservices via HTTP requests tying up the whole network. This is naturally how most of us think of structuring microservice communication until we dig much deeper and find videos like this that explain why it is bad. At ~14:00 Jimmy explains how we can be naive as developers by only locally testing our microservices with mock outs and never tapping into a live system for testing, this is a very important point. At ~15:00 he points out an older Microservice design that failed. So many of the ideas on how to design Microservice architectures is wrong even though they seem logical. At ~17:00 he points out what a mega failure the management of the IT industry is, basically how not to run a company. At ~19:45 he gives the definition of a Microservice. At ~20:30 he defines and explains what a DDD bounded context is and how it relates to Microservice Architectures (MSA) At ~24:00 Jimmy gets to data duplication and why it is needed in Microservice architectures. At ~27:00 he explains Service Dependency Inversion and why it is needed in a Microservice Architecture. This dependency inversion is a great design for searches especially. At ~38:00 he again points out how not to run a business, especially not a software business. Idiot managers. At ~39:00 Jimmy’s law.
GOTO 2017 • The Seven (More) Deadly Sins of Microservices • Daniel Bryant
Link to the slides used in the video.
Continuous Delivery with Containers:The Good, the Bad, and the Ugly free book by Daniel Bryant
Containerizing Continuous Delivery in Java – another free book by Daniel Bryant
Daniel starts out by telling a short story about microservices, a very good talk about some of the history. At ~6:30 he makes an excellent point about moving to Microservices, you really need to plan your architecture well. You need to think about what technologies you will need in your architecture to make it function, the more you plan the better. At ~8:45 to containerize or not, he talks about Docker and containerization. GRPC ~12:00 quickly mentioned. Service Mesh’s covered ~14:00 More about RPC ~15:00 comparing it to event sourcing Link to the article mentioned at ~15:30 From Microliths to Microsystems: Jonas Bonér at QCon London Next up ESB ( Enterprise Service Buses) ~16:30 these are used in SOA (Service Oriented Architectures) which were popular before Microservices became a word. Microservices are a continuation of the SOA concepts, taking the good leaving the bad basically. Non functional requirements ~23:00 He makes another good point of things you need to consider upfront when designing your microservice system. At ~26:00 he mentions Performance and Load testing with Gatling, JMeter and Flood.io. He then covers security testing ~27:15 Owasp Dependency Check, Docker Bench etc. The discussion about Wrath begins ~28:30 He talks about Devops starting ~30:30 some useful information about books there. PAAS Platforms as a Service ~35:00 Data Models ~36:00 Domain Driven Design DDD mentioned ~36:30 Context Mapping and Event Storming ~38:00 Data bases and Data Stores, RDBMS, NoSQL etc. ~39:00 At ~40:00 he mentions testing At ~41:30 he mentions API Simulation and Service Virtualization this is a very interesting concept. At ~43:00 Service Virtualization software information listed.
From Microliths to Microsystems GOTO 2017 – Jonas Boner
This video is mentioned in articles and videos above. Jonas starts the video with a discussion about Monoliths and how they don’t work. This is an excellent video that points out some bad habits to avoid, things that seem intuitive in Microservice systems is actually the wrong way to do it. At ~5:00 he starts talking about the most common way we developers begin creating microservices. At ~6:00 he begins to define what a Microlith is. Reactive Programing vs Reactive Systems ~16:30 Async discussed ~18:30 Back pressure ~20:00 Microservices come as Systems ~23:00 Separating the Stateless behavior from the Stateful entity ~23:30 Asynchronous Message Passing ~28:00 An excellent book on this subject is Reactive Messaging Patterns with the Actor Model: Applications and Integration in Scala and Akka by Vaugh Vernon . Thinking in terms of Consistent Boundaries ~34:00 Inside Data vs Outside Data ~35:00 DDD and Events first design ~36:00 Defining Bounded contexts by Events ~37:00 Event Storming ~38:00 Event Logging ~41:30 CQRS and Event Sourcing ~43:30 decouple read from write. What about Transactions? ~47:30
Microservices, Kubernetes, and Application Modernization Done Right
This video covers multiple subjects, many listed individually above. This is sort of like tying it all together. It talks about the mistakes of the SOA era.
Leave a Reply