Categories
Resources Software Development

Microservice Communication Resources

Microservice Communication resources

In order to completely understand microservices you also have to understand networking. Communication is one of the hardest parts of microservice design, get it wrong and your system simply won’t perform. Below are some useful links that explain networking concepts.

Understanding the basics of RESTful APIs. This is a great article that covers how REST works and how to design a REST API. This article contains some very useful information and is a very quick introduction to the concept.


Understanding And Using REST APIs This is another really great article that covers API consumption. This article is quite a bit longer and more involved than the one above. This article covers REST in depth and is one of the best articles I have found on the subject.

Covered in the article is:

  • What is a REST API
  • The anatomy of a request
  • Testing Endpoints with Curl ( if you have not played with Curl then you absolutely should) CURL website
  • JSON
  • Authentication
  • HTTP Status Codes and Errors
  • Messages
  • API Versioning

Communication in a microservice architecture This article covers some basic information about Microservice communication. It makes some valid points and discusses some of the techniques and technology used for Microservice communication. This Article has some really good links listed throughout it.


Asynchronous Request-Reply pattern  This is a very useful article that covers communication between the frontend client aka browser and the server backend.


Should you use MQ/Brokers or Web services? and how they are different

 


Apache Kafka vs. Enterprise Service Bus (ESB) – Friends, Enemies or Frenemies?

This article covers :

  1. Some of the failures of SOA
  2. Streaming as a replacement for ESB
  3. Compares Apache Kafka to previous SOA architecture that used ESB
  4. Talks about Streaming and why it is the next great thing you must do
  5. Talks about large companies that use Kafka and how
  6. Talks about using Apache Kafka as a dumb pipeline for streaming apps
  7. Talks about the key differences between Kafka and ESB’s
  8. Talks about using Kafka in Legacy systems along side ESB’s and previous generation technologies

 


Application Integration for Microservices Architectures: A Service Mesh Is Not an ESB

This excellent article explains SOA, ESB, Microservices and how the hell we ended up in the current confusing word soup.
This article covers the following:

  1. The basic history of Software Oriented Architecture and Event Service Bus usage
  2. Compares a service mesh to an ESB, similarities and differences
  3. Gives a decent overview of the past usage of ESB’s
  4. Has a wonderful diagram of how ESB’s operate in a SOA system.
  5. Explains what a service mesh is

Kubernetes Service Mesh: A Comparison of Istio, Linkerd and Consul 

This article explains exactly what a service mesh is and compares a few of them. This is a very good, very informative article, one of my favorite.


REST, RPC, and Brokered Messaging – This article explores three popular styles of communication in service-oriented architectures and how to chose the appropriate style for a given use case.


Remote Procedural Calls explained in depth.


This video covers RPC even deeper than the above video. This video also covers Java RMI remote method invocation.

Categories
Resources Software Development Web Development

scala web development resources

Resources for web development with Scala

Scala tags. This library is for creating HTML from Scala. Thanks lihaoyi

ScalaCSS a library for creating CSS from Scala

Categories
Random

Scala Java Joke

Interviewer: How much do you know about Java?

Me : Enough to decide to learn Scala

Interviewer: Do you not like Java?

Me: I’d rather not write a paragraph to simply create and use an object

Categories
Resources Software Development

stream processing software resources and links

Stream processing is useful for many things, including real-time suggestions to users about content that may interest them based on their actions. You can feed streams of events to an event processor to have it search text. Designing Data Intensive Applications by Martin Kleppmann has great information about Streaming and Stream Processing in Chapter 11 Stream processing, this chapter covers the subject in great depth. If you don’t own that book you absolutely should if you are a software engineer. This book doesn’t cover Microservices, however it covers all of the software you use to create microservices and explains each in depth.  If you are engineering any kind of modern app you either have that book or you are lost as to what you are doing and haphazardly choose software because it is all hypie and cool sounding. This book explains the internals of most Databases, NoSQL, RDBMS etc., it really gives you the info you need to make sound decisions. This book will save you hundreds of hours of raw research.

Stream processor list

There are many stream processors, below I list a few.

  1. Apache Flink – accepts input from message queue or file system, can save output to a message queue, database file system and more. Easily connects to KAFKA and Cassandra. Processes streams event by event
  2. Apache SparkSpark website.  processes streams in mini batches Interesting point from Wikipedia article “Spark Streaming has support built-in to consume from Kafka, Flume, Twitter, ZeroMQ, Kinesis, and TCP/IP sockets” Another interesting point from the wikipedia is the Spark MLib a machine learning framework built on top of Spark Core
  3. Apache Storm – Apache storm website  Integrates with KAFKA, Databases and other Messaging Queues.
  4. Apache Kafka – This excellent piece of software functions as a Stream Processor, Message Broker and more.

 

 

Categories
Resources Software Development

event storming resources

Event Storming Resources

Event storming is a process of modelling software that works well with domain driven design principles. During an event storming session all involved parties get together and help model a domain, this helps everyone from the developer to the business decision makers understand the system and from a ubiquitous language ( common language). Event storming is just another modelling tool to help with quick design and development.

Articles

Event Storming 101: Accelerating Your Software Development in Domain-Driven Design This article covers the absolute basics of Event Storming what it is and how it is used.

Here is another short article about event sourcing. This article includes many useful links at the bottom.


 Videos

Alberto Brandolini – 50,000 Orange Stickies Later

This video is from the man who created the concept of Event Storming. In this video Alberto discusses how he invented this technique of Event Storming with PostIt notes. He describes and demonstrates how this sort of Brain Storming session is a great way to get all involved parties together.  This technique will help you more quickly and completely model your software. I’ll be writing about this technique in my articles about microservices with Scala.

Categories
Resources

Lagom notes and facts

This is just a list of notes and facts I have gathered about Lagom. I’m the kind of person that asks 1 million questions and likes to know why things are done.

Why does Lagom use Cassandra as an event store system?

Lagom uses Cassandra as it’s event store. I wanted to know why they would choose Cassandra so I did some digging.  I found the following video helpful in understanding why anyone would use Cassandra for an event store instead of EventStore itself.  The video is not about Lagom, however it is about Cassandra and CQRS/Event Sourcing and why you would choose Cassandra as an event store. Basically Cassandra has features that are required for Event Sourcing and make it a good fit for it.

Why does Lagom use KAFKA as a Message Broker?

The documentation mentions using KAFKA as a Message Broker. I wanted to know why?  This article helped clarify what the difference between a message bus and a message broker.  Basically a broker gives a little less coupling with some other nice features. A message bus is more strict with a Schema like a traditional RDBMS database system, a message broker is more free like a NoSql database it has no Schema. Read the article for the other benefits and differences.

Codecentric: CQRS and Event Sourcing Applications with Cassandra

This video explains how to handle event sourcing with Cassandra

I also found this link to be helpful in understanding why Lagom would use Cassandra instead of EventStore. That article has a nice comparison of Cassandra vs other similar software used for event storage. A quick look at the features is all it takes.

Categories
Resources Software Development

functional programming resources and notes

Before you get too excited and go all functional in Scala you should know that 100% Functional Scala can be serious Memory hog because you are making copies of everything everywhere because functional means immutable data. Be careful with things such as map and flatmap they create lots of extra objects.

Avoiding loops for the sake of Functional programming strictness is a smooth brained action. When programming avoid hype, understand what you are doing, what your goals are and figure out how to get there. Never blindly ride the hype train.

This article lihaoyi points out some of the inefficiencies of Scala collections for example.

Articles

Scala Closures – article about closures in Scala

Pure functions in Scala – Article about what pure functions are in Scala

Basic Functional Programming in Scala – a short article covering some of the basics of Function programming with Scala

Videos

GOTO 2018 • Functional Programming in 40 Minutes • Russ Olsen

Categories
Resources Software Development

notes facts and resources about domain driven design aggregates

DDD Domain Driven Design Aggregates

When learning about Domain Driven Design one of the harder things is wrapping your head around what aggregates are. In this post I list some facts, some may be redundant, but redundancy helps reinforce concepts.

Here is my personal list of facts and notes about domain driven design aggregates in no general order. This list will be updated as time goes on.

List of Aggregate facts

  1. An aggregate is a consistency boundary that decomposes large models into smaller clusters of domain objects that are technically easier to manage. Basically I refer to these as modules/services, modules should be easily hot swappable.
  2. An Aggregate is a DDD pattern, a cluster of related domain objects that work together as a single unit
  3. Aggregates will have one component object that acts as an interface to the aggregate known as the Aggregate root All outside interaction with the aggregate should be through the aggregate root. An aggregate root is the contract for the entire aggregate, the API interface for the Aggregate/Module/Service
  4. Aggregates are the inner bounded contexts that build the complete bounded context of the entire domain, like building blocks.
  5. An aggregate is a way to group objects inside of a specific domain bounded context. They are a way to modularize things that have to work together to perform a specific task of the system
  6. Aggregates are collections of objects that perform the functions of a particular bounded context.
  7. Aggregates are stored in a repository, aka database
  8. Aggregates are the only thing that can be persisted and retrieved ( re-hydrated )
  9. Repositories are used to manage the persistence of aggregates and ensure a clear separation between the data and the domain
  10. Aggregates communicate with other aggregates via Events,  creating event driven systems
  11. Try to make aggregates as small and specific as possible, this creates less room for bugs and makes the system easier to maintain. Stick to the Single Responsibility Principle  
  12. Large aggregates can suffer from performance issues when they span multiple entities, data stores, database tables Only make aggregates as large as they absolutely have to be, eliminate all unrelated code and actions.
  13. An Aggregate lives in a Bounded Context. Each bounded context is made up of multiple aggregates. Each a aggregate within a bounded context should have a Single Responsibility.
  14. When two or more objects/entities need to interact with each other to perform a task, make them part of the same aggregate.
  15. Aggregates represent concepts in the larger domain.
  16. Aggregates should be behavior focused
  17. Each object within an aggregate should be required to do some action for the aggregate to be included. Don’t have redundant aggregates within other bounded contexts, this is a clue that a new aggregate needs to be born into its own bounded context. Create another module and make the redundant aggregate it’s own.
  18. An aggregate root should be the only entry point for an aggregate. The aggregate root is like an application programmers interface to the aggregate API.
  19. The aggregate root is the coordinator for the aggregate. It handles all of the events by calling the aggregate objects to perform the required tasks/actions.
  20. The aggregate root should expose only the behaviors required by other aggregates. The aggregate root is the contract of the aggregate.  Removing behaviors makes breaking changes and can destroy a system. However, adding new behaviors any time is safe.
  21. Objects within aggregates should not hold references to objects in other aggregates, this creates tight coupling, exactly what we are trying to prevent. This means no object in one module/service/microservice should directly call an object or refer to one inside another module/service/microservice.   This means no communication outside one bounded context to another, the aggregate root is the only way to communicate.
  22. An aggregate can span several database tables when persisted, but this can cause problems.
  23. Domain objects within an aggregate can have direct object references to each other. This means that aggregates ( objects ) within a single bounded context (module/service) can have references to each other. Meaning within a service you can call other objects within the service.
  24. Repositories should hold aggregates. Aggregates can be reconstructed at any time from a repository.
  25. DDD repositories are not the same as code repositories like github
  26. The aggregate roots are AKKA actors that should accept commands and produce events
  27. Repositories are used to manage domain object/aggregate root persistence
  28. A repository manages the retrieval of aggregate/domain objects while ensuring a separation between the domain and data models
  29. A repository is a pattern for storing and retrieving the pieces of an aggregate in a database
  30. A repository enforces the aggregate roots contract by providing the interface to store and retrieve the aggregate parts
  31. Repositories map to the aggregates not the actual data-store such as database tables

Videos

Developing microservices with aggregates – Chris Richardson
I found this to be one of the best explanations of aggregates and how they relate in microservice based systems. Here is a link to the site mentioned in the video microservices.io  where you can find a lot more useful information about microservices.

Categories
Resources

Scala tools and libraries

ScalaSTM Software transaction memory