Monday, March 31, 2014

Functional Patterns in Domain Modeling - The Specification Pattern

When you model a domain, you model its entities and behaviors. As Eric Evans mentions in his book Domain Driven Design, the focus is on the domain itself. The model that you design and implement must speak the ubiquitous language so that the essence of the domain is not lost in the myriads of incidental complexities that your implementation enforces. While being expressive the model needs to be extensible too. And when we talk about extensibility, one related attribute is compositionality.

Functions compose more naturally than objects and In this post I will use functional programming idioms to implement one of the patterns that form the core of domain driven design - the Specification pattern, whose most common use case is to implement domain validation. Eric's book on DDD says regarding the Specification pattern ..
It has multiple uses, but one that conveys the most basic concept is that a SPECIFICATION can test any object to see if it satisfies the specified criteria.
A specification is defined as a predicate, whereby business rules can be combined by chaining them together using boolean logic. So there's a concept of composition and we can talk about Composite Specification when we talk about this pattern. Various literature on DDD implement this using the Composite design pattern so commonly implemented using class hierarchies and composition. In this post we will use function composition instead.

Specification - Where ?

One of the very common confusions that we have when we design a model is where to keep the validation code of an aggregate root or any entity, for that matter.
  • Should we have the validation as part of the entity ? No, it makes the entity bloated. Also validations may vary based on some context, while the core of the entity remains the same.
  • Should we have validations as part of the interface ? May be we consume JSON and build entities out of it. Indeed some validations can belong to the interface and don't hesitate to put them there.
  • But the most interesting validations are those that belong to the domain layer. They are business validations (or specifications), which Eric Evans defines as something that "states a constraint on the state of another object". They are business rules which the entity needs to honor in order to proceed to the next stage of processing.
We consider the following simple example. We take an Order entity and the model identifies the following domain "specifications" that a new Order must satisfy before being thrown into the processing pipeline:

  1. it must be a valid order obeying the constraints that the domain requires e.g. valid date, valid no of line items etc.
  2. it must be approved by a valid approving authority - only then it proceeds to the next stage of the pipeline
  3. customer status check must be passed to ensure that the customer is not black-listed
  4. the line items from the order must be checked against inventory to see if the order can be fulfilled
These are the separate steps that are to be done in sequence by the order processing pipeline as pre-order checks before the actual order is ready for fulfilment. A failure in any of them takes the order out of the pipeline and the process stops there. So the model that we will design needs to honor the sequence as well as check all constraints that need to be done as part of every step.

An important point to note here is that none of the above steps mutate the order - so every specification gets a copy of the original Order object as input, on which it checks some domain rules and determines if it's suitable to be passed to the next step of the pipeline.

Jumping on to the implementation ..

Let's take down some implementation notes from what we learnt above ..

  • The Order can be an immutable entity at least for this sequence of operations
  • Every specification needs an order, can we can pull some trick out of our hat which prevents this cluttering of API by passing an Order instance to every specification in the sequence ?
  • Since we plan to use functional programming principles, how can we model the above sequence as an expression so that our final result still remains composable with the next process of order fulfilment (which we will discuss in a future post) ?
  • All these functions look like having similar signatures - we need to make them compose with each other
Before I present more of any explanation or theory, here are the basic building blocks which will implement the notes that we took after talking to the domain experts ..

type ValidationStatus[S] = \/[String, S]

type ReaderTStatus[A, S] = ReaderT[ValidationStatus, A, S]

object ReaderTStatus extends KleisliInstances with KleisliFunctions {
  def apply[A, S](f: A => ValidationStatus[S]): ReaderTStatus[A, S] = kleisli(f)
}

ValidationStatus defines the type that we will return from each of the functions. It's either some status S or an error string that explains what went wrong. It's actually an Either type (right biased) as implemented in scalaz.

One of the things which we thought will be cool is to avoid repeating the Order parameter for every method when we invoke the sequence. And one of the idioamtic ways of doing it is to use the Reader monad. But here we already have a monad - \/ is a monad. So we need to stack them together using a monad transformer. ReaderT does this job and ReaderTStatus defines the type that somehow makes our life easier by combining the two of them.

The next step is an implementation of ReaderTStatus, which we do in terms of another abstraction called Kleisli. We will use the scalaz library for this, which implements ReaderT in terms of Kleisli. I will not go into the details of this implementation - in case you are curious, refer to this excellent piece by Eugene.

So, how does one sample specification look like ?

Before going into that, here are some basic abstractions, grossly simplified only for illustration purposes ..

// the base abstraction
sealed trait Item {
  def itemCode: String
}

// sample implementations
case class ItemA(itemCode: String, desc: Option[String], 
  minPurchaseUnit: Int) extends Item
case class ItemB(itemCode: String, desc: Option[String], 
  nutritionInfo: String) extends Item

case class LineItem(item: Item, quantity: Int)

case class Customer(custId: String, name: String, category: Int)

// a skeleton order
case class Order(orderNo: String, orderDate: Date, customer: Customer, 
  lineItems: List[LineItem])

And here's a specification that checks some of the constraints on the Order object ..

// a basic validation
private def validate = ReaderTStatus[Order, Boolean] {order =>
  if (order.lineItems isEmpty) left(s"Validation failed for order $order") 
  else right(true)
}

It's just for illustration and does not contain much domain rules. The important part is how we use the above defined types to implement the function. Order is not an explicit argument to the function - it's curried. The function returns a ReaderTStatus, which itself is a monad and hence allows us to sequence in the pipeline with other specifications. So we get the requirement of sequencing without breaking out of the expression oriented programming style.

Here are a few other specifications based on the domain knowledge that we have gathered ..

private def approve = ReaderTStatus[Order, Boolean] {order =>
  right(true)
}

private def checkCustomerStatus(customer: Customer) = ReaderTStatus[Order, Boolean] {order =>
  right(true)
}

private def checkInventory = ReaderTStatus[Order, Boolean] {order =>
  right(true)
}

Wiring them together

But how do we wire these pieces together so that we have the sequence of operations that the domain mandates and yet all goodness of compositionality in our model ? It's actually quite easy since we have already done the hard work of defining the appropriate types that compose ..

Here's the isReadyForFulfilment method that defines the composite specification and invokes all the individual specifications in sequence using for-comprehension, which, as you all know does the monadic bind in Scala and gives us the final expression that needs to be evaluated for the Order supplied.

def isReadyForFulfilment(order: Order) = {
  val s = for {
    _ <- validate
    _ <- approve
    _ <- checkCustomerStatus(order.customer)
    c <- checkInventory
  } yield c
  s(order)
}

So we have the monadic bind implement the sequencing without breaking the compositionality of the abstractions. In the next instalment we will see how this can be composed with the downstream processing of the order that will not only read stuff from the entity but mutate it too, of course in a functional way.

15 comments:

Joheinz said...

In the code for

private def approve = ReaderTStatus[Order, Boolean] {case order =>
right(true)
}


is there a reason you use a partial function instead of a ordinary function from order to ValidationStatus?

And I right(true) should be true.right, right? Or I have been missing some imports from scalaz...

Very enjoyable read BTW, and I looking forward to follow ups.

Unknown said...

I curried the order argument for dependency injection so that I don't have to explicitly pass it in every invocation within the for comprehension.

And right(true) is ok and works for me ..

David Fox said...

I actually don't like to use the Specification pattern to test if a domain entity is "valid". I think consistency enforcing code does belong in the entity itself. The rationale is that if you do that, the entity can never be placed into an inconsistent state. Therefore whenever you use the entity, you can be sure it is valid, you don't need to test it before you use it.

I have also been using immutable entities, and so the validation code that might otherwise "bloat" the entity is contained in the constructor. This also encourages a functional aspect, as methods which would mutate the entity return a new instance rather than modify the original object's state.

You mention that the specification can change in different contexts in the application, and in DDD that usually implies a separate bounded context, and hence a separate domain model. Not sure if that's what you meant though. I do use Specification to centralize code which tests that an entity matches different criteria, but I think this is different than validation.

Love to hear your thoughts on this.

Leif Wickland said...

Minor typo: scalaz' Either is right, not left, biased.

Joheinz said...

right(true) works for you, as you seem to do something like "with DisjunctionFunctions" somewhere in your code, if I am not mistaken. Similiar to what you do with the StateFunctions.

For the second question:
I understood that you do not do def approve(order: Order) = ... since we have a ReaderT here, but is there an ordinary reason why you do { case order => ...} instead of just { order => ... } As far as I know the case would match always, and I was wondering if there reason (maybe because of the things in the second part).

Thanks again!

Unknown said...

@Joheinz : oops that was a typo replicated by copy/paste. Thanks for pointing out. Now fixed the partial function point :-)

Regarding the first point, I am doing import \/._ .. hence getting that right.

Unknown said...

@Leif : Thanks for pointing out .. fixed now.

Unknown said...

@David: In many cases it makes sense to do all validations when the aggregate is constructed. But consider the case where you construct the aggregate in th Web layer from JSON structures. As I mentioned in the post, there are some validations which you can do there. But some of the business validations need to be done in the domain layer as it relates to the domain and the Web layer should not be polluted with them. May be u need to access the database or interact with external system. e.g. in the example which I gave one validation is to check the customer background, if he is blacklisted. For that you may have to contact an external system. So the "specifications" that I stated in the post are "domain" validations. It's not that I construct the object in the Web layer with an invalid date or an invalid string in a numeric field and write specifications in the domain layer. Possibly another solution will be to use another set of DTOs for transporting data from the Web layer and construct the domain objects in the domain layer. But I am not a big fan of DTOs - leads to proliferation of objects.

David Fox said...

Gotcha. I do similar things as well: Using validation facilities to validate incoming data from the web layer. But I treat those objects as commands which are applied to the aggregates rather than aggregates themselves.

Anyways, good article as always. I've followed your blog for a while now.

JB said...

in this design, right(false) could be returned by a specification. Would it make any sense?

Unknown said...

@JB scalaz \/ is a right biased Either and the standard convention is to return a "right" in case of a pass and return a "left" in case of a fail with an error message. So "right(false)" will not make much sense here.

Unknown said...

Hi

Just wanted to pitch in, I don't get the upside of this method.

Using the previous example of a person having to be validated against an external service to see if that person is blacklisted:

I would first use one case class Person with require-methods for validating the JSON input then wrap that object in another monadic object describing the new state, ie Approved[Person] after verifying against an external service.

That pattern retains type safety, models the current business state in an explicit way (Verb[Noun]) and does not have the throw away DTO-problem.

I feel that this makes it easy to understand where this domain object came from and where it is going next. You can also retain the history, when it makes sense, ie LegalAge[Verified[Person]].

Is there a downside of this approach compared to the one described. I feel like I am probably missing something?

Unknown said...

Hi Magnus -

It's definitely another approach of validation. In my approach I have tried to show 2 things working together. The Reader monad that makes the sequence of validations less verbose by not having to repeat the order argument. And secondly the composition part of applying the validations in sequence. This is just one approach that I have used and found quite useful.

Would be interested to see how your approach composes multiple validation methods applied in sequence.

Having said that I like the stacking the history of validations in your approach in a completely typesafe way.

Jamie Pullar said...

Is there a formalised approach when you want to aggregate failures, rather than stop on the first one?
I threw together a Monadic structure to use instead of ValidationStatus, which seems to play nicely with the Kleisi compositions, though I wonder if there is something more standard from scalaz which could be used...
case class Collector[+E, +A](e:Vector[E], a:A) {
def flatMap[EE >: E, B](f:A => Collector[EE, B]):Collector[EE, B] = {
val result = f(a)
Collector(e ++ result.e, result.a)
}
def map[B](f: A => B):Collector[E, B] = this.copy(a = f(a))
}

Unknown said...

@Jamie - You can use the applicative functor pattern for accumulating failures. There are standard idioms for this in scalaz and shapeless. Have a look at this reply in SoF by Travis Brown http://stackoverflow.com/questions/16930347/scalaz-how-can-i-accumulate-failures-or-apply-a-function-to-validations-with-di and also the linked blog post .. http://meta.plasm.us/posts/2013/06/05/applicative-validation-syntax/ ..