Foundation Services (Broker-Neighboring)

2.1.0 Introduction

Foundation services are the first point of contact between your business logic and the brokers.

In general, the broker-neighboring services are a hybrid of business logic and an abstraction layer for the processing operations where the higher-order business logic happens, which we will talk about further when we start exploring the processing services in the next section.

Broker-neighboring services main responsibility is to ensure the incoming and outgoing data through the system is validated and vetted structurally, logically and externally.

You can also think of broker-neighboring services as a layer of validation on top of the primitive operations the brokers already offer.

For instance, if a storage broker is offering InsertStudentAsync(Student student) as a method, then the broker-neighboring service will offer something as follows:

public async ValueTask<Student> AddStudentAsync(Student student)
{
	ValidateStudent(student);

	return await this.storageBroker.InsertStudentAsync(student);
}

This makes broker-neighboring services nothing more than an extra layer of validation on top of the existing primitive operations brokers already offer.

2.1.1 On The Map

The broker-neighboring services reside between your brokers and the rest of your application, on the left side higher-order business logic processing services, orchestration, coordination, aggregation or management services may live, or just simply a controller, a UI component or any other data exposure technology.

2.1.2 Characteristics

Foundation or Broker-Neighboring services in general have very specific characteristics that strictly govern their development and integration.

Foundation services in general focus more on validations than anything else - simply because that's their purpose, to ensure all incoming and outgoing data through the system is in a good state for the system to process it safely without any issues.

Here's the characteristics and rules that govern broker-neighboring services:

2.1.2.0 Pure-Primitive

Broker-neighboring services are not allowed to combine multiple primitive operations to achieve a higher-order business logic operation.

For instance, broker-neighboring services cannot offer an upsert function, to combine a Select operations with an Update or Insert operations based on the outcome to ensure an entity exists and is up to date in any storage.

But they offer a validation and exception handling (and mapping) wrapper around the dependency calls, here's an example:

public ValueTask<Student> AddStudentAsync(Student student) =>
TryCatch(async () =>
{
	ValidateStudent(student);

	return await this.storageBroker.InsertStudentAsync(student);
});

In the above method, you can see ValidateStudent function call preceded by a TryCatch block. The TryCatch block is what I call Exception Noise Cancellation pattern, which we will discuss soon in this very section.

But the validation function ensures each and every property in the incoming data is validated before passing it forward to the primitive broker operation, which is the InsertStudentAsync in this very instance.

2.1.2.1 Single Entity Integration

Services strongly ensure the single responsibility principle is implemented by not integrating with any other entity brokers except for the one that it supports.

This rule doesn't necessarily apply to support brokers like DateTimeBroker or LoggingBroker since they don't specifically target any particular business entity and they are almost generic across the entire system.

For instance, a StudentService may integrate with a StorageBroker as long as it only targets only the functionality offered by the partial class in the StorageBroker.Students.cs file.

Foundation services should not integrate with more than one entity broker of any kind simply because it will increase the complexity of validation and orchestration which goes beyond the main purpose of the service which is just simply validation. We push this responsibility further to the orchestration-type services to handle it.

2.1.2.2 Business Language

Broker-neighboring services speak primitive business language for their operations. For instance, while a Broker may provide a method with the name InsertStudentAsync - the equivelant of that on the service layer would be AddStudentAsync.

In general, most of the CRUD operations shall be converted from a storage lanaugage to a business language, and the same goes for non-storage operations such as Queues, for instance we say PostQueueMessage but on the business layer we shall say EnqueueMessage.

Since the CRUD operations the most common ones in every system, our mapping to these CRUD operations would be as follows:

Brokers	Services
Insert	Add
Select	Retrieve
Update	Modify
Delete	Remove

As we move forward towards higher-order business logic services, the language of the methods beings used will lean more towards a business language rather than a technology language as we will see in the upcoming sections.

2.1.3 Responsibilities

Broker-neighboring services play three very important roles in any system. The first role is to abstract away native broker operations from the rest of the system. Irregardless of whether a broker is a communication between a local or external storage or an API - broker-neighboring services will always have the same contract/verbiage to expose to upper stream services such as processing, orchestration or simply exposers like controllers or UI components. The second and most important role is to offer a layer of validation on top of the existing primitive operations a broker already offers to ensure incoming and outgoing data is valid to be processed or persisted by the system. The third role is to play the role of a mapper of all other native models and contracts that may be needed to completed any given operation while interfacing with a broker. Foundation services are the last point of abstraction between the core business logic of any system and the rest of the world, let's discuss these roles in detail.

2.1.3.0 Abstraction

The first and most important responsibility for foundation/broker-neighboring services is to ensure a level of abstraction exists between the brokers and the rest of your system. This abstraction is necessary to ensure the pure business logic layer in any system is verbally and functionally agnostic to whichever dependencies the system is relying on to communicate with the outside world.

Let's visualize a concrete example of the above principle. Let's assume we have a StudentProcessingService which implements an UpsertStudentAsync functionality. Somewhere in that implementation there will be a dependency on AddStudentAsync which is exposed and implemented by some StudentService as a foundation service. Take a look at this snippet:

public async ValueTask<Student> UpsertStudentAsync(Student student)
{
	...

	return await this.studentService.AddStudentAsync(student);
}

The contract between a processing or an orchestration service and a foundation service will always be the same irregardless of what type of implementation or what type of brokers the foundation service is using. For example, AddStudentAsync could be a call to a database or an API endpoint or simply putting a message on a queue. It all doesn't impact in any way, shape or form the upstream processing service implementation. here's an example of three different implementations of a foundation service that wouldn't change anything in the implementation of it's upstream services:

With a storage broker:

public async ValueTask<Student> AddStudentAsync(Student student)
{
	...
	return await this.storageBroker.InsertStudentAsync(student);
}

Or with a queue broker:

public async ValueTask<Student> AddStudentAsync(Student student)
{
	...

	return await this.queueBroker.EnqueueStudentAsync(student);
}

or with an API broker:

public async ValueTask<Student> AddStudentAsync(Student student)
{
	...

	return await this.apiBroker.PostStudentAsync(student);
}

here's a visualization of that concept:

In all of these above cases, the underlying implementation may change, but the exposed contract will always stay the same for the rest of the system. We will discuss in later chapters how the core, agnostic and abstreact business logic of your system starts with Processing services and ends with Management or Aggregation services.

2.1.3.0.1 Implementation

Let's talk about a real-life example of implementing a simple Add function in a foundation service. Let's assume we have the following contract for our StudentService:

public IStudentService
{
	ValueTask<Student> AddStudentAsync(Student student);
}

For starters, let's go ahead and write a failing test for our service as follows:

public async Task ShouldAddStudentAsync()
{
	// given
	Student randomStudent = CreateRandomStudent();
	Student inputStudent = randomStudent;
	Student storageStudent = inputStudent;
	Student expectedStudent = storageStudent.DeepClone();

	this.storageBrokerMock.Setup(broker =>
		broker.InsertStudentAsync(inputStudent))
			.ReturnsAsync(storageStudent);

	// when
	Student actualStudent =
		await this.studentService.AddStudentAsync(inputStudent);

	// then
	actualStudent.Should().BeEquivelantTo(expectedStudent);

	this.storageBroker.Verify(broker =>
		broker.InsertStudentAsync(inputStudent),
			Times.Once);

	this.storageBroker.VerifyNoOtherCalls();
	this.loggingBroker.VerifyNoOtherCalls();
}

In the above test, we defined four variables with the same value. Each variable contains a name that best fits the context it will be used in. For instance, inputStudent best fits in the input parameter position, while storageStudent best first what gets returned from the storage broker after a student is persisted sucessfully.

You will also notice that we deep cloned the expectedStudent variable to ensure no modifications have happened to the originally passed in student. For instance, assume an input student value has changed for any of it's attributes internally within the AddStudentAsync function. That change won't trigger a failing test unless we dereference the expectedStudent variable from the input and returned variables.

We mock the response from the storage broker and execute our subject of test AddStudentAsync then we verify the returned student value actualStudent matches the expected value expectedStudent regardless of the reference.

Finally, we verify all calls are done properly and no additional calls has been made to any of the service dependencies.

Let's make that test pass by writing in an implementation that only satisfies the requirements of the aforementioned test:

public async ValueTask<Student> AddStudentAsync(Student student) =>
	await this.storageBroker.InsertStudentAsync(student);

This simple implementation should make our test pass sucessfully. It's important to understand that any implementation should be only enough to pass the failing tests. Nothing more and nothing less.

2.1.3.1 Validation

Foundation services are required to ensure incoming and outgoing data from and to the system are in a good state - they play the role of a gatekeeper between the system and the outside world to ensure the data that goes through is structurally, logically and externally valid before performing any further operations by upstream services. The order of validations here is very intentional. Structural validations are the cheapest of all three types. They ensure a particular attribute or piece of data in general doesn't have a default value if it's required. the opposite of that is the logical validations, where attributes are compared to other attributes with the same entity or any other. Additional logical validations can also include a comparison with a constant value like comparing a student enrollment age to be no less than 5 years of age. Both strucural and logical validations come before the external. As we said, it's simply because we don't want to pay the cost of communicating with an external resource including latency tax if our request is not in a good shape first. For instance, we shouldn't try to post some Student object to an external API if the object is null. Or if the Student model is invalid structurally or logically.

For all types of validations, it's important to understand that some validations are circuit-breaking or requiring an immediate exit from the current flow by throwing an exception or returning a value in some cases. And some other validations are continuous. Let's talk about these two sub categories of validations first.

2.1.3.1.0 Circuit-Breaking Validations

Circuit-breaking validations require an immediate exit from the current flow. For instance, if an object being passed into a function is null - there will be no further operations required at that level other than exiting the current flow by throwing an exception or returning a value of some type. Here's an example: In some validation scenario, assume that our AddStudent function has a student of value null passed into it as follows:

Student noStudent = null;

await this.studentService.AddStudentAsync(noStudent);

Our AddStudentAsync function in this scenario is now required to validate whether the passed in parameter is null or not before going any further with any other type of validations or the business logic itself. Something like this:

public Student AddStudentAsync(Student student) =>
TryCatch(async () =>
{
	ValidateStudent(student);

	return await this.storageBroker.InsertStudentAsync(student);
});

The statement in focus here is ValidateStudent function and what it does. Here's an example of how that routine would be implemented:

private void ValidateStudent(Student student)
{
	if(student is null)
	{
		throw new NullStudentException();
	}
}

In the function above, we decided to throw the exception immediately instead of going in further. That's an example of circuit-breaking validation type.

But with validations, circuit-breaking isn't always the wise thing to do. Sometimes we want to collect all the issues within a particular request before sending the error report back to the request submitter. Let's talk about that in this next section.

2.1.3.1.1 Continuous Validations

Continuous validations are the opposite of circuit-breaking validations. They don't stop the flow of validations but they definitely stop the flow of logic. In other words, continuous validations ensure no business logic will be executed but they also ensure other validations of the same type can continue to execute before breaking the circuit. Let's materialize this theory with an example: Assume our student model looks like this:

public class Student
{
	public Guid Id {get; set;}
	public string Name {get; set;}
}

Assuming that the passed in Student model is not null, but it has default values across the board for all it's properties. We want to collect all these issues for however many attributes/properties this object has and return a full report back to the requestor. Here's how to do it.

2.1.3.1.1.0 Upsertable Exceptions

A problem of that type requires a special type of exceptions that allow collecting all errors in it's Data property. Every native exception out there will contain the Data property which is basically a dictionary for a key/value pairs for collecting more information about the issues that caused that exception to occur. The issue with these native exceptions is that they don't have native support for upsertion. Being able to append to an existing list of values against a particular key at any point of time. Here's a native implementation of upserting values in some given dictionary:

var someException = new Exception();

if(someException.Data.Contains(someKey))
{
	(someException.Data[someKey] as List<string>)?.Add(someValue);
}
else
{
	someException.Data.Add(someKey, new List<string>{ someValue });
}

This implementation can be quite daunting for engineers to think about and test in their service-level implementation. It felt more appropriate to introduce a simple library Xeptions to simplify the above implementation into something as simple as:

var someException = new Xeption();
someException.UpsertData(someKey, someValue);

Now that we have this library to utilize, the concern of implementing upsertable exceptions has been addressed. This means that we have what it takes to collect our validation errors. But that's not good enough if we don't have a mechanism to break the circuit when we believe that the time is right to do so. We can simply use the native offerings to implement the circuit-breaking directly as follows:

if(someException.Data.Count > 0)
{
	throw someException;
}

And while this can be easily baked into any existing implementation. It still didn't contribute much to overall look-n-feel of the code. Therefore I have decided to make it a part of the Xeptions library to be simplified to the following:

someException.ThrowIfContainsErrors();

That would make our custom validations look something like this:

public class InvalidStudentException : Xeption
{
	public InvalidStudentException()
		: base ("Student is invalid. Please fix the errors and try again.")
	{ }
}

But with continuous validations, the process of collecting these errors conveys more than just a special exception implementation. Let's talk about that in the next section.

2.1.3.1.1.1 Dynamic Rules

A non-circuit-breaking or continuous validation process will require the ability to pass in dynamic rules at any count or capacity to add these validation errors. A validation rule is a dynamic structure that reports whether the rule has been violated for its condition; and also the error message that should be reported to the end user to help them fix that issue.

In a scenario where we want to ensure any given Id is valid, a dynamic continuous validation rule would look something like this:

private static dynamic IsInvalid(Guid id) => new
{
	Condition = id == Guid.Empty,
	Message = "Id is required"
};

Now our Rule doesn't just report whether a particular attribute is invalid or not. It also has a meaningful human-readable message that helps the consumer of the service understand what makes that very attribute invalid.

It's really important to point out the language engineers must use for validation messages. It will all depend on the potential consumers of your system. A non-engineer will not understand a message such as Text cannot be null, empty or whitespace - null as a term isn't something that is very commonly used. Engineers must work closely with their meatware (The people using the system) to ensure the language makes sense to them.

Dynamic rules by design will allow engineers to modify both their inputs and outputs without breaking any existing functionality as long as null values are considered across the board. Here's another manifestation of a Dynamic Validation Rule:

private static dynamic IsNotSame(
	Guid firstId,
	Guid secondId,
	string secondIdName) => new
{
	Condition = firstId != secondId,
	Message = $"Id is not the same as {secondIdName}.",
	HelpLink = "/help/code1234"
};

Our dynamic rule now can offer more input parameters and more helpful information in terms of more detailed exception message with links to helpful documentation sites or references for error codes.

2.1.3.1.1.2 Rules & Validations Collector

Now that have the advanced exceptions and the dynamic validation rules. It's time to put it all together in terms of accepting infinite number of validation rules, examining their condition results and finally break the circuit when all the continuous validations are done. here's how to do that:

private void Validate(params (dynamic Rule, string Parameter)[] validations)
{
	var invalidStudentException = new InvalidStudentException();

	foreach((dynamic rule, string parameter) in validations)
	{
		if(rule.Condition)
		{
			invalidStudentException.UpsertData(parameter, rule.Message);
		}
	}

	invalidStudentException.ThrowIfContainsErrors();
}

The above function now will take any number of validation rules, and the parameters the rule is running against then examine the conditions and upsert the report of errors. This is how we can use the method above:

private void ValidateStudent(Student student)
{
	Validate(
		(Rule: IsInvalid(student.Id), Parameter: nameof(Student.Id)),
		(Rule: IsInvalid(student.Name), Parameter: nameof(Student.name)),
		(Rule: IsInvalid(student.Grade), Parameter: nameof(Student.Grade))
	);
}

This simplification of writing the rules and validations is the ultimate goal of continuing to provide value to the end users while making the process of engineering the solution pleasant to the engineers themselves.

Now, let's dive deeper into the types of validations that our systems can offer and how to handle them.

2.1.3.1.2 Structural Validations

Validations are three different layers. the first of these layers is the structural validations. to ensure certain properties on any given model or a primitive type are not in an invalid structural state.

For instance, a property of type String should not be empty, null or white space. another example would be for an input parameter of an int type, it should not be at it's default state which is 0 when trying to enter an age for instance.

The structural validations ensure the data is in a good shape before moving forward with any further validations - for instance, we can't possibly validate a student has the minimum number of characters (which is a logical validation) in their names if their first name is structurally invalid structurally by being null, empty or whitespace.

Structural validations play the role of identifying the required properties on any given model, and while a lot of technologies offer the validation annotations, plugins or libraries to globally enforce data validation rules, I choose to perform the validation programmatically and manually to gain more control of what would be required and what wouldn't in a TDD fashion.

The issue with some of the current implementations on structural and logical validations on data models is that it can be very easily changed under the radar without any unit tests firing any alarms. Check this example for instance:

public Student
{
	[Required]
	public string Name {get; set;}
}

The above example can be very enticing at a glance from an engineering standpoint. All you have to do is decorate your model attribute with a magical annotation and then all of the sudden your data is being validated.

The problem here is that this pattern combines two different responsibilities or more together all in the same model. Models are supposed to be just a representation of objects in reality - nothing more and nothing less. Some engineers call them anemic models which focuses the responsibility of every single model to only represent the attributes of the real world object it's trying to simulate without any additional details.

But the annotated models now try to inject business logic into their very definitions. This business logic may or may not be needed across all services, brokers or exposing components that uses them.

Structural validations on models may seem like extra work that can be avoided with magical decorations. But in the case of trying to diverge slightly from these validations into a more customized validations, now you will see a new anti-pattern emerge like custom annotations that may or may not be detectable through unit tests.

Let's talk about how to test a structural validation routine:

2.1.3.1.2.0 Testing Structural Validations

Because I truly believe in the importance of TDD, I am going to start showing the implementation of structural validations by writing a failing test for it first.

Let's assume we have a student model, with the following details:

public class Student
{
	public Guid Id {get; set;}
}

We want to validate that the student Id is not a structurally invalid Id - such as an empty Guid - therefore we would write a unit test in the following fashion:

[Fact]
public async void ShouldThrowValidationExceptionOnRegisterWhenIdIsInvalidAndLogItAsync()
{
	// given
	Student randomStudent = CreateRandomStudent();
	Student inputStudent = randomStudent;
	inputStudent.Id = Guid.Empty;

	var invalidStudentException = new InvalidStudentException();

	invalidStudentException.AddData(
		key: nameof(Student.Id),
		value: "Id is required"
	);

	var expectedStudentValidationException =
		new StudentValidationException(invalidStudentException);

	// when
	ValueTask<Student> registerStudentTask =
		this.studentService.RegisterStudentAsync(inputStudent);

	// then
	await Assert.ThrowsAsync<StudentValidationException>(() =>
		registerStudentTask.AsTask());

	this.loggingBrokerMock.Verify(broker =>
		broker.LogError(It.Is(SameExceptionAs(expectedStudentValidationException))),
			Times.Once);

	this.storageBrokerMock.Verify(broker =>
		broker.InsertStudentAsync(It.IsAny<Student>()),
			Times.Never);

	this.dateTimeBrokerMock.VerifyNoOtherCalls();
	this.loggingBrokerMock.VerifyNoOtherCalls();
	this.storageBrokerMock.VerifyNoOtherCalls();
}

In the above test, we created a random student object then assigned the an invalid Id value of Guid.Empty to the student Id.

When the structural validation logic in our foundation service examines the Id property, it should throw an exception property describing the issue of validation in our student model. in this case we throw InvalidStudentException.

The exception is required to briefly describe the whats, wheres and whys of the validation operation. in our case here the what would be the validation issue occurring, the where would be the Student service and the why would be the property value.

Here's how an InvalidStudentException would look like:

public class InvalidStudentException : Xeption
{
	public InvalidStudentException()
		:base ("Student is invalid. Please fix the errors and try again.")
	{ }
}

The above unit test however, requires our InvalidStudentException to be wrapped up in a more generic system-level exception, which is StudentValidationException - these exceptions is what I call outer-exceptions, they encapsulate all the different situations of validations regardless of their category and communicates the error to upstream services or controllers so they can map that to the proper error code to the consumer of these services.

Our StudentValidationException would be implemented as follows:

public class StudentValidationException : Exception
{
	public StudentValidationException(Exception innerException)
		: base("Student validation error occurred, please check your input and then try again.", innerException) { }
}

The message in the outer-validation above indicates that the issue is in the input, and therefore it requires the input submitter to try again as there are no actions required from the system side to be adjusted.

2.1.3.1.2.1 Implementing Structural Validations

Now, let's look at the other side of the validation process, which is the implementation. Structural validations always come before each and every other type of validations. That's simply because structural validations are the cheapest from an execution and asymptotic time perspective. For instance, It's much cheaper to validation an Id is invalid structurally, than sending an API call across to get the exact same answer plus the cost of latency. This all adds up when multi-million requests per second start flowing in. Structural and logical validations in general live in their own partial class to run these validations, for instance, if our service is called StudentService.cs then a new file should be created with the name StudentService.Validations.cs to encapsulate and visually abstract away the validation rules to ensure clean data are coming in and going out. Here's how an Id validation would look like:

StudentService.Validations.cs

private void ValidateStudent(Student student)
{
	Validate((Rule: IsInvalid(student.Id), Parameter: nameof(Student.Id)));
}

private static dynamic IsInvalid(Guid id) => new
{
	Condition = id == Guid.Empty,
	Message = "Id is required"
};

private void Validate(params (dynamic Rule, string Parameter)[] validations)
{
	var invalidStudentException = new InvalidStudentException();

	foreach((dynamic rule, string parameter) in validations)
	{
		if(rule.Condition)
		{
			invalidStudentException.UpsertData(parameter, rule.Message);
		}
	}

	invalidStudentException.ThrowIfContainsErrors();
}

We have implemented a method to validate the entire student object, with a compilation of all the rules we need to setup to validate structurally and logically the student input object. The most important part to notice about the above code snippet is to ensure the encapsulation of any finer details further away from the main goal of a particular method.

That's the reason why we decided to implement a private static method IsInvalid to abstract away the details of what determines a property of type Guid is invalid or not. And as we move further in the implementation, we are going to have to implement multiple overloads of the same method to validate other value types structurally and logically.

The purpose of the ValidateStudent method is to simply set up the rules and take an action by throwing an exception if any of these rules are violated. There's always an opportunity to aggregate the violation errors rather than throwing too early at the first sign of structural or logical validation issue to be detected.

Now, with the implementation above, we need to call that method to structurally and logically validate our input. Let's make that call in our RegisterStudentAsync method as follows:

StudentService.cs

public ValueTask<Student> RegisterStudentAsync(Student student) =>
TryCatch(async () =>
{
	ValidateStudent(student);

	return await this.storageBroker.InsertStudentAsync(student);
});

At a glance, you will notice that our method here doesn't necessarily handle any type of exceptions at the logic level. That's because all the exception noise is also abstracted away in a method called TryCatch.

TryCatch is a concept I invented to allow engineers to focus on what matters that most based on which aspect of the service that are looking at without having to take any shortcuts with the exception handling to make the code a bit more readable.

TryCatch methods in general live in another partial class, and an entirely new file called StudentService.Exceptions.cs - which is where all exception handling and error reporting happens as I will show you in the following example.

Let's take a look at what a TryCatch method looks like:

StudentService.Exceptions.cs

private delegate ValueTask<Student> ReturningStudentFunction();

private async ValueTask<Student> TryCatch(ReturningStudentFunction returningStudentFunction)
{
	try
	{
		return await returningStudentFunction();
	}
	catch (InvalidStudentException invalidStudentInputException)
	{
		throw CreateAndLogValidationException(invalidStudentInputException);
	}
}

private StudentValidationException CreateAndLogValidationException(Exception exception)
{
	var studentValidationException = new StudentValidationException(exception);
	this.loggingBroker.LogError(studentValidationException);

	return studentValidationException;
}

The TryCatch exception noise-cancellation pattern beautifully takes in any function that returns a particular type as a delegate and handles any thrown exceptions off of that function or it's dependencies.

The main responsibility of a TryCatch function is to wrap up a service inner exceptions with outer exceptions to ease-up the reaction of external consumers of that service into only one of the three categories, which are Service Exceptions, Validations Exceptions and Dependency Exceptions. there are sub-types to these exceptions such as Dependency Validation Exceptions but these usually fall under the Validation Exception category as we will discuss in upcoming sections of The Standard.

In a TryCatch method, we can add as many inner and external exceptions as we want and map them into local exceptions for upstream services not to have a strong dependency on any particular libraries or external resource models, which we will talk about in detail when we move on to the Mapping responsibility of broker-neighboring (foundation) services.

2.1.3.1.3 Logical Validations

Logical validations are the second in order to structural validations. their main responsibility by definition is to logically validate whether a structurally valid piece of data is logically valid. For instance, a date of birth for a student could be structurally valid by having a value of 1/1/1800 but logically, a student that is over 200 years of age is an impossibility.

The most common logical validations are validations for audit fields such as CreatedBy and UpdatedBy - it's logically impossible that a new record can be inserted with two different values for the authors of that new record - simply because data can only be inserted by one person at a time.

Let's talk about how we can test-drive and implement logical validations:

2.1.3.1.3.0 Testing Logical Validations

In the common case of testing logical validations for audit fields, we want to throw a validation exception that the UpdatedBy value is invalid simply because it doesn't match the CreatedBy field.

Let's assume our Student model looks as follows:

public class Student {
	Guid CreatedBy {get; set;}
	Guid UpdatedBy {get; set;}
}

Our test to validate these values logically would be as follows:

[Fact]
public async Task ShouldThrowValidationExceptionOnRegisterIfUpdatedByNotSameAsCreatedByAndLogItAsync()
{
	// given
	Student randomStudent = CreateRandomStudent();
	Student inputStudent = randomStudent;
	inputStudent.UpdatedBy = Guid.NewGuid();

	var invalidStudentException = new InvalidStudentException();

	invalidStudentException.AddData(
		key: nameof(Student.UpdatedBy),
		value: $"Id is not the same as {nameof(Student.CreatedBy)}.");

	var expectedStudentValidationException =
		new StudentValidationException(invalidStudentException);

	// when
	ValueTask<Student> registerStudentTask =
		this.studentService.RegisterStudentAsync(inputStudent);

	// then
	await Assert.ThrowsAsync<StudentValidationException>(() =>
		registerStudentTask.AsTask());

	this.loggingBrokerMock.Verify(broker =>
		broker.LogError(It.Is(
			SameExceptionAs(expectedStudentValidationException))),
				Times.Once);

	this.storageBrokerMock.Verify(broker =>
		broker.InsertStudentAsync(It.IsAny<Student>()),
			Times.Never);

	this.loggingBrokerMock.VerifyNoOtherCalls();
	this.dateTimeBrokerMock.VerifyNoOtherCalls();
	this.storageBrokerMock.VerifyNoOtherCalls();
}

In the above test, we have changed the value of the UpdatedBy field to ensure it completely differs from the CreatedBy field - now we expect an InvalidStudentException with the CreatedBy to be the reason for this validation exception to occur.

Let's go ahead an write an implementation for this failing test.

2.1.3.1.3.1 Implementing Logical Validations

Just like we did in the structural validations section, we are going to add more rules to our validation switch case as follows:

StudentService.Validations.cs

private void ValidateStudent(Student student)
{
	Validate(
		(Rule: IsNotSame(
			firstId: student.UpdatedBy,
			secondId: student.CreatedBy,
			secondIdName: nameof(student.CreatedBy)),
		Parameter: nameof(Student.UpdatedBy))
	);
}

private static dynamic IsNotSame(
	Guid firstId,
	Guid secondId,
	string secondIdName) => new
	{
		Condition = firstId != secondId,
		Message = $"Id is not the same as {secondIdName}."
	};

private void Validate(params (dyanamic Rule, string Parameter)[] validations)
{
	var invalidStudentException = new Exception();

	foreach((dynamic rule, string parameter) in validations)
	{
		if(rule.Condition)
		{
			invalidStudentException.UpsertData(
				key: parameter,
				value: rule.Message);
		}
	}
}

Everything else in both StudentService.cs and StudentService.Exceptions.cs continues to be exactly the same as we've done above in the structural validations.

Logical validations exceptions, just like any other exceptions that may occur are usually non-critical. However, it all depends on your business case to determine whether a particular logical, structural or even a dependency validation are critical or not, this is when you might need to create a special class of exceptions, something like InvalidStudentCriticalException then log it accordingly.

2.1.3.1.4 External Validations

The last type of validations that are usually performed by foundation services is external validations. I define external validations as any form of validation that requires calling an external resource to validate whether a foundation service should proceed with processing incoming data or halt with an exception.

A good example of dependency validations is when we call a broker to retrieve a particular entity by it's id. If the entity returned is not found, or the API broker returns a NotFound error - the foundation service is then required to wrap that error in a ValidationException and halts all following processes.

External validation exceptions can occur if the returned value did not match the expectation, such as an empty list returned from an API call when trying to insert a new coach of a team - if there is no team members, there can be no coach for instance. The foundation service in this case will be required to raise a local exception to explain the issue, something like NoTeamMembersFoundException or something of that nature.

Let's write a failing test for an external validation example:

2.1.3.1.4.0 Testing External Validations

Let's assume we are trying to retrieve a student with an Id that doesn't match any records in the database. Here's how we would go about testing this scenario. First off, let's define a NotFoundStudentException model as follows:

using Xeption;

public class NotFoundStudentException : Xeption
{
	public NotFoundStudentException(Guid id)
		: base (message: $"Couldn't find a student with id: {id}.")
	{}
}

The above model is the localization aspect of handling the issue. Now let's write a failing test as follows:

public async Task ShouldThrowValidationExceptionOnRetrieveByIdIfStudentNotFoundAndLogItAsync()
{
	// given
	Guid randomStudentId = Guid.NewGuid();
	Guid inputStudentId = randomStudentId;
	Student noStudent = null;

	var notFoundStudentException =
		new NotFoundStudentException(inputStudentId);

	var expectedStudentValidationException =
		new StudentValidationException(notFoundStudentException);

	this.storageBrokerMock.Setup(broker =>
		broker.SelectStudentByIdAsync(inputStudentId))
			.ReturnsAsync(noStudent);

	// when
	ValueTask<Student> retrieveStudentByIdTask =
		this.studentService.RetrieveStudentByIdAsync(inputStudentId);

	// then
	await Assert.ThrowsAsync<StudentValidationException>(() =>
		retrieveStudentByIdTask.AsTask());

	this.storageBrokerMock.Verify(broker =>
		broker.SelectStudentByIdAsync(inputStudentId),
			Times.Once);

	this.loggingBrokerMock.Verify(broker =>
		broker.LogError(It.Is(SameExceptionAs(
			expectedStudentValidationException))),
				Times.Once);

	this.storageBrokerMock.VerifyNoOtherCalls();
	this.loggingBrokerMock.VerifyNoOtherCalls();
	this.dateTimeBrokerMock.VerifyNotOtherCalls();
}

The test above requires us to throw a localized exception as in NotFoundStudentException when the storage broker returns no values for the given studentId and then wrap or categorize this up in StudentValidationException.

We choose to wrap the localized exception in a validation exception and not in a dependency validation exception because the initiation of the error originated from our service not from the external resource. If the external resource is the source of the error we would have to categorize this as a DependencyValidationException which we will discuss shortly.

Now let's get to the implementation part of this section to make our test pass.

2.1.3.1.4.1 Implementing External Validations

In order to implement an external validation we will need to touch on all different aspects of our service. The core logic, the validation and the exception handling aspects are as follows.

First off, let's build a validation function that will throw a NotFoundStudentException if the passed-in parameter is null.

StudentService.Validations.cs

private static void VerifyStudentExists(Student maybeStudent, Guid studentId)
{
	if (maybeStudent is null)
	{
		throw new NotFoundStudentException(studentId);
	}
}

This implementation will take care of detecting an issue and issuing a local exception NotFoundStudentException. Now let's jump over to the exception handling aspect of our service.

StudentService.Exceptions.cs

private async ValueTask<Student> TryCatch(ReturningStudentFunction returningStudentFunction)
{
	try
	{
		return await returningStudentFunction();
	}
	..
	catch (NotFoundStudentException notFoundStudentException)
	{
		throw CreateAndLogValidationException(notFoundStudentException);
	}
}

private StudentValidationException CreateAndLogValidationException(Exception exception)
{
	var studentValidationException = new StudentValidationException(exception);
	this.loggingBroker.LogError(studentValidationException);

	return studentValidationException;
}

The above implementation will take care of categorizing a NotFoundStudentException to StudentValidationException. The last part is to put the pieces together as follows.

StudentService.cs

public ValueTask<Student> RetrieveStudentByIdAsync(Guid studentId) =>
TryCatch(async () =>
{
	ValidateStudentId(studentId);

	Student maybeStudent =
		await this.storageBroker.SelectStudentByIdAsync(studentId);

	ValidateStudentExists(maybeStudent, studentId);

	return maybeStudent;
});

The above implementation will ensure that the id is valid, but more importantly that whatever the storageBroker returns will be checked for whether it's an object or null. Then issue the exception.

There are situations where attempting to retrieve an entity then finding out that it doesn't exist is not necessarily erroneous. This is where Processing Services come in to leverage a higher-order business logic to deal with this more complex scenario.

2.1.3.1.5 Dependency Validations

Dependency validation exceptions can occur because you called an external resource and it returned an error, or returned a value that warrants raising an error. For instance, an API call might return a 404 code, and that's interpreted as an exception if the input was supposed to correspond to an existing object.

A more common example is when a particular input entity is using the same id as an existing entity in the system. In a relational database world, a duplicate key exception would be thrown. In a RESTful API scneario, programmatically applying the same concept also achieves the same goal for API validations assuming the granularity of the system being called weaken the referential integrity of the overall system data.

There are situations where the faulty response can be expressed in a fashion other than exceptions, but we shall touch on that topic in a more advanced chapters of this Standard.

Let's write a failing test to verify whether we are throwing a DependencyValidationException if Student model already exists in the storage with the storage broker throwing a DuplicateKeyException as a native result of the operation.

2.1.3.1.5.0 Testing Dependency Validations

Let's assume our student model uses an Id with the type Guid as follows:

public class Student
{
	public Guid Id {get; set;}
	public string Name {get; set;}
}

our unit test to validate a DependencyValidation exception would be thrown in a DuplicateKey situation would be as follows:

[Fact]
public async void ShouldThrowDependencyValidationExceptionOnRegisterIfStudentAlreadyExistsAndLogItAsync()
{
	// given
	Student someStudent = CreateRandomStudent();
	string someMessage = GetRandomMessage();
	var duplicateKeyException = new DuplicateKeyException(exceptionMessage);

	var alreadyExistsStudentException =
		new AlreadyExistsStudentException(duplicateKeyException);

	var expectedStudentValidationException =
		new StudentDependencyValidationException(alreadyExistsStudentException);

	this.storageBrokerMock.Setup(broker =>
		broker.InsertStudentAsync(It.IsAny<Student>()))
			.ThrowsAsync(duplicateKeyException);

	// when
	ValueTask<Student> registerStudentTask =
		this.studentService.RegisterStudentAsync(someStudent);

	// then
	await Assert.ThrowsAsync<StudentValidationException>(() =>
		registerStudentTask.AsTask());

	this.storageBrokerMock.Verify(broker =>
		broker.InsertStudentAsync(It.IsAny<Student>()),
			Times.Once);

	this.loggingBrokerMock.Verify(broker =>
		broker.LogError(It.Is(
			SameExceptionAs(expectedStudentValidationException))),
				Times.Once);

	this.storageBrokerMock.VerifyNoOtherCalls();
	this.loggingBrokerMock.VerifyNoOtherCalls();
}

In the above test, we validate that we wrap a native DuplicateKeyException in a local model tailored to the specific model case which is the AlreadyExistsStudentException in our example here. then we wrap that again with a generic category exception model which is the StudentDependencyValidationException.

There's a couple of rules that govern the construction of dependency validations, which are as follows:

• Rule 1: If a dependency validation is handling another dependency validation from a downstream service, then the inner exception of the downstream exception should be the same for the dependency validation at the current level.

In other words, if some StudentService is throwing a StudentDepdendencyValidationException to an upstream service such as StudentProcessingService - then it's important that the StudentProcessingDependencyValidationException contain the same inner exception as the StudentDependencyValidationException. That's because once these exception are mapped into exposure components, such as API controller or UI components, the original validation message needs to propagate through the system and be presented to the end user no matter where it originated from.

Additionally, maintaining the original inner exception guarantees the ability to communicate different error messages through API endpoints. For instance, AlreadyExistsStudentException can be communicated as Conflict or 409 on an API controller level - this differs from another dependency validation exception such as InvalidStudentReferenceException which would be communicated as FailedDependency error or 424.

• Rule 2: If a dependency validation exception is handling a non-dependency validation exception it should take that exception as it's inner exception and not anything else.

These rules ensures that only the local validation exception is what's being propagated not it's native exception from a storage system or an API or any other external dependency.

Which is the case that we have here with our AlreadyExistsStudentException and it's StudentDependencyValidationException - the native exception is completely hidden away from sight, and the mapping of that native exception and it's inner message is what's being communicated to the end user. This gives the engineers the power to control what's being communicated from the other end of their system instead of letting the native message (which is subject to change) propagate to the end-users.

2.1.3.0.5.1 Implementing Dependency Validations

Depending on where the validation error originates from, the implementation of dependency validations may or may not contain any business logic. As we previously mentioned, if the error is originating from the external resource (which is the case here) - then all we have to do is just wrap that error in a local exception then categorize it with an external exception under dependency validation.

To ensure the aforementioned test passed, we are going to need few models. For the AlreadyExistsStudentException the implementation would be as follows:

public class AlreadyExistsStudentException : Exception
{
	public AlreadyExistsStudentException(Exception innerException)
		: base($"Student with the same Id already exists", innerException){ }
}

We also need the StudentDependencyValidationException which should be as follows:

public class StudentDependencyValidationException : Exception
{
	public StudentDependencyValidationException(Exception innerException)
		: base($"Student dependency validation error occurred, please try again.", innerException){ }
}

Now, let's go to the implementation side, let's start with the exception handling logic:

StudentService.Exceptions.cs

private delegate ValueTask<Student> ReturningStudentFunction();

private async ValueTask<Student> TryCatch(ReturningStudentFunction returningStudentFunction)
{
	try
	{
		return await returningStudentFunction();
	}
	...
	catch (DuplicateKeyException duplicateKeyException)
	{
		var alreadyExistsStudentException = new AlreadyExistsStudentException(duplicateKeyException);
		throw CreateAndLogDependencyValidationException(alreadyExistsStudentException);
	}
}

...

private StudentDependencyValidationException CreateAndLogDependencyValidationException(Exception exception)
{
	var studentDependencyValidationException = new StudentDependencyValidationException(exception);
	this.loggingBroker.LogError(studentDependencyValidationException);

	return studentDependencyValidationException;
}

We created the local inner exception in the catch block of our exception handling process to allow the reusability of our dependency validation exception method for other situations that require that same level of external exceptions.

Everything else stays the same for the referencing of the TryCatch method in the StudentService.cs file.

2.1.3.2 Mapping

The second responsibility for a foundation service is to play the role of a mapper both ways between local models and non-local models. For instance, if you are leveraging an email service that provides it's own SDKs to integrate with, and your brokers are already wrapping and exposing the APIs for that service, your foundation service is required to map the inputs and outputs of the broker methods into local models. the same situation and more commonly between native non-local exceptions such as the ones we mentioned above with the dependency validation situation, the same aspect applies to just dependency errors or service errors as we will discuss shortly.

2.1.3.2.0 Non-Local Models

Its very common for modern applications to require integration at some point with external services. these services can be local to the overall architecture or distributed system where the application lives, or it can be a 3rd party provider such as some of the popular email services for instance. External services providers invest a lot of effort in developing fluent APIs, SDKs and libraries in every common programming language to make it easy for the engineers to integrate their applications with that 3rd party service. For instance, let's assume a third party email service provider is offering the following API through their SDKs:

public interface IEmailServiceProvider
{
	ValueTask<EmailMessage> SendEmailAsync(EmailMessage message);
}

Let's consider the model EmailMessage is a native model, it comes with the email service provider SDK. your brokers might offer a wrapper around this API by building a contract to abstract away the functionality but can't do much with the native models that are passed in or returned out of these functionality. therefore our brokers interface would look something like this:

public interface IEmailBroker
{
	ValueTask<EmailMessage> SendEmailMessageAsync(EmailMessage message);
}

Then the implementation would something like this:

public class EmailBroker : IEmailBroker
{
	public async ValueTask<EmailMessage> SendEmailMessageAsync(EmailMessage message) =>
		await this.emailServiceProvider.SendEmailAsync(message);
}

As we said before, the brokers here have done their part of abstraction by pushing away the actual implementation and the dependencies of the native EmailServiceProvider away from our foundation serviecs. But that's only 50% of the job, the abstraction isn't quite fully complete yet until there are no tracks of the native EmailMessage model. This is where the foundation services come in to do a test-driven operation of mapping between the native non-local models and your application's local models. therefore its very possible to see a mapping function in a foundation service to abstract away the native model from the rest of your business layer services.

Your foundation service then will be required to support a new local model, let's call it Email. your local model's property may be identical to the external model EmailMessage - especially on a primitive data type level. But the new model would be the one and only contract between your pure business logic layer (processing, orchestration, coordination and management services) and your hybrid logic layer like the foundation services. Here's a code snippet for this operation:

public async ValueTask<Email> SendEmailMessageAsync(Email email)
{
	EmailMessage inputEmailMessage = MapToEmailMessage(email);
	EmailMessage sentEmailMessage = await this.emailBroker.SendEmailMessageAsync(inputEmailMessage);

	return MapToEmail(sentEmailMessage);
}

Depending on whether the returned message has a status or you would like to return the input message as a sign of a successful operation, both practices are valid in my Standard. It all depends on what makes more sense to the operation you are trying to execute. the code snippet above is an ideal scenario where your code will try to stay true to the value passed in as well as the value returned with all the necessary mapping included.

2.1.3.2.1 Exceptions Mappings

Just like the non-local models, exceptions that are either produced by the external API like the EntityFramework models DbUpdateException or any other has to be mapped into local exception models. Handling these non-local exceptions that early before entering the pure-business layer components will prevent any potential tight coupling or dependency on any external model. as it may be very common, that exceptions can be handled differently based on the type of exception and how we want to deal with it internally in the system. For instance, if we are trying to handle a UserNotFoundException being thrown from using Microsoft Graph for instance, we might not necessarily want to exit the entire procedure. we might want to continue by adding a user in some other storage for future Graph submittal processing. External APIs should not influence whether your internal operation should halt or not. and therefore handling exceptions on the Foundation layer is the guarantee that this influence is limited within the borders of our external resources handling area of our application and has no impact whatsoever on our core business processes. The following illustration should draw the picture a bit clearer from that perspective:

Here's some common scenarios for mapping native or inner local exceptions to outer exceptions:

Exception	Wrap Inner Exception With	Wrap With	Log Level
NullStudentException	-	StudentValidationException	Error
InvalidStudentException	-	StudentValidationException	Error
SqlException	FailedStudentStorageException	StudentDependencyException	Critical
HttpResponseUrlNotFoundException	FailedStudentApiException	StudentDependencyException	Critical
HttpResponseUnauthorizedException	FailedStudentApiException	StudentDependencyException	Critical
NotFoundStudentException	-	StudentValidationException	Error
HttpResponseNotFoundException	NotFoundStudentException	StudentDependencyValidationException	Error
DuplicateKeyException	AlreadyExistsStudentException	StudentDependencyValidationException	Error
HttpResponseConflictException	AlreadyExistsStudentException	StudentDependencyValidationException	Error
ForeignKeyConstraintConflictException	InvalidStudentReferenceException	StudentDependencyValidationException	Error
DbUpdateConcurrencyException	LockedStudentException	StudentDependencyValidationException	Error
DbUpdateException	FailedStudentStorageException	StudentDependencyException	Error
HttpResponseException	FailedStudentApiException	StudentDependencyException	Error
Exception	FailedStudentServiceException	StudentServiceException	Error

[*] Standerdizing Validations & Exceptions

[*] Test-Driving Non-Circuit-Breaking Validations