Purposing, Modeling and Simulation

0.0.0 Introduction

The Standard defines the process of software engineering in three main categories:

• Purposing,

• Modeling

• Simulation

The order in which these aspects are followed is also intentional.

A purpose must exist to shape the modeling process.

And one can't simulate interactions with no models.

While that order at the initiation of the engineering process is crucial, It's important to understand that the process itself is selectively iterative. A change in the purpose may reflect as a change in the simulation but not necessarily the modeling. A change in the models may not neccessarily require a change in either the purpose nor the simulation.

0.0.1 Purposing

The purposing process is our ability to ==find out why we need a solution.==

For instance:

• We are having an issue with knowing how many items are on the shelf in some grocery store.

• We deem the manual process of counting to be inefficient.

• A system needs to be put in place to ensure we have the proper count of items.

Purposing therefore, is to find a reason to take an action.

Reasoning relies heavily on observability. Our ability to observe problems and then being able to articulate the problem to better device a solution that addresses the given problem.

So, we have the:

• 0bservation

• Articulation or reflexion on the problem

• Intent for a solution.

0.0.1.0 Observation

We live in a world full of observables. Our inspiration is triggered by our ambition to achieve more. Our dreams reveal blockers in our way that we need to solve to continue our journey and fulfill our dreams. From the moment a young student uses a calculator to solve a complex equation to the moment that very student becomes an astronaut, calculatoring the trajectory of satellites orbiting our planet.

Observation is our ability to detect an issue that's blocking a goal from being achieved. Issues can be as simple as coming up with the proper count of items on a grocery store shelf. All the way up to understanding why we can't capture images of planets millions of light years away from us. These are all what engineers would describe an observable problem.

The greater the purpose, the more complex a problem is going to be. But these smaller purposes are our way to train our minds to tackle bigger ones. Step by step, one problem at a time.

0.0.1.1 Articulation

Describing the observable is an art in and of itself. Because describing a problem properly is half way to its solution. The clearer the articulation of the problem is, the more likely the problem is to be understood by others helping us to solve that very same problems.

Articulation isn't always with words. It's also with figures and shapes. It is not an accident that some of the most advanced ancient cultures have used figures and shapes to describe their times and their history. Figures are a universal language, understood and interpreted by anyone who could relate to them much faster than learning a spoken language. In fact, a figure or shape might be the most optimum way to illustrate an idea as it pictures are worthy of thousands of words.

Articulation requires the passion to solve the issue. Whether it's written, said or illustrated. A passionate mind shall convey the hidden message of the criticality of the problem to be solved. Articulating a problem is a big part of how one can sell their solution. Our ability to convey an idea towards other engineers and then those who will be investing and using this solution is one of the most important aspects of engineering software.

0.0.1.2 Solutioning

A part of the purpose is the way to fulfil it. In the engineering industry, fulfilling the goals can't just be by -any- means. A huge aspect of why so many massive pieces of software fails around the world is because the solutioning aspect was overlooked as a trivial part of the purpose. You may have heard of those who were pressured against a deadline so they decided to cut corners to achieve the goal. In our Standard, this is a violation. A solution must be not just fulfilling to a goal, but it must be a purpose in and of itself. This is in terms of optimization, readability, configurability and longevity. Solutioning as part of the purpose is software craftmanship.

0.0.2 Modeling

Modeling is the second most important aspect of software engineering. It's our ability to extract models from the actors in any problem. Whether these actors are living beings, objects or other. For instance, in a problem where we are trying to count the items on a grocery store shelf, a model would be for these items. Extracting only the attributes that are relevant to the problem we are trying to solve, and discarding everything else.

A simpler example would be detecting the items in a grocery store that are perishable. The only attribute we are concerned with here is the expiration date on the item. Everything else including the label, color, weight or any other details are out of the scope of the modeling process and the solution.

Modeling then cannot exist without a purpose. As the purpose defines the scope or the framework of which the modeling should occur. Modeling without a purpose leaves the door open for attracting infinite number of attributes every single element in the observable universe may have.

The relationship between the purposing and modeling attributes is proportional. The more complex the purpose is, the more likely the modeling process will require more attributes from the real world to resemble in a prototype.

We express our models in programming languages as a class. The aforementioned perishable items problem above can be represented as follows:

public class Item
{
	public DateTimeOffset ExpirationDate {get; set;}
}

The name of the class represents the overall type of the item. Since all items have the exact same attribute of ExpirationDate then the name shall stay as generic as it can be.

Now, imagine if our purpose grew a bit more complex. Let's assume the new problem is to be able to identify the more expensive perishable items so the store can put them up front for selling before the less expensive items. In this case our model would require a new attribute such as Price so a computer program or a solution can determine which is more valuable. This is how our new model would look like:

public class Item
{
	public double Price {get; set;}
	public DateTimeOffset ExpirationDate {get; set;}
}

0.0.2.0 Model Types

Models govern the entire process of simulating a problem (and its solution). Models themselves break into three main categories, Data Carriers, Operational and Configurations. Let's discuss those types in the following sections:

0.0.2.0.0 Data Carrier Models

Data carrier models have one main purpose which is to carry data points across systems. Data carrier models can vary based on the type of data they carry. Some data carrier models will carry other models to represent a complex system. Some others will just represent references to the original data points it represents.

Data carrier models in a relational fashion can be broken into three different categories. These categories makes it a lot clearer what the areas of priority are in terms of development, design and engineering. For instance, we can't start developing secondary/supporting models if we don't have our primary models in place first. Let's talk about these categoris in details:

0.0.2.0.0.0 Primary Models

Primary models are the pillars of every system. Any given system cannot proceed in terms of design and engineering without a clear definition and a materialization of these primary models. For instance, if we are building a schooling system, models like Student, Teacher and Course are considered Primary models.

We call these models Primary because they are self-sufficient. They don't rely physically on some other model in order for them to exist. Which means that a given Primary model like Student may still exist in a schooling system whether a Teacher record existed or not. This is called physical dependency. Primary models in a relational storage schema do not contain any foreign keys or references to any other physical model.

Primary models however may rely conceptually or logically on other models. For instance, a Student model has a logical relationship to a Teacher. Simply because there can never be a student without a teacher and vice versa. A Student model also has a conceptual relationship with its host and neighboring hosting services. For instance, there's a conceptual relationship between a Student model and a Notification model in terms of business flow. Any given student in any school conceptually relies on notifications to attend certain classes, complete certain assignments or any other events.

0.0.2.0.0.1 Secondary Models

Secondary models on the other hand, have a hard-dependency on Primary models. In a relational database model, secondary models usually have foreign keys referencing another model in the overall database schema. But even in non-relational storage systems, secondary models can be represented as nested entities within a given larger entity or have a loose reference to another entity.

Let's talk about some examples for secondary models. a Comment model in a social media platform cannot exist without a Post model. You simply cannot comment on something that doesn't exist. In a relational database, the comments model would look something like this:

In the example above, a secondary model Comment has a foreign key PostId referencing the primary key Id in a Post model. In a non-relational system, secondary models can easily be identified as nested objects within a given entity. Here's an example:

{
	"id": "some-id",
	"content": "some post",
	"comments": [
		{
			"id": "comment-id",
			"content": "some comment"
		}
	]
}

Secondary models in general may have logical and conceptual relations to other models within their host, neighboring or external systems. However, their chances of having these conceptual relations are much less than the Primary models.

0.0.2.0.0.2 Relational Models

Relational models are connectors between two Primary models. Their main responsibility is to materialize a many-to-many relationship between two entities. For instance, a Student may have multiple teachers; and a Teacher may have multiple students. In this case an intermediary model.

Relational models are not supposed to have any details in them. They only contain references to other models and that is their primary key. A composite key that aggregates two or more foreign keys within it. Let's take a look at an example:

0.0.2.0.0.3 Hybrid Models

There's a situation where a model is connecting multiple entities but is also carrying its own data. I highly advise against following that path to maintain a level of purity in your system design and control the complexity of your models. However, sometimes this approach is a necessary option to proceed with a certain implementation or a business flow. In which case, we can propose a hybrid model that can carry certain details about the relationship between two independent entities.

Here's an example of a hybrid model that can occur in reality. In a soft-delete scenario, a hybrid model can describe the detachment between two entities in a many-to-many relationship. Let's assume a group member does not want to be a part of a certain group anymore. We consider their group membersip as Deactivated with a reason attached without actually fully delete the record. Here's how would that look like:

In a non-relational data model the refrencing integrity might become a bit more looser given the linear nature of that schema. Hybrid models combine both secondary models in the way they reference Primary models, and they implement a relational nature in which they allow multiple entities to relate to each other with no exclusivity.

0.0.2.0.1 Operational Models

Operational models mainly target the simulation aspect of any software system. Think about all primitive, complex and exposure operations a simple scenario could require for a successful simulation to be implemented. Let's assume we are trying to solve a problem where we can simplify students registrations in some school. The registration process will require some simulation to add these students information into a computerized system.

Operational models will handle the exposure, processing and integration of that entire process, by offering services that offers APIs/UIs to enter, post, add and insert/persist students information into some schooling system.

Operational models can be broken into three main categories, which are: Integration, Processing and Exposure. The Standard focuses heavily on operational models because they represent the core of any system in terms of business flows. Operational models are also where most of the resources of development and design go in any software development effort.

Let's talk the operational models here.

0.0.2.0.1.0 Integration Models (Brokers)

Integration operational models main responsibility is to connect any existing system with external resources. These resources can be localized to the environment of that system like reading the current date or time or remote like calling an external API or persisting data in some database.

We call these integration models Brokers. They play the role of a laiason between processing operational models and external systems. here's an exmaple:

public partial class ApiBroker
{
	public async ValueTask<Student> PostStudentAsync(Student student) =>
		this.apiBoroker.PostAsync<Student>(student, url);
}

The integration model above, offers a capability to call an external API, while abstracting the configuration details away from the processing operational models.

Just like any other operational model type, they don't hold data within them, but rather use private class members and constants to share internal data across it's public and private methods. The ApiBroker here as a model represents a simulation of integration with an external system.

We will discuss Brokers extensively in upcoming chapters to shed some light on the rules and guidelines in developing brokers with any external resources or systems.

0.0.2.0.1.1 Processing Models (Services)

Processing models are the holders of all business-specific simulations. Things like student registrations, requesting a new library card or simply retrieving some student information based on a certain criteria. Processing models themselves can be either primitive/foundational, high-order/processing or advanced/orchesterators.

Processing models in general either rely on integration models, or self-relying like computational processing services or rely on each other.

Here's an example of a simple foundational/primtive service:

public partial class StudentService : IStudentService
{
	private readonly IStorageBroker storageBroker;
	...

	public async ValueTask<Student> AddStudentAsync(Student student) =>
		await this.storageBroker.InsertStudentAsync(student);
}

A higher-order service would do/look as follows:

public partial class StudentProcessingService : IStudentProcessingService
{
	private readonly IStudentService studentService;
	...

	public async ValueTask<Student> UpsertStudentAsync(Student student)
	{
		....

		Student maybeStudent = await this.studentService
			.RetrieveStudentByIdAsync(student.Id);
		
		return maybeStudent switch
		{
			null => await this.studentService.AddStudentAsync(student),
			_ => await this.studentService.ModifyStudentAsync(student)
		}
	}
}

A more advanced orchestration-type services would combine from multiple processing or foundational services as follows:

public partial class StudentOrchestrationService : IStudentOrchestrationService
{
	private readonly IStudentProcessingService studentProcessingService;
	private readonly IStudentLibraryCardProcessingService studentLibraryCardProcessingService;
	...

	public async ValueTask<Student> RegisterStudentAsync(Student student)
	{
		....
		Student upsertedStudent = await this.studentProcessingService
			.UpsertStudentAsync(student);

		...

		await this.studentLibraryProcessingService.AddStudentLibraryCardAsync(studentLibraryCard);
	}
}

In general, operational models are only concerned with the nature of simulation or processing of certain data carrier models, they are not concerned with withholding data, neither retaining a status. In general, operational models are stateless in the sense that they don't retain none of the details that went through them other than delegating logging for observability and monitoring purposes.

0.0.2.0.1.2 Exposure Models (Exposers)

Exposure models handle the HMI in all scenarios where a human and a system have to interact. They could be simple RESTful APIs, they could be SDKs or just UI like in web, mobile or desktop applications including commandline-based systems/terminals.

Exposure operational models are just like the integration models, they allow the outside world to interact with your system. They sit on the other end of any system and they are responsible for routing every request, communication or call to the proper operational models. Exposure models never communicate directly with integration models, and they don't have any configuration within them other then their dependencies being injected through their constructors.

Exposure models may have their own language in terms of operations, for instance, an integration model might use a language like InsertStudent while an exposure model for an API endpoint would use a language such as PostStudent for express the very same operation in an exposure context.

Here's an example for exposure models:

public class StudentsController
{
	private readonly IStudentOrchestrationService studentOrchestrationService;

	[HttpGet]
	public async ValueTask<ActionResult<Student>> PostStudentAsync(Student student)
	{
		Student registeredStudent = await this.studentOrchestrationService
			.RegisterStudentAsync(student);

		return Ok(registeredStudent);
	}
}

The above model exposes an API endpoint for RESTful communication to offer a capbility to register students into some schooling system. We will be discussing further the types of exposure models based on the context and the system they are implemented within.

0.0.2.0.2 Configuration Models

The last type of models in any system are the configuration models. They can represent the entry point in any system, or register dependencies for any system or simply play the role of middleware to route URLs into their respective function within an exposure model.

Configuration models usually sit at the beginning of launching a system, or handling incoming and ongoing communications or simply handle underlying system operations like memory caching, thread management and so on.

In a simple API application you may see models that look like this:

public class Startup
{
	public void ConfigureServices(IServices services)
	{
		services.AddTransient<IStorageBroker, StorageBroker>();
		services.AddOAuth();
	}	
}

As you can see the code snippet above, the configuration model Startup offers capabilities to handle dependency injection-based registration of contracts to their concerete implementations. They may handle adding security or setting up a middleware pipeline. Configuration models are technology-specific. They may differ from a Play framework in Scala to a Spring or Flex in Paython or Java. We will outline high-level rules according to The Standard for configuration models but we will not dive deeper into the details of implementing any of it.

0.0.3 Simulation

The simulation aspect of software engineering, is our ability to resemble the interactions to and from the models. For instance, in the grocery store example, a simulation would be the act of selling the item. Selling the item requires multiple modifications to the item in terms of deducting the count of the available items and reordering the items left based on the most valuable available item.

The simulation process can be described as illustrating the relationships between models. they are programmed as functions, methods or routines and these all mean the same thing. If we have a software service that is responsible for items sales, a simulation process would look like this:

public class SaleService
{
	public void Sell(Item item) => Items.Remove(item);
}

In the example above, we have a model called SaleService that offers a functionality to simulate the sales process in the real world on a model of an item. And this is how you describe everything in object-oriented programming. Everything is an object (from a model) and these objects interact with each other (simulation).

Objects interacting in general can be observed in three different types. A model taking an action on another model. For instance, the SaleService is executing an action of Sell on an Item model. That's a model interacting with another model. In the very same example, a simulation could be something happening to the model from another model such as the Item in the aforementioned example. And the last type of simulation is a model interacting with itself. Models that self-dispose once their purpose is fulfilled and they are no longer needed for example. They will self-destruct.

The simulation process is the third and last aspect of software engineering. Which we will dive deeply in when we talk about brokers, services and exposers to illustrate how the modeling and simulation process happens in the industrial software.

0.0.4 Summary

If we considered purposing to be the domain or the framework in which models interact. Then the following illustration should simplify and convey the picture a bit clearer:

It's important to understand that a computer software can serve multiple purposes. Computer software can interact with other software that share common purposes. The purpose of the software becomes the model and the integrations becomes the simulations in that aspect. Here's a 10,000 feet example:

The complexity of any large system can be very easily broken down into smaller problems if the single-purpose or single-responsibility aspect was enforced for each and every sub-system. That's what modern software achitectures would call granularity and modularization. Which we will be discussing briefly throughout the architecture aspect of The Standard.

[*] Purposing, Modeling & Simulation (Part 1)