Rough Design Up Front
by John R. Harris
Emergent Design
Lean Agile adherents advocate emergent design over design up front. When employing a lean approach, features are built incrementally, one iteration at a time, and continually re-factored, based on feedback, until a design emerges. This evolutionary approach makes sense for situations where the solution is poorly understood and there are few, if any, experts to guide the team. But this is seldom the case outside startups or research departments.
Most businesses understand, at least partially, what their customers need, how soon they need it, and what the business can afford. Most businesses have an existing customer base, product, technology stack, and workforce. In short they have a business and technology context that the new solution must complement and extend. There are few green fields left.
Rough Design up Front
When armed with well defined goals and objectives, and a solid understanding of the existing business and technology context, experienced lead software engineers and product managers can develop successful strategies and designs for an initiative. Incrementally evolving a design is perverse, if a rough design, that is fit for purpose, can be rapidly defined upfront. Skilled professionals are more than capable of anticipating problems and identifying exploitable opportunities in advance.
Significant parts of a solution can, and should, be designed up front. Not every project is so complex and uncertain that it must be evolved from scratch - In fact most are not. Experienced Product Managers and Technical Leads can usually create a rough design for up to 80% of a solution in the first few iterations of a project.
Example Design Artifacts
Discussing the merits of Rough Design Up Front is difficult without showing some actual design artifacts. This article presents some example artifacts from projects we have completed at SPAN. These have been deliberately obfuscated and redacted, for obvious reasons, but hopefully still illustrate the type of artifact and the amount of detail necessary for a rough design.
System Architecture Diagrams
Based loosely on the C4 Architecture Model we typically create a combined context, container, and component diagram. We seldom include any classes on these diagrams. This is a single diagram that shows the system under construction; the external systems it interacts with, the containers it is made up of (web applications, mobile apps, desktop applications, databases, file systems, etc), and the components that comprise each container. The diagram is usually arranged into layers and functional areas. Its purpose is to define the key abstractions and concepts, major functions, dependencies, data flows, implementation technologies (where known) and potential construction sequence. Other factors can be shown on the diagram as needed.
Architecture Diagram
A typical C4 based architecture diagram color coded by implementation technology showing many text labels linked to key Jira tickets
Architecture Diagram
The evolution of a data model over several days. From a hand drawn diagram on a white board created during a design session to a fairly detailed model with working notes in google drive
Architecture Diagram
A group of interoperating systems and the data items (green) they manage.
Architecture Diagram
A simple components diagram with two important scenarios that describe the basic operation of the system.
Entity Relationship Diagrams
The data entities within the system are represented by a logical entity relationship diagram. It is important to stress, this is a data model NOT a database design. Regardless of how the data will eventually be stored a logical ERD identifies the key entities and their relationships. This helps the team define terminology, start to group together tightly coupled entities into resources that can be managed collectively, and begin to understand the life-cycle of the resources that the system will manage.
Entity Relationship Diagram
A typical ERD diagram with it’s ongoing Google docs conversation.
State Transition Diagrams
The behavior of key resources, entities, and objects within the system is often represented via one or more state transition diagrams. These diagrams show the allowable transitions between resource and entity states, the component or person that can initiate each transition, and the interdependencies between state transitions of different resources or entities. State diagrams help the team understand the life-cycle of resources and are often labeled with estimates of event frequency and geographic distribution.
State Transition Diagram
A typical state transition diagram showing the transitions for all the major objects in a systems and how these transitions affect each other.
State Transition Diagram
Account state transitions in a subscription payment system based on credit card processing status labeled with the three outstanding issues that need to be resolved.
Rough Design in Practice
Most of the artifacts we produce are Diagrams. These are created in GSuite because it supports contemporaneous multi-user collaboration. These documents are frequently used in team meetings and are under continuous revision. Diagrams are often color coded and annotated and usually act as host to discussions about various design issues. Over time objects and labels on each diagram are converted into links to Jira tickets, Github repositories, and other important resources. These design artifacts are few in number, very high in information content, and create a common baseline from which engineers can evolve the solution. Attempting to design everything up front assumes perfect foresight on the part of the team and little value to be gained from learning as we proceed. This is obviously the epitome of hubris - we can’t anticipate everything and should be eager to learn as we go along. But equally, designing nothing, or too little, up front ignores the benefits to be gained from a rapidly produced rough design. Such a design saves time, reduces cost and risk, and leads to a better end result.