Select Final Design

Abstract

In this activity, you will learn how to use a decision matrix to evaluate design options. It explains the steps of identifying criteria, ranking their importance, assigning weightings, and scoring design ideas. Examples are provided to illustrate the process, followed by a task where you can create your own decision matrix. Potential drawbacks of the method are listed.

The learning outcomes for this activity are that by the end of this activity you will be able to:

• Perform a structured design selection process
• Determine, rank, and weight design criteria
• Recognize the responsibilities, benefits, and importance of considering sustainability, diversity, inclusion, cultural, and societal factors
• Understand and apply a decision matrix method to select a final conceptual design

Before you begin this activity, you will need to have:

• A previously generated range of conceptual design ideas
• A way to evaluate these ideas

Estimated time to complete: 30 minutes

Introduction

Once you have generated a range of design ideas, the next step in the process is to select the best one to develop further. Making this decision can be hard. There may be many factors to consider when selecting the 'best' design, and different members of the team may disagree with one another. Instead of relying on opinions, it is better to use a systematic approach to evaluate the design ideas. This will help you objectively justify the final design decision and move forward in the process.

Using a decision matrix

One method to use is the decision matrix method, also known as a selection matrix or Pugh matrix (named after Scottish scientist Dr. Stuart Pugh). In this approach, different design ideas are scored against predetermined criteria to determine the overall best design. The process for creating a decision matrix is as follows:

Each of these steps will be discussed in detail below.

Determine and agree on a list of "design criteria"

Design criteria are the factors that you will use to evaluate your design ideas. These can be technical, economic, social, or environmental factors. They can also be related to the user experience, aesthetics, or manufacturability of the design. Examples of design criteria are:

• Low cost
• Easily scalable
• Pretty
• Efficient
• Easy to maintain
• Non-toxic
• Low emissions

Investing time to capture appropriate and well-defined criteria will help ensure that the evaluation process is fair and consistent. All chosen design criteria should be:

Written positively

The criteria should be written in a way that ensures they perform well if the design meets them. For example, the criterion of "cost" is ambiguous. Should the design aim to be low or high cost? Instead, you could specify "low cost" or "affordable" if this is advantageous to the design.

Not overlapping

In the next stage, weighting will be applied to the criteria. If two criteria overlap, there is a danger their weighting will be accidentally increased due to double counting. For example, the criteria for a car could include "fuel-efficient" and "low running costs." Low fuel costs may contribute to low running costs, and so this would be double counted. An alternative could be to use "fuel-efficient" and "low maintenance costs." Similarly, "long lifespan" and "durable material" could overlap and be replaced with "long lifespan" and "not breakable."

Related to user requirements and the context in which the design will be used

Chosen criteria should be based on attributes that users of the design will value. Consideration should be given to the context in which the design will operate and any specific needs of the users. For example, a design requiring access to a water supply may only be convenient in certain countries, and a design requiring written documentation to operate may not be suitable for people who are unable to read.

Agreed upon

Before moving to the next step in using a decision matrix, all the criteria should be agreed upon by the team.

Reflect on the chosen criteria to ensure sustainability and inclusivity are considered

Order the criteria from most to least important

Once you have agreed on your design criteria, the next task is to assign each with a weighting. This can be challenging, and there can be competing opinions within the design team. One approach to take as a first step is to rank the criteria from most to least important, before deciding the weightings. This can help to ensure consistency.

For example, if you have chosen four criteria as "sustainable," "smooth," "easy to use," and "cheap," you may decide the order from most to least important is:

1. Cheap (most important)
2. Sustainable
3. Smooth
4. Easy to use (least important)

Assign each criterion importance weightings relative to one another

Once you have ranked the criteria, you can then assign each with weightings. This should be done by giving each criterion a score depending on how important the design team agrees it to be. The greater the importance, the higher the score. The process of ranking helps to bound the possible score, as the value will be between that of the more important and less important criteria.

Each criterion is given a weighting relative to the others. The absolute value of the weightings is not important, provided the relative weightings are appropriate for the context and users of the design. For example, you may decide that "cheap" is twice as important as "sustainable," so you would give "cheap" a weighting of double the value of "sustainable."

Strategies you could adopt to assign the weightings include:

Start with the most important

You may decide the most important criterion is worth 10 or 100 and consider the other criteria relative to this.

Use a number line

Draw a number line from zero to 10 or zero to 100 and place each criterion somewhere on the line (with the most important near the high end). The order should be the same as the initial ranking.

Distribute a total

Select a total number of points, for example 100, and distribute a proportion of the total to each criterion.

Score each design idea against each criterion

The next task is to score each conceptual design idea against each of the criteria. If, for example, you have 3 conceptual design ideas and 4 criteria, you will need to give 4x3=12 scores.

Each score should be based on how well the design idea meets the criteria. For example, if the criterion is "cheap" and "design idea 1" is very cheap, you may give it a score of 10. If "design idea 2" is very expensive, you may give it a score of 1. If "design idea 3" is neither cheap nor expensive, you may give it a score of 5.

The maximum score for each criterion for each design can be any number, for example, 5, 10, or 100, but it must be the same maximum possible for every score given.

Multiply each score by the criteria weighting

When all the scores have been given, the next step is to calculate the weighted score for each criterion for each design idea. This is done by multiplying each score by the weighting for the criterion. For example, if the criterion is "cheap" and the weighting is 10, and the cheapness score for "design idea 1" is 7, the score for "design idea 1" for the criterion "cheap" is 10x7=70.

Sum the weighted scores for each idea

The final task is to sum all the weighted scores for each design idea. The design with the highest total is systematically selected as the best concept to take forward to more detailed planning.

Example of a decision matrix

In this example, a decision matrix has been used to decide what drink to have during a long meeting with a project design team. After some brainstorming, you decide on four options: coffee, juice, pop (soda for non-UK people), or water. The most important design criterion for the drink is that it provides stimulation to get through the meeting. Further criteria of taste, health, convenience, and low cost have been selected and ranked in that order. The weightings for each criterion have been decided and are shown below.

Each drink has been scored against each criterion, as shown below. When considering how to score taste, it was important to take into account the views of stakeholders. Some people may prefer coffee to juice, while others may prefer pop to coffee. It was also necessary to consider the context of the problem when scoring the convenience criterion. Certain drinks may be easier to obtain depending on whether there is convenient access to a café, vending machine, or tap (faucet).

Each score has been multiplied by the weightings for the criterion, shown on the right-hand side of the decision matrix below. For example, the health score for juice is "7" (out of 10), and the weight of this criterion is "6" (out of 10), so the weighted score is 7x6=42.

The total weighted score for each drink is calculated by summing each weighted criterion score, shown at the bottom of the table below. The best drink to meet the design criteria is coffee, with a score of 222, while the worst drink to meet the design criteria is water, with a score of 136.

This illustrates the importance of selecting appropriate weightings. While it is not the tastiest, healthiest, most convenient, or cheapest drink, the winner is coffee. The high weighting for stimulation, combined with the high score on this criterion for coffee, makes a significant contribution to the overall score.

Try creating a decision matrix

Now that you have learned about the decision matrix method, try practicing by following the steps described above to create your own decision matrix and select a final design. To do this, use the template below. You will need to think of three design ideas to overcome a problem and five criteria to evaluate them against. You can either think of your own problem to solve or choose one of the following:

• Transport options to get from Sheffield to London
• Solutions to help amputees play a musical instrument
• Choosing the best engineering discipline

Drawbacks of a decision matrix

There are drawbacks to using the decision matrix method. These include:

• "Must-have" criteria are not included. It is possible for a design to be fundamentally unfeasible but still score highly enough in other categories to be selected.
• There is no way to ensure that all design criteria have been considered.
• Weightings and scoring are usually based on the opinions of the design team (rather than measurable data) and could be subject to biases.