Signal from the Noise

The Art of Choosing Concepts That Win

Concept: Generating many concepts is crucial, but selecting the best ones is particularly important. This post explores structured methodologies for evaluating and selecting product concepts, including screening and scoring matrices and the assessment of technology readiness levels (TRL).

Generating a variety of alternative concepts has a higher chance of success in the ideation process. Once potential solutions or partial concepts for sub-functions are identified, they need to be integrated. Combining these partial solutions systematically, using for example combination "trees" or "tables," can generate numerous alternative concepts. However, not all concepts resulting from these combinations are feasible, either technically or economically. Therefore, the design team must make choices and decisions that will impact the product's life cycle cost.

Evaluating concepts at an early stage, when they are still abstract with little detail, can be challenging. Nevertheless, the process of evaluating competing concepts involves comparison and informed decision-making. This evaluation process helps to:

• Improve the understanding of requirements.

• Increase understanding of the problem and potential solutions.

• Uncover key interactions, potentially leading to new solutions.

• Identify the strengths and weaknesses of the concept.

There are several types of concept comparison techniques. These include absolute judgment, Technology Readiness Level (TRL), filtering (pass/fail), relative comparison using a decision matrix, and absolute or relative comparison using metrics.

Qualitative assessment, based on the team's experience and tacit knowledge (intuition/instinct or feeling), can be used to analyze concept feasibility. Even concepts initially seeming unfeasible might be innovative or trigger new ideas, especially if the team can gather additional information through experimentation or other means for more informed decisions.

For certain products, especially those incorporating new technologies, assessing the Technology Readiness Level (TRL) is necessary. Technology Readiness Assessment (TRA) is a systematic, evidence-based process to estimate the maturity of critical hardware and software technologies that could affect system performance or project objectives. A validated technology with high maturity is generally preferred to reduce future risks. TRL uses a scale from 1 to 9 to indicate maturity, but not necessarily quality when integrated into a product. Assessing readiness involves reviewing the technology's status based on documentary evidence to ensure it works as intended before a final decision on incorporation. This helps identify risks early, avoiding high costs later.

The application of criteria based on concept and technology maturity allows for a pass/fail filtering process. Aspects considered include identifying critical factors for overall function, knowing operational conditions for high fidelity tests, identifying main failure modes (e.g., through FMEA), possibility of production with known processes, and control of the technology throughout the life cycle.

Structured methods for concept assessment can be very useful. Screening matrices, proposed by Pugh (1991), offer a controlled convergence procedure for agile and effective concept selection. A matrix is used to compare concepts against a reference concept based on predefined criteria derived from customer requirements. Scoring matrices offer a quantitative approach, assigning scores to criteria and concepts, with weighted criteria possible. While simple and easy to apply, other decision-aid tools like Analytical Hierarchy Process (AHP) or utility functions exist for more complex problems with many criteria and alternatives.

Using a structured selection process has advantages, though it can be time-consuming. Problems that can invalidate these advantages include fixating on the first idea, ignoring customer requirements, allowing a single person (even an expert) to decide for the team, relying on "gut instinct," or ignoring cost. Despite potential difficulties, continuous use of structured methods is verified to be efficient in resource management and effective in results. It imposes discipline, stimulates creativity by allowing iterations and combinations, and helps overcome potential impatience from experienced team members by demonstrating inherent advantages.

For greater details study Chapter 3 of my book for exploration of these tools, complete with practical examples.