In this blog, we preview a section of our recent customer story, “Ultra Maritime UK Enlists Jama Connect for Naval Systems Requirements Management” – Click HERE to read it in its entirety.

Ultra Maritime UK Enlists Jama Connect for Naval Systems Requirements Management

UK operation chooses Jama Connect for its ease-of-use and administration.

About Ultra Maritime UK

• Founded in 1944 and acquired in 2021 by Advent International
• Over 2,300 employees across fifteen locations worldwide
• Premier provider of undersea warfare systems, products, and solutions to US, UK, Canada, Australia, and allied navies worldwide.

CUSTOMER STORY OVERVIEW

This customer story is about Ultra Maritime UK, a division of Ultra Maritime, which develops equipment for surface, subsurface, and unmanned platforms, including acoustic and sonar systems and torpedo defense and radar sensor solutions. Customers include the Royal Navy of the UK, the U.S. Navy, the Royal Canadian Navy, the Royal Australian Navy, and the Indian Navy.

Since the Ultra Group’s acquisition by U.S. private equity firm Advent International in 2021, Ultra Maritime has operated as an integrated company with lines of business headquartered in the USA, UK, Canada, and Australia, responsible for developing both worldwide and localized solutions.

Ultra Maritime UK’s products and solutions must adhere to the highest quality, security, and safety standard,s including ISO 9001-2015, ISO 14001:2015, ISO/IEC 27001:2013, and ISO 45001:2018. In addition, the products must satisfy customer requirements and regional naval standards, including U.S. DoD/MIL-STD and UK MOD DStan.

Challenges

• Enabling collaborative remote working on requirements
• Managing both project and product requirements effectively
• Becoming more product-focused to balance global and local requirements

Evaluation Criteria

• Intuitive user interface for quick adoption
• Low administrative overhead
• Support for reuse of requirements and test evidence

Outcomes

• Consistency across projects
• Business mandatory requirements tool for all new projects
• Quick user adoption with minimal training
• Easy tracking of progress with dashboards and standardization
• Reduced risk with pre-built standardized project structures
• Easy management of large numbers of objects and changes
• Easy initiation and completion of reviews to action items sooner in development

RELATED: IBM Customers Choose Jama Connect the Most

CHALLENGES

• After years of managing project requirements with IBM^® DOORS^® Classic, these challenges drove Ultra Maritime UK to find a user-friendly collaborative tool that would enable them to increase their requirements management effectiveness and deliver projects faster.
• Enabling collaborative remote working on requirements
• Managing both project and product requirements effectively
• Becoming more product-focused to balance global and local requirements

EVALUATION CRITERIA

Ultra Maritime UK identified several requirements management tools as potential replacements for DOORS Classic, including Jama Connect, which a member of the engineering team suggested. They then established criteria to be used for the evaluation.

• Intuitive user interface for quick adoption
• Low administrative overhead
• Integrable with development and test software tools
• Support for reuse of requirements and test evidence

First, a top priority was for the new solution to have a modern, intuitive user experience for teams to get up and running quickly with their new projects. They needed software that people would want to work in. Otherwise, people might opt out of using the tool and work in disparate documents, which would introduce risk, impede productivity, and hamper efficiency. Second, it would need to have low administrative overhead that did not require team members to become full-time administrators. Third, it would need to be integrable with development and test software tools from different vendors. Fourth, it would need to support the reuse of requirements and test evidence from past development programs when starting new products or projects.

During the evaluation process, Jama Connect stood out from the competition as the solution that would best meet the company’s needs. “Looking at all the features, the user’s ease of use, and the low level of administrative time required, Jama Connect came out on top compared to the other tools reviewed,” says the Senior Systems Engineer.

In its search for a modern solution that would be quickly adopted, Ultra Maritime UK found that Jama Connect’s intuitive user experience made adoption extremely easy for engineers to get started managing requirements and test evidence more efficiently and intelligently. Tracking and finding information quickly and easily was achievable with Jama Connect’s powerful filtering and the ability to add hyperlinks to any architectural elements, requirements, test items or other objects. In addition, the ability to create a Definitions database and Glossary in Jama Connect was particularly useful for getting everyone informed and up-to-speed about projects. “Jama Connect has a highly intuitive user interface and allows for engineers to quickly and easily become accustomed to using it,” says the Senior Systems Engineer.

Having a simple and quick way for systems engineers to initiate reviews and for stakeholders to complete their reviews in a timely manner was a key area where Jama Connect’s Review Center led the way. The fact that reviewers were not required to be licensed as full-time users made Jama Connect more attractive. “Jama Connect certainly makes it much easier to initiate and manage reviews and be aware of progress through them,” says the System Design Authority and Functional Lead.

“Jama Connect proved to be extremely useful for making sure that we’ve got complete coverage and traceability of a given set of artifacts to see which ones have or haven’t reached the approved step. This helps ensure we haven’t missed any anomalies such as system requirements missing verification cases to avoid rework,” says the System Design Authority and Functional Lead.

Support for multiple IDs for the same object in Jama Connect made it easy to identify opportunities for reuse of older product requirements and test evidence for new products to efficiently manage shared elements of core and variant products. “The ability to identify, distinguish, and reuse global requirements across products and projects to reduce development time and cost is a strength of Jama Connect,” says the System Design Authority and Functional Lead.

In addition to product capabilities, team expertise, and training resources provided during the evaluation demonstrated that Jama Software would be a good fit as a partner for Ultra Maritime UK. “We were impressed by Jama Software’s responsiveness to our questions and the online training, forums, and support available to our team,” says the System Design Authority and Functional Lead.

RELATED: Jama Connect for Air, Land, Sea, and Space Datasheet

OUTCOMES

With Jama Connect, Ultra Maritime UK has benefited from:

• Greater consistency across projects
• One mandatory requirements management tool for all new projects
• Quicker user adoption with minimal training
• Easier tracking of progress with dashboards and standardization
• Reduced risk with pre-built standardized project structures
• Easier management of large numbers of objects and changes
• Easier initiation and completion of reviews

TO READ THE ENTIRE CUSTOMER STORY, VISIT:
Ultra Maritime UK Enlists Jama Connect for Naval Systems Requirements Management

Community Event Week 2025: Exclusive Roundtables for Jama Connect^® Users

At Jama Software, we know the power of community. That’s why we’re thrilled to invite you to our Community Event Week, happening from May 12 to May 16! This exclusive event is just for our valued Jama Connect users, offering attendee-led sessions packed with insights, opportunities to grow, and meaningful connections with other professionals in your industry.

Each day focuses on a specific vertical, ensuring that the topics, challenges, and triumphs discussed are truly relevant to you. Whether you’re tackling compliance hurdles, managing intricate development cycles, or looking to exchange ideas with peers, our Community Event Week is your chance to level up your expertise and meet like-minded professionals.

Why Attend Community Event Week?

This isn’t your typical webinar or conference. Community Event Week is all about fostering connections, sharing real-world solutions, and growing together. Here’s what you can expect:

• Relevant Sessions: Each day is tailored to a specific industry vertical, so the topics resonate with your professional challenges and goals.
• Expert Guidance: Gain insights from Jama Software solutions leads who’ll moderate discussions and help answer your burning questions.
• Peer Connection: Build relationships with other professionals in your field who use Jama Connect.
• Practical Takeaways: Leave your session armed with actionable insights and strategies to tackle your day-to-day challenges.

Community Connect Schedule

May 12 – Community Connect for Automotive & Semiconductor

Kick off the week with professionals from the automotive and semiconductor industries. This session is all about addressing the intricacies of these fast-evolving fields. Share your experiences, discuss challenges, and get answers to your questions, all while gaining expert insights from fellow attendees and our Jama Software Solutions Lead.

• Who it’s for: Automotive and semiconductor professionals
• Why attend: Network with peers, talk about the complexities of the industry, and exchange real-world solutions

Learn More and Register HERE 

RELATED: Buyer’s Guide: Selecting a Requirements Management and Traceability Solution for Automotive

May 13 – Community Connect for Medical Devices & Life Sciences

Dive into the complex world of medical devices and life sciences. Whether you’re managing compliance challenges, curious about effective regulatory strategies, or simply looking for innovative perspectives, this session offers something for everyone.

• Who it’s for: Medical device industry professionals
• Why attend: Take part in attendee-led discussions with peers, exploring strategies to deliver safe, effective products in a fast-paced and evolving market.

Learn More and Register HERE 

RELATED: Jama Connect® for Medical Device & Life Sciences Development Datasheet

May 14 – Community Connect for Oil, Gas, Architecture, Engineering, and Software

Join a diverse group of professionals from industries like oil, gas, engineering, construction, and software. This open discussion will tackle both common and industry-specific challenges, sparking ideas and solutions to help you move forward.

• Who it’s for: Professionals in software, oil & gas, engineering, and construction
• Why attend: Tap into collective expertise across these important and complex industries

Learn More and Register HERE 

RELATED: Buyer’s Guide: Selecting a Requirements Management and Traceability Solution for Aerospace

May 15 – Community Connect for Aerospace & Defense

Aerospace and defense teams know the stakes are high. From managing complex requirements to complying with strict regulations and timelines, this industry requires precision. This session offers a space for collaboration and innovative problem-solving with fellow experts.

• Who it’s for: Aerospace and defense professionals
• Why attend: Swap ideas and discover new strategies for mission-critical projects

Learn More and Register HERE 

RELATED: Buyer’s Guide: Selecting a Requirements Management and Traceability Solution for Aerospace

May 16 – Community Connect for Industrial, Consumer Electronics, & Energy

Wrap up the week with an opportunity for cross-industry connection. Professionals in industrial, energy, and consumer electronics will share their experiences, lessons learned, and fresh ideas. Expand your network while learning from peers in diverse yet complementary fields.

• Who it’s for: Industrial, electronics, and energy professionals
• Why attend: Engage in attendee-led sessions that foster peer collaboration on creating reliable, high-quality products while navigating the complexities of a dynamic marketplace.

Learn More and Register HERE 

RELATED: Functional Safety in Industrial Manufacturing: Navigating IEC 61508, ISO 13849, IEC 10218 for Safer, Smarter Operations

Exclusive to Jama Connect Users

These roundtables are designed for current Jama Connect customers to provide the most relevant and enriching experience possible. If you’re a current user, you’re in for some invaluable sessions that could transform the way you work.

However, if you’re not yet a Jama Software client, but are exploring how we can help your organization streamline complex development, we’ve got plenty of resources to get you started:

• Download our Buyer’s Guide to Selecting a Requirements Management and Traceability Solution eBook
• Explore our Essential Guide to Requirements Management
• Read Our Customer Success Stories
• Visit our Resources and Events page

And if you’d like to speak directly with our team about your specific challenges, we’d love to set up a personalized consultation. Contact us here.

Jama Software Provides a Single, All-in-One Solution for Requirements, Risk Management, and Validation

Organizing requirements, managing risk, and ensuring validation are complex processes on their own. For teams in the medical device industry, these tasks are even more challenging due to stringent regulatory standards and the critical nature of the products themselves. Jama Connect^®, a robust software tool designed specifically for such pressures, offers a seamless, all-in-one solution.

What to Look for In a Requirements Management Tool

At the 2024 INCOSE Healthcare Conference, one of the presenters delivered an insightful presentation highlighting the essential features their organization identified as necessary in a requirements management solution to effectively support complex risk management procedures.

This blog post provides a detailed breakdown of how Jama Connect is specifically designed to address each of these critical areas discussed during the presentation.

End-to-End Traceability for Risk Analysis and Validation

Why It’s Important: Without clear traceability, unverified requirements or overlooked risks can result in delays, missed compliance, or worse, product failure.

How Jama Connect Helps:

• Jama Connect provides traceability across all items in the development process, linking risk analysis elements to requirements, mitigations, and verification items.
• Suspect triggers and impact analysis. Any upstream change triggers “suspect links,” highlighting potential downstream impacts, so no critical changes are overlooked. Impact analysis features allow for a proactive approach to understanding the scope of a change BEFORE it occurs.
• Configurable views and export templates ensure efficient reporting to compliance agencies, showing validation evidence clearly and concisely.

Example: Teams can easily export views with verification columns from Jama Connect into Word or Excel for seamless external reporting.

Integrated FMEA Capabilities

Why It’s Important: Proper Failure Mode and Effects Analysis (FMEA) evaluates risks effectively and shows compliance with ISO 14971 for medical devices.

How Jama Connect Helps:

• Full support for FMEA tracking, including automated RPN (Risk Priority Number) calculations and risk matrices to simplify decision-making.
• Pre-configured medical device frameworks allow quicker adoption while still being customizable to match specific workflows.
• Tools like exportable templates make it easy to share mitigation strategies with stakeholders.

For complex risk calculations, teams can leverage Jama Connect Interchange™, ensuring that even intricate needs are met reliably.

• Additional Resource: Watch this video to see how Jama Connect facilitates FMEA.
• Additional Resource: Read about FMEA in our Requirements Guide

RELATED: Accelerate the Development of Safe and Effective Medical Devices & Life Science Products with Jama Connect

Standard Frameworks for Risk Management

Why It’s Important: Medical device standards aren’t optional; they’re essential for safety and compliance.

How Jama Connect Helps:

• Enables compliance with key standards like ISO 14971, integrating these directly into risk frameworks along with thorough hazard documentation.
• With pre-configured templates, teams can efficiently manage risk assessments without manually recreating workflows.

For development teams, having standards built into the process reduces human error and increases efficiency.

Version Control and Change Management

Why It’s Important: Medical devices often undergo project iterations where changes can impact earlier decisions. Keeping track of these is vital.

How Jama Connect Helps:

• Change management item types can trace changes back to risk assessments, ensuring traceability.
• Built-in features allow versioned tracking for every modification, maintaining compliance and detailed documentation.
• Connections between mitigations, risks, and validations ensure transparency in every decision made.

Change management is a core feature that proves invaluable when audits require detailed project histories.

Usability and Configuration

Why It’s Important: Even the most advanced tool is counterproductive if it’s hard to use or implement.

How Jama Connect Helps:

• Jama Connect is celebrated on the G2 Grid as the highest-ranked requirements management tool, with accolades for its ease of use.
• Supports seamless data imports from applications like Microsoft Excel or Word, using reusable import wizards for faster and repeatable imports.
• Pre-configured framework for Medical Device & Life Sciences, reduces time-to-value, allowing teams to get up and running quickly.
• SOC2 Compliance, combined with the robust capabilities of a Validated Cloud and Validation Kit, provides systems engineers with effective solutions to meet stringent security and regulatory compliance requirements.

Additionally, the focus on user experience allows for a fast time to value across project teams.

Flexible Configurations for Different Workflows

Why It’s Important: Medical device companies don’t all operate under the same workflow, and tools must accommodate that diversity.

How Jama Connect Helps:

• Pre-configured frameworks for compliance streamline setup time while allowing for configuration to suit unique organizational processes.
• Provides users with the Traceability Information Model, visualizing traceability chains to ensure nothing falls through the cracks.
• From aligning with new SOPs to creating testing and validation workflows, customization ensures this tool adapts as projects evolve.

Scalable Solution for Large Projects

Why It’s Important: Medical device development often spans multiple teams and product lines, increasing complexity.

How Jama Connect Helps:

• Provides for the ability to break large projects down into smaller, manageable subcomponents while maintaining traceability across product lines.
• Supports cross-project traceability and reusability, which is ideal for future scaling.
• This modular approach makes it easy for businesses to simplify operations without losing sight of critical compliance elements.

RELATED: Align Safety Hazards, Security Threat Analysis, Risk Assessments, and Functional Safety Directly into Your Engineering Workstream

Tool Configuration for Effective Risk Management

Risk management and requirements traceability go hand-in-hand, forming a critical component of any successful project. A well-configured tool should seamlessly align with your existing processes while offering the flexibility to adapt to your specific needs.

To illustrate, here are three examples of how Jama Connect’s Traceability Information Model can be configured to enhance and support a comprehensive risk management process. These configurations are designed to streamline workflows, improve traceability, and ensure better alignment with your risk management objectives.

Example 1 (from Jama Connect’s Medical Device Framework)

Example 2 (from Jama Connect’s Medical Device Framework)

Example 3: Customized Workflow 

Conclusion:

Jama Connect is a purpose-built solution designed to meet the complex and stringent requirements of the medical device industry. Offering robust capabilities like end-to-end traceability — from risk analysis through to verification — it ensures seamless oversight across the entire development lifecycle. With capabilities, such as FMEA and alignment with critical regulatory standards for the medical device industry, Jama Connect simplifies compliance and elevates requirements management to the next level.

With award winning usability, Jama Connect features an intuitive interface, customizable configurations, and powerful change management capabilities. These features enable teams to work efficiently while staying aligned with regulatory needs. By fostering real-time collaboration and bridging the gaps between stakeholders, Jama Connect empowers organizations to accelerate product development without compromising on safety or quality.

Jama Software is always looking for news that would benefit and inform our industry partners. As such, we’ve curated a series of customer and industry spotlight articles that we found insightful. In this blog post, we share an article from Synopsys^®, titled “Synopsys Bold Prediction: 50% of New HPC Chip Designs Will Be Multi-Die in 2025”, and written by Michael Posner and Shekhar Kapoor and published on January 21, 2025.

Synopsys Bold Prediction: 50% of New HPC Chip Designs Will Be Multi-Die in 2025

Monolithic chips have been the workhorses behind decades of technological advancement. But just as the industrial revolution saw workhorses replaced with more efficient and powerful machinery, the semiconductor industry is on the cusp of a similar revolution.

Multi-die and chiplet-based designs — which integrate multiple specialized dies in a single package or stack integrated circuits vertically — stand to deliver far greater performance and flexibility than monolithic chips, capable of supporting the insatiable processing demands of high-performance computing (HPC) and AI-driven workloads. But pursuing these advanced chip designs has required the deepest of pockets and most advanced R&D capabilities.

Until now.

Multi-die technologies, tools, flows, and IP have matured rapidly. Engineering expertise has evolved. And foundry capacity continues to expand. With this in mind, we predict 50% of new HPC chip designs will be 2.5D or 3D multi-die in 2025.

RELATED: The Benefits of Jama Connect®: Supercharge Your Systems Development and Engineering Process

Foundries preparing for wave of 2.5D and 3D multi-die designs

It takes more than R&D to bring 2.5D and 3D multi-die designs to market. It also requires high-bandwidth, low-latency interconnects (3DIO), advanced manufacturing processes with sufficient capacity, and sophisticated design tools and IP.

Open industry standards like UCIe (Universal Chiplet Interconnect Express) continue to mature, helping simplify and strengthen the connectivity between heterogeneous chiplets while reducing risk and accelerating design cycles. The increased adoption of UCIe for HPC, AI, data center, and edge applications is driving significant demand for 2.5D and 3D multi-die designs.

In addition to the maturation and proliferation of advanced interconnects, foundries are preparing for the oncoming wave of 2.5D and 3D multi-die designs. This includes new manufacturing processes that offer denser bumps and higher performance. Additional package, interposer, and integration options provide cost and architectural flexibility. And expanded production capacity means more designs and prototypes can be brought to market.

RELATED: How to Manage Cybersecurity in Jama Connect^® for Automotive and Semiconductor Industries

Advanced multi-die design tools and IP

Developing these cutting-edge chips would not be possible without the most advanced design solutions, and Synopsys remains at the forefront of 2.5D and 3D multi-die innovation. Our comprehensive and scalable multi-die solutions, including design automation tools and IP, enable:

• Early architecture exploration
• Rapid software development and system validation
• Efficient die/package co-design
• Robust die-to-die and chip-to-chip connectivity
• Improved manufacturing and reliability

In particular, our 3DIC Compiler is the industry’s only unified exploration-to-signoff platform for 2.5D and 3D multi-die designs. Certified by all major foundries, it supports feasibility exploration, multi-die partitioning, and foundry technology selection for prototyping and floorplanning. This enables analysis-driven design implementation (including advanced packaging and die-to-die routing) with golden signoff verification.

3DIC Compiler also integrates with 3DSO.ai, our AI-driven system analysis and optimization solution. The integrated solutions help maximize system performance and quality of results for thermal integrity, signal integrity, and power network design.

We also offer the highest performance, lowest latency, lowest power, and smallest area die-to-die IP solutions, including UCIe and proprietary controllers, physical layer devices (PHYs), and verification IP. The UCIe-based IP is compliant with the latest UCIe specification, and the proprietary die-to-die IP delivers 40Gbps performance, maximum die-edge and power efficiency, low latency, and support for standard and advanced packaging technologies.

Our 2.5D and 3D multi-die solutions have helped achieve several silicon successes across multiple foundry processes. Customer adoption and foundry capacity continue to increase. And 3DIO standards continue to mature.

For these reasons and more, we believe at least half of new HPC chip designs will be 2.5D or 3D multi-die in 2025.

In this blog, we’ll recap a section of our recent Expert Perspectives video, “A Method to Asses Benefit-Risk More Objectively for Healthcare Applications” – Click HERE to watch it in it entirety.

Expert Perspectives: A Method to Assess Benefit-Risk More Objectively for Healthcare Applications

Welcome to our Expert Perspectives Series, where we showcase insights from leading experts in complex product, systems, and software development. Covering industries from medical devices to aerospace and defense, we feature thought leaders who are shaping the future of their fields.

In the complex world of healthcare, evaluating benefit-risk is crucial to successful product development and patient outcomes. Our expert perspectives video, “A Method to Assess Benefit-Risk More Objectively for Healthcare Applications,” offers actionable insights for healthcare innovators aiming to meet rigorous regulatory requirements while ensuring patient safety and efficacy.

In this episode of Expert Perspectives, Richard Matt breaks down a streamlined, objective method for benefit-risk analysis. He explores a structured frameworks and data-driven approach that help teams make balanced decisions, mitigate risks early, and stay compliant with regulatory standards, including FDA and ISO guidelines.

This patent-pending approach helps organizations navigate challenges, foster innovation, and ultimately bring safer, more effective healthcare solutions to market.

Below is a preview of our interview. Click HERE to watch it in its entirety.

Kenzie Jonsson: Welcome to our expert perspective series where we showcase insights from leading experts in complex product, systems, and software development. Covering industries from medical devices to aerospace and defense, we feature thought leaders who are shaping the future in their fields. I’m Kenzie, your host, and today, I’m excited to welcome Richard Matt. Formerly educated in mechanical, electrical, and software engineering and mathematics, Richard has more than thirty years of experience in product development and product remediation. Richard has worked with everyone from Honeywell to Pfizer and is now a renowned risk management consultant. Today, Richard will be speaking with us about his patent pending method to assess benefit-risk more objectively in health care. Without further ado, I’d like to welcome Richard Matt.

Richard Matt: Hello. My name is Richard Matt, and I’m delighted to be speaking with you about our general solution to the problem of assessing whether the benefit of a medical action will outweigh its risk. I’ll start my presentation by saying a few words about my background and how this background led to the benefit-risk method you’ll be seeing in the presentation.

To understand my background, it really helps to go back to the first job I got out of undergraduate school. I graduated with a degree in mechanical engineering and an emphasis in fluid flow. And my first job was in the aerospace industry at Arnold Engineering Development Center, at a wind tunnel that Baron von Braun designed. I worked there as a project manager, coordinating various departments with the needs of a client who brought models to be tested. These are pictures of the ADC’s transonic wind tunnel with its twenty-foot by forty-foot long test section that consumes over a quarter million horsepower when running flat out. Those dots in the walls are holes, and a slight suction would pull the out on the outside of the wall to suck the air’s boundary layer through the holes. So a flight vehicle appeared more closely to match its flight air characteristics in free air. It was amazing place to work.

We could talk about aerodynamic issues and thermodynamic issues like why nitrogen condenses out of the air at mach speeds above six or why every jet fighter in every country’s air force has a maximum speed of about mach three and a half. But to stay on the topic of benefit-risk, the reason or my intro to this, the reason I was brought this up was that I saw here firsthand the long looping iterations that came from different technical specialties, each approaching the same problem from the respective of their technical specialty. I found it very frustrating and the, following analogy very apt, after getting, so each of our technical specialties would look at the same problem, the elephant from their own view. And I found myself getting frustrated with my electrical and software engineering coworkers, that they didn’t understand what I was talking about, but I knew realized soon I didn’t understand what they were talking about either.

So I decided I wanted to become part of the solution to that problem by going back to graduate school and getting myself rounded out and my education so I could talk to these folks from their perspective also. So I went back to grad after mechanical and undergraduate, went back to graduate school in electrical and mathematics and picked up enough software. I started teaching, programming also in college. I developed there a solution for the robot arms in those wind tunnels to to control a robot arm for every possible one, two, or three rotational degree of freedom arm, and that was my graduate thesis. After I completed my thesis, I felt empowered to start, my work doing going wherever I wanted doing whatever I wanted to do and realized that if I wanted to do anything significant, it would take many years, and I decided to focus on teamwork. Does that sound pretty good?

RELATED: Jama Connect^® for Medical Device & Life Sciences Development Datasheet

Matt: My ability to work across technical boundaries enabled me to bring exceptional products to the market. For instance, I brought an Internet of Thing  (IoT) device to the market during the nineteen nineties before Internet of Things was a thing. And I rapidly advanced while I was working as a VP of engineering at a boutique design firm in the Silicon Valley. These are a few of the clients that I had, through the work that I’ve done over the years.

And, the combination of the breadth of my formal training and my system perspective for solving problems has really helped me work across continue to work across boundaries, so that I’ve worked for companies to help them establish their pro product requirements, trace requirements, do V and V work. I’ve done a lot of post-market surveillance work. I established internal audit programs. I’ve been the lead auditee when my firm is audited. Done had significant success accelerating product development and has been on work on. So mixed in with all of these works, I special I started specializing into risk management as consulting focus versus something I just did normally during development.

And since the defense of a patent requires notice, I’ll mention that the material here is being pursued on the patent, and, would like to talk with anyone who finds this interesting to pursue after you’ve learned about it. So let me start my presentation on benefit risk analysis by talking about how important it is to all branches of medicine and the many problems we have implementing it. The solution I’m gonna come up with, I’ll just outline here briefly so you can follow as we’re going through the presentation. I’m gonna first establish a single and much more objective metric to measure benefit and risk than people traditionally use. I’ll be accumulating overall benefit and risk with sets of metric values from this first metric. And finally, we’ll show how to draw a conclusion from the overall benefits and risk measurements of which is bigger benefit or risk.

So in terms of importance, historically, benefit-risk has been with medicine for millennia. It’s a basic tenant to all of medicine. The first do no harm goes all the way back to the quarter of Hammurabi 2,000 BC, and it legally required physicians to think not just about how they can help patients with treatment or what harm they might cause to treatment and making sure that the balance of those two favor the patient is very much the benefit-risk balance that we look at today. The result we’re gonna talk about is gonna be used everywhere throughout medicine with devices, with drugs, with biologics, even with clinical trials.

So is that fundamental cross medicine? How it’s used currently?

If you are in one of the ways developing new products, benefit-risk determinations have to be used in clinical trials to show that they’re ethical to perform, that we’re not putting people in danger needlessly. Benefit-risk determinations are the final gate before a new product is released for use to patients. And I have a quote here from a paper put out by AstraZeneca saying the benefit-risk determination is the Apex deliverable of any r and d organization. There’s a lot of truth to that. It’s the final thing that’s being put together to justify a product’s release. And so it has a very important role here for FDA and has a very important role for pretty much the regulatory structure of every country, including the EU.

RELATED: Buyer’s Guide: Selecting a Requirements Management and Traceability Solution for Medical Device & Life Sciences

Matt: In terms of creating a quality system, every medical company is required to have one. Benefit-risk determinations are used to assess a company’s quality system. This is per the FDA notice about factors on benefit-risk analysis. When regulators are evaluating company’s quality system, they’ll use benefit-risk to determine if nothing should be done, if a product should be redesigned, if they should take legal actions against a company of a range of possibilities from replacing things in the field to stopping products from being shipped. It’s also a key in favorite target for product liability lawsuits, because of how subjective it is, and we’ll get to that in a moment. It can also be used for legal actions against officers. So benefit risk is a really foundational concept for getting products out and keeping products out and keeping companies running well. Just a bit of historical perspective of medical documentation and development. We have here, I cited four different provisions of the laws, regarding medical devices in the United States. This is a small sampling.

The point I’m trying to make here is that each of these summaries of the laws discuss continually evolving, continually growing, more rigorous standards for evidence, more detailed requests for information from the regulators to the instrumentation development companies to the product development companies. So first, medical products are heavily regulated. We have the trend of increasing analysis and rigor. Per ISO 142471, and this is an application standard that is highly respected in the medical device field. A decision as to whether risks are outweighed with benefits is essentially a matter of judgment by experienced and knowledgeable individuals.

And this is our current state of the art.

Not that everybody does it this way, but this is the most common method of performing benefit-risk analysis. And benefit-risk analysis by this method, has a lot of problems because it’s based on the judgment and it’s based on individuals, and both of those can change with different settings. That’s why it’s a favorite point of attack for product liability lawsuits.

This quote was true in 1976, when medical devices were put under FDA regulation, but significantly remains unchanged nearly fifty years laters. Benefit-risk determinations are an aberration and that unlike the rest of medicine, they have not improved over time. They’ve remained a judgment by a group of individuals. In, twenty eighteen, FDA was, approached by congress to set a goal for itself of increasing the clarity, transparency, and consistency of benefit risk assessments from the FDA.

This was in human drug review as the subject, and the issue was that various drug companies had gotten very frustrated with the FDA for disagreeing with their assessments of what benefit-risk should look like. And to repeat again, when you have a group of individuals making a judgment, that’s gonna lead to inconsistencies because both the group and their own individual judgment will vary from one situation to the next. I have another, quote here from the article from AstraZeneca. The field of formal and structured benefit-risk assessments is relatively new.

RELATED: Application of Risk Analysis Techniques in Jama Connect to Satisfy ISO 14971

Matt: Over the last twenty years, there’s still a lack of consistent operating detail in terms of best practice by sponsors and health authorities. So this is an understatement, but a true statement. We have had a lot of increasing effort over the last few years because if people are dissatisfied with the state of benefit-risk assessments, they want to do better than this judgment approach. And so there have been a plethora of new methods developed. I’ve found one survey here that summarize fifty different methods just to give you an idea of how many attempts there are. And I went through those fifty methods.

The other thing that’s interesting to see is the FDA’s attempt to clarify benefit-risk assessments. I have here five guidance documents from the FTA, and I would put forth the proposition that anytime you need five temps five attempts to explain something, it means you didn’t understand the thing well in the first place or failing about a bit trying to get it done right. I think this is also held up by the drug companies, pressure on congress to get FDA to improve their clarity and consistency of benefit-risk assessments.

So here’s the, fifty methods that I found in one study of benefit-risk assessments. They have them grouped into, a framework, metrics, estimate techniques, and utility surveys. These are the fifty different methods, and I’ve gone through each one of them. And they all have fundamental problems. They, I’m going through them a bit slowly. Like, here’s one, from the FDA, another benefit risk assessment. Health-adjusted life years are one of the few that uses the same metric for benefit and risk. Number needed to treat is a very popular indication for a single characteristic, but you can’t integrate that across the many factors that needed to do benefit-risk assessment.

And so we’ve gone down the rest of these, methods. If I group these fifty methods by how they accumulate risk, I get a rather useful collection. Most of the methods do not consider all the risk-benefit factors for benefit-risk situation. They will pick on just one factor. And you can’t combine the factors with themselves or with others. It’s simply looking at one factor by itself. So it’s an extremely narrow view of benefit-risk for most of these. The few methods that do look at all the risk-benefit factors, most of them start with what I call the judgment method, where you’re forced to distill all the factors down to the most significant few, only four maybe four to seven methods, four to seven factors.

So either the methods consider only one type of, one factor at a time, or they force you to throw away most of the methods and consider maybe four or seven factors is the second method. The third method is they assign numbers to the factors, they’ll add the factors together, and they’ll divide the benefit sum by the risk sum. And if the division is bigger than one, they’ll say the benefit’s bigger than the risk. And if the division is less than one, they’ll say the risk is bigger than the benefit.

CLICK HERE TO WATCH THIS WEBINAR IN ITS ENTIRETY:
Expert Perspectives: A Method to Assess Benefit-Risk More Objectively for Healthcare Applications

Learn about the critical role of benefit-risk analysis in the development of safe and effective medical devices, including the use of ISO 14971, regulatory requirements, and optimizing for patient needs and healthcare costs.

The Importance of Benefit-Risk Analysis in Medical Device Development

Benefit-risk analysis is a crucial stage in the creation of medical devices. It entails evaluating the device’s possible benefits as well as drawbacks and deciding if the advantages outweigh the disadvantages. This examination aids in ensuring that medical devices are reliable, safe, and capable of being used by patients without harm.

The global standard for risk management of medical devices is ISO 14971. It offers a framework for recognizing, assessing, and managing hazards related to medical devices. Manufacturers must follow the standard and conduct a benefit-risk analysis as part of the risk management procedure. To ensure that the level of risk connected with a medical device are acceptable and that the benefits outweigh the risks, this analysis is crucial.

To start a benefit-risk analysis, it is important to first determine the device’s intended use(s). The device’s intended use should be defined in detail and contain information on the patient group it is meant for, the medical problem it is intended to support, and the clinical environment in which it will be used.

RELATED: Jama Connect^® Features in Five: Risk Management for Medical Device

Finding the potential advantages of the device is the next step after defining its intended usage. Benefits could include better health outcomes, more comfortable patients, and lower healthcare expenses. These advantages should be measured and contrasted with any possible risks connected to using the technology.

A medical device’s dangers may include physical harm to the patient, adverse events, and device failure, according to the definition of harm within ISO 14971. The possibility and seriousness of each risk should be evaluated, and these risks should be identified and quantified.

Even after risks have been lowered as far as possible with risk controls, there may still be some unacceptable level of risks. This is why a benefit-risk analysis is so important in medical device development.

The following stage is to assess the benefit-risk balance after the potential advantages and risks have been determined. This entails weighing the device’s possible advantages and disadvantages to decide whether the benefits outweigh the risks.

If the benefits outweigh the risks, the device may be considered safe and effective for use in the intended patient population. However, if the risks outweigh the benefits, the device may not be considered safe or effective and may need to be redesigned or modified to reduce the risks, a medical device manufacturer might also make the decision not to launch the product to market.

Benefit-risk analysis must be optimized to guarantee the safety and efficacy of medical devices.

RELATED: Validation Kit for Medical Device & Life Sciences

The benefit-risk analysis should be an ongoing process throughout the development and life cycle of the device. As new information becomes available, the analysis should be updated to ensure that the benefits still outweigh the risks, as prescribed in various regulations and standards such as 14971:2019, and EU MDR/IVDR.

Regulatory standards for medical devices should also be considered in the benefit-risk analysis. The Food and Drug Administration (FDA), which oversees medical device regulation in the US, requires manufacturers to prove their devices are safe and effective before they can be marketed or sold.

The FDA requires med device manufacturers to perform a benefit-risk analysis as part of the product development process. This analysis is used to determine whether the benefits of the device outweigh the risks and whether the device is safe and effective for use in the intended patient population.

Conducting a benefit-risk analysis is a critical step in the development of med devices. It involves identifying and evaluating potential benefits and risks and determining whether the benefits outweigh the risks. ISO 14971 provides a framework for performing a benefit-risk analysis as part of the risk management process.

Optimizing benefit-risk analysis is important for ensuring that medical devices are safe and effective, meet regulatory requirements, and are reimbursed by healthcare payers. A systematic approach that considers all relevant factors, including patient needs and preferences, and clinical outcomes.

Note: This article was drafted with the aid of AI. Additional content, edits for accuracy, and industry expertise by McKenzie Jonsson and Vincent Balgos.

Ready to learn more about managing risk in medical device development?
Watch this short video: Jama Connect^® Features in Five: Risk Management for Medical Device

In this blog, we recap the “An Overview of the EU Medical Device Regulation (MDR) and In-Vitro Diagnostics Regulation (IVDR)” webinar.

Looking to stay ahead of ever-evolving regulations governing medical devices?

In this webinar, we discuss the continual rollout of the EU Medical Device Regulation (MDR) and In-Vitro Device Regulation (IVDR) and the impact they’re having on the medical device industry.

Vincent Balgos, Director of Medical Device Solutions at Jama Software and Saby Agai, Sr. Consultant at Jama Software, provide a high-level overview of the new regulations, along with general industry observations and future considerations for organizations with medical products marketed in the EU market area, including:

• New classifications, grandfathering clause, and risk management requirements
• The number of notified bodies, backlog, and remediation efforts for placed products
• Future considerations regarding the compliance compatibility of IVDR & FDA and traceability
• Finally, learn how the Medical Device Framework in Jama Connect® can help streamline your compliance efforts and ensure your products meet all the necessary regulatory requirements.

Below is an abbreviated transcript and a recording of our webinar.

Why it Makes Sense to Store Cybersecurity Risk Management Items Inside a Requirements Management System

Saby Agai: So, in the first part of the webinar we will talk about the EU medical device regulations. There is a small agenda to that. Basically, we would like to show the key changes and challenges that the MDR means compared to the MDD. We would also like to talk a little bit about what we see as the challenges for the process transformation of the medical device developers and also a bit of discussions with the race on the timeline for the MDR. The second section is on the MDR for the MedDev engineering. So basically, how the engineering teams can do anything with the MDR. We’ll talk about harmonized standardization. How does that fit in the concept of the MDR? And some of the medical device best practices that we would recommend. So the medical device regulations now has quite a bit of history because the MDR is valued also for existing [inaudible 00:04:15] devices and also for all the new devices.

The medical device regulations entered into force historically in May 2017, and there was a bit of extension period in 2020 that the certificates issued under the MDD before the MDR remained valid up to four additional years. So it was a bit of a time extension for manufacturers to migrate the legacy devices to the MDR. Recently 2023, the EU commission had the new rule based on 607 was the number of it on the time extension for the medical device regulations. So there are two-time extension now in force for December 2027 and 2028 for all devices. As part of this modification, the commission removed the sale of period from the original context of the Medicaid device regulation.

Three key area that we would mention that we see as key challenges with the MDR is first is the technical documentation. So because of the legacy medical devices has to be reclassified in a context of the new MDR, those manufacturers highly likely will face it extended set of documentation for market clearances in the EU. It’s particularly true for software as a medical device because, basically the class one level has removed by legislation for all software as a medical device. The other thing on the technical documentation is that the MDR is far more prescriptive about the requirement content of the technical documentation, and it’s particularly true and there are more detailed requirements needed for the quality management system. So the manufacturer will have to ensure that they not only have full access and control for the documentation of the device, but also they should keep the eye on the market and the vigilance market, post-market vigilance area, as well as publication or new common specifications.

RELATED: Buyer’s Guide: Selecting a Requirements Management and Traceability Solution for Medical Device & Life Sciences

Agai: So there is a bit of higher focus on the post-market activities in the context of the MDR. And the technical documentation basically has two key parts in Annex II, annex III it’s detailed. Annex II there is a list of requirements for the technical documentation itself for the device design, and also, in Annex III, we see details or requirements for technical documentation post-market surveillance. So nothing particularly new, only an extended set of expectation and requirements for all these contents. Little note something on the technical documentation. So historically, the technical documentation has a tradition to be seen as a burden on the med tech developers and additional administrative work. And quite often, at the end of the development cycle, there is a massive effort made to make what is documentation available for regulators and also for market clearance. It definitely could require very intense administrative work from the engineering sometimes stress involved and also, the content creation has not much help or not much support for the regional engineering activities, which is the development.

So these content created purely to support the market access activities and it should not necessarily should be a case, though. So, for example, we in Jama has a medical solution. It’s a example proven a tool actually can support both med tech developers to enhance the development efficiency as they develop a new device as well as to support these technical documentation needs at the same time. So it’s a opportunity also for organizations to get most out of using a tool when they thinking about to ease the burden of the medical device regulation technical documentation part.

RELATED: Jama Connect^® Validated Cloud Package for Medical Device and Life Sciences

Agai: Thirdly but not lastly, there was a new, particularly in EU requirements for the unique device identifier. So basically, 2021 was a deadline to register an MDR UD MDR devices with the UDI in the [inaudible 00:09:02] framework, and for the IVDR, it’s 2022. Looking into purely on the numbers, we could say that the content of the MDR compared to the MDD is actually four time heavier. So the extent and the legal tax basically is four time more. There are five plus on axis that we can see, and there is a special attention on safety and patient safety particularly because 293 times mentioned the safety word in MDR where in the MDD was the 24. All these numbers also telling that the regulators in EU want to have higher scrutiny compared to the MDD, and they also have more details on that level of expectation that they would like to see from manufacturer. And there is a definite focus on patient safety that we can see.

Two more things to mention is that quite often, in the context of the MDR, the legacy devices should be reclassified into a higher level class. So it means that the quality management process support is more intense, and more support expected. More activities and works are expected from the manufacturer to keep the same device basically on the market. It also could mean that companies should take a step for high-level maturity as an organization and it’s true also for the design and device development activities. So one of the challenge with that is that if we talk about the same device with higher regulatory scrutiny, how do we retain and enhance profitability? Because the administrative burden is definitely something that goes towards the cost part of the profitability. So the design and development goes under higher level of process expectation in that sense, and it goes higher level of design documentation needs as well. So one of the advantages using the tool in general medical device environment that the medical solution can ease actually this work and enable a bit quicker, and the developers can leverage a little bit more help on these challenges.

To watch the entire webinar, visit
An Overview of the EU Medical Device Regulation (MDR) and In-Vitro Diagnostics Regulation (IVDR)

In this blog, we recap the “Why it Makes Sense to Store Cybersecurity Risk Management Items Inside a Requirements Management System” webinar.

In this webinar, “Why it Makes Sense to Store Cybersecurity Risk Management Items Inside a Requirements Management System”, learn about the implementation of the Threat and Risk Analysis (TARA), the centerpiece of the new Automotive Cybersecurity standard ISO 21434.

Many companies currently use spreadsheets to develop TARAs, which can be challenging when managing large sets of requirements across distributed teams and car line variants. In this webinar, we’ll examine why a requirement management system (RMS) is well-suited to manage the TARA work product and can make a significant impact on managing this data across teams, supporting compliance audits, and assessments.

Attendees will gain insights into TARA’s complexities and how the right tooling solution can make a difference in managing this data across teams, supporting compliance audits and assessments.

Key Takeaways:

• The Threat and Risk Analysis or TARA is the centerpiece of the ISO 21434 Automotive Cybersecurity standard
• Overview of TARA
• ISO 21434 compliance requirements when implementing TARAs
• Why an RMS is well-suited to manage TARAs

Below is an abbreviated transcript and a recording of our webinar.

Why it Makes Sense to Store Cybersecurity Risk Management Items Inside a Requirements Management System

Kevin Dibble: Thanks, Juliet. Okay. I’m going to just go through the agenda and then get right into 21434. I’ll start with a high-level introduction and then get into the focus of our topic today, which is the threat and risk analysis, which is a centerpiece of 21434, also known as the TARA. And then make an argument for the management of a TARA using an RMS or a requirement management tool. And then Steve will take over and talk about what that would look like in Jama software and summarize with some key points of managing TARAs in Jama versus some traditional methods. And then we’ll have time for some questions.

So with that, again, this is going to be a very high-level overview of 21434. I have a feeling that some of you have worked in cybersecurity for some time, others are just brand new to the term. And so, I want to touch on this as a basis for the rest of the discussion.

And so, first, what is 21434? It is the automotive industry standard for developing cyber secure systems. After several years of review, it was approved in August of 2021 as the method for developing cyber secure systems. In terms of the standard itself, it’s structured and uses a lot of the same terminology as the functional safety standard called ISO 26262. So if you’re familiar with functional safety, then this standard will make a lot of sense the way that it’s organized. Some of the terms such as an item definition, a concept phase, a cybersecurity goal, even TARA parallels functional safety terms like functional safety concept, functional safety goals, or the HARA, Hazard and Risk Analysis. And so, that’s just a reference point as you’re learning about this new standard. Now as far as its scope, it covers or it applies to passenger vehicles and cargo vehicles.

So just a little bit different than ISO 262 there, passengers would include buses, commercial or non-commercial. I think even tripods and some of those other types of motorcycle hybrid type of devices are in or vehicles are in scope as well. It applies to series production and it uses a lifecycle that starts at the request for a quotation for an item. And I’ll define that in a little bit and goes all the way through to the end of cybersecurity support. So like functional safety, we’re not talking about supporting the risks and the hazards associated in this case with threats from attackers leading up to SOP, but it extends far past that. In fact, in 21434, instead of using the term SOP or start of production, which is a critical milestone in any automotive product development program, they call that milestone the release to post-development.

RELATED: Functional Safety (FuSA) Explained: The Vital Role of Standards and Compliance in Ensuring Critical Systems’ Safety

Dibble: And I want to camp on that for just a second because it raises a really important point and it’s very relevant to what we’re going to talk about regarding the TARA. Release to post-development. So the automotive industry is under a lot of change and OEMs want to be or are becoming mobility providers and services will be sold after the car is released. And some of those services weren’t even imagined at the time the car was sold. That’s so different than where the automotive industry was even five years ago. And this standard recognizes that and embraces it along with another important concept, which is that the world of cybersecurity and the landscape for threats and the technologies and the tools that are used to attack vehicles is constantly changing. And so, at the release to production, what is assumed to be protected in terms of say a set of cryptographic keys or a communication bus might be more vulnerable in five years than it was when the car was released because of new techniques, new methods, new tricks, new hacks, and other things that have been discovered.

And so, that’s an important concept because it feeds to our idea that we’re going to get into about the TARA as a living document, as a living asset that begins all the way at the concept phase at the beginning of the high-level architectures of the item or the system in the car. And extends all the way until the end of life for cybersecurity support, which is 10, 15 years down the road. Now, the 21434 has both requirements for developing cyber secure systems, is kind of what I’m showing you on the right, but it also has process requirements. And to that end, there is an audit of the process and an assessment of the results of your project according to 21434. That assessment piece is important for our discussion because when we think about the TARA and the pieces of it or the items of the TARA, then we have to think in terms of what are the evidences we need to leave behind and produce in order to pass an assessment, very important consideration.

RELATED: A Guide to Road Vehicle Cybersecurity and Risk Management: Part 1

Dibble: And so, we have audits for the processes and then assessments for the end result. So that’s very brief overview of 21434. I want to make sure I leave you with the… If you remember anything about 21434 besides the TARA, you’ll hopefully remember this, is to manage unreasonable risk of damage to road users due to a malicious attack to a vehicle or a vehicle data, confidentiality, integrity and availability. Let me unpack that for just a second. Unreasonable risk, this is when you get into a car, when you operate a vehicle, you assume some risk. But that risk doesn’t include driving down the highway at 70 miles an hour, turning right and the car going left or the headlights going off while you’re on the highway at night. It applies to road users. That’s the people that use the road, the driver, the passengers, and the people surrounding it.

All of that is our scope for how we’re going to define threats according to 262 and then mitigate them against malicious attack due to… That’s the cyber aspect of this. And then what’s being attacked and what are we protecting? We’re protecting vehicle systems, functions, data, et cetera. We call them assets according to their properties, confidentiality, integrity and availability. There could be more properties, that’s the CIA that we’re protecting. Why is cyber such a hot topic? Well, I would say there’s several reasons, but here’s two of the big ones. On the left of my slide, the advent of the connected car coupled with the automated driving functions. I’m not going to read through all the stats here, but the connected car is here. It’s 2 billion in terms of the market in 2021 to grow to $5.3 billion in 2026. And the connected car is accessible via the internet, accessible via Bluetooth and other network interfaces, which all result in attack services. It also has a lot more software.

To watch the entire webinar, visit
Why it Makes Sense to Store Cybersecurity Risk Management Items
Inside an Requirements Management System

Trace Score™ – An Empirical Way to Reduce the Risk of Late Requirements

Executive Summary

One of the main causes of rework, delays, and cost overruns in product development is the creation of new requirements late in the process. This is a well-known risk in product development, but what management practices can empirically be shown to reduce this known risk?

Using our proprietary database of metadata from over 50,000 complex product development projects, we were able to determine that the Trace Score is an empirical method to reduce late requirements. In fact, teams that maintain a high Trace Score reduce the burden late requirements have on their project by 67% compared to teams with low trace scores.

• With this knowledge, our recommendation is that practitioners measure and monitor the Trace Score of their projects to resolve issues early and ensure that the risk of late requirements is kept to a minimum.

Dataset Background

Jama Software has the world’s largest, live dataset of engineering process performance with over 50,000 engineering projects updated and growing continuously. Leveraging this dataset, it is now possible to determine empirically which management practices improve the performance of the product development process. To learn more about our benchmarking, please review our Traceability Benchmarking Report.

The Empirical Questions

In this analysis we will explore three key questions:

1. What are late requirements?
2. How do late requirements negatively impact projects?
3. Does maintaining a high Trace Score reduce the risk of late requirements?

What are late requirements?

For the purpose of this analysis, we define “late requirements” as those requirements created after the completion of a project’s requirement decomposition phase which we estimate as spanning the middle 50% of all requirement creation activity (creation and refinement). To illustrate what late requirements look like, we show two actual projects below with requirement activity plotted over time.

Requirement Creation Over Time

In the Timely Project, requirement creation occurs in a defined requirement decomposition phase to form a necessary and sufficient set of requirements, with very few requirements being added after the fact (e.g. in fig (a), only 1.3% of requirements created late). In the Late Project’, requirement creation bleeds into future phases of the project, leading to a significant amount of late requirements (e.g. in fig (b), 9.2% of requirements are created late).

RELATED: Requirements Traceability Benchmark

How do late requirements negatively impact projects?

We can measure the outsized burden late requirements have on project teams, which we have illustrated for our two projects below. We define late requirement burden as the total number of requirement activities (creation and refinement) attributed to late requirements as a percentage of all requirement activity.

Impact of Late Requirements on Project Team Activity Burden

In the Timely Project, minimal late requirements enable better forecasting of project completion and limit the rework and cost brought on by late requirements (e.g. in fig (c), late requirements only create an additional 8% burden).

In the Late Project, the high volume of late requirements makes it much harder to forecast project completion as the scope of the project is constantly changing, and project teams need to accommodate the late requirements (e.g. in fig (d), late requirements contribute an additional 31% burden).

Unsurprisingly, this additional burden of late requirements has an impact during testing for requirement validation. In our actual project examples, the Late Project has a test failure rate over 3x that of the Timely Project.

RELATED: Unlocking The Power of Live Traceability with Jama Connect

Does maintaining a high Trace Score reduce the risk of late requirements?

A core theorem of Systems Engineering is that maintaining high requirement traceability from the start of a project reduces the risk of late requirements and negative product outcomes. With our project dataset we can now test this theorem empirically. We define traceability as a measure of a project’s ‘expected’ traceability that has actually been established and calculate the Trace Score as follows:(1)

For our example projects, the Timely Project achieved a Trace Score over 6X that of the Late Project; suggesting that maintaining a high Trace Score throughout the project reduces the risk of late requirements.

To further determine if Trace Score correlates to late requirements, we divided our dataset of projects into quartiles based on their Trace Scores (Quartile 1 = bottom 25% trace score, Quartile 4 = top 25% trace score) and then compared the distribution of ‘Late Requirements Burden’ across these quartile groups. What we found is that projects within the bottom traceability quartile had a median Late Requirements Burden 3x greater than those in the top traceability quartile. In other words, the evidence supports that projects managed with higher traceability generally experience less risk from late requirements.

Recommendation

Our analysis has shown that late requirements negatively impact projects and that managing projects through a Trace Score is the only empirical way to reduce the risk of late requirements. Below you can see how one can measure the Trace Score over time as a project progresses to ensure system engineering best practices are being followed. A low or falling Trace Score can quickly identify areas to address to reduce the risk of late requirements.

Here you can see how managing the Trace Score directly as the project is underway would have identified the risk early in the Late Project.

To learn more about achieving Live Traceability™ on your projects, please reach out for a consultation.

Interested in learning more? Download the entire Research Notes: Trace Score datasheet HERE.

In this blog, we’ll recap our whitepaper, “Understanding Integrated Risk Management for Medical Devices” – To read the entire paper, click HERE.

Understanding Integrated Risk Management for Medical Devices

Knowledge on best practices, how to integrate risk-based thinking into product development cycles, and the importance of having end-to-end traceability to improve risk management, shared by industry and solution experts.

A level of risk exists with all medical devices, no matter how simple they are.Companies developing medical devices are constantly considering who (or what environment, facility, etc.) could potentially be hurt by a device so they can help reduce risk and meet regulatory requirements. Risk management in the context of ISO 14971 is designed to support medical device manufacturers with these tasks — but not all approaches are equal.

The amount of time it takes to manage risks, connect specific risks to specific requirement tasks, and pull together required documents to respond to an audit varies slightly depending on the approach. The risk management process is an integrated process that not only includes teams in product development, quality, but also many other parts of an organization.

This whitepaper taps into the knowledge of industry and solution experts to uncover best practices, how to integrate risk-based thinking into product development cycles, and the importance of having end-to end traceability to improve risk management. Before we dig into integrated risk management, let’s first define some key terms.

RELATED: Jama Connect^® Features in Five: Risk Management for Medical Device

Risk Management Terms According to ISO 14971

Harm – Harm occurs when people are injured physically or their health is compromised or when property or the environment is damaged.

Hazard – A hazard is a potential source of harm. Annex E.2 categorizes hazards in the following way: energy hazards, chemical hazards, biological hazards, operationalhazards, and informational hazards.

Hazardous – A hazardous situation occurs when people are exposed to a hazard or when property or the environment is threatened. A hazardous situation exists when a vulnerable entity is exposed to a hazard.

Situation – According to ISO 14971, the concept of risk combines two variables: the probability of harm and the severity of harm.

Risk – For example, if a particular hazardous situation is very likely to cause harm and would be very harmful if it actually occurred, then it would be a high risk situation. Conversely, if it’s very unlikely to cause harm and would be only slightly harmful if it actually occurred, then it would be a trivial risk.

Risk Analysis – Risk analysis is a systematic process that is used to identify hazards and to estimate risk. It includes an examination of every reasonably foreseeable sequence or combination of events that could produce a hazardous situation and cause harm.

Risk Assessment – Risk assessment is a process that is, in turn, made up of two interconnected processes: risk analysis and risk evaluation.

Risk Evaluation – Risk evaluation is a process that is used to examine the estimated risk for each hazardous situation and then to use risk acceptability criteria to determine whether
or not the estimated risk is acceptable and to decide if risk reduction is required.

Risk Control – Risk control is a process that is used to consider risk control options and to select and implement risk control measures that will reduce risk or maintain risk within
specified levels. ISO 14971 expects you to consider the following risk control options and, if possible, to apply them in the following order:

1. Design safety into the product.
2. Establish protective measures.
3. Provide safety information.

Risk Estimation – Risk estimation is a process that is used to assign qualitative or quantitative probability values and severity values to each hazardous situation. These values are then used to estimate risk.

Risk Management – Risk management uses policies, procedures, and practices to systematically analyze, evaluate, control, and monitor risk.

Safety – Safety is freedom from unacceptable risk. Risk acceptability criteria are used to help decide whether or not a risk is unacceptable.

Severity – Severity is a measure of the possible harmful consequences that a hazard could potentially cause.

RELATED: Download our whitepaper, Application of Risk Analysis
Techniques in Jama Connect® to Satisfy ISO 14971

The Risk Management Process

During risk management — after one defines a device’s intended use(s) — risk analysis can begin with identifying all potential hazards, and hazardous situations. Once this is defined, risk can be estimated and can determine the type of appropriate risk control required. Once the risk controls are implemented, residual risk needs to be analyzed to ensure that the benefits outweigh the risks. Let’s take a look at what’s involved in the risk management process.

Identifying Hazards

“Risk” is defined as the severity and probability that harm will occur. Defining the severity of harm requires you to identify all the known and foreseeable hazards for both intended and unintended uses.

For example, let’s say you have an infusion pump, and that pump has air in the line, which creates a hazardous situation for the patient. Different levels of patient harm can occur, so it’s about uncovering the possible scenarios and the likelihood of a situation’s occurring.

Understanding Harm

Understanding harm includes both people and property. A medical device that catches fire might threaten property, while an infusion pump with air in the line might threaten human life. Think about what could cause harm to people, like a shark swimming in the water. A shark that attacks a person could create different levels of harm. A few examples include loss of a limb, an infection from getting bitten and loss of life. The various levels of harm result from the hazardous situation, which is the shark in the water.

Risk Evaluation

Risk evaluation involves comparing an estimated risk against a specific criterion to determine if a risk is acceptable. Five different levels to evaluate risk are common practice, but you can use as many as you’d like. The most severe risk (level five) might include death or impairment. Level one might include no risk to a patient or operator. The levels inbetween include all the other varying
degrees of risk.

Sequence of Events

A hazardous event includes a number of steps, which is the sequence of events. A risk situation might have two, three, or more steps that, when aligned, create a hazardous event. Risk management tools such as fault trees and failure modes and effects analysis (FMEA) help identify these steps.

Previous version of ISO 14971 used terms like “acceptable” and “unacceptable” to describe risks, but that language has since been removed and the most current version maintains as low as possible (ALAP). The goal of every manufacturer is to lower the risk as much as possible and rethinking how to prioritize risk controls can help.

This has been a preview of the content in our whitepaper, Understanding Integrated Risk Management for Medical Devices to read the entire paper, click HERE