Evaluate and Evolve Policies

Organizations and the teams that operate within them are complex systems nested within wider complex systems. In such environments, circumstances can change rapidly, making it unrealistic to expect that every decision or policy will be entirely suitable from the outset or will remain effective over time.

An iterative process of regularly evaluating policies and adapting them as necessary ensures that policies are improved, so that they become and remain fit for purpose despite changes in the organization or the wider environment. The compound effect of regularly evaluating and evolving all policies prevents organizations from accumulating useless policies that hinder productivity. It also ensures that people regularly refresh their knowledge about the policies that affect them, enabling them to better keep to them, too.

An effective evaluation integrates empirical data derived from metrics, systematically collected qualitative data, and personal experiences related to the policy being evaluated. These inputs guide participants toward understanding how effective the policy was, and reveal insights into worthwhile ways it might be improved.

To ensure policies are evaluated as necessary, it’s important to keep a written record of each policy, including an archive of previous versions, and to set a suitable review date or cadence for when each policy should be reviewed.

Ideally, when evaluating a policy, you will already have a written record, not only describing the policy itself but also its purpose and metrics for evaluation. However, many teams may not have these details fully defined or recorded. If this is the case, you can use the format provided at the end of this pattern description to assess and improve your existing policies while you are putting these other elements in place.

When deciding on a suitable review date or frequency, consider factors like the level of uncertainty surrounding the policy and the potential impact of its application. Make arrangements to alert those responsible as the review date approaches, and remain adaptable by initiating reviews earlier if conditions change or if new insights emerge.

Note: While this pattern is designed for teams, individuals can also apply elements to evaluate their own decisions, fostering greater awareness of personal decision-making effectiveness.

How to Evaluate and Evolve Policies

Evaluating a policy can be as simple as checking whether it is still relevant and suitable for achieving the outcomes intended and whether there are any objections to keeping it as it is. Policies are often reviewed in Governance Meetings. However, for more extensive or particularly complicated policies, or when a policy affects multiple stakeholders, it’s sometimes more effective to schedule a session dedicated solely to the review.

Before evolving the policy itself, consider that the policy is designed to fulfill a requirement, which in turn is meant to provide a suitable response to an organizational driver. Therefore, making necessary changes to the driver and requirement takes precedence over changes in the policy itself:

• Changes to the driver can alter opinions about what is required.
• Changes to the requirement can affect whether the policy remains suitable or needs adjustment.

This implies there is a simple and natural order to the steps of a review:

First, ensure the purpose is still relevant to fulfill (or retire and archive the policy if it is not); second, evaluate how effective the current version of the policy has been in fulfilling the requirement; third, assess whether the purpose (driver and requirement) needs updating; and finally adapt or replace the current version of the policy based on insights from the evaluation, to support another iteration of learning and improvement.

General Format for Evaluating and Evolving Policies

1. Prepare
   • Schedule the review
   • Ensure all necessary information is available to participants
   • As a participant, ensure to review the policy and related information and prepare reports if relevant
2. Hear any prepared reports about the policy, e.g. usage data, observed outcomes, or incidents.
3. Check the purpose of the policy to ensure it’s still relevant. If the situation is no longer relevant to respond to (because the situation or the context has changed), retire and archive the policy, unless a compelling reason for its continuation is identified. You may still choose to evaluate what worked and what didn’t if you have the time, especially if doing so may reveal learning that informs future governance.
4. Evaluate actual outcomes and whether requirement(s) were fulfilled.
   • Evaluate how the intervention played out in practice: what was actually done, what happened as a result, and whether this aligned with what was expected.
   • Review recorded numbers for metrics and any other relevant data to determine whether intended outcomes were achieved, and to what extent, and which acceptance criteria were met.
   • Identify any significant unintended outcomes of implementing the policy, both positive and negative, and extract learning.
   • Assess whether the conditions described in the requirement(s) were established or maintained.
   • Where intended outcomes were not achieved, or conditions were not established or maintained, assess whether the issue lay in implementation, an inappropriate policy, or unsuitable conditions defined in the requirements.
     • Take note of any findings that might inform updates to the requirement, description of the driver, or the policy itself.
5. Evaluate and evolve purpose: Review and — if necessary — change the description of the driver and/or update the requirement.
   • Review driver: Has the situation changed, and is it still relevant?
   • Are the intended outcomes valuable in relation to the driver, and reasonably complete?
   • Are the conditions suitable for achieving the intended outcome?
6. Evolve the policy
   • Evolve the actual policy: What aspects were effective for achieving the intended outcomes? What can be improved? What needs to change to account for updates made to the purpose? Is there anything missing?
   • Review the metrics: do they sufficiently determine progress in relation to the intended outcomes, conditions, and driver? Do they warn us when things go wrong?
   • If the policy is made up of sub-policies, you can apply this same process (for evaluating and evolving policies) to each of the sub-policies (starting from step 2 or 3).
   • Check for coherence of all sub-policies, and add what is still missing.
   • Whenever you learn anything new about the driver or requirement, you need to update the purpose first, and then check for any consequences this might have on the policy.
7. Follow-up
   • Agree on the date for the next evaluation.
   • Decide on all necessary follow-up activities.
   • Ensure that all follow-up decisions and tasks are documented and shared to maintain accountability and track progress.
   • Consider effects on any related agreements.