Measuring Design-Pathway Transformation

Purpose

This note provides a practical method to measure transformation in design pathways through possibility space. It operationalises three criteria:

1. Scope of change
2. Speed of change
3. Depth of change

It also supports direct comparison between human-led and nonhuman-led pathways.

Why These Criteria Are Measurable

The criteria are measurable when modelling represents pathways as time-sequenced transitions between states in a bounded but partly latent possibility space. This is consistent with existing notes, cf.:

• Conceptual framing of phase and possibility space
• Innovation distance, adjacent possible, and proxy limits
• Transformation framing
• Measurement value and selection effects
• Deliberation pipeline and validity criteria
• Participation ladders and transformation constraints
• Model plurality and performativity
• Network analysis methods

Because possibility spaces are partly unobservable, design all indicators below as defensible proxies rather than direct measurements.

Unit of Analysis

Use one unit consistently across the study:

1. A project
2. A policy cycle
3. A place-based intervention
4. A species-management programme

Each unit needs:

1. Baseline state at time $t_0$
2. Ordered sequence of pathway states $P_t$
3. Logged interventions and observed responses

Core Operational Definitions

Scope

Scope is the extent of explored or newly reachable possibility space.

Operationally, scope increases when pathways:

1. Reach states far from baseline configurations
2. Activate additional adjacent possibilities
3. Expand into new social, ecological, institutional, or technical domains

Speed

Speed is the rate of pathway movement, branching, and adaptation.

Operationally, speed increases when pathways:

1. Shift sooner after intervention
2. Update quickly after new signals
3. Reduce lag between nonhuman feedback and governance action

Depth

Depth is the degree of structural and normative reconfiguration.

Operationally, depth increases when change:

1. Rewires actor-resource or decision networks
2. Changes institutional rules and persists across cycles
3. Alters human-nonhuman relations and ecological feedback integration

Human-Led and Nonhuman-Led Pathways

Human-led pathway

Primary pathway direction is set by human agenda. Humans treat nonhuman data mainly as constraints or monitoring inputs.

Nonhuman-led pathway

Pathway direction changes materially because nonhuman signals guide decisions, mandates, and revisions.

Decision rule for classification

Classify a pathway segment as nonhuman-led only if both conditions hold:

1. A nonhuman signal is documented before a key decision
2. The decision outcome differs from the counterfactual human-only plan

Indicator Set

Score each indicator on a 0-5 scale. Use explicit coding notes for each score.

E.g.,

• Score: 4/5
• Evidence: “3 documented pathway forks in 6 months; two linked to nonhuman signals.”
• Rule: “Score 4 = high branching with at least 2 confirmed redirects.”
• Exclusion: “Not 5 because no full institutional lock-in yet.”

A. Scope Indicators

1. Novelty distance from baseline. Definition: Distance between current state and baseline state in a feature space. Possible measures: semantic distance, design-feature distance, policy-instrument distance.

2. Adjacent possible activation. Definition: Count of newly reachable state classes after each transition. Possible measures: new option classes unlocked per period.

3. Domain span. Definition: Number of domains with material change. Domains: social, ecological, institutional, technical, economic, legal.

B. Speed Indicators

1. Time-to-first-deviation. Definition: Time from intervention to first measurable pathway deviation from baseline.

2. Branching rate. Definition: Number of forks, reversals, or major redirects per period.

3. Signal-to-decision latency. Definition: Time from nonhuman signal detection to corresponding decision update.

C. Depth Indicators

1. Network reconfiguration. Definition: Structural shift in actor-resource or actor-decision networks. Possible measures: modularity shift, centrality redistribution, density change.

2. Institutional embedding. Definition: Degree to which changes are codified in standards, policy, budget, and governance procedure.

3. Feedback integration quality. Definition: Extent to which ecological or behavioural feedback loops modify subsequent pathway decisions.

Deliberative Integrity Layer

Apply this layer to all nonhuman-led claims. Score each criterion 0-5:

1. Legibility
2. Contestability
3. Reversibility
4. Non-coercion
5. Temporal fit
6. Accountability

These criteria derive from repository deliberation notes and are needed to avoid false attribution of agency.

Composite Scoring Model

Let $S$, $V$, and $D$ be dimension scores on 0-100 scales after normalisation.

Where:

1. $T$ is transformation score
2. $\alpha, \beta, \gamma$ are dimension weights

Default weighting for balanced use:

1. $\alpha = 1$
2. $\beta = 1$
3. $\gamma = 1$

If the study prioritises structural transition, increase $\gamma$.

Nonhuman Agency Influence Metrics

Use both metrics together.

Nonhuman Agency Influence Ratio (NAIR)

Translation Integrity Index (TII)

Where $c_i$ are the six deliberative integrity criteria scored 0-5.

Interpretation:

1. High NAIR, high TII: strong evidence for nonhuman-led influence
2. High NAIR, low TII: influence claims likely unstable or weakly justified
3. Low NAIR, high TII: robust process, but still predominantly human-led direction

Scoring Template

Copy this table per case and time window.

Summary block template

Data Logging Template

Use one row per key event.

Recommended Analysis Workflow

1. Define baseline and pathway boundaries.
2. Build a state-transition log and event timeline.
3. Score all nine core indicators plus NAIR and TII.
4. Normalise to 0-100 scales and calculate $S$, $V$, $D$, and $T$.
5. Compare human-led and nonhuman-led segments over equal windows.
6. Report uncertainty bands and proxy limitations.
7. Run sensitivity checks on $\alpha, \beta, \gamma$.

Validation and Robustness Checks

1. Inter-rater reliability for coded indicators.
2. Counterfactual review for decision-change attribution.
3. Lag sensitivity using alternative temporal windows.
4. Model sensitivity across weighting schemes.
5. Triangulation across textual, behavioural, ecological, and institutional data.

Interpretation Guide

High transformation

High scope, high speed, high depth, with non-trivial NAIR and strong TII.

Fast but shallow change

High speed with low depth, often indicating rapid adaptation without structural transition.

Broad but inert change

High scope with low speed and low depth, often indicating rhetorical expansion without durable reconfiguration.

Agency washout risk

Apparent nonhuman influence with low TII indicates translation distortion or symbolic inclusion.

Limits and Justification

Why proxies are necessary

Possibility spaces are partly latent and cannot be fully observed directly. Proxy indicators remain justified when:

1. They are explicitly linked to theory
2. They are consistently operationalised
3. Their uncertainty is reported

Why agency needs integrity checks

Nonhuman leadership claims can be overstated when signals are selectively interpreted by human institutions. The deliberative integrity layer is therefore a validity requirement, not an optional addition.

References

Edelblutte, Émilie, Roopa Krithivasan, and Matthew Nassif Hayek. “Animal Agency in Wildlife Conservation and Management.” Conservation Biology 37, no. 1 (2023): e13853. https://doi.org/10.1111/cobi.13853.

Sharpe, Bill, Anthony Hodgson, Graham Leicester, Andrew Lyon, and Ioan Fazey. “Three Horizons: A Pathways Practice for Transformation.” Ecology and Society 21, no. 2 (2016). https://doi.org/10.5751/es-08388-210247.

Steffen, Will, Katherine Richardson, Johan Rockström, Sarah E. Cornell, Ingo Fetzer, Elena M. Bennett, Reinette Biggs, et al. “Planetary Boundaries: Guiding Human Development on a Changing Planet.” Science 347, no. 6223 (2015): 1259855. https://doi.org/10.1126/science.1259855.

Kelly, Bryan, Dimitris Papanikolaou, Amit Seru, and Matt Taddy. “Measuring Technological Innovation over the Long Run.” American Economic Review: Insights 3, no. 3 (2021): 303–20. https://doi.org/10.1257/aeri.20190499.

O’Brien, Karen, Lucas Garibaldi, Arun Agrawal, Elena Bennett, Reinette Biggs, Rafael Calderón Contreras, Edward R. Carr, et al. Transformative Change Assessment: Summary for Policymakers. IPBES/11/12/Add.2. Bonn: Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services (IPBES) Secretariat, 2025.