$ cat ./reports/defospam-caa-swarm.json

DeFOSPAMREQUIREMENTS REPORT

Source: CAA_User_Stories_Swarm_BVLOS.docx (32 user stories) · 7 analysts · Timestamp 2026-04-22T00:00:00Z · Based on the Business Story Method by Paul Gerrard & Susan Windsor.

37
Findings
8
Critical
22
Major
7
Minor
7
Analysts
17
Features
17
Scenarios
30
Glossary
// glossary

PROPOSED GLOSSARY

cat ./glossary.md
Swarm
A group of uncrewed aircraft operating as a coordinated single operation under a common autonomy stack. In the input, the term is used throughout but is never numerically bounded; CAA-US-014 caps the airborne count but no story defines the minimum or the coordination protocol that qualifies a group as a 'swarm' vs. a 'formation'.
unverified
BVLOS
Beyond Visual Line of Sight. Operation where the remote pilot in command cannot see the aircraft directly. Used throughout but not defined in the stories.
unverified
Operational Design Domain (ODD)
The set of conditions under which the autonomous system is designed to operate safely (lat/long polygon, altitude band, wind, visibility, daylight). Named in CAA-US-005 but the schema of a 'machine-checkable' ODD is not prescribed.
unverified
SORA
Specific Operations Risk Assessment — the JARUS 2.0 methodology the CAA uses to determine Ground Risk Class, Air Risk Class and Specific Assurance and Integrity Level for Specific-category UAS operations.
verified
GRC
Ground Risk Class — SORA Step 2 output expressing intrinsic ground risk before mitigation.
verified
ARC
Air Risk Class — SORA Step 4 output expressing intrinsic airspace encounter risk.
verified
SAIL
Specific Assurance and Integrity Level — SORA Step 7 output mapping residual risk to required Operational Safety Objectives (OSO).
verified
OSO
Operational Safety Objective — SORA-prescribed mitigation required at a given SAIL and robustness level.
verified
ConOps
Concept of Operations — document describing mission profile, ODD, roles and contingencies.
verified
TDA
Temporary Danger Area — airspace construct applied via CAP 1616 / ANO to segregate a defined volume.
verified
C2L
Command and Control Link — the datalink over which the RPIC commands the aircraft. Classed under RTCA DO-377 / STANAG 4660.
verified
DAA
Detect and Avoid — the function that identifies and resolves a collision hazard with other traffic, per DO-365C / ASTM F3442.
verified
Autonomous
Not defined in the corpus. Stories conflate 'autonomous', 'automated', 'AI-driven' and 'learned' without drawing the line between pre-coded behaviour and machine-learned behaviour. This is significant because assurance technique (AMLAS vs. conventional software assurance) hinges on the classification.
unverified
Emergent behaviour
Behaviour that arises from interaction of swarm members and cannot be traced to an explicit rule in any single member. Used in CAA-US-013 and CAA-US-015 but the 'emergent vs. coded' distinction is not operationalised.
unverified
Meaningful Human Control (MHC)
UK MOD policy position under which a human retains authority over any engagement decision and can intervene. Cited at CAA-US-019; however, the input does not define which classes of decision are 'in scope' for human authorisation vs. delegable to the autonomy.
unverified
RAISO
Responsible AI Senior Officer — defined under JSP 936 Part 1. Cited in CAA-US-001, CAA-US-002, CAA-US-030 but the CAA-facing responsibilities of the RAISO (what they counter-sign, when) are not fully enumerated.
unverified
Go/no-go gate
A decision point at the end of a trial phase. Cited in CAA-US-032 but the decision criteria are not specified.
unverified
Containment
Action that keeps the aircraft within declared volumes, e.g. geofence-triggered RTB, autoland or flight termination. Cited at CAA-US-008 and CAA-US-021. The input does not clarify whether 'containment' and 'termination' are equivalent or hierarchical.
unverified
Fly-away
An unintended departure from the authorised airspace. Used in CAA-US-008 and CAA-US-023 without formal definition; no numeric threshold distinguishes a 'fly-away' from a 'containment event'.
unverified
Adversarial input
An input crafted to induce incorrect ML behaviour (e.g. jamming, spoofed ADS-B, adversarial EO patches). Cited in CAA-US-004 and CAA-US-015 but not defined operationally.
unverified
Out-of-distribution (OOD) input
An input outside the statistical distribution of the training data. Cited in CAA-US-015; detection mechanism is required but no performance target.
unverified
Model freeze
Prohibition on changing an ML model during the trial window. Cited in CAA-US-017; 'cryptographic hash' is required but the hash-verification workflow is not defined.
unverified
Explainability evidence
Artefacts (SHAP, LIME, attention maps) that justify a decision. Cited in CAA-US-018. The input does not specify which decisions are 'safety-relevant' enough to require explainability artefacts.
unverified
Workload limit (RPIC)
Named in CAA-US-019 as 'published workload limits' but no published reference is cited. The CAA has no published RPAS-swarm workload limit.
unverified
Accountable Manager
Role named in CAA-US-002 — conventional AOC term, but meaning under a Specific Category OA is not defined by the input.
unverified
Swarm Commander
Role named in CAA-US-002 and CAA-US-029 — not a CAP 722 term; the input introduces it without definition or competency reference.
unverified
Autonomy stack
The software that implements the aircraft's decision-making. Used informally; distinction from 'flight control software' is unclear.
unverified
Authorisation (OA)
Operational Authorisation issued under UK Reg 2019/947 Art 12. Referenced but not defined for swarm activity specifically.
verified
Engagement
Used in CAA-US-019 ('an engagement or threat response') — a defence-specific term the civil regulator is unlikely to apply; overloads the civil reviewer's vocabulary.
unverified
Responsible Kilometre / ground environment
The ground area under the aircraft's trajectory. CAA-US-028 uses 'ground-environment survey' but does not define the survey depth, vintage or recency requirement.
unverified
// stories

BUSINESS STORIES

Feature: Application Intake & Completeness Check

As a CAA case officer, I want to receive a complete Operational Authorisation pack so that I can open a valid case.

traces → CAA-US-001, CAA-US-002
Feature: SORA 2.0 Assessment for Swarm

As a CAA SORA assessor, I want swarm-aware SORA evidence so that I can set SAIL and OSO requirements defensibly.

traces → CAA-US-003, CAA-US-004
Feature: ConOps & ODD Review

As a CAA safety-case reviewer, I want a clear ConOps and ODD so that I can judge nominal and worst-credible behaviour.

traces → CAA-US-005, CAA-US-006
Feature: Airspace Change & TDA Coordination

As a CAA airspace regulator, I want a TDA and consultation package so that I can segregate the trial from other users.

traces → CAA-US-007, CAA-US-031
Feature: Geofence / Containment Enforcement

As a CAA airspace regulator, I want automatic containment so that a fly-away cannot become uncontrolled BVLOS.

traces → CAA-US-008
Feature: C2 Link and Spectrum Assurance

As a CAA spectrum specialist, I want authorised, monitored C2 and inter-vehicle links so that link loss is survivable and non-interfering.

traces → CAA-US-009, CAA-US-010
Feature: Detect-and-Avoid Performance

As a CAA DAA assessor, I want DO-365 / F3442-aligned evidence for each aircraft and coordinated swarm de-confliction so that cooperative and non-cooperative encounters are resolved predictably.

traces → CAA-US-011, CAA-US-012
Feature: Swarm Decision-Making Architecture Review

As a CAA autonomy reviewer, I want the decision-making architecture, airborne limits and emergent-behaviour evidence so that I can assure what the swarm will do collectively.

traces → CAA-US-013, CAA-US-014, CAA-US-015
Feature: ML Assurance Case

As a CAA AI/ML assurance specialist, I want an AMLAS / SACE case, a frozen-model manifest and explainability evidence so that ML components are assurable and forensically explainable.

traces → CAA-US-016, CAA-US-017, CAA-US-018
Feature: Human Oversight and Crew Fitness

As a CAA human factors inspector, I want workload, duty and take-over evidence so that meaningful human control is demonstrated throughout.

traces → CAA-US-019, CAA-US-020
Feature: Contingency and Flight Termination

As a CAA flight-ops inspector, I want independent termination, a contingency matrix and ATSU coordination so that the trial can be made safe in any off-nominal state.

traces → CAA-US-021, CAA-US-022, CAA-US-023
Feature: Data, MOR & DPIA Obligations

As a CAA flight safety investigator, I want complete recorded evidence, accepted MOR obligations and a DPIA so that any incident is investigable and privacy is protected.

traces → CAA-US-024, CAA-US-025, CAA-US-026
Feature: Insurance & Ground-Environment Survey

As a CAA authorisation officer, I want insurance and a ground survey so that third-party risk is both transferable and factually grounded.

traces → CAA-US-027, CAA-US-028
Feature: Crew Training & Competency

As a CAA licensing inspector, I want evidenced competency so that the trial is flown by qualified crew.

traces → CAA-US-029
Feature: CAA/MAA Interface Management

As a CAA/MAA liaison, I want a signed apportionment so that regulatory ownership is gap-free.

traces → CAA-US-030, CAA-US-031
Feature: Phased Trial Build-Up

As a CAA programme manager, I want phased gates so that evidence builds progressively and is reversible.

traces → CAA-US-032
Gap: MISSING: Public and Stakeholder Engagement

No user story covers community consultation, press/media handling or prior-notification to landowners. Strongly implied by CAP 1616 and GDPR but absent.

Gap: MISSING: Security Classification Handling

No user story addresses how classified MOD evidence is presented to the CAA under CAP 722H / JSP 440 without compromise to either side.

Feature: MISSING: Environmental Impact

No user story covers noise, species disturbance at dusk/night, or battery-fire environmental risk on ground.

Gap: MISSING: Post-Trial Assurance & Close-out

No user story covers the post-trial lessons-learned, evidence preservation and authorisation close-out.

// scenarios

GIVEN / WHEN / THEN

#1Complete pack submitted
Application Intake & Completeness Check
GIVEN
A CAA case officer receives an OA pack for a BVLOS swarm trial.
WHEN
The officer runs the intake checklist.
THEN
Every required artefact (SORA workbook, ConOps, RAISO letter, CAA/MAA routing memo, safety case, ML manifest, insurance certificate, DPIA) is present or a waiver is recorded, and the case is assigned a review team within 10 working days.
#2Incomplete pack
Application Intake & Completeness Check
GIVEN
The pack is missing the RAISO nomination and the ML deployment manifest.
WHEN
The officer runs the intake checklist.
THEN
The case is not progressed, a deficiency letter is issued, and the applicant is given a defined window to resubmit.
#3Aggregated kinetic energy
SORA 2.0 Assessment for Swarm
GIVEN
The applicant declares N airframes each with mass M and terminal velocity V.
WHEN
The SORA assessor calculates worst-credible simultaneous loss of control.
THEN
The assessor uses the aggregated KE envelope rather than the single-aircraft KE, and the GRC is recomputed accordingly.
#4Common-cause failure
SORA 2.0 Assessment for Swarm
GIVEN
The swarm uses identical hardware and identical autonomy stacks.
WHEN
The assessor evaluates independence of airframe reliability figures.
THEN
Independence is rejected unless the applicant has enumerated common-cause failure modes (software defect, GNSS, RF, adversarial) and mitigated each.
#5Late TDA application
Airspace Change & TDA Coordination
GIVEN
An airspace change is proposed with less than 90 days to the first sortie.
WHEN
The airspace regulator receives the application.
THEN
The application is returned to the applicant because AIP SUP / NOTAM cycles cannot be met; a revised trial start date is required.
#6Geofence breach
Geofence / Containment Enforcement
GIVEN
One airborne member approaches the TDA boundary.
WHEN
Pre-boundary geofence condition triggers.
THEN
Automatic containment executes before the boundary is crossed; the trigger-to-actuation budget is less than the time-to-boundary at max airspeed; NATS and the CAA duty officer are notified within the documented SLA.
#7C2 performance degradation
C2 Link and Spectrum Assurance
GIVEN
C2 link quality drops below the declared threshold.
WHEN
The GCS detects the degradation.
THEN
The RPIC is alerted with time, latency and recoverability data, and the documented contingency (hover, orbit, RTB, terminate) auto-initiates if a secondary threshold is crossed.
#8Inter-vehicle datalink loss
C2 Link and Spectrum Assurance
GIVEN
Two or more members lose the inter-vehicle link.
WHEN
Link timeout expires.
THEN
Each member falls back to standalone DO-365-compliant DAA behaviour, and the swarm degrades gracefully to independent operation.
#9Non-cooperative traffic encounter
Detect-and-Avoid Performance
GIVEN
A GA aircraft without ADS-B enters the Operational Volume.
WHEN
At least one swarm member detects the intruder.
THEN
The coordinated de-confliction protocol resolves a unique, predictable manoeuvre within the reaction-time budget, and no other member creates a cascading conflict.
#10Out-of-distribution input
Swarm Decision-Making Architecture Review
GIVEN
A swarm member encounters a weather or sensor condition outside its ODD or training distribution.
WHEN
The OOD detector fires.
THEN
The member executes a pre-declared safe response (hover, orbit, return, terminate) and the swarm coordinator propagates the state change to all members.
#11Model hash mismatch
ML Assurance Case
GIVEN
A pre-flight deployment check runs against the ML manifest.
WHEN
The hash of a flown model does not match the manifest.
THEN
Flight is prohibited until the discrepancy is resolved, and the occurrence is reported as an ML-specific anomaly.
#12RPIC take-over request
Human Oversight and Crew Fitness
GIVEN
The RPIC issues a take-over command during active autonomy.
WHEN
The command is received.
THEN
The swarm transitions to the commanded state within the declared latency budget and the event is logged with a time-stamp.
#13Concurrent contingencies
Contingency and Flight Termination
GIVEN
A C2 loss and a sensor failure occur on the same airframe within the same decision cycle.
WHEN
The contingency matrix is evaluated.
THEN
Priority ordering applies deterministically (termination > containment > RTB > orbit), and the outcome is logged.
#14Occurrence reporting
Data, MOR & DPIA Obligations
GIVEN
An occurrence that meets Reg 376/2014 criteria happens in flight.
WHEN
Post-flight debrief identifies it.
THEN
An MOR is filed via ECCAIRS within 72 hours.
#15ML-specific anomaly (not a Reg 376 occurrence)
Data, MOR & DPIA Obligations
GIVEN
Emergent swarm behaviour is observed but no safety event results.
WHEN
Post-flight analysis confirms the anomaly.
THEN
A supplementary report is sent to the CAA even though it is not a Reg 376 occurrence.
#16Cross-boundary incident
CAA/MAA Interface Management
GIVEN
An incident straddles civil and military airspace.
WHEN
It is detected.
THEN
The joint CAA/MAA investigation path pre-agreed in the apportionment memo is activated; DSA and RAISO are notified simultaneously.
#17Exit criteria failure
Phased Trial Build-Up
GIVEN
A phase exit criterion is not met.
WHEN
The CAA reviews the evidence.
THEN
The pre-declared regression path is activated — the next phase is not authorised and the applicant returns to a safer phase.
// findings

ANALYST FINDINGS

DO
F-D-01 — The core noun 'swarm' is never formally defined
Found by Dorothy — Definitions Analystundefined_term
criticalC10D
Detail
Every other story leans on this term. Without a definition, 'swarm limits' (CAA-US-014), 'swarm-level DAA' (CAA-US-012) and 'swarm-wide termination' (CAA-US-021) are each interpretable differently.
Current usage
Used as if self-explanatory in every story, but with no lower/upper bounds on member count, no coordination-protocol threshold, and no distinction from 'formation' or 'coordinated flight'.
Proposed definition
A group of two or more uncrewed aircraft operating under a single OA, sharing a common coordination protocol and a single RPIC, such that loss of coordination with the group is itself a safety-relevant event.
PA
F-P-02 — 'Defined latency budget' is used without a budget
Found by Paul — Prediction Analystincomplete_rule
criticalC10P
Detail
A 'budget' without a number is an uncheckable acceptance criterion.
Scenario
Multiple stories (CAA-US-008, CAA-US-012, CAA-US-019, CAA-US-021) cite 'declared latency budget' / 'reaction-time budget' without a numeric value.
Realistic alternative
Follow DO-365C Well-Clear timing (approx. 35 s) for DAA, and DO-377 for C2 latency.
Absurd alternative (DeFOSPAM provocation)
The latency budget is whatever the autonomy stack happens to achieve on the day.
OL
F-O-01 — The 'go/no-go' outcome has no enumerated decision set
Found by Olivia — Outcomes Analystmissing_outcome
criticalC9O
Detail
An outcome that is enumerable as a choice but is not enumerated cannot be implemented consistently across CAA reviewers.
Outcome (output)
CAA-US-032 declares that 'the CAA reviews the evidence and issues an explicit go/no-go' but does not enumerate the set of possible decisions (go, no-go, go-with-conditions, pause, terminate).
OL
F-O-04 — Outcome of a model update during the trial is under-specified
Found by Olivia — Outcomes Analysthanging_outcome
criticalC9O
Detail
Detecting the breach without a stated response risks ad-hoc handling and regulatory inconsistency.
Outcome (state_change)
CAA-US-017 says updates are 'not permitted without a fresh authorisation' — but does not say what happens if an update is detected mid-trial.
SO
F-S-01 — No scenario for partial-swarm return after partial fly-away
Found by Sophia — Scenarios Analystmissing_scenario
criticalC9S
Detail
CAA-US-008 covers the containment of a member and CAA-US-021 covers group termination, but the middle case (one down, rest continue) is unspecified.
Missing scenario
Given the swarm has N members airborne and one experiences a containment event; When that member initiates containment; Then the remaining members' behaviour (continue mission, hold, group-return) must be specified.
PA
F-P-01 — 'Published workload limits' is not predictable
Found by Paul — Prediction Analystunpredictable_outcome
criticalC9P
Detail
Neither the CAA nor EASA publishes workload limits for multi-vehicle uncrewed operations; the acceptance criterion is therefore unprovable as written.
Scenario
CAA-US-019: 'a single RPIC can monitor all airborne members within published workload limits'.
Realistic alternative
The applicant uses NASA-TLX or SART to demonstrate workload under a stated threshold, and proposes a cap based on that.
Absurd alternative (DeFOSPAM provocation)
The limit is whatever the applicant's own CEO decides is acceptable.
MI
F-M-01 — No numeric thresholds for SAIL assignment
Found by Milarna — Missing Data Analystmissing_scenario
criticalC9M
Detail
SAIL is foundational to every downstream OSO level of evidence.
What is missing
The stories invoke SORA SAIL but never specify the expected SAIL band for this trial (likely SAIL III or IV).
Where expected
CAA-US-003 or a new constraint story
Impact if left unaddressed
Applicant and reviewer negotiate SAIL from scratch, costing months.
MI
F-M-03 — No story for pre-flight airworthiness release
Found by Milarna — Missing Data Analystmissing_feature
criticalC9M
Detail
The stories assume CAA is sufficient; for this trial the MAA is the airworthiness regulator.
What is missing
For a military RPAS under RA 1600 and DEFSTAN 00-970 Part 9, a Release-to-Service / Authorised Service Modification is required before first flight. No story captures the airworthiness release chain.
Where expected
Between CAA-US-001 and CAA-US-003
Impact if left unaddressed
Without the release, the trial cannot legally launch even if all CAA evidence is complete.
DO
F-D-02 — 'Autonomous' / 'automated' / 'AI-driven' are used interchangeably
Found by Dorothy — Definitions Analystsynonym_collision
majorC9D
Detail
Assurance method (AMLAS vs. conventional 00-055) hinges on this distinction.
Current usage
CAA-US-013 refers to 'autonomy stack'; CAA-US-016 says 'AI/ML Assurance'; CAA-US-017 says 'ML model'. The collapse of autonomy-level vocabulary obscures which assurance regime applies.
Proposed definition
Use JSP 936 definitions: 'ML component' = trained model; 'deterministic component' = coded rules; 'autonomy level' per ALFUS scale.
DO
F-D-03 — 'Swarm Commander' is coined without a competency reference
Found by Dorothy — Definitions Analystundefined_term
majorC9D
Detail
The competency evidence at CAA-US-029 cannot be assessed without a definition of the role.
Current usage
Introduced in CAA-US-002 and CAA-US-029 as a role distinct from RPIC and Accountable Manager but not defined anywhere in CAP 722.
Proposed definition
A named role holding command authority over swarm-level decisions (formation change, role assignment, group contingency) while the RPIC retains individual-airframe authority.
FL
F-F-01 — Community consultation / public engagement is a missing feature
Found by Flo — Features Analystmissing_feature
majorC9F
Detail
CAP 1616 airspace change explicitly requires consultation. CAA-US-007 mentions 'affected airspace users' but not the public or landowners under the Operational Volume.
Proposed business story
As a CAA airspace regulator, I want evidence that the public, landowners and affected airspace users have been consulted, so that CAP 1616 requirements are met and the trial has social licence.
FL
F-F-02 — Post-trial close-out is a missing feature
Found by Flo — Features Analystmissing_feature
majorC9F
Detail
Every story covers pre- or in-flight activity. No story covers ending the trial, archiving evidence, or feeding lessons back into CAP 722.
Proposed business story
As a CAA programme manager, I want a defined close-out artefact (lessons-learned, evidence preservation, authorisation withdrawal), so that the trial authorisation does not silently over-run its validity.
FL
F-F-04 — Security-classification handling is absent
Found by Flo — Features Analystmissing_feature
majorC9F
Detail
The trial is MOD-sponsored (Alvina context). CAP 722H is cited but no story covers how OFFICIAL-SENSITIVE / SECRET evidence is provided to the CAA.
Proposed business story
As a CAA safety-case reviewer, I want a defined pathway to receive, store and review classified MOD evidence, so that I can assure the trial without either side compromising its security regime.
PA
F-P-03 — 'Demonstrated link margin' has no numeric threshold
Found by Paul — Prediction Analystincomplete_rule
majorC9P
Detail
Without a number the acceptance criterion is not falsifiable.
Scenario
CAA-US-009: 'a minimum link margin has been demonstrated at maximum range'.
Realistic alternative
Require ≥ 6 dB fade margin at maximum range with 95 % availability over the sortie window, per typical CAP 722 practice.
Absurd alternative (DeFOSPAM provocation)
Any positive link budget is acceptable.
AL
F-A-01 — Vague quantifier 'extensive'
Found by Alexa — Ambiguity Analystvague_quantifier
majorC9A
Detail
'Extensive' is the textbook DeFOSPAM weasel word — it shifts the burden of proof to the regulator.
Text excerpt
CAA-US-013: 'extensive Monte Carlo or agent-based simulation'
Interpretation A
Sufficient to demonstrate convergence at 95 % CI over the ODD.
Interpretation B
A weekend run on a laptop because it 'felt extensive'.
AL
F-A-02 — 'Where practicable' is a get-out clause
Found by Alexa — Ambiguity Analystweasel_word
majorC9A
Detail
'Where practicable' without a definition of impracticability permits circular reasoning.
Text excerpt
CAA-US-021: 'functionally and physically independent of the autonomy software and uses a separate RF path where practicable.'
Interpretation A
If the applicant can show a separate RF path is technically infeasible, a shared path is acceptable.
Interpretation B
The applicant self-declares 'not practicable' and the CAA cannot challenge it.
DO
F-D-04 — 'Containment' and 'termination' are used without a hierarchy
Found by Dorothy — Definitions Analystconflicting_definition
majorC8D
Detail
A developer cannot implement the stories without knowing whether containment succeeds include termination or stop short of it.
Current usage
CAA-US-008 uses 'containment' for geofence-triggered actions; CAA-US-021 uses 'termination' as an independent function; CAA-US-022 references both without ordering.
Proposed definition
Containment: keep the aircraft within declared volumes (may include RTB, orbit, autoland). Termination: intentional cessation of flight (forced landing or destructive). Termination is a subset of containment but is always available independently.
FL
F-F-03 — Night operations and dusk/dawn are implicit
Found by Flo — Features Analystimplied_feature
majorC8F
Detail
CAA-US-005 mentions 'daylight' in the ODD example but no story constrains dusk, dawn or night explicitly — yet EO-sensor-based DAA (CAA-US-011) is strongly light-dependent.
Proposed business story
As a CAA flight-ops inspector, I want the authorisation to state permitted light levels, so that the DAA / EO sensor performance envelope is matched to the actual flight window.
OL
F-O-02 — Null outcome for inadvertent personal-data capture is not stated
Found by Olivia — Outcomes Analystnull_outcome
majorC8O
Detail
Without a declared in-flight response, post-flight redaction is the only control — too late if storage itself is the risk.
Outcome (state_change)
CAA-US-026 asks for a DPIA and redaction procedure but does not state what the system does at the moment of inadvertent capture (e.g. whether the sensor continues recording or auto-masks).
SO
F-S-02 — No scenario for concurrent DAA and C2-loss events
Found by Sophia — Scenarios Analystmissing_scenario
majorC8S
Detail
CAA-US-022 lists the matrix generically but does not test this combination explicitly.
Missing scenario
Given a DAA conflict has been detected and a C2 link loss occurs within the resolution time-budget; When the swarm executes a cooperative avoidance manoeuvre; Then the avoidance manoeuvre must complete autonomously without RPIC concurrence.
SO
F-S-03 — No scenario for operations in reduced visibility or icing
Found by Sophia — Scenarios Analystedge_case
majorC8S
Detail
CAA-US-005 mentions wind and visibility as ODD parameters but does not specify the response to degraded conditions.
Missing scenario
Given visibility or icing conditions exceed the declared ODD; When any member encounters these conditions; Then a coordinated response moves the swarm out of the deteriorating volume or lands it.
PA
F-P-04 — 'Confidence intervals' without a level are unpredictable
Found by Paul — Prediction Analystunpredictable_outcome
majorC8P
Detail
The acceptance criterion rewards or penalises applicants arbitrarily.
Scenario
CAA-US-016 requires ML metrics 'expressed as confidence intervals' but no level (90 %, 95 %, 99 %) is stated.
Realistic alternative
Require 95 % CI or matched to SAIL level.
Absurd alternative (DeFOSPAM provocation)
Any non-trivial interval is accepted.
MI
F-M-02 — No CRUD coverage for authorisation conditions
Found by Milarna — Missing Data Analystmissing_crud
majorC8M
Detail
The OA is a living instrument; the stories treat it as static.
What is missing
Stories describe issue of conditions (create) and review at phase gates (read), but there is no 'update a condition mid-trial' or 'withdraw an authorisation' story.
Where expected
Adjacent to CAA-US-032
Impact if left unaddressed
Neither the CAA nor the applicant knows how conditions change when new evidence emerges.
MI
F-M-04 — No non-functional performance envelope declared
Found by Milarna — Missing Data Analystmissing_nfr
majorC8M
Detail
DO-178C, DEFSTAN 00-055 and AMLAS all require timing assurance for safety-relevant software.
What is missing
Performance NFRs for the autonomy stack (decision cycle period, worst-case execution time, CPU/GPU utilisation) are absent.
Where expected
CAA-US-013 or a new story
Impact if left unaddressed
A timing-induced failure (deadline miss) is indistinguishable from an algorithmic failure in post-incident analysis.
MI
F-M-05 — No cyber-resilience test story
Found by Milarna — Missing Data Analystmissing_feature
majorC8M
Detail
The autonomy stack, GCS and C2 link are all soft attack surfaces.
What is missing
CAP 722H is cited in passing but no story asks for penetration testing, a cyber-incident response plan, or SBOM.
Where expected
Adjacent to CAA-US-004 / CAA-US-009
Impact if left unaddressed
A cyber event during the trial has no pre-declared containment.
MI
F-M-07 — No authority to inspect, suspend or revoke between phases
Found by Milarna — Missing Data Analystmissing_feature
majorC8M
Detail
CAP 722 and ANO give the CAA powers of inspection; the stories do not exercise them.
What is missing
The CAA's right to inspect the applicant's facility, suspend flights pending inspection, or revoke between phases is absent.
Where expected
CAA-US-032 or a new powers story
Impact if left unaddressed
Phase gating without inspection is a rubber-stamp exercise.
OL
F-O-03 — No outcome stated when the applicant refuses an authorisation condition
Found by Olivia — Outcomes Analystmissing_outcome
majorC7O
Detail
The CAA's escalation path (refuse OA, offer a reduced scope OA, refer to MAA) is itself an outcome class.
Outcome (output)
The stories describe conditions the CAA will impose (CAA-US-008 SLA, CAA-US-014 numeric caps, CAA-US-017 model freeze) but never state what happens if the applicant refuses.
SO
F-S-04 — No data-driven scenario table for airborne-count limits
Found by Sophia — Scenarios Analystdata_driven
majorC7S
Detail
CAA-US-014 specifies the limit as a numeric condition but does not table the enforcement behaviour.
Missing scenario
Scenario Outline: airborne count limits — Given N airborne members, When a Nth+1 launch is attempted, Then the launch is inhibited at the GCS.
PA
F-P-05 — 'Monte Carlo statistics' without sample-size floor
Found by Paul — Prediction Analystunpredictable_outcome
majorC7P
Detail
Without a floor the regulator cannot compare applicants.
Scenario
CAA-US-013 asks for 'behavioural envelope statistics' from Monte Carlo or agent-based simulation.
Realistic alternative
Require sample size sufficient to estimate failure rate at SAIL-appropriate confidence (e.g. 3 × 10⁻⁴ for SAIL IV).
Absurd alternative (DeFOSPAM provocation)
A single simulation run is treated as evidence.
MI
F-M-06 — No story for human-in-the-loop testing of the GCS
Found by Milarna — Missing Data Analystmissing_feature
majorC7M
Detail
Measured workload under realistic failure scenarios is the HF community's expected evidence.
What is missing
CAA-US-019 references workload limits but does not require live human-in-the-loop testing of the GCS with the operational crew under off-nominal scenarios.
Where expected
Adjacent to CAA-US-019/020
Impact if left unaddressed
Workload analysis is modelled but not measured.
DO
F-D-05 — The defence term 'engagement' is used in a civil-regulator context
Found by Dorothy — Definitions Analystundefined_term
minorC8D
Detail
Mixing civil and defence vocabulary invites misreading and undermines the CAA/MAA apportionment memo in CAA-US-030.
Current usage
CAA-US-019 acceptance criterion references 'an engagement or threat response' — this is defence vocabulary that does not map onto CAA decision-making.
Proposed definition
Replace with 'a safety-critical or use-of-force decision' so the CAA reviewer does not have to interpret kinetic terminology.
AL
F-A-03 — 'Pre-briefed' vs. 'pre-declared' used inconsistently
Found by Alexa — Ambiguity Analystinconsistency
minorC8A
Detail
Auditability hinges on this distinction — oral briefings are not evidence.
Text excerpt
CAA-US-023 uses 'pre-briefed'; CAA-US-008, CAA-US-015 and CAA-US-022 use 'pre-declared'. CAA-US-032 uses 'pre-declared'.
Interpretation A
Pre-briefed means an oral briefing occurred; pre-declared means a written declaration exists.
Interpretation B
They are the same thing and the drafter switched words for variety.
FL
F-F-05 — Environmental and noise assessment is missing
Found by Flo — Features Analystmissing_feature
minorC7F
Detail
CAA-US-028 touches ground environment but not wider environmental impact.
Proposed business story
As a CAA environmental policy team member, I want an environmental assessment (noise, species disturbance, battery-fire ground risk), so that the trial does not breach DEFRA / Natural England expectations.
AL
F-A-04 — Weasel word 'reasonable' hidden in story 029
Found by Alexa — Ambiguity Analystweasel_word
minorC7A
Detail
'Recent' without a period is the most common currency-definition trap in UK aviation.
Text excerpt
CAA-US-029: 'recent simulator and live-flying currency records exist'.
Interpretation A
Recent = within the last 90 days per CAP 722 analogues.
Interpretation B
Recent = any time this calendar year.
AL
F-A-05 — 'Significant multipath' contains undefined adjective
Found by Alexa — Ambiguity Analystambiguous_language
minorC7A
Detail
The phrase passes any review that the applicant writes.
Text excerpt
CAA-US-009: 'fade and multipath analysis' — the absence of a threshold turns the analysis into a narrative, not a pass/fail.
Interpretation A
An FDTD-based multipath simulation against the terrain model, at 1 m resolution.
Interpretation B
A back-of-envelope calculation of expected reflection loss.
AL
F-A-06 — 'Declared' vs. 'documented' used as synonyms but differ in legal weight
Found by Alexa — Ambiguity Analystinconsistency
minorC7A
Detail
Audit evidence turns on this distinction.
Text excerpt
Multiple stories use both words. In regulator practice, a 'declaration' is a regulator-facing legal instrument; 'documented' can be internal-only.
Interpretation A
Declared items must be counter-signed; documented items need only exist in the applicant's files.
Interpretation B
They are treated as interchangeable by the reviewer.
MI
F-M-08 — No explicit evidence-preservation period post-trial
Found by Milarna — Missing Data Analystmissing_feature
minorC7M
Detail
AMLAS and CAP 722 both expect long-term evidence preservation.
What is missing
CAA-US-024 sets retention for flight data, but wider evidence (assurance case, training records, ML manifests) has no retention period.
Where expected
CAA-US-024 or close-out story
Impact if left unaddressed
Post-trial re-investigation (e.g. if a failure manifests later) cannot be supported.