Sequence view of uss_qualifier test run

Test run

Test characteristic Value
Test run identifier TR-67e83e0
Start time 2024-10-14 12:06:11 UTC
End time 2024-10-14 12:08:57 UTC
Test baseline identifier TB-e968fca
Environment identifier TE-03ef179
Codebase version interuss/monitoring/v0.0.0-2ca7dee
Commit hash 2ca7dee5c4f881fe08c6d4572a40c755b1d98ed6
Artifact generated by interuss/monitoring/v0.0.0-2ca7dee

Resources configuration

  • Baseline
    :
    • kentland_service_area (resources.netrid.ServiceAreaResource)
      :
      • Specification
        :
        • base_url
          :
          https://testdummy.interuss.org/interuss/monitoring/uss_qualifier/configurations/dev/library/resources/kentland_service_area
        • footprint
          :
          • item_1
            :
            • lat
              :
              37.1853
            • lng
              :
              -80.614
          • item_2
            :
            • lat
              :
              37.2148
            • lng
              :
              -80.614
          • item_3
            :
            • lat
              :
              37.2148
            • lng
              :
              -80.544
          • item_4
            :
            • lat
              :
              37.1853
            • lng
              :
              -80.544
        • altitude_min
          :
          0.0
        • altitude_max
          :
          3048.0
        • reference_time
          :
          2023-01-10T00:00:00.123456+00:00
        • time_start
          :
          2023-01-10T00:00:01.123456+00:00
        • time_end
          :
          2023-01-10T01:00:01.123456+00:00
    • kentland_planning_area (resources.astm.f3548.v21.PlanningAreaResource)
      :
      • Specification
        :
        • base_url
          :
          https://testdummy.interuss.org/interuss/monitoring/uss_qualifier/configurations/dev/library/resources/kentland_planning_area
        • volume
          :
          • outline_polygon
            :
            • vertices
              :
              • item_1
                :
                • lat
                  :
                  37.1853
                • lng
                  :
                  -80.614
              • item_2
                :
                • lat
                  :
                  37.2148
                • lng
                  :
                  -80.614
              • item_3
                :
                • lat
                  :
                  37.2148
                • lng
                  :
                  -80.544
              • item_4
                :
                • lat
                  :
                  37.1853
                • lng
                  :
                  -80.544
          • altitude_lower
            :
            • value
              :
              0.0
            • reference
              :
              W84
            • units
              :
              M
          • altitude_upper
            :
            • value
              :
              3048.0
            • reference
              :
              W84
            • units
              :
              M
          • outline_circle
            :
            None
    • kentland_problematically_big_area (resources.VerticesResource)
      :
      • Specification
        :
        • vertices
          :
          • item_1
            :
            • lat
              :
              38.0
            • lng
              :
              -81.0
          • item_2
            :
            • lat
              :
              37.0
            • lng
              :
              -81.0
          • item_3
            :
            • lat
              :
              37.0
            • lng
              :
              -80.0
          • item_4
            :
            • lat
              :
              38.0
            • lng
              :
              -80.0
    • utm_client_identity (resources.communications.ClientIdentityResource)
      :
      • Specification
        :
        • whoami_audience
          :
          localhost
        • whoami_scope
          :
          rid.display_provider
      • Dependencies
        :
        • auth_adapter
          :
          From utm_auth resource
    • id_generator (resources.interuss.IDGeneratorResource)
      :
      • Specification
        :
      • Dependencies
        :
        • client_identity
          :
          From utm_client_identity resource
    • che_non_conflicting_flights (resources.flight_planning.FlightIntentsResource)
      :
      • Specification
        :
        • file
          :
          • path
            :
            test_data.flight_intents.standard.non_conflicting
          • hash_sha512
            :
            9e49a876b0aaf78e224a6851900b06bb57f6d3b4ec6daaf7166d48a0d5e6450e595ff396c7c0c34f2ad87ea61d0f765490c38fb79573ffa921465b0750c3074a
        • transformations
          :
          • item_1
            :
            • relative_translation
              :
              • degrees_north
                :
                46.9748
              • degrees_east
                :
                7.4774
              • meters_up
                :
                605.0
    • test_exclusions (resources.dev.TestExclusionsResource)
      :
      • Specification
        :
        • allow_private_addresses
          :
          True
        • allow_cleartext_queries
          :
          True
  • Environment
    :
    • utm_auth (resources.communications.AuthAdapterResource)
      :
      • Specification
        :
        • environment_variable_containing_auth_spec
          :
          AUTH_SPEC
        • scopes_authorized
          :
          • item_1
            :
            rid.inject_test_data
          • item_2
            :
            dss.read.identification_service_areas
          • item_3
            :
            rid.service_provider
          • item_4
            :
            rid.display_provider
          • item_5
            :
            dss.write.identification_service_areas
          • item_6
            :
            dss.read.identification_service_areas
          • item_7
            :
            interuss.flight_planning.direct_automated_test
          • item_8
            :
            interuss.flight_planning.plan
          • item_9
            :
            utm.inject_test_data
          • item_10
            :
            utm.strategic_coordination
          • item_11
            :
            utm.conformance_monitoring_sa
          • item_12
            :
            utm.availability_arbitration
          • item_13
            :
            utm.constraint_management
          • item_14
            :
            interuss.versioning.read_system_versions
          • item_15
            :
            interuss.geospatial_map.query
          • item_16
            :
    • second_utm_auth (resources.communications.AuthAdapterResource)
      :
      • Specification
        :
        • environment_variable_containing_auth_spec
          :
          AUTH_SPEC_2
        • scopes_authorized
          :
          • item_1
            :
            utm.strategic_coordination
    • scd_dss_instances (resources.astm.f3548.v21.DSSInstancesResource)
      :
      • Specification
        :
        • dss_instances
          :
          • item_1
            :
            • participant_id
              :
              uss1
            • user_participant_ids
              :
              • item_1
                :
                mock_uss
            • base_url
              :
              http://dss.uss1.localutm
          • item_2
            :
            • participant_id
              :
              uss2
            • base_url
              :
              http://dss.uss2.localutm
      • Dependencies
        :
        • auth_adapter
          :
          From utm_auth resource
    • netrid_dss_instances_v22a (resources.astm.f3411.DSSInstancesResource)
      :
      • Specification
        :
        • dss_instances
          :
          • item_1
            :
            • participant_id
              :
              uss1
            • rid_version
              :
              F3411-22a
            • base_url
              :
              http://dss.uss1.localutm/rid/v2
          • item_2
            :
            • participant_id
              :
              uss2
            • rid_version
              :
              F3411-22a
            • base_url
              :
              http://dss.uss2.localutm/rid/v2
      • Dependencies
        :
        • auth_adapter
          :
          From utm_auth resource
    • netrid_dss_instances_v19 (resources.astm.f3411.DSSInstancesResource)
      :
      • Specification
        :
        • dss_instances
          :
          • item_1
            :
            • participant_id
              :
              uss1
            • rid_version
              :
              F3411-19
            • base_url
              :
              http://dss.uss1.localutm
          • item_2
            :
            • participant_id
              :
              uss2
            • rid_version
              :
              F3411-19
            • base_url
              :
              http://dss.uss2.localutm
      • Dependencies
        :
        • auth_adapter
          :
          From utm_auth resource

Full configuration

  • v1
    :
    • test_run
      :
      • resources
        :
        • resource_declarations
          :
          • kentland_service_area
            :
            • resource_type
              :
              resources.netrid.ServiceAreaResource
            • specification
              :
              • base_url
                :
                https://testdummy.interuss.org/interuss/monitoring/uss_qualifier/configurations/dev/library/resources/kentland_service_area
              • footprint
                :
                • item_1
                  :
                  • lat
                    :
                    37.1853
                  • lng
                    :
                    -80.614
                • item_2
                  :
                  • lat
                    :
                    37.2148
                  • lng
                    :
                    -80.614
                • item_3
                  :
                  • lat
                    :
                    37.2148
                  • lng
                    :
                    -80.544
                • item_4
                  :
                  • lat
                    :
                    37.1853
                  • lng
                    :
                    -80.544
              • altitude_min
                :
                0.0
              • altitude_max
                :
                3048.0
              • reference_time
                :
                2023-01-10T00:00:00.123456+00:00
              • time_start
                :
                2023-01-10T00:00:01.123456+00:00
              • time_end
                :
                2023-01-10T01:00:01.123456+00:00
            • dependencies
              :
          • kentland_planning_area
            :
            • resource_type
              :
              resources.astm.f3548.v21.PlanningAreaResource
            • specification
              :
              • base_url
                :
                https://testdummy.interuss.org/interuss/monitoring/uss_qualifier/configurations/dev/library/resources/kentland_planning_area
              • volume
                :
                • outline_polygon
                  :
                  • vertices
                    :
                    • item_1
                      :
                      • lat
                        :
                        37.1853
                      • lng
                        :
                        -80.614
                    • item_2
                      :
                      • lat
                        :
                        37.2148
                      • lng
                        :
                        -80.614
                    • item_3
                      :
                      • lat
                        :
                        37.2148
                      • lng
                        :
                        -80.544
                    • item_4
                      :
                      • lat
                        :
                        37.1853
                      • lng
                        :
                        -80.544
                • altitude_lower
                  :
                  • value
                    :
                    0.0
                  • reference
                    :
                    W84
                  • units
                    :
                    M
                • altitude_upper
                  :
                  • value
                    :
                    3048.0
                  • reference
                    :
                    W84
                  • units
                    :
                    M
                • outline_circle
                  :
                  None
            • dependencies
              :
          • kentland_problematically_big_area
            :
            • resource_type
              :
              resources.VerticesResource
            • specification
              :
              • vertices
                :
                • item_1
                  :
                  • lat
                    :
                    38.0
                  • lng
                    :
                    -81.0
                • item_2
                  :
                  • lat
                    :
                    37.0
                  • lng
                    :
                    -81.0
                • item_3
                  :
                  • lat
                    :
                    37.0
                  • lng
                    :
                    -80.0
                • item_4
                  :
                  • lat
                    :
                    38.0
                  • lng
                    :
                    -80.0
            • dependencies
              :
          • utm_auth
            :
            • resource_type
              :
              resources.communications.AuthAdapterResource
            • specification
              :
              • environment_variable_containing_auth_spec
                :
                AUTH_SPEC
              • scopes_authorized
                :
                • item_1
                  :
                  rid.inject_test_data
                • item_2
                  :
                  dss.read.identification_service_areas
                • item_3
                  :
                  rid.service_provider
                • item_4
                  :
                  rid.display_provider
                • item_5
                  :
                  dss.write.identification_service_areas
                • item_6
                  :
                  dss.read.identification_service_areas
                • item_7
                  :
                  interuss.flight_planning.direct_automated_test
                • item_8
                  :
                  interuss.flight_planning.plan
                • item_9
                  :
                  utm.inject_test_data
                • item_10
                  :
                  utm.strategic_coordination
                • item_11
                  :
                  utm.conformance_monitoring_sa
                • item_12
                  :
                  utm.availability_arbitration
                • item_13
                  :
                  utm.constraint_management
                • item_14
                  :
                  interuss.versioning.read_system_versions
                • item_15
                  :
                  interuss.geospatial_map.query
                • item_16
                  :
            • dependencies
              :
          • second_utm_auth
            :
            • resource_type
              :
              resources.communications.AuthAdapterResource
            • specification
              :
              • environment_variable_containing_auth_spec
                :
                AUTH_SPEC_2
              • scopes_authorized
                :
                • item_1
                  :
                  utm.strategic_coordination
            • dependencies
              :
          • utm_client_identity
            :
            • resource_type
              :
              resources.communications.ClientIdentityResource
            • dependencies
              :
              • auth_adapter
                :
                utm_auth
            • specification
              :
              • whoami_audience
                :
                localhost
              • whoami_scope
                :
                rid.display_provider
          • id_generator
            :
            • resource_type
              :
              resources.interuss.IDGeneratorResource
            • dependencies
              :
              • client_identity
                :
                utm_client_identity
            • specification
              :
          • scd_dss_instances
            :
            • resource_type
              :
              resources.astm.f3548.v21.DSSInstancesResource
            • dependencies
              :
              • auth_adapter
                :
                utm_auth
            • specification
              :
              • dss_instances
                :
                • item_1
                  :
                  • participant_id
                    :
                    uss1
                  • user_participant_ids
                    :
                    • item_1
                      :
                      mock_uss
                  • base_url
                    :
                    http://dss.uss1.localutm
                • item_2
                  :
                  • participant_id
                    :
                    uss2
                  • base_url
                    :
                    http://dss.uss2.localutm
          • netrid_dss_instances_v22a
            :
            • resource_type
              :
              resources.astm.f3411.DSSInstancesResource
            • dependencies
              :
              • auth_adapter
                :
                utm_auth
            • specification
              :
              • dss_instances
                :
                • item_1
                  :
                  • participant_id
                    :
                    uss1
                  • rid_version
                    :
                    F3411-22a
                  • base_url
                    :
                    http://dss.uss1.localutm/rid/v2
                • item_2
                  :
                  • participant_id
                    :
                    uss2
                  • rid_version
                    :
                    F3411-22a
                  • base_url
                    :
                    http://dss.uss2.localutm/rid/v2
          • netrid_dss_instances_v19
            :
            • resource_type
              :
              resources.astm.f3411.DSSInstancesResource
            • dependencies
              :
              • auth_adapter
                :
                utm_auth
            • specification
              :
              • dss_instances
                :
                • item_1
                  :
                  • participant_id
                    :
                    uss1
                  • rid_version
                    :
                    F3411-19
                  • base_url
                    :
                    http://dss.uss1.localutm
                • item_2
                  :
                  • participant_id
                    :
                    uss2
                  • rid_version
                    :
                    F3411-19
                  • base_url
                    :
                    http://dss.uss2.localutm
          • che_non_conflicting_flights
            :
            • resource_type
              :
              resources.flight_planning.FlightIntentsResource
            • specification
              :
              • file
                :
                • path
                  :
                  test_data.flight_intents.standard.non_conflicting
                • hash_sha512
                  :
                  9e49a876b0aaf78e224a6851900b06bb57f6d3b4ec6daaf7166d48a0d5e6450e595ff396c7c0c34f2ad87ea61d0f765490c38fb79573ffa921465b0750c3074a
              • transformations
                :
                • item_1
                  :
                  • relative_translation
                    :
                    • degrees_north
                      :
                      46.9748
                    • degrees_east
                      :
                      7.4774
                    • meters_up
                      :
                      605.0
            • dependencies
              :
          • test_exclusions
            :
            • resource_type
              :
              resources.dev.TestExclusionsResource
            • specification
              :
              • allow_private_addresses
                :
                True
              • allow_cleartext_queries
                :
                True
            • dependencies
              :
      • non_baseline_inputs
        :
        • item_1
          :
          v1.test_run.resources.resource_declarations.utm_auth
        • item_2
          :
          v1.test_run.resources.resource_declarations.second_utm_auth
        • item_3
          :
          v1.test_run.resources.resource_declarations.scd_dss_instances
        • item_4
          :
          v1.test_run.resources.resource_declarations.netrid_dss_instances_v22a
        • item_5
          :
          v1.test_run.resources.resource_declarations.netrid_dss_instances_v19
      • action
        :
        • test_suite
          :
          • suite_type
            :
            suites.interuss.dss.all_tests
          • resources
            :
            • f3411v19_dss_instances
              :
              netrid_dss_instances_v19
            • f3411v22a_dss_instances
              :
              netrid_dss_instances_v22a
            • f3548v21_dss_instances
              :
              scd_dss_instances
            • utm_client_identity
              :
              utm_client_identity
            • id_generator
              :
              id_generator
            • service_area
              :
              kentland_service_area
            • planning_area
              :
              kentland_planning_area
            • problematically_big_area
              :
              kentland_problematically_big_area
            • second_utm_auth
              :
              second_utm_auth
            • flight_intents
              :
              che_non_conflicting_flights
            • test_exclusions
              :
              test_exclusions
        • on_failure
          :
          ReactionToFailure.Continue
      • execution
        :
        • stop_fast
          :
          True
        • stop_when_resource_not_created
          :
          False
        • include_action_when
          :
          None
        • skip_action_when
          :
          None
    • artifacts
      :
      • raw_report
        :
        • redact_access_tokens
          :
          True
        • indent
          :
          None
      • sequence_view
        :
        • redact_access_tokens
          :
          True
        • render_kml
          :
          True
      • tested_requirements
        :
        • item_1
          :
          • report_name
            :
            requirements
          • requirement_collections
            :
            • all_astm_dss_requirements
              :
              • requirement_collections
                :
                • item_1
                  :
                  • requirement_sets
                    :
                    • item_1
                      :
                      astm.f3411.v22a.dss_provider
                    • item_2
                      :
                      astm.f3411.v19.dss_provider
                    • item_3
                      :
                      astm.f3548.v21.dss_provider
          • participant_requirements
            :
            • uss1
              :
              None
            • uss2
              :
              all_astm_dss_requirements
      • templated_reports
        :
        None
      • report_html
        :
        None
    • validation
      :
      • criteria
        :
        • item_1
          :
          • applicability
            :
            • test_scenarios
              :
          • pass_condition
            :
            • each_element
              :
              • has_execution_error
                :
                False
        • item_2
          :
          • applicability
            :
            • failed_checks
              :
              • has_severity
                :
                • higher_than
                  :
                  Severity.Low
          • pass_condition
            :
            • elements
              :
              • count
                :
                • equal_to
                  :
                  0.0
        • item_3
          :
          • applicability
            :
            • skipped_actions
              :
          • pass_condition
            :
            • elements
              :
              • count
                :
                • equal_to
                  :
                  6.0

Scenarios executed

Suite / action generator Scenario mock_uss uss1 uss2 <None> Time
ASTM DSS tests For each F3548 DSS instance DSS testing for ASTM NetRID F3548-21 1) ASTM SCD DSS: Implicit Subscription handling 0:01
2) ASTM SCD DSS: Operational Intent Reference Simple 0:00
3) ASTM SCD DSS: Constraint Reference Simple 0:00
4) ASTM SCD DSS: Constraint Reference Synchronization 0:01
5) ASTM SCD DSS: USS Availability Synchronization 0:00
6) ASTM F3548-21 UTM DSS Operational Intent Reference State Transitions 0:00
7) ASTM SCD DSS: Subscription and entity deletion interaction 0:00
8) ASTM SCD DSS: Subscription and entity interaction 0:07
9) ASTM SCD DSS: Operational Intent Reference Key Validation 0:00
10) ASTM SCD DSS: Operational Intent Reference Synchronization 0:00
11) ASTM SCD DSS: Interfaces authentication 0:01
12) ASTM SCD DSS: Subscription Simple 0:01
13) ASTM SCD DSS: Subscription Validation 0:00
14) ASTM F3548-21 UTM DSS Operational Intent Reference Access Control 0:00
15) ASTM F3548-21 UTM DSS interoperability 0:00
16) ASTM SCD DSS: Subscription Synchronization 0:00

Skipped: ASTM UTM DSS: Direct CRDB access

Test suite action to run ActionType.TestScenario scenarios.astm.utm.dss.CRDBAccess could not find required resource ID "dss_crdb_cluster" used to populate child resource ID "crdb_cluster"

17) ASTM SCD DSS: Report 0:00
DSS testing for ASTM NetRID F3548-21 18) ASTM SCD DSS: Implicit Subscription handling 0:00
19) ASTM SCD DSS: Operational Intent Reference Simple 0:00
20) ASTM SCD DSS: Constraint Reference Simple 0:00
21) ASTM SCD DSS: Constraint Reference Synchronization 0:00
22) ASTM SCD DSS: USS Availability Synchronization 0:00
23) ASTM F3548-21 UTM DSS Operational Intent Reference State Transitions 0:00
24) ASTM SCD DSS: Subscription and entity deletion interaction 0:00
25) ASTM SCD DSS: Subscription and entity interaction 0:07
26) ASTM SCD DSS: Operational Intent Reference Key Validation 0:00
27) ASTM SCD DSS: Operational Intent Reference Synchronization 0:00
28) ASTM SCD DSS: Interfaces authentication 0:01
29) ASTM SCD DSS: Subscription Simple 0:01
30) ASTM SCD DSS: Subscription Validation 0:00
31) ASTM F3548-21 UTM DSS Operational Intent Reference Access Control 0:00
32) ASTM F3548-21 UTM DSS interoperability 0:00
33) ASTM SCD DSS: Subscription Synchronization 0:01

Skipped: ASTM UTM DSS: Direct CRDB access

Test suite action to run ActionType.TestScenario scenarios.astm.utm.dss.CRDBAccess could not find required resource ID "dss_crdb_cluster" used to populate child resource ID "crdb_cluster"

34) ASTM SCD DSS: Report 0:00
For each F3411 DSS instance DSS testing for ASTM NetRID F3411-19 35) ASTM NetRID DSS: Simple ISA 0:00
36) ASTM NetRID DSS: Submitted ISA Validations 0:00
37) ASTM NetRID DSS: ISA Expiry 0:05
38) ASTM NetRID DSS: ISA Subscription Interactions 0:00
39) ASTM NetRID DSS: Subscription Validation 0:00
40) ASTM NetRID DSS: Subscription Simple 0:00
41) ASTM F3411-19 NetRID DSS interoperability 🌐 0:11
42) ASTM NetRID DSS: Token Validation 0:00

Skipped: ASTM NetRID DSS: Direct CRDB access

Test suite action to run ActionType.TestScenario scenarios.astm.netrid.v19.dss.CRDBAccess could not find required resource ID "dss_crdb_cluster" used to populate child resource ID "crdb_cluster"

43) ASTM NetRID DSS: Concurrent Requests 0:16
DSS testing for ASTM NetRID F3411-19 44) ASTM NetRID DSS: Simple ISA 0:00
45) ASTM NetRID DSS: Submitted ISA Validations 0:00
46) ASTM NetRID DSS: ISA Expiry 0:05
47) ASTM NetRID DSS: ISA Subscription Interactions 0:00
48) ASTM NetRID DSS: Subscription Validation 0:00
49) ASTM NetRID DSS: Subscription Simple 0:00
50) ASTM F3411-19 NetRID DSS interoperability 🌐 0:11
51) ASTM NetRID DSS: Token Validation 0:00

Skipped: ASTM NetRID DSS: Direct CRDB access

Test suite action to run ActionType.TestScenario scenarios.astm.netrid.v19.dss.CRDBAccess could not find required resource ID "dss_crdb_cluster" used to populate child resource ID "crdb_cluster"

52) ASTM NetRID DSS: Concurrent Requests 0:16
For each F3411 DSS instance DSS testing for ASTM NetRID F3411-22a 53) ASTM NetRID DSS: Simple ISA 0:00
54) ASTM NetRID DSS: Submitted ISA Validations 0:00
55) ASTM NetRID DSS: ISA Expiry 0:05
56) ASTM NetRID DSS: ISA Subscription Interactions 0:00
57) ASTM NetRID DSS: Subscription Validation 0:00
58) ASTM NetRID DSS: Subscription Simple 0:00
59) ASTM F3411-22a NetRID DSS interoperability 🌐 0:11
60) ASTM NetRID DSS: Token Validation 0:00

Skipped: ASTM NetRID DSS: Direct CRDB access

Test suite action to run ActionType.TestScenario scenarios.astm.netrid.v22a.dss.CRDBAccess could not find required resource ID "dss_crdb_cluster" used to populate child resource ID "crdb_cluster"

61) ASTM NetRID DSS: Concurrent Requests 0:22
DSS testing for ASTM NetRID F3411-22a 62) ASTM NetRID DSS: Simple ISA 0:00
63) ASTM NetRID DSS: Submitted ISA Validations 0:00
64) ASTM NetRID DSS: ISA Expiry 0:05
65) ASTM NetRID DSS: ISA Subscription Interactions 0:00
66) ASTM NetRID DSS: Subscription Validation 0:00
67) ASTM NetRID DSS: Subscription Simple 0:00
68) ASTM F3411-22a NetRID DSS interoperability 🌐 0:11
69) ASTM NetRID DSS: Token Validation 0:00

Skipped: ASTM NetRID DSS: Direct CRDB access

Test suite action to run ActionType.TestScenario scenarios.astm.netrid.v22a.dss.CRDBAccess could not find required resource ID "dss_crdb_cluster" used to populate child resource ID "crdb_cluster"

70) ASTM NetRID DSS: Concurrent Requests 0:21