test: OVOS-PIPELINE-1/STOP-1/INTENT-4 conformance suites by JarbasAl · Pull Request #765 · OpenVoiceOS/ovos-core

JarbasAl · 2026-06-05T11:45:43Z

What

Two commits:

test: — clause-labelled ovoscope conformance suites for OVOS-PIPELINE-1,
OVOS-STOP-1, OVOS-INTENT-4 (test/end2end/test_*_conformance.py).
feat: — a deployment-wide legacy_namespace config flag (default True)
selecting the bus-topic namespace during the mycroft.* → ovos.* transition. The
orchestrator and stop pipeline emit exactly one namespace (never both);
subscribers listen on both.

Logical event	legacy	spec (`legacy_namespace: false`)
entry (§9.1)	`recognizer_loop:utterance`	also subscribes `ovos.utterance.handle`
no-match (§9.3)	`complete_intent_failure`	`ovos.intent.unmatched`
match notif (§9.2) / `pipeline_id` (§7.1)	—	`ovos.intent.matched` / `context.pipeline_id`
stop broadcast (§5.3)	`mycroft.stop`	`ovos.stop`
stop ping (§4.2)	per-skill `{skill}.stop.ping`	broadcast `ovos.stop.ping`

The dispatch topic <skill_id>:<intent_name> and the ovos.utterance.handled end-marker
are unchanged (the spec keeps both).

Conformance results

Run in the spec namespace (legacy_namespace=False): 21 passed, 7 xfail, 0 failed.
Remaining xfail: STOP-1 <id>:global_stop dispatch rename, STOP-1 reserved-stop
rejection, and the whole OVOS-INTENT-4 registration/introspection contract (larger,
future work). Legacy-namespace end2end tests are unchanged and green (no test hacks).

The §8 handler-trio and §9.6 speak conformance tests skipif an ovos-workshop without the
companion flag is installed, so core CI stays green until ovos-workshop#415 releases.

Companions

ovos-workshop#415 (merge first), ovos-dinkum-listener, ovos-simple-listener, ovos-audio.

🤖 Generated with Claude Code

coderabbitai · 2026-06-05T11:45:51Z

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 162f6f4a-336f-4868-90e6-bd7eec150aaa

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch test/spec-conformance-suites

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-06-05T11:46:21Z

The automated pipeline has reached its destination. 🏁

I've aggregated the results of the automated checks for this PR below.

📋 Repo Health

Checking for any potential maintenance bottlenecks. 🚧

✅ All required files present.

Latest Version: 2.2.0a2

✅ ovos_core/version.py — Version file
✅ README.md — README
✅ LICENSE — License file
✅ pyproject.toml — pyproject.toml
⚠️ setup.py — setup.py
✅ CHANGELOG.md — Changelog
✅ ovos_core/version.py has valid version block markers

🏷️ Release Preview

Setting the stage for the upcoming deployment. 🎭

Current: 2.2.0a2 → Next: 2.2.0a3

Signal	Value
Label	(none)
PR title	`test: OVOS-PIPELINE-1/STOP-1/INTENT-4 conformance suites`
Bump	alpha

✅ PR title follows conventional commit format.

🚀 Release Channel Compatibility

Predicted next version: 2.2.0a3

Channel	Status	Note	Current Constraint
Stable	❌	Too new (must be <1.4.0)	`ovos-core>=1.3.1,<1.4.0`
Testing	✅	Compatible	`ovos-core>=2.1.1,<3.0.0`
Alpha	✅	Compatible	`ovos-core>=2.2.0a2`

🌍 Locale Build

I've performed a quick audit of the latest commit. 🕵️

✅ Locale properly configured (64 files, 17 languages)

Locale directories found:

ovos_core/intent_services/locale

Localization coverage:

ovos_core/intent_services/locale: 64 files in 17 languages (eu-ES, es-es, it-it, de-de, uk-ua...)

pyproject.toml: ✅ [tool.setuptools.package-data.ovos_core] includes locale

intent_services/locale/*/*.voc

Build manifest: ✅ 31 locale files included in package

🔒 Security (pip-audit)

Scanning the horizon for any zero-day threats. 🌅

✅ No known vulnerabilities found (109 packages scanned).

📊 Coverage

Mapping out the 'known' vs 'unknown' in your code. 🗺️

⚠️ 61.0% total coverage

Files below 80% coverage (8 files)

File	Coverage	Missing lines
`ovos_core/__init__.py`	0.0%	7
`ovos_core/__main__.py`	0.0%	26
`ovos_core/intent_services/__init__.py`	0.0%	1
`ovos_core/version.py`	0.0%	18
`ovos_core/skill_installer.py`	42.3%	139
`ovos_core/intent_services/service.py`	50.8%	164
`ovos_core/skill_manager.py`	58.8%	173
`ovos_core/transformers.py`	66.0%	49

Full report: download the coverage-report artifact.

🔌 Skill Tests (ovoscope)

Does the skill understand what we're saying? Let's find out. 👂

❌ 43/99 passed, 33 failed, 9 skipped

❌ **TestActiveHandlerRecency** — 2/3

Test	Result
`test_reactivation_is_head_first_dedup`	✅ passed
`test_active_handlers_spec_field`	⚠️ skipped
`test_activation_updates_session_active_skills`	✅ passed

❌ **TestAdaptIntent** — 0/4

Test	Result
`test_adapt_match`	❌ failed
`test_skill_blacklist`	❌ failed
`test_intent_blacklist`	❌ failed
`test_padatious_no_match`	❌ failed

❌ **TestCancelIntentMidSentence** — 0/1

Test	Result
`test_cancel_match`	❌ subtests passed

❌ **TestConverse** — 0/1

Test	Result
`test_parrot_mode`	❌ failed

❌ **TestConverseOwnerOrdering** — 1/2

Test	Result
`test_converse_handlers_spec_field`	⚠️ skipped
`test_owner_ordering_most_recent_first`	✅ passed

❌ **TestCountSkills** — 0/4

Test	Result
`test_count_infinity_global`	❌ failed
`test_count`	❌ failed
`test_count_infinity_active`	❌ failed
`test_count_infinity_stop_low`	❌ failed

❌ **TestDeactivate** — 2/3

Test	Result
`test_deactivate`	✅ passed
`test_activate`	✅ passed
`test_deactivate_inside_converse`	❌ failed

❌ **TestFallback** — 0/1

Test	Result
`test_fallback_match`	❌ failed

❌ **TestFallbackHandlersField** — 0/1

Test	Result
`test_fallback_handlers_spec_field`	⚠️ skipped

❌ **TestGlobalStopVocWithActiveSkill** — 0/1

Test	Result
`test_global_stop_voc_with_active_skill`	❌ failed

❌ **TestGlobalStopVocabulary** — 0/2

Test	Result
`test_stop_voc_exact_still_works`	❌ failed
`test_global_stop_voc_no_active_skills`	❌ failed

❌ **TestIntentPipelineRouting** — 0/4

Test	Result
`test_padatious_intent_matched`	❌ failed
`test_no_match_produces_intent_failure`	❌ failed
`test_high_priority_stage_handles_before_low`	❌ failed
`test_blacklisted_skill_falls_through_to_failure`	❌ failed

❌ **TestLangDisambiguation** — 0/4

Test	Result
`test_stt_lang`	❌ failed
`test_lang_text_detection`	❌ failed
`test_invalid_lang_detection`	❌ failed
`test_metadata_preferred_over_text_detection`	❌ failed

❌ **TestNoSkills** — 0/2

Test	Result
`test_complete_failure`	❌ failed
`test_routing`	❌ failed

❌ **TestPadatiousIntent** — 0/4

Test	Result
`test_skill_blacklist`	❌ failed
`test_intent_blacklist`	❌ failed
`test_padatious_match`	❌ failed
`test_adapt_no_match`	❌ failed

❌ **TestResponseMode** — 1/2

Test	Result
`test_get_response_enable_sets_response_state`	✅ passed
`test_response_mode_spec_field`	⚠️ skipped

❌ **TestSec10Introspection** — 0/2

Test	Result
`test_intent_describe_responds`	❌ xfailed
`test_intent_list_responds`	❌ xfailed

test_intent_describe_responds failure:

ance.TestSec10Introspection testMethod=test_intent_describe_responds>

    @pytest.mark.xfail(strict=False,
                       reason="ovos-core does not serve 'ovos.intent.describe'; "
                              "INTENT-4 §10.2")
    def test_intent_describe_responds(self):
        """A query on ``ovos.intent.describe`` yields an
        ``ovos.intent.describe.response`` (§10.2)."""
        recs = capture(_MC, Message("ovos.intent.describe", {
            "skill_id": TEMPLATE_SKILL, "intent_name": TEMPLATE_NAME, "lang": "en-US",
        }, {"source": "A"}), 2.0)
>       self.assertIn("ovos.intent.describe.response", types(recs))
E       AssertionError: 'ovos.intent.describe.response' not found in ['ovos.intent.describe']

test/end2end/test_intent4_conformance.py:334: AssertionError

test_intent_list_responds failure:

self = <test.end2end.test_intent4_conformance.TestSec10Introspection testMethod=test_intent_list_responds>

    @pytest.mark.xfail(strict=False,
                       reason="ovos-core serves the legacy 'intent.service.intent.get'; "
                              "INTENT-4 §10.1 defines 'ovos.intent.list'")
    def test_intent_list_responds(self):
        """A query on ``ovos.intent.list`` yields an ``ovos.intent.list.response`` (§10.1)."""
        recs = capture(_MC, Message("ovos.intent.list", {}, {"source": "A"}), 2.0)
>       self.assertIn("ovos.intent.list.response", types(recs))
E       AssertionError: 'ovos.intent.list.response' not found in ['ovos.intent.list']

test/end2end/test_intent4_conformance.py:323: AssertionError

❌ **TestSec21OwnerOrdering** — 1/2

Test	Result
`test_most_recent_owner_first`	✅ passed
`test_converse_handlers_reflects_owner`	⚠️ skipped

✅ **TestSec2FireAndForget** — 1/1

❌ **TestSec2ReservedName** — 0/1

Test	Result
`test_reserved_stop_registration_not_dispatched`	❌ xfailed

test_reserved_stop_registration_not_dispatched failure:

st_reserved_stop_registration_not_dispatched(self):
        """A registered intent named ``stop`` must not become matchable (§2)."""
        register_padatious_intent(_MC.bus, "rogue.skill:stop",
                                  ["please halt everything now"])
        time.sleep(1)
        from ._conformance import PADACIOSO_HIGH
        recs = capture(_MC,
                       utterance("please halt everything now", "stop-reserved",
                                 [PADACIOSO_HIGH]),
                       3.0)
>       self.assertNotIn("rogue.skill:stop", types(recs))
E       AssertionError: 'rogue.skill:stop' unexpectedly found in ['recognizer_loop:utterance', 'rogue.skill.activate', 'ovos.intent.matched', 'rogue.skill:stop']

test/end2end/test_stop1_conformance.py:185: AssertionError

✅ **TestSec3Activation** — 1/1 ✅ **TestSec42PingPong** — 2/2 ✅ **TestSec43PerSkillStop** — 3/3 ✅ **TestSec4ConverseRoundTrip** — 2/2 ✅ **TestSec4Decline** — 1/1

❌ **TestSec4Registration** — 0/2

Test	Result
`test_fallback_handlers_session_field`	⚠️ skipped
`test_spec_register_topic_consumed`	❌ xfailed

test_spec_register_topic_consumed failure:

ALLBACK-1 §4 defines 'ovos.fallback.register'")
    def test_spec_register_topic_consumed(self):
        """Registering on the spec topic ``ovos.fallback.register`` makes the
        handler poolable; querying yields a pong from it (§4)."""
        _MC.bus.emit(Message("ovos.fallback.register",
                             {"skill_id": "spec.fb.skill", "priority": 50},
                             {"skill_id": "spec.fb.skill"}))
        time.sleep(1.0)
        recs = capture(_MC, utterance("zxqw blah blah", "fb-spec-reg", PIPELINE), 5.0)
        # the spec-registered skill should be polled (appear as a pool member)
>       self.assertTrue(any("spec.fb.skill" in t for t in types(recs)))
E       AssertionError: False is not true

test/end2end/test_fallback1_conformance.py:192: AssertionError

✅ **TestSec53BlacklistBackstop** — 1/1 ✅ **TestSec5EffectivePipeline** — 2/2 ✅ **TestSec5FallbackOrdering** — 2/2

❌ **TestSec5GlobalStop** — 2/3

Test	Result
`test_global_stop_terminates`	✅ passed
`test_global_stop_broadcast_topic`	✅ passed
`test_global_stop_dispatch_topic`	❌ xfailed

test_global_stop_dispatch_topic failure:

self = <test.end2end.test_stop1_conformance.TestSec5GlobalStop testMethod=test_global_stop_dispatch_topic>

    @pytest.mark.xfail(strict=False,
                       reason="ovos-core self-dispatches the legacy 'stop:global'; "
                              "STOP-1 §3.1/§5.2 use '<stop_plugin_id>:global_stop'")
    def test_global_stop_dispatch_topic(self):
        """Global stop is dispatched on ``<stop_plugin_id>:global_stop`` (§3.1, §5.2)."""
        recs = capture(_MC, utterance("stop", "stop-global-disp", [STOP_HIGH]), 4.0)
>       self.assertTrue(any(t.endswith(":global_stop") for t in types(recs)))
E       AssertionError: False is not true

test/end2end/test_stop1_conformance.py:91: AssertionError

❌ **TestSec5KeywordRegistration** — 0/1

Test	Result
`test_spec_keyword_registration_is_matchable`	❌ xfailed

test_spec_keyword_registration_is_matchable failure:

tering via the spec keyword topic makes the intent matchable (§5)."""
        _MC.bus.emit(Message("ovos.intent.register.keyword", {
            "skill_id": KEYWORD_SKILL,
            "intent_name": KEYWORD_NAME,
            "lang": "en-US",
            "samples": KEYWORD_SAMPLES,
        }, {"skill_id": KEYWORD_SKILL}))
        time.sleep(1.5)
        recs = capture(
            _MC,
            utterance("turn off the lights", "i4-kw", [PADACIOSO_HIGH]),
            3.0,
        )
>       self.assertIn(KEYWORD_INTENT, types(recs))
E       AssertionError: 'intent4.skill:lights_off' not found in ['recognizer_loop:utterance', 'mycroft.audio.play_sound', 'complete_intent_failure', 'ovos.intent.unmatched', 'ovos.utterance.handled']

test/end2end/test_intent4_conformance.py:204: AssertionError

✅ **TestSec5Priority** — 1/1 ✅ **TestSec64Cancelled** — 3/3 ✅ **TestSec6QueryResponse** — 3/3

❌ **TestSec6TemplateRegistration** — 0/1

Test	Result
`test_spec_topic_registration_is_matchable`	❌ xfailed

test_spec_topic_registration_is_matchable failure:

'padatious:register_intent'; INTENT-4 §6 defines "
                              "'ovos.intent.register.template'")
    def test_spec_topic_registration_is_matchable(self):
        """Registering via the spec topic makes the intent matchable (§6)."""
        _register_template("ovos.intent.register.template", TEMPLATE_INTENT, TEMPLATE_SAMPLES)
        recs = capture(
            _MC,
            utterance("turn on the lights", "i4-tmpl", [PADACIOSO_HIGH]),
            3.0,
        )
>       self.assertIn(TEMPLATE_INTENT, types(recs))
E       AssertionError: 'intent4.skill:lights_on' not found in ['recognizer_loop:utterance', 'mycroft.audio.play_sound', 'complete_intent_failure', 'ovos.intent.unmatched', 'ovos.utterance.handled']

test/end2end/test_intent4_conformance.py:128: AssertionError

✅ **TestSec7Dispatch** — 4/4

❌ **TestSec7EntityRegistration** — 0/1

Test	Result
`test_spec_entity_registration_no_ack`	❌ xpassed

❌ **TestSec81HandlerError** — 1/2

Test	Result
`test_error_still_terminates_once`	✅ passed
`test_error_topic_emitted`	⚠️ skipped

❌ **TestSec82Deregister** — 0/1

Test	Result
`test_spec_deregister_removes_intent`	❌ xfailed

test_spec_deregister_removes_intent failure:

working mechanism so there is something to remove
        register_padatious_intent(_MC.bus, TEMPLATE_INTENT, TEMPLATE_SAMPLES)
        time.sleep(1.5)
        _MC.bus.emit(Message("ovos.intent.deregister", {
            "skill_id": TEMPLATE_SKILL, "intent_name": TEMPLATE_NAME, "lang": "en-US",
        }, {"skill_id": TEMPLATE_SKILL}))
        time.sleep(1.5)
        recs = capture(
            _MC,
            utterance("turn on the lights", "i4-dereg", [PADACIOSO_HIGH]),
            3.0,
        )
>       self.assertNotIn(TEMPLATE_INTENT, types(recs))
E       AssertionError: 'intent4.skill:lights_on' unexpectedly found in ['recognizer_loop:utterance', 'intent4.skill.activate', 'ovos.intent.matched', 'intent4.skill:lights_on']

test/end2end/test_intent4_conformance.py:152: AssertionError

❌ **TestSec83EntityDeregister** — 0/1

Test	Result
`test_spec_entity_deregister_no_error`	❌ xpassed

❌ **TestSec84SkillDeregister** — 0/1

Test	Result
`test_spec_skill_deregister_removes_all_intents`	❌ xpassed

❌ **TestSec85Disable** — 0/1

Test	Result
`test_spec_disable_suppresses_intent`	❌ xfailed

test_spec_disable_suppresses_intent failure:

intent is excluded from match candidacy (§8.5)."""
        register_padatious_intent(_MC.bus, TEMPLATE_INTENT, TEMPLATE_SAMPLES)
        time.sleep(1.5)
        _MC.bus.emit(Message("ovos.intent.disable", {
            "skill_id": TEMPLATE_SKILL, "intent_name": TEMPLATE_NAME, "lang": "en-US",
        }, {"skill_id": TEMPLATE_SKILL}))
        time.sleep(1.5)
        recs = capture(
            _MC,
            utterance("turn on the lights", "i4-disable", [PADACIOSO_HIGH]),
            3.0,
        )
>       self.assertNotIn(TEMPLATE_INTENT, types(recs))
E       AssertionError: 'intent4.skill:lights_on' unexpectedly found in ['recognizer_loop:utterance', 'intent4.skill.activate', 'ovos.intent.matched', 'intent4.skill:lights_on']

test/end2end/test_intent4_conformance.py:175: AssertionError

❌ **TestSec85Enable** — 0/1

Test	Result
`test_spec_enable_rearms_intent`	❌ xpassed

❌ **TestSec8HandlerTrio** — 0/1

Test	Result
`test_handler_trio_topics`	⚠️ skipped

✅ **TestSec91Entry** — 1/1 ✅ **TestSec92Matched** — 1/1 ✅ **TestSec93Unmatched** — 1/1 ✅ **TestSec95EndMarker** — 2/2

❌ **TestSec96Speak** — 0/1

Test	Result
`test_speak_topic`	⚠️ skipped

❌ **TestStopNoSkills** — 0/3

Test	Result
`test_not_exact_med`	❌ failed
`test_not_exact_high`	❌ failed
`test_exact`	❌ failed

❌ **TestStopServiceAsSkill** — 0/1

Test	Result
`test_stop_service_emits_activate_and_stop_response`	❌ failed

❌ **TestStopSkillCanHandleFalse** — 0/1

Test	Result
`test_stop_with_active_skill_ping_pong`	❌ failed

✅ **TestUpdatedSessionEcho** — 2/2

🚌 Bus Coverage

A deep dive into the skill's communication patterns. 🌊

🔴 Coverage Summary

Metric	Status	Coverage
Listeners	`██░░░░░░░░` 27.5%	69/251 handlers
Emitters	`██████████` 100%	13/13 observed
Assertions	`██████████` 100%	13/13 asserted

📊 Per-Skill Breakdown

Skill	Listeners	Observed	Asserted
`AdaptPipeline`	4/7 (57.1%)	0/0	0/0
`ConverseService`	3/4 (75.0%)	0/0	0/0
`DomainAdaptPipeline`	4/7 (57.1%)	0/0	0/0
`FallbackService`	2/2 (100.0%)	0/0	0/0
`HierarchicalAdaptPipeline`	4/7 (57.1%)	0/0	0/0
`IntentService`	3/4 (75.0%)	0/0	0/0
`Model2VecIntentPipeline`	3/5 (60.0%)	0/0	0/0
`PadaciosoPipeline`	2/4 (50.0%)	0/0	0/0
`PadatiousPipeline`	4/8 (50.0%)	0/0	0/0
`SkillManager`	0/4 (0.0%)	0/0	0/0
`__core__`	9/44 (20.5%)	7/7	7/7
`ovos-conformance-echo.test`	2/17 (11.8%)	0/0	0/0
`ovos-skill-count.openvoiceos`	4/16 (25.0%)	0/0	0/0
`ovos-skill-fallback-unknown.openvoiceos`	3/17 (17.6%)	0/0	0/0
`ovos-skill-hello-world.openvoiceos`	2/24 (8.3%)	0/0	0/0
`ovos-skill-parrot.openvoiceos`	8/29 (27.6%)	0/0	0/0
`stop.openvoiceos`	3/21 (14.3%)	0/0	0/0
`test_activation.openvoiceos`	9/27 (33.3%)	6/6	6/6
`type`	0/4 (0.0%)	0/0	0/0

🔍 Detailed Message Type Breakdown

`AdaptPipeline`

⚠️ Uncovered Listeners:

❌ detach_intent
❌ intent.service.adapt.get
❌ intent.service.adapt.vocab.manifest.get
✅ Covered Listeners:
✅ detach_skill (443x)
✅ intent.service.adapt.manifest.get (863x)
✅ register_intent (108x)
✅ register_vocab (14716x)

`ConverseService`

⚠️ Uncovered Listeners:

❌ intent.service.active_skills.get
✅ Covered Listeners:
✅ converse:skill (36x)
✅ intent.service.skills.activate (5x)
✅ intent.service.skills.deactivate (14x)

`DomainAdaptPipeline`

⚠️ Uncovered Listeners:

❌ detach_intent
❌ intent.service.adapt.get
❌ intent.service.adapt.vocab.manifest.get
✅ Covered Listeners:
✅ detach_skill (443x)
✅ intent.service.adapt.manifest.get (863x)
✅ register_intent (108x)
✅ register_vocab (14716x)

`FallbackService`

✅ Covered Listeners:

✅ ovos.skills.fallback.deregister (12x)
✅ ovos.skills.fallback.register (12x)

`HierarchicalAdaptPipeline`

⚠️ Uncovered Listeners:

❌ detach_intent
❌ intent.service.adapt.get
❌ intent.service.adapt.vocab.manifest.get
✅ Covered Listeners:
✅ detach_skill (443x)
✅ intent.service.adapt.manifest.get (863x)
✅ register_intent (108x)
✅ register_vocab (14716x)

`IntentService`

⚠️ Uncovered Listeners:

❌ intent.service.intent.get (Intent)
✅ Covered Listeners:
✅ intent.service.pipelines.reload (277x)
✅ intent.service.skills.deactivate (14x)
✅ ovos.utterance.handle (163x)

`Model2VecIntentPipeline`

⚠️ Uncovered Listeners:

❌ detach_intent
❌ mycroft.ready
✅ Covered Listeners:
✅ detach_skill (443x)
✅ padatious:register_intent (312x)
✅ register_intent (108x)

`PadaciosoPipeline`

⚠️ Uncovered Listeners:

❌ padatious:register_entity (Intent)
❌ detach_intent
✅ Covered Listeners:
✅ detach_skill (443x)
✅ padatious:register_intent (312x)

`PadatiousPipeline`

⚠️ Uncovered Listeners:

❌ padatious:register_entity (Intent)
❌ detach_intent
❌ intent.service.padatious.entities.manifest.get
❌ intent.service.padatious.get
✅ Covered Listeners:
✅ detach_skill (443x)
✅ intent.service.padatious.manifest.get (863x)
✅ mycroft.skills.train (277x)
✅ padatious:register_intent (312x)

`SkillManager`

⚠️ Uncovered Listeners:

❌ skillmanager.activate
❌ skillmanager.deactivate
❌ skillmanager.keep
❌ skillmanager.list

`core`

⚠️ Uncovered Listeners:

❌ add_context
❌ clear_context
❌ fake.skill.stop.response
❌ message
❌ mycroft.ovos-conformance-echo.test.all_loaded
❌ mycroft.ovos-conformance-echo.test.is_alive
❌ mycroft.ovos-conformance-echo.test.is_ready
❌ mycroft.ovos-skill-count.openvoiceos.all_loaded
❌ mycroft.ovos-skill-count.openvoiceos.is_alive
❌ mycroft.ovos-skill-count.openvoiceos.is_ready
❌ mycroft.ovos-skill-fallback-unknown.openvoiceos.all_loaded
❌ mycroft.ovos-skill-fallback-unknown.openvoiceos.is_alive
❌ mycroft.ovos-skill-fallback-unknown.openvoiceos.is_ready
❌ mycroft.ovos-skill-hello-world.openvoiceos.all_loaded
❌ mycroft.ovos-skill-hello-world.openvoiceos.is_alive
❌ mycroft.ovos-skill-hello-world.openvoiceos.is_ready
❌ mycroft.ovos-skill-parrot.openvoiceos.all_loaded
❌ mycroft.ovos-skill-parrot.openvoiceos.is_alive
❌ mycroft.ovos-skill-parrot.openvoiceos.is_ready
❌ mycroft.test_activation.openvoiceos.all_loaded
❌ mycroft.test_activation.openvoiceos.is_alive
❌ mycroft.test_activation.openvoiceos.is_ready
❌ ovos-conformance-echo.test.set
❌ ovos-skill-count.openvoiceos.set
❌ ovos-skill-fallback-unknown.openvoiceos.set
❌ ovos-skill-hello-world.openvoiceos.set
❌ ovos-skill-parrot.openvoiceos.set
❌ ovos.session.sync
❌ ovos.skills.fallback.force_timeout
❌ ovos.skills.fallback.ovos-skill-fallback-unknown.openvoiceos
❌ remove_context
❌ skill.converse.get_response.disable
❌ skill.converse.get_response.enable
❌ stoppable.test.stop.response
❌ test_activation.openvoiceos.set
✅ Covered Listeners:
✅ ovos-skill-count.openvoiceos.stop.response (60x)
✅ ovos.session.update_default (305x)
✅ ovos.skills.converse.force_timeout (36x)
✅ ovos.skills.fallback.pong (24x)
✅ ovos.stop.ping (40x)
✅ ovos.stop.pong (4x)
✅ ovos.utterance.handled (408x)
✅ skill.converse.pong (44x)
✅ skill.stop.pong (36x)

📤 Emitters:

✅ mycroft.audio.play_sound (Asserted ✅)
✅ ovos.utterance.cancelled (Asserted ✅)
✅ ovos.utterance.handle (Asserted ✅)
✅ ovos.utterance.handled (Asserted ✅)
✅ recognizer_loop:utterance (Asserted ✅)
✅ test_activate (Asserted ✅)
✅ test_deactivate (Asserted ✅)

`ovos-conformance-echo.test`

⚠️ Uncovered Listeners:

❌ question:action (Intent)
❌ question:action.ovos-conformance-echo.test (Intent)
❌ question:query (Intent)
❌ conformance.echo
❌ homescreen.metadata.get
❌ mycroft.skill.disable_intent
❌ mycroft.skill.enable_intent
❌ mycroft.skill.remove_cross_context
❌ mycroft.skill.set_cross_context
❌ mycroft.skills.settings.changed
❌ ovos-conformance-echo.test.converse.get_response
❌ ovos-conformance-echo.test.stop
❌ ovos-conformance-echo.test.stop.ping
❌ ovos.common_query.ping
❌ ovos.skills.settings_changed
✅ Covered Listeners:
✅ conformance.skill:boom (4x)
✅ mycroft.stop (76x)

`ovos-skill-count.openvoiceos`

⚠️ Uncovered Listeners:

❌ question:action (Intent)
❌ question:action.ovos-skill-count.openvoiceos (Intent)
❌ question:query (Intent)
❌ homescreen.metadata.get
❌ mycroft.skill.disable_intent
❌ mycroft.skill.enable_intent
❌ mycroft.skill.remove_cross_context
❌ mycroft.skill.set_cross_context
❌ mycroft.skills.settings.changed
❌ ovos-skill-count.openvoiceos.converse.get_response
❌ ovos.common_query.ping
❌ ovos.skills.settings_changed
✅ Covered Listeners:
✅ mycroft.stop (104x)
✅ ovos-skill-count.openvoiceos.stop (36x)
✅ ovos-skill-count.openvoiceos.stop.ping (36x)
✅ ovos-skill-count.openvoiceos:count_to_N.intent (60x)

`ovos-skill-fallback-unknown.openvoiceos`

⚠️ Uncovered Listeners:

❌ question:action (Intent)
❌ question:action.ovos-skill-fallback-unknown.openvoiceos (Intent)
❌ question:query (Intent)
❌ homescreen.metadata.get
❌ mycroft.skill.disable_intent
❌ mycroft.skill.enable_intent
❌ mycroft.skill.remove_cross_context
❌ mycroft.skill.set_cross_context
❌ mycroft.skills.settings.changed
❌ ovos-skill-fallback-unknown.openvoiceos.converse.get_response
❌ ovos-skill-fallback-unknown.openvoiceos.stop
❌ ovos-skill-fallback-unknown.openvoiceos.stop.ping
❌ ovos.common_query.ping
❌ ovos.skills.settings_changed
✅ Covered Listeners:
✅ mycroft.stop (76x)
✅ ovos.skills.fallback.ovos-skill-fallback-unknown.openvoiceos.request (24x)
✅ ovos.skills.fallback.ping (24x)

`ovos-skill-hello-world.openvoiceos`

⚠️ Uncovered Listeners:

❌ ovos-skill-hello-world.openvoiceos:Greetings.intent (Intent)
❌ ovos-skill-hello-world.openvoiceos:HowAreYou.intent (Intent)
❌ ovos-skill-hello-world.openvoiceos:ThankYouIntent (Intent)
❌ question:action (Intent)
❌ question:action.ovos-skill-hello-world.openvoiceos (Intent)
❌ question:query (Intent)
❌ hello.world
❌ homescreen.metadata.get
❌ mycroft.ovos-skill-hello-world.openvoiceos.all_loaded
❌ mycroft.ovos-skill-hello-world.openvoiceos.is_alive
❌ mycroft.ovos-skill-hello-world.openvoiceos.is_ready
❌ mycroft.skill.disable_intent
❌ mycroft.skill.enable_intent
❌ mycroft.skill.remove_cross_context
❌ mycroft.skill.set_cross_context
❌ mycroft.skills.settings.changed
❌ ovos-skill-hello-world.openvoiceos.converse.get_response
❌ ovos-skill-hello-world.openvoiceos.set
❌ ovos-skill-hello-world.openvoiceos.stop
❌ ovos-skill-hello-world.openvoiceos.stop.ping
❌ ovos.common_query.ping
❌ ovos.skills.settings_changed
✅ Covered Listeners:
✅ mycroft.stop (104x)
✅ ovos-skill-hello-world.openvoiceos:HelloWorldIntent (12x)

`ovos-skill-parrot.openvoiceos`

⚠️ Uncovered Listeners:

❌ ovos-skill-parrot.openvoiceos:did.you.hear.me.intent (Intent)
❌ ovos-skill-parrot.openvoiceos:repeat.stt.intent (Intent)
❌ ovos-skill-parrot.openvoiceos:repeat.tts.intent (Intent)
❌ ovos-skill-parrot.openvoiceos:speak.intent (Intent)
❌ ovos-skill-parrot.openvoiceos:stop_parrot.intent (Intent)
❌ question:action (Intent)
❌ question:action.ovos-skill-parrot.openvoiceos (Intent)
❌ question:query (Intent)
❌ homescreen.metadata.get
❌ mycroft.skill.disable_intent
❌ mycroft.skill.enable_intent
❌ mycroft.skill.remove_cross_context
❌ mycroft.skill.set_cross_context
❌ mycroft.skills.settings.changed
❌ ovos-skill-parrot.openvoiceos.converse.get_response
❌ ovos-skill-parrot.openvoiceos.deactivate
❌ ovos-skill-parrot.openvoiceos.stop
❌ ovos-skill-parrot.openvoiceos.stop.ping
❌ ovos.common_query.ping
❌ ovos.skills.settings_changed
❌ speak
✅ Covered Listeners:
✅ intent.service.skills.activated (5x)
✅ intent.service.skills.deactivated (14x)
✅ mycroft.stop (104x)
✅ ovos-skill-parrot.openvoiceos.activate (66x)
✅ ovos-skill-parrot.openvoiceos.converse.ping (36x)
✅ ovos-skill-parrot.openvoiceos.converse.request (28x)
✅ ovos-skill-parrot.openvoiceos:start_parrot.intent (38x)
✅ recognizer_loop:utterance (329x)

`stop.openvoiceos`

⚠️ Uncovered Listeners:

❌ question:action (Intent)
❌ question:action.stop.openvoiceos (Intent)
❌ question:query (Intent)
❌ homescreen.metadata.get
❌ mycroft.skill.disable_intent
❌ mycroft.skill.enable_intent
❌ mycroft.skill.remove_cross_context
❌ mycroft.skill.set_cross_context
❌ mycroft.skills.settings.changed
❌ mycroft.stop.openvoiceos.all_loaded
❌ mycroft.stop.openvoiceos.is_alive
❌ mycroft.stop.openvoiceos.is_ready
❌ ovos.common_query.ping
❌ ovos.skills.settings_changed
❌ stop.openvoiceos.converse.get_response
❌ stop.openvoiceos.set
❌ stop.openvoiceos.stop
❌ stop.openvoiceos.stop.ping
✅ Covered Listeners:
✅ mycroft.stop (104x)
✅ stop:global (104x)
✅ stop:skill (52x)

`test_activation.openvoiceos`

⚠️ Uncovered Listeners:

❌ question:action (Intent)
❌ question:action.test_activation.openvoiceos (Intent)
❌ question:query (Intent)
❌ homescreen.metadata.get
❌ mycroft.skill.disable_intent
❌ mycroft.skill.enable_intent
❌ mycroft.skill.remove_cross_context
❌ mycroft.skill.set_cross_context
❌ mycroft.skills.settings.changed
❌ mycroft.test_activation.openvoiceos.all_loaded
❌ mycroft.test_activation.openvoiceos.is_alive
❌ mycroft.test_activation.openvoiceos.is_ready
❌ ovos.common_query.ping
❌ ovos.skills.settings_changed
❌ test_activation.openvoiceos.converse.get_response
❌ test_activation.openvoiceos.set
❌ test_activation.openvoiceos.stop
❌ test_activation.openvoiceos.stop.ping
✅ Covered Listeners:
✅ intent.service.skills.activated (5x)
✅ intent.service.skills.deactivated (14x)
✅ mycroft.stop (104x)
✅ test_activate (5x)
✅ test_activation.openvoiceos.activate (13x)
✅ test_activation.openvoiceos.converse.ping (8x)
✅ test_activation.openvoiceos.converse.request (8x)
✅ test_activation.openvoiceos.deactivate (14x)
✅ test_deactivate (6x)

📤 Emitters:

✅ intent.service.skills.activate (Asserted ✅)
✅ intent.service.skills.activated (Asserted ✅)
✅ intent.service.skills.deactivate (Asserted ✅)
✅ intent.service.skills.deactivated (Asserted ✅)
✅ test_activation.openvoiceos.activate (Asserted ✅)
✅ test_activation.openvoiceos.deactivate (Asserted ✅)

`type`

⚠️ Uncovered Listeners:

❌ recognizer_loop:audio_output_end (Intent)
❌ recognizer_loop:audio_output_start (Intent)
❌ recognizer_loop:record_begin (Intent)
❌ recognizer_loop:record_end (Intent)

🔨 Build Tests

Checking if the code is ready for prime time. 📺

✅ All versions pass

Python	Build	Install	Tests
3.10	✅	✅	✅
3.11	✅	✅	✅
3.12	✅	✅	✅
3.13	✅	✅	✅
3.14	✅	✅	✅

⚖️ License Check

Everything looks good on the legal front. ✅

✅ No license violations found.

Policy: Apache 2.0 (universal donor). StrongCopyleft / NetworkCopyleft / WeakCopyleft / Other / Error categories fail. MPL allowed.

Your automated companion in the OpenVoiceOS journey. 🤝

Add clause-labelled ovoscope end-to-end conformance suites mapping the normative Conformance clauses of OVOS-PIPELINE-1, OVOS-STOP-1 and OVOS-INTENT-4 to tests against the ovos-core orchestrator. Clauses the orchestrator already satisfies pass; clauses pending the mycroft.* -> ovos.* topic migration are marked xfail and flip to passing once the impl is updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

The orchestrator and stop pipeline emit either the legacy mycroft.* topics or the OVOS spec ovos.* topics, chosen by the deployment 'legacy_namespace' config (default True) — never both. Subscribers listen on both namespaces. - entry: also subscribe ovos.utterance.handle (PIPELINE-1 §9.1) - no-match: complete_intent_failure <-> ovos.intent.unmatched (§9.3) - match notification + context.pipeline_id stamp (§9.2 / §7.1, spec namespace) - stop: mycroft.stop <-> ovos.stop (§5.3); per-skill pings <-> broadcast ovos.stop.ping (§4.2) Conformance suites run in the spec namespace (legacy_namespace=False) and are green (21 passed, 7 xfail); legacy-namespace end2end tests are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

handle_global_stop and _collect_stop_skills called self._legacy_namespace(), which does not exist on StopService — the AttributeError crashed the handler before it emitted the universal ovos.utterance.handled end-marker. Use the shared module-level helper from the intent service instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…PIPELINE-1, STOP-1, INTENT-4) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…OP-1 §4.2) _collect_stop_skills now subscribes/removes both skill.stop.pong and ovos.stop.pong; the unit tests asserted a single subscription. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…(pre_release) register_padatious_intent (ovoscope >=0.15.0a1) is dev-only; the default ovoscope install resolves to a stable release without it. A conformance suite asserts against current ecosystem dev, so pre_release: true is the correct install mode and makes the e2e job green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…IPELINE-1 §8) Under pre_release the dev count-skill's handler completes when stopped and emits mycroft.skill.handler.complete, which is orthogonal to the stop-cascade message sequence these tests assert. Add the handler-lifecycle events to ignore_messages. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

….4 deregister, §8.5 enable Adds the remaining INTENT-4 registration bus-contract clauses as ovoscope end-to-end assertions, following the existing xfail discipline (ovos-core still consumes the legacy padatious:/detach_ topics, so the spec ovos.intent.* / ovos.entity.* / ovos.skill.* topics xfail until the contract lands): - §5 ovos.intent.register.keyword makes a keyword intent matchable - §7 ovos.entity.register value-set hint accepted fire-and-forget - §8.3 ovos.entity.deregister removes an entity - §8.4 ovos.skill.deregister removes all of a skill's intents - §8.5 ovos.intent.enable re-arms a disabled intent Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…ler error - §6.4: an utterance flagged context['canceled'] emits ovos.utterance.cancelled, terminates with exactly one ovos.utterance.handled, and never dispatches. - §8.1: a padatious-matched handler that raises emits ovos.intent.handler.error (lifecycle-trio terminal) yet the utterance still ends with exactly one end-marker. Adds _ErrorSkill handler (BOOM_INTENT) to _EchoSkill. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…active-vs-none - §4.2: an active stoppable skill answers ovos.stop.ping with ovos.stop.pong {skill_id, can_handle}; the collected skill is told to stop. - §4.3: with an active skill a generic 'stop' dispatches <skill_id>.stop and does NOT broadcast the universal ovos.stop. - §4/§5.1: with no active skill 'stop' escalates to the global ovos.stop broadcast and no per-skill stop fires. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

End-to-end converse clauses driven by the real ovos-skill-parrot fixture: - §3 activating a skill records it as an active converse owner - §4 an active owner consumes the follow-up before intent matching (parrot echoes the utterance via converse:skill) with exactly one ovos.utterance.handled - §4 a declining owner falls through to the normal pipeline (no converse:skill) - §2.1 owners polled most-recently-active first (head-first active_skills); converse_handlers session field reflects the owner (skipped until the spec session field is populated; legacy active_skills today) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

End-to-end fallback clauses driven by the real ovos-skill-fallback-unknown fixture (priority 100 catch-all), with the fallback stage pinned after the matcher: - §5 fallback runs only after the matchers decline; query precedes dispatch - §6 query/response trio (ovos.skills.fallback.ping/pong + per-skill .response) and exactly one ovos.utterance.handled - §5 pool sorted by priority ascending (high-confidence before low-confidence) - §4 spec register topic ovos.fallback.register + session.fallback_handlers asserted as xfail/skip (core consumes ovos.skills.fallback.register; the session field is absent until feat/session-spec-fields bus-client) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Asserts session state carries/updates through real ovos-core interactions: - PIPELINE-1 §7.1 active-handler recency + head-first dedup (active_skills) - CONVERSE-1 §2.1 converse owner ordering (most-recent-first) - CONVERSE-1 §2.2 response-mode capture via utterance_states RESPONSE - FALLBACK-1 §4 fallback_handlers field - SESSION-2 updated_session echoed on responses (id preserved, mutation rides forward) Spec-field-named assertions (active_handlers/converse_handlers/fallback_handlers/ response_mode) skip cleanly until feat/session-spec-fields bus-client is installed. Also drop an unused import left in the converse suite. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…shop cap - ovoscope.yml: add pre_install_pip force-reinstalling the producer/session/ plugin/fixture branches that emit the spec topics and session fields the conformance suites assert (bus-client feat/session-spec-fields, workshop feat/intent-4-producer, adapt/padatious fix/allow-ovos-workshop-9, and the parrot/fallback-unknown/count/hello-world fixtures fix/allow-ovos-workshop-10). - pyproject: lift ovos-workshop cap >=8.3.0a1,<9.0.0 -> <10.0.0 so the workshop 9.x producer branch resolves. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The .[test] resolution downgrades ovos-workshop to a <9 published wheel (fixture skills + transitive ovos-core cap it), uninstalling the 9.x INTENT-4 producer so its bus topics never emit. Re-pin the integrated stack after the extras install via the new post_install_pip input (gh-automations#58). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

JarbasAl force-pushed the test/spec-conformance-suites branch from 54d1ed1 to 65f1680 Compare June 5, 2026 14:43

JarbasAl force-pushed the test/spec-conformance-suites branch from 65f1680 to 5ef4fd5 Compare June 23, 2026 16:16

JarbasAl and others added 11 commits June 27, 2026 04:38

docs: document spec conformance and the legacy_namespace transition (…

3d609a2

…PIPELINE-1, STOP-1, INTENT-4) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Update service.py

98443a1

Update stop_service.py

90fafb5

Refactor stop service ping handling for clarity

a36a109

Remove legacy namespace function and clean up code

d28603a

JarbasAl force-pushed the test/spec-conformance-suites branch from 8266722 to d28603a Compare June 27, 2026 03:43

JarbasAl and others added 8 commits June 27, 2026 04:46

JarbasAl force-pushed the test/spec-conformance-suites branch 2 times, most recently from b1d9879 to 284f116 Compare June 27, 2026 05:41

Uh oh!

Uh oh!

Conversation

JarbasAl commented Jun 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Conformance results

Companions

Uh oh!

coderabbitai Bot commented Jun 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

github-actions Bot commented Jun 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The automated pipeline has reached its destination. 🏁

📋 Repo Health

🏷️ Release Preview

🌍 Locale Build

🔒 Security (pip-audit)

📊 Coverage

🔌 Skill Tests (ovoscope)

🚌 Bus Coverage

🔴 Coverage Summary

📊 Per-Skill Breakdown

AdaptPipeline

ConverseService

DomainAdaptPipeline

FallbackService

HierarchicalAdaptPipeline

IntentService

Model2VecIntentPipeline

PadaciosoPipeline

PadatiousPipeline

SkillManager

__core__

ovos-conformance-echo.test

ovos-skill-count.openvoiceos

ovos-skill-fallback-unknown.openvoiceos

ovos-skill-hello-world.openvoiceos

ovos-skill-parrot.openvoiceos

stop.openvoiceos

test_activation.openvoiceos

type

🔨 Build Tests

⚖️ License Check

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

JarbasAl commented Jun 5, 2026 •

edited

Loading

coderabbitai Bot commented Jun 5, 2026 •

edited

Loading

github-actions Bot commented Jun 5, 2026 •

edited

Loading

`AdaptPipeline`

`ConverseService`

`DomainAdaptPipeline`

`FallbackService`

`HierarchicalAdaptPipeline`

`IntentService`

`Model2VecIntentPipeline`

`PadaciosoPipeline`

`PadatiousPipeline`

`SkillManager`

`core`

`ovos-conformance-echo.test`

`ovos-skill-count.openvoiceos`

`ovos-skill-fallback-unknown.openvoiceos`

`ovos-skill-hello-world.openvoiceos`

`ovos-skill-parrot.openvoiceos`

`stop.openvoiceos`

`test_activation.openvoiceos`

`type`