Skip to content

[ci] Add mock LLM for Lightspeed deployment testing - #4222

Draft
malingatembo wants to merge 5 commits into
openstack-k8s-operators:mainfrom
malingatembo:feat/OSPRH-33344-mock-llm-deployment
Draft

malingatembo wants to merge 5 commits into
openstack-k8s-operators:mainfrom
malingatembo:feat/OSPRH-33344-mock-llm-deployment

Conversation

@malingatembo

@malingatembo malingatembo commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

[ci] Add mock LLM for Lightspeed deployment testing

Deploy a mock LLM so Lightspeed can be tested without external provider
credentials. The mock supports ordinary JSON and streamed chat completions.

Validated locally on CRC:

  • Deployment and an unchanged rerun passed.
  • Authenticated query returned HTTP 200 with the mock answer.
  • Unauthenticated query returned HTTP 401.
  • Broken provider connection returned HTTP 500.
  • Restored provider connection returned HTTP 200.

Syntax and formatting checks passed. Ansible lint did not complete.

Remaining before final review:

  • Address the fresh-install readiness timeout.
  • Restart the mock automatically when its code changes.

Query checks were manual; automated lightspeed-tests integration remains
separate work..

Add a controller hook that runs the prepared lightspeed-tests checkout after
Lightspeed reaches Ready. The hook supplies client authentication, retains
JUnit reports and runner logs, propagates test failures, and cleans up its
temporary token files and API tunnel.

Test-runner changes:
openstack-k8s-operators/lightspeed-tests#17

Combined upstream integration validation remains pending.

Related-Issue: OSPRH-33344

Deploy a mock provider so Lightspeed can be tested without external LLM credentials. Support ordinary and streamed chat completions required by the current provider adapter.

Related-Issue: OSPRH-33344
Signed-off-by: malingatembo <tembomalinga@gmail.com>
@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@openshift-ci

openshift-ci Bot commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign fultonj for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build succeeded (check pipeline).
https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/7cec839c08e14eb588dd6a0737736eb1

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 35m 50s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 20m 21s
✔️ podified-multinode-edpm-deployment-crc-centos-10 SUCCESS in 1h 24m 09s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 30m 17s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 01m 12s
✔️ openstack-k8s-operators-content-provider-bootc SUCCESS in 3h 00m 42s
✔️ cifmw-crc-podified-edpm-baremetal-bootc SUCCESS in 1h 30m 44s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 04s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 31s

The upstream mock-provider deployment timed out while application images
were still starting. The collected custom resource became Ready about
50 seconds after the playbook failed.

Increase the default CR readiness retries from 30 to 120 while retaining
the ten-second polling interval, Ready condition and bounded failure.

Related-Issue: #OSPRH-33344

Signed-off-by: malingatembo <tembomalinga@gmail.com>
@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/0bec6a0613c64ce69be3c3912004fdeb

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 47m 27s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 22m 21s
✔️ podified-multinode-edpm-deployment-crc-centos-10 SUCCESS in 1h 28m 51s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 28m 02s
❌ cifmw-crc-podified-edpm-baremetal-minor-update NODE_FAILURE Node(set) request 099-0000224023 failed in 0s
✔️ openstack-k8s-operators-content-provider-bootc SUCCESS in 3h 09m 22s
✔️ cifmw-crc-podified-edpm-baremetal-bootc SUCCESS in 1h 25m 23s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 42s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 17s

The operator now requires models and defaultModel. Move the existing
provider settings into one configured model and select it as the default
so the deployment hook can create the CR against the current operator.

Related-Issue: OSPRH-33344
Signed-off-by: malingatembo <tembomalinga@gmail.com>
@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/e074a5e4b9764fc6822dd5201a637b1d

✔️ openstack-k8s-operators-content-provider SUCCESS in 3h 13m 42s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 25m 33s
✔️ podified-multinode-edpm-deployment-crc-centos-10 SUCCESS in 1h 29m 45s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 33m 06s
❌ cifmw-crc-podified-edpm-baremetal-minor-update FAILURE in 2h 04m 07s
✔️ openstack-k8s-operators-content-provider-bootc SUCCESS in 2h 51m 42s
✔️ cifmw-crc-podified-edpm-baremetal-bootc SUCCESS in 1h 23m 57s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 43s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 12s

Wait for mock readiness, reload changed mock code, and allow bounded
application startup. Run the prepared Lightspeed test checkout with an
explicit client identity and retain JUnit/logs before propagating failure.
Remove private client token files and API tunnels on success or failure.

Related-Issue: OSPRH-33344
Signed-off-by: malingatembo <tembomalinga@gmail.com>
@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/b9a9fb66e3b446d79f970656578a8147

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 03m 01s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 22m 35s
✔️ podified-multinode-edpm-deployment-crc-centos-10 SUCCESS in 1h 19m 42s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 30m 48s
❌ cifmw-crc-podified-edpm-baremetal-minor-update NODE_FAILURE Node(set) request 099-0000228726 failed in 0s
✔️ openstack-k8s-operators-content-provider-bootc SUCCESS in 2h 25m 43s
✔️ cifmw-crc-podified-edpm-baremetal-bootc SUCCESS in 1h 27m 09s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 55s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 25s

Serve GET /v1/models with the configured provider

Related-Issue: OSPRH-33344
Signed-off-by: malingatembo <tembomalinga@gmail.com>
@centosinfra-prod-github-app

Copy link
Copy Markdown

Build succeeded (check pipeline).
https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/5cb59774eb6d40c5aaf37f683b53eb7c

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 16m 19s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 23m 56s
✔️ podified-multinode-edpm-deployment-crc-centos-10 SUCCESS in 1h 20m 59s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 35m 45s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 1h 57m 21s
✔️ openstack-k8s-operators-content-provider-bootc SUCCESS in 2h 14m 25s
✔️ cifmw-crc-podified-edpm-baremetal-bootc SUCCESS in 1h 27m 55s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 10m 39s
✔️ cifmw-pod-pre-commit SUCCESS in 9m 55s

@openshift-ci

openshift-ci Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant