Repository navigation
feat!: Add a local instance pool selector and name pool peers by instance_id - #22
Merged
Merged
Conversation
…n counts - Updated `forecast_samples.ipynb`, `max_utilization.ipynb`, `pandas_result_conversion.ipynb`, `predict_json.ipynb`, `service_health.ipynb` to clear output cells and set execution counts to null for a cleaner notebook state. - This change enhances the readability and maintainability of the notebooks by removing unnecessary output clutter.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
BREAKING CHANGE: JointFMInstanceSettings, InstanceHealth, endpoint_client(),
and the CLI health output name peers by instance_id instead of deployment_id.
HealthMetadata requires the /healthz generation field.
Note
Medium Risk
Breaking renames and a new required health field can break existing integrations; pooling and health-probe paths changed for multi-local setups, affecting routing and capacity behavior without touching auth secrets.
Overview
Breaking: Pool and health APIs now identify peers with
instance_id(hosted deployment ID or normalized local base URL) instead ofdeployment_id. That applies toJointFMInstanceSettings,InstanceHealth,JointFMClient.endpoint_client(), and thejointfm-client healthJSON output.HealthMetadatanow requires agenerationfield (jointfm-1/jointfm-2) from service health payloads.Adds
JOINTFM_LOCAL_BASE_URLS: a comma-separated list of at least two local inference base URLs for round-robin pooling with the same semantics asJOINTFM_DEPLOYMENT_IDS(health aggregation, sample batch splitting, failover, max-utilization). Configuration flows throughconfig.sample.yaml, unified selector resolution (mutually exclusive selectors), andJointFMSettings.is_local/instance_idhelpers. Local pool peers are probed viaPOST {"request_type": "health"}on each service’s predict URL; a single local URL still usesGET /healthz.Docs, notebooks, and samples are updated (generic placeholders, equality-condition behavior).
CLAUDE.mdis replaced by a symlink toAGENTS.md. Tests cover local pool loading, routing, checkpoint mismatch, and transport auth omission for local targets.Reviewed by Cursor Bugbot for commit 14be330. Configure here.