From abaeb0323d4182f7bc4dd3775a8ba9209117d1cf Mon Sep 17 00:00:00 2001 From: Alex Cheema <41707476+AlexCheema@users.noreply.github.com> Date: Sun, 21 Dec 2025 23:09:44 +0000 Subject: [PATCH] Update README.md. (#956) ## Motivation Made a mistake on the merge of the last PR. ## Changes ## Why It Works ## Test Plan ### Manual Testing ### Automated Testing --- README.md | 99 ------------------------------------------------------- 1 file changed, 99 deletions(-) diff --git a/README.md b/README.md index 7d5d6837..f8fe3241 100644 --- a/README.md +++ b/README.md @@ -90,105 +90,6 @@ Download the latest build here: [EXO-latest.dmg](https://assets.exolabs.net/EXO- The app will ask for permission to modify system settings and install a new Network profile. Improvements to this are being worked on. -### Using the API - -If you prefer to interact with exo via the API, here is a complete example using `curl` and a real, small model (`mlx-community/Llama-3.2-1B-Instruct-4bit`). All API endpoints and request shapes match `src/exo/master/api.py` and `src/exo/shared/types/api.py`. - ---- - -**1. Preview instance placements** - -Obtain valid deployment placements for your model. This helps you choose a valid configuration: - -```bash -curl "http://localhost:52415/instance/previews?model_id=mlx-community/Llama-3.2-1B-Instruct-4bit" -``` - -Sample response: - -```json -{ - "previews": [ - { - "model_id": "mlx-community/Llama-3.2-1B-Instruct-4bit", - "sharding": "Pipeline", - "instance_meta": "MlxRing", - "instance": {...}, - "memory_delta_by_node": {"local": 734003200}, - "error": null - } - // ...possibly more placements... - ] -} -``` - -This will return all valid placements for this model. Pick a placement that you like. - ---- - -**2. Create a model instance** - -Send a POST to `/instance` with your placement in the `instance` field (the full payload must match types as in `CreateInstanceParams`): - -```bash -curl -X POST http://localhost:52415/instance \ - -H 'Content-Type: application/json' \ - -d '{ - "instance": { - "model_id": "mlx-community/Llama-3.2-1B-Instruct-4bit", - "instance_meta": "MlxRing", - "sharding": "Pipeline", - "min_nodes": 1 - } - }' -``` - -Sample response: - -```json -{ - "message": "Command received.", - "command_id": "e9d1a8ab-...." -} -``` - ---- - -**3. Issue a chat completion** - -Now, make a POST to `/v1/chat/completions` (the same format as OpenAI's API): - -```bash -curl -N -X POST http://localhost:52415/v1/chat/completions \ - -H 'Content-Type: application/json' \ - -d '{ - "model": "mlx-community/Llama-3.2-1B-Instruct-4bit", - "messages": [ - {"role": "user", "content": "What is Llama 3.2 1B?"} - ] - }' -``` - -You will receive a streamed or non-streamed JSON reply. - ---- - -**4. Delete the instance** - -When you're done, delete the instance by its ID (find it via `/state` or `/instance` endpoints): - -```bash -curl -X DELETE http://localhost:52415/instance/YOUR_INSTANCE_ID -``` - -**Tip:** - -- List all models: `curl http://localhost:52415/models` -- Inspect instance IDs and deployment state: `curl http://localhost:52415/state` - -For further details, see API types and endpoints in `src/exo/master/api.py`. - - --- ### Using the API