electron-rare/exo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update README.md. (#956)

Browse Source 
## Motivation

<!-- Why is this change needed? What problem does it solve? -->
Made a mistake on the merge of the last PR.
<!-- If it fixes an open issue, please link to the issue here -->

## Changes

<!-- Describe what you changed in detail -->

## Why It Works

<!-- Explain why your approach solves the problem -->

## Test Plan

### Manual Testing
<!-- Hardware: (e.g., MacBook Pro M1 Max 32GB, Mac Mini M2 16GB,
connected via Thunderbolt 4) -->
<!-- What you did: -->
<!-- - -->

### Automated Testing
<!-- Describe changes to automated tests, or how existing tests cover
this change -->
<!-- - -->
This commit is contained in:
Alex Cheema
2025-12-21 23:09:44 +00:00
committed by GitHub
parent 7d15fbdaab
commit abaeb0323d
1 changed files with 0 additions and 99 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
-99
View File
@@ -90,105 +90,6 @@ Download the latest build here: [EXO-latest.dmg](https://assets.exolabs.net/EXO-
The app will ask for permission to modify system settings and install a new Network profile. Improvements to this are being worked on.
### Using the API
If you prefer to interact with exo via the API, here is a complete example using `curl` and a real, small model (`mlx-community/Llama-3.2-1B-Instruct-4bit`). All API endpoints and request shapes match `src/exo/master/api.py` and `src/exo/shared/types/api.py`.
---
**1. Preview instance placements**
Obtain valid deployment placements for your model. This helps you choose a valid configuration:
```bash
curl "http://localhost:52415/instance/previews?model_id=mlx-community/Llama-3.2-1B-Instruct-4bit"
```
Sample response:
```json
{
"previews": [
{
"model_id": "mlx-community/Llama-3.2-1B-Instruct-4bit",
"sharding": "Pipeline",
"instance_meta": "MlxRing",
"instance": {...},
"memory_delta_by_node": {"local": 734003200},
"error": null
}
// ...possibly more placements...
]
}
```
This will return all valid placements for this model. Pick a placement that you like.
---
**2. Create a model instance**
Send a POST to `/instance` with your placement in the `instance` field (the full payload must match types as in `CreateInstanceParams`):
```bash
curl -X POST http://localhost:52415/instance \
-H 'Content-Type: application/json' \
-d '{
"instance": {
"model_id": "mlx-community/Llama-3.2-1B-Instruct-4bit",
"instance_meta": "MlxRing",
"sharding": "Pipeline",
"min_nodes": 1
}
}'
```
Sample response:
```json
{
"message": "Command received.",
"command_id": "e9d1a8ab-...."
}
```
---
**3. Issue a chat completion**
Now, make a POST to `/v1/chat/completions` (the same format as OpenAI's API):
```bash
curl -N -X POST http://localhost:52415/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
"messages": [
{"role": "user", "content": "What is Llama 3.2 1B?"}
]
}'
```
You will receive a streamed or non-streamed JSON reply.
---
**4. Delete the instance**
When you're done, delete the instance by its ID (find it via `/state` or `/instance` endpoints):
```bash
curl -X DELETE http://localhost:52415/instance/YOUR_INSTANCE_ID
```
**Tip:**
- List all models: `curl http://localhost:52415/models`
- Inspect instance IDs and deployment state: `curl http://localhost:52415/state`
For further details, see API types and endpoints in `src/exo/master/api.py`.
---
### Using the API