blanchon commited on
Commit
67a499d
·
1 Parent(s): a957565
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .prettierignore +9 -0
  2. README.md +85 -79
  3. src/app.css +105 -105
  4. src/app.d.ts +1 -1
  5. src/lib/components/3d/elements/compute/ComputeGridItem.svelte +0 -1
  6. src/lib/components/3d/elements/compute/Computes.svelte +21 -5
  7. src/lib/components/3d/elements/compute/GPU.svelte +11 -14
  8. src/lib/components/3d/elements/compute/GPUModel.svelte +167 -182
  9. src/lib/components/3d/elements/compute/modal/AISessionConnectionModal.svelte +11 -9
  10. src/lib/components/3d/elements/compute/status/ComputeBoxUIKit.svelte +1 -5
  11. src/lib/components/3d/elements/compute/status/ComputeStatusBillboard.svelte +13 -7
  12. src/lib/components/3d/elements/compute/status/RobotOutputBoxUIKit.svelte +20 -14
  13. src/lib/components/3d/elements/compute/status/VideoInputBoxUIKit.svelte +18 -16
  14. src/lib/components/3d/elements/robot/RobotGridItem.svelte +11 -8
  15. src/lib/components/3d/elements/robot/URDF/primitives/UrdfJoint.svelte +2 -8
  16. src/lib/components/3d/elements/robot/URDF/primitives/UrdfLink.svelte +1 -1
  17. src/lib/components/3d/elements/robot/URDF/utils/UrdfParser.ts +0 -2
  18. src/lib/components/3d/elements/robot/modal/InputConnectionModal.svelte +6 -5
  19. src/lib/components/3d/elements/robot/modal/OutputConnectionModal.svelte +11 -10
  20. src/lib/components/3d/elements/robot/status/ConnectionFlowBoxUIkit.svelte +21 -7
  21. src/lib/components/3d/elements/robot/status/RobotBoxUIKit.svelte +17 -28
  22. src/lib/components/3d/elements/robot/status/RobotStatusBillboard.svelte +28 -28
  23. src/lib/components/3d/elements/video/modal/VideoInputConnectionModal.svelte +6 -5
  24. src/lib/components/3d/elements/video/modal/VideoOutputConnectionModal.svelte +6 -6
  25. src/lib/components/3d/elements/video/status/VideoBoxUIKit.svelte +17 -27
  26. src/lib/components/3d/elements/video/status/VideoConnectionFlowBoxUIKit.svelte +20 -7
  27. src/lib/components/3d/elements/video/status/VideoStatusBillboard.svelte +26 -26
  28. src/lib/components/3d/ui/StatusArrow.svelte +10 -28
  29. src/lib/components/3d/ui/StatusButton.svelte +3 -9
  30. src/lib/components/3d/ui/StatusContent.svelte +14 -18
  31. src/lib/components/3d/ui/icons.ts +4 -3
  32. src/lib/components/3d/ui/index.ts +6 -6
  33. src/lib/components/interface/overlay/AddAIButton.svelte +9 -7
  34. src/lib/components/interface/overlay/AddRobotButton.svelte +4 -3
  35. src/lib/components/interface/overlay/AddSensorButton.svelte +28 -29
  36. src/lib/components/interface/overlay/Overlay.svelte +9 -7
  37. src/lib/components/interface/overlay/SettingsSheet.svelte +2 -1
  38. src/lib/configs/robotUrdfConfig.ts +2 -2
  39. src/lib/elements/compute/RemoteCompute.svelte.ts +150 -144
  40. src/lib/elements/compute/RemoteComputeManager.svelte.ts +486 -477
  41. src/lib/elements/compute/index.ts +11 -5
  42. src/lib/elements/robot/Robot.svelte.ts +535 -494
  43. src/lib/elements/robot/RobotManager.svelte.ts +269 -256
  44. src/lib/elements/robot/calibration/CalibrationState.svelte.ts +290 -269
  45. src/lib/elements/robot/calibration/USBCalibrationPanel.svelte +14 -5
  46. src/lib/elements/robot/calibration/index.ts +2 -2
  47. src/lib/elements/robot/components/ConnectionPanel.svelte +5 -7
  48. src/lib/elements/robot/components/RobotGrid.svelte +101 -96
  49. src/lib/elements/robot/components/RobotItem.svelte +212 -200
  50. src/lib/elements/robot/components/index.ts +4 -4
.prettierignore CHANGED
@@ -10,3 +10,12 @@ src-python/
10
  node_modules/
11
  build/
12
  .svelte-kit/
 
 
 
 
 
 
 
 
 
 
10
  node_modules/
11
  build/
12
  .svelte-kit/
13
+
14
+ # External
15
+ external/
16
+
17
+ # Build
18
+ build/
19
+
20
+ # Package
21
+ packages/
README.md CHANGED
@@ -15,14 +15,14 @@ app_port: 8000
15
  pinned: true
16
  license: mit
17
  fullWidth: true
18
- short_description: Web interface of the RobotHub platform
19
  ---
20
 
21
  # 🤖 RobotHub Arena – Frontend
22
 
23
  RobotHub is an **open-source, end-to-end robotics stack** that combines real-time communication, 3-D visualisation, and modern AI policies to control both simulated and physical robots.
24
 
25
- **This repository contains the *Frontend*** – a SvelteKit web application that runs completely in the browser (or inside Electron / Tauri). It talks to two backend micro-services that live in their own repositories:
26
 
27
  1. **[RobotHub Transport Server](https://github.com/julien-blanchon/RobotHub-TransportServer)**
28
  – WebSocket / WebRTC switch-board for video streams & robot joint messages.
@@ -46,22 +46,22 @@ RobotHub is an **open-source, end-to-end robotics stack** that combines real-tim
46
  ## ✨ Key Features
47
 
48
  • **Digital-Twin 3-D Scene** – inspect robots, cameras & AI compute blocks in real-time.
49
- • **Multi-Workspace Collaboration** – share a hash URL and others join the *same* WS rooms instantly.
50
  • **Drag-&-Drop Add-ons** – spawn robots, cameras or AI models from the toolbar.
51
  • **Transport-Agnostic** – control physical hardware over USB, or send/receive via WebRTC rooms.
52
  • **Model Agnostic** – any policy exposed by the Inference Server can be used (ACT, Diffusion, …).
53
- • **Reactive Core** – built with *Svelte 5 runes* – state is automatically pushed into the UI.
54
 
55
  ---
56
 
57
  ## 📂 Repository Layout (short)
58
 
59
- | Path | Purpose |
60
- |-------------------------------|---------|
61
- | `src/` | SvelteKit app (routes, components) |
62
- | `src/lib/elements` | Runtime domain logic (robots, video, compute) |
63
- | `external/RobotHub-*` | Git sub-modules for the backend services – used for generated clients & tests |
64
- | `static/` | URDFs, STL meshes, textures, favicon |
65
 
66
  A more in-depth component overview can be found in `/src/lib/components/**` – every major popup/modal has its own Svelte file.
67
 
@@ -96,34 +96,34 @@ $ python launch_simple.py # → http://localhost:8001
96
  $ bun run dev -- --open # → http://localhost:5173 (hash = workspace-id)
97
  ```
98
 
99
- The **workspace-id** in the URL hash ties all three services together. Share `http://localhost:5173/#<id>` and a collaborator instantly joins the same set of rooms.
100
 
101
  ---
102
 
103
  ## 🛠️ Usage Walk-Through
104
 
105
- 1. **Open the web-app** → a fresh *workspace* is created (☝ left corner shows 🌐 ID).
106
- 2. Click *Add Robot* → spawns an SO-100 6-DoF arm (URDF).
107
- 3. Click *Add Sensor → Camera* → creates a virtual camera element.
108
- 4. Click *Add Model → ACT* → spawns a *Compute* block.
109
- 5. On the Compute block choose *Create Session* – select model path (`LaetusH/act_so101_beyond`) and cameras (`front`).
110
  6. Connect:
111
- *Video Input* – local webcam → `front` room.
112
- *Robot Input* – robot → *joint-input* room (producer).
113
- *Robot Output* – robot ← AI predictions (consumer).
114
- 7. Press *Start Inference* – the model will predict the next joint trajectory every few frames. 🎉
115
 
116
- All modals (`AISessionConnectionModal`, `RobotInputConnectionModal`, …) expose precisely what is happening under the hood: which room ID, whether you are *producer* or *consumer*, and the live status.
117
 
118
  ---
119
 
120
  ## 🧩 Package Relations
121
 
122
- | Package | Role | Artifacts exposed to this repo |
123
- |---------|------|--------------------------------|
124
- | **Transport Server** | Low-latency switch-board (WS/WebRTC). Creates *rooms* for video & joint messages. | TypeScript & Python client libraries (imported from sub-module) |
125
- | **Inference Server** | Loads checkpoints (ACT, Pi-0, …) and manages *sessions*. Each session automatically asks the Transport Server to create dedicated rooms. | Generated TS SDK (`@robothub/inference-server-client`) – auto-called from `RemoteComputeManager` |
126
- | **Frontend (this repo)** | UI + 3-D scene. Manages *robots*, *videos* & *compute* blocks and connects them to the correct rooms. | – |
127
 
128
  > Because the two backend repos are included as git sub-modules you can develop & debug the whole trio in one repo clone.
129
 
@@ -135,7 +135,7 @@ All modals (`AISessionConnectionModal`, `RobotInputConnectionModal`, …) expose
135
  • `RobotManager` – talks to Transport Server and USB drivers.
136
  • `VideoManager` – handles local/remote camera streams and WebRTC.
137
 
138
- Each element is a small class with `$state` fields which Svelte 5 picks up automatically. The modals listed below are *thin* UI shells around those classes:
139
 
140
  ```
141
  AISessionConnectionModal – create / start / stop AI sessions
@@ -163,9 +163,9 @@ See `Dockerfile` for the full build – it also performs `bun test` & `bun run b
163
 
164
  ## 🧑‍💻 Contributing
165
 
166
- PRs are welcome! The codebase is organised into **domain managers** (robot / video / compute) and **pure-UI** components. If you add a new feature, create a manager first so that business logic can be unit-tested without DOM.
167
 
168
- 1. `bun test` – unit tests.
169
  2. `bun run typecheck` – strict TS config.
170
 
171
  Please run `bun format` before committing – ESLint + Prettier configs are included.
@@ -176,7 +176,8 @@ Please run `bun format` before committing – ESLint + Prettier configs are incl
176
 
177
  Huge gratitude to [Tim Qian](https://github.com/timqian) ([X/Twitter](https://x.com/tim_qian)) and the
178
  [bambot project](https://bambot.org/) for open-sourcing **feetech.js** – the
179
- delightful js driver that powers our USB communication layer.
 
180
  ---
181
 
182
  ## 📄 License
@@ -187,34 +188,34 @@ MIT – see `LICENSE` in the root.
187
 
188
  RobotHub follows a **separation-of-concerns** design:
189
 
190
- * **Transport Server** is the single source of truth for *real-time* data – video frames, joint values, heart-beats. Every participant (browser, Python script, robot firmware) only needs one WebSocket/WebRTC connection, no matter how many peers join later.
191
- * **Inference Server** is stateless with regard to connectivity; it spins up / tears down *sessions* that rely on rooms in the Transport Server. This lets heavy AI models live on a GPU box while cameras and robots stay on the edge.
192
- * **Frontend** stays 100 % in the browser – no secret keys or device drivers required – and simply wires together rooms that already exist.
193
 
194
  > By decoupling the pipeline we can deploy each piece on separate hardware or even different clouds, swap alternative implementations (e.g. ROS bridge instead of WebRTC) and scale each micro-service independently.
195
 
196
  ---
197
 
198
- ## 🛰 Transport Server – Real-Time Router
199
 
200
  ```
201
  Browser / Robot ⟷ 🌐 Transport Server ⟷ Other Browser / AI / HW
202
  ```
203
 
204
- * **Creates rooms** – `POST /robotics/workspaces/{ws}/rooms` or `POST /video/workspaces/{ws}/rooms`.
205
- * **Manages roles** – every WebSocket identifies as *producer* (source) or *consumer* (sink).
206
- * **Does zero processing** – it only forwards JSON (robotics) or WebRTC SDP/ICE (video).
207
- * **Health-check** – `GET /api/health` returns a JSON heartbeat.
208
 
209
  Why useful?
210
 
211
- * You never expose robot hardware directly to the internet – it only speaks to the Transport Server.
212
- * Multiple followers can subscribe to the *same* producer without extra bandwidth on the producer side (server fans out messages).
213
- * Works across NAT thanks to WebRTC TURN support.
214
 
215
- ## 🏢 Workspaces – Lightweight Multi-Tenant Isolation
216
 
217
- A **workspace** is simply a UUID namespace in the Transport Server. Every room URL starts with:
218
 
219
  ```
220
  /robotics/workspaces/{workspace_id}/rooms/{room_id}
@@ -223,105 +224,110 @@ A **workspace** is simply a UUID namespace in the Transport Server. Every room
223
 
224
  Why bother?
225
 
226
- 1. **Privacy / Security** – clients in workspace *A* can neither list nor join rooms from workspace *B*. A workspace id is like a private password that keeps the rooms in the same workspace isolated from each other.
227
  2. **Organisation** – keep each class, project or experiment separated without spinning up extra servers.
228
- 3. **Zero-config sharing** – the Frontend stores the workspace ID in the URL hash (e.g. `/#d742e85d-c9e9-4f7b-…`). Send that link to a teammate and they automatically connect to the *same* namespace – all existing video feeds, robot rooms and AI sessions become visible.
229
  4. **Stateless Scale-out** – Transport Server holds no global state; deleting a workspace removes all rooms in one call.
230
 
231
  Typical lifecycle:
232
 
233
- * **Create** – Frontend generates `crypto.randomUUID()` if the hash is empty. Back-end rooms are lazily created when the first producer/consumer calls the REST API.
234
- * **Share** – click the *#workspace* badge → *Copy URL* (handled by `WorkspaceIdButton.svelte`)
235
 
236
  > Practical tip: Use one workspace per demo to prevent collisions, then recycle it afterwards.
237
 
238
  ---
239
 
240
- ## 🧠 Inference Server – Session Lifecycle
241
 
242
  1. **Create session**
243
  `POST /api/sessions` with JSON:
244
  ```jsonc
245
  {
246
- "session_id": "pick_place_demo",
247
- "policy_path": "LaetusH/act_so101_beyond",
248
- "camera_names": ["front", "wrist"],
249
- "transport_server_url": "http://localhost:8000",
250
- "workspace_id": "<existing-or-new>" // optional
251
  }
252
  ```
253
- 2. **Receive response**
254
  ```jsonc
255
  {
256
- "workspace_id": "ws-uuid",
257
- "camera_room_ids": { "front": "room-id-a", "wrist": "room-id-b" },
258
- "joint_input_room_id": "room-id-c",
259
- "joint_output_room_id": "room-id-d"
260
  }
261
  ```
262
  3. **Wire connections**
263
- * Camera PC joins `front` / `wrist` rooms as **producer** (WebRTC).
264
- * Robot joins `joint_input_room_id` as **producer** (joint states).
265
- * Robot (or simulator) joins `joint_output_room_id` as **consumer** (commands).
266
  4. **Start inference**
267
  `POST /api/sessions/{id}/start` – server loads the model and begins publishing commands.
268
- 5. **Stop / delete** as needed. Stats & health are available via `GET /api/sessions`.
269
 
270
- The Frontend automates steps 1-4 via the *AI Session* modal – you only click buttons.
271
 
272
  ---
273
 
274
  ## 🌐 Hosted Demo End-Points
275
 
276
- | Service | URL | Status |
277
- |---------|-----|--------|
278
- | Transport Server | <https://blanchon-robothub-transportserver.hf.space/api> | Public & healthy |
279
- | Inference Server | <https://blanchon-robothub-inferenceserver.hf.space/api> | `{"status":"healthy"}` |
280
- | Frontend (read-only preview) | <https://blanchon-robothub-frontend.hf.space> | latest `main` |
281
 
282
- Point the *Settings → Server Configuration* panel to these URLs and you can play without any local backend.
283
 
284
  ---
285
 
286
  ## 🎯 Main Use-Cases
287
 
288
- Below are typical connection patterns you can set-up **entirely from the UI**. Each example lists the raw data-flow (→ = producer to consumer/AI) plus a video placeholder you can swap for a screen-capture.
289
 
290
  ### Direct Tele-Operation (Leader ➜ Follower)
291
- *Leader PC* `USB` ➜ **Robot A** ➜ `Remote producer` → **Transport room** → `Remote consumer` ➜ **Robot B** (`USB`)
 
292
 
293
  > One human moves Robot A, Robot B mirrors the motion in real-time. Works with any number of followers – just add more consumers to the same room.
294
  >
295
- > 📺 *demo-teleop-1.mp4*
296
 
297
  ### Web-UI Manual Control
 
298
  **Browser sliders** (`ManualControlSheet`) → `Remote producer` → **Robot (USB)**
299
 
300
  > No physical master arm needed – drive joints from any device.
301
  >
302
- > 📺 *demo-webui.mp4*
303
 
304
  ### AI Inference Loop
 
305
  **Robot (USB)** ➜ `Remote producer` → **joint-input room**
306
  **Camera PC** ➜ `Video producer` → **camera room(s)**
307
  **Inference Server** (consumer) → processes → publishes to **joint-output room** → `Remote consumer` ➜ **Robot**
308
 
309
  > Lets a low-power robot PC stream data while a beefy GPU node does the heavy lifting.
310
  >
311
- > 📺 *demo-inference.mp4*
312
 
313
  ### Hybrid Classroom (Multi-Follower AI)
314
- *Same as AI Inference Loop* with additional **Robot C, D…** subscribing to `joint_output_room_id` to run the same policy in parallel.
 
315
 
316
  > Useful for swarm behaviours or classroom demonstrations.
317
  >
318
- > 📺 *demo-classroom.mp4*
319
 
320
  ### Split Video / Robot Across Machines
 
321
  **Laptop A** (near cameras) → streams video → Transport
322
- **Laptop B** (near robot) → joins joint rooms
323
- **Browser** anywhere → watches video consumer & sends manual overrides
324
 
325
  > Ideal when the camera PC stays close to sensors and you want minimal upstream bandwidth.
326
  >
327
- > 📺 *demo-splitio.mp4*
 
15
  pinned: true
16
  license: mit
17
  fullWidth: true
18
+ short_description: Web interface of the RobotHub platform
19
  ---
20
 
21
  # 🤖 RobotHub Arena – Frontend
22
 
23
  RobotHub is an **open-source, end-to-end robotics stack** that combines real-time communication, 3-D visualisation, and modern AI policies to control both simulated and physical robots.
24
 
25
+ **This repository contains the _Frontend_** – a SvelteKit web application that runs completely in the browser (or inside Electron / Tauri). It talks to two backend micro-services that live in their own repositories:
26
 
27
  1. **[RobotHub Transport Server](https://github.com/julien-blanchon/RobotHub-TransportServer)**
28
  – WebSocket / WebRTC switch-board for video streams & robot joint messages.
 
46
  ## ✨ Key Features
47
 
48
  • **Digital-Twin 3-D Scene** – inspect robots, cameras & AI compute blocks in real-time.
49
+ • **Multi-Workspace Collaboration** – share a hash URL and others join the _same_ WS rooms instantly.
50
  • **Drag-&-Drop Add-ons** – spawn robots, cameras or AI models from the toolbar.
51
  • **Transport-Agnostic** – control physical hardware over USB, or send/receive via WebRTC rooms.
52
  • **Model Agnostic** – any policy exposed by the Inference Server can be used (ACT, Diffusion, …).
53
+ • **Reactive Core** – built with _Svelte 5 runes_ – state is automatically pushed into the UI.
54
 
55
  ---
56
 
57
  ## 📂 Repository Layout (short)
58
 
59
+ | Path | Purpose |
60
+ | --------------------- | ----------------------------------------------------------------------------- |
61
+ | `src/` | SvelteKit app (routes, components) |
62
+ | `src/lib/elements` | Runtime domain logic (robots, video, compute) |
63
+ | `external/RobotHub-*` | Git sub-modules for the backend services – used for generated clients & tests |
64
+ | `static/` | URDFs, STL meshes, textures, favicon |
65
 
66
  A more in-depth component overview can be found in `/src/lib/components/**` – every major popup/modal has its own Svelte file.
67
 
 
96
  $ bun run dev -- --open # → http://localhost:5173 (hash = workspace-id)
97
  ```
98
 
99
+ The **workspace-id** in the URL hash ties all three services together. Share `http://localhost:5173/#<id>` and a collaborator instantly joins the same set of rooms.
100
 
101
  ---
102
 
103
  ## 🛠️ Usage Walk-Through
104
 
105
+ 1. **Open the web-app** → a fresh _workspace_ is created (☝ left corner shows 🌐 ID).
106
+ 2. Click _Add Robot_ → spawns an SO-100 6-DoF arm (URDF).
107
+ 3. Click _Add Sensor → Camera_ → creates a virtual camera element.
108
+ 4. Click _Add Model → ACT_ → spawns a _Compute_ block.
109
+ 5. On the Compute block choose _Create Session_ – select model path (`LaetusH/act_so101_beyond`) and cameras (`front`).
110
  6. Connect:
111
+ _Video Input_ – local webcam → `front` room.
112
+ _Robot Input_ – robot → _joint-input_ room (producer).
113
+ _Robot Output_ – robot ← AI predictions (consumer).
114
+ 7. Press _Start Inference_ – the model will predict the next joint trajectory every few frames. 🎉
115
 
116
+ All modals (`AISessionConnectionModal`, `RobotInputConnectionModal`, …) expose precisely what is happening under the hood: which room ID, whether you are _producer_ or _consumer_, and the live status.
117
 
118
  ---
119
 
120
  ## 🧩 Package Relations
121
 
122
+ | Package | Role | Artifacts exposed to this repo |
123
+ | ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ |
124
+ | **Transport Server** | Low-latency switch-board (WS/WebRTC). Creates _rooms_ for video & joint messages. | TypeScript & Python client libraries (imported from sub-module) |
125
+ | **Inference Server** | Loads checkpoints (ACT, Pi-0, …) and manages _sessions_. Each session automatically asks the Transport Server to create dedicated rooms. | Generated TS SDK (`@robothub/inference-server-client`) – auto-called from `RemoteComputeManager` |
126
+ | **Frontend (this repo)** | UI + 3-D scene. Manages _robots_, _videos_ & _compute_ blocks and connects them to the correct rooms. | – |
127
 
128
  > Because the two backend repos are included as git sub-modules you can develop & debug the whole trio in one repo clone.
129
 
 
135
  • `RobotManager` – talks to Transport Server and USB drivers.
136
  • `VideoManager` – handles local/remote camera streams and WebRTC.
137
 
138
+ Each element is a small class with `$state` fields which Svelte 5 picks up automatically. The modals listed below are _thin_ UI shells around those classes:
139
 
140
  ```
141
  AISessionConnectionModal – create / start / stop AI sessions
 
163
 
164
  ## 🧑‍💻 Contributing
165
 
166
+ PRs are welcome! The codebase is organised into **domain managers** (robot / video / compute) and **pure-UI** components. If you add a new feature, create a manager first so that business logic can be unit-tested without DOM.
167
 
168
+ 1. `bun test` – unit tests.
169
  2. `bun run typecheck` – strict TS config.
170
 
171
  Please run `bun format` before committing – ESLint + Prettier configs are included.
 
176
 
177
  Huge gratitude to [Tim Qian](https://github.com/timqian) ([X/Twitter](https://x.com/tim_qian)) and the
178
  [bambot project](https://bambot.org/) for open-sourcing **feetech.js** – the
179
+ delightful js driver that powers our USB communication layer.
180
+
181
  ---
182
 
183
  ## 📄 License
 
188
 
189
  RobotHub follows a **separation-of-concerns** design:
190
 
191
+ - **Transport Server** is the single source of truth for _real-time_ data – video frames, joint values, heart-beats. Every participant (browser, Python script, robot firmware) only needs one WebSocket/WebRTC connection, no matter how many peers join later.
192
+ - **Inference Server** is stateless with regard to connectivity; it spins up / tears down _sessions_ that rely on rooms in the Transport Server. This lets heavy AI models live on a GPU box while cameras and robots stay on the edge.
193
+ - **Frontend** stays 100 % in the browser – no secret keys or device drivers required – and simply wires together rooms that already exist.
194
 
195
  > By decoupling the pipeline we can deploy each piece on separate hardware or even different clouds, swap alternative implementations (e.g. ROS bridge instead of WebRTC) and scale each micro-service independently.
196
 
197
  ---
198
 
199
+ ## 🛰 Transport Server – Real-Time Router
200
 
201
  ```
202
  Browser / Robot ⟷ 🌐 Transport Server ⟷ Other Browser / AI / HW
203
  ```
204
 
205
+ - **Creates rooms** – `POST /robotics/workspaces/{ws}/rooms` or `POST /video/workspaces/{ws}/rooms`.
206
+ - **Manages roles** – every WebSocket identifies as _producer_ (source) or _consumer_ (sink).
207
+ - **Does zero processing** – it only forwards JSON (robotics) or WebRTC SDP/ICE (video).
208
+ - **Health-check** – `GET /api/health` returns a JSON heartbeat.
209
 
210
  Why useful?
211
 
212
+ - You never expose robot hardware directly to the internet – it only speaks to the Transport Server.
213
+ - Multiple followers can subscribe to the _same_ producer without extra bandwidth on the producer side (server fans out messages).
214
+ - Works across NAT thanks to WebRTC TURN support.
215
 
216
+ ## 🏢 Workspaces – Lightweight Multi-Tenant Isolation
217
 
218
+ A **workspace** is simply a UUID namespace in the Transport Server. Every room URL starts with:
219
 
220
  ```
221
  /robotics/workspaces/{workspace_id}/rooms/{room_id}
 
224
 
225
  Why bother?
226
 
227
+ 1. **Privacy / Security** – clients in workspace _A_ can neither list nor join rooms from workspace _B_. A workspace id is like a private password that keeps the rooms in the same workspace isolated from each other.
228
  2. **Organisation** – keep each class, project or experiment separated without spinning up extra servers.
229
+ 3. **Zero-config sharing** – the Frontend stores the workspace ID in the URL hash (e.g. `/#d742e85d-c9e9-4f7b-…`). Send that link to a teammate and they automatically connect to the _same_ namespace – all existing video feeds, robot rooms and AI sessions become visible.
230
  4. **Stateless Scale-out** – Transport Server holds no global state; deleting a workspace removes all rooms in one call.
231
 
232
  Typical lifecycle:
233
 
234
+ - **Create** – Frontend generates `crypto.randomUUID()` if the hash is empty. Back-end rooms are lazily created when the first producer/consumer calls the REST API.
235
+ - **Share** – click the _#workspace_ badge → _Copy URL_ (handled by `WorkspaceIdButton.svelte`)
236
 
237
  > Practical tip: Use one workspace per demo to prevent collisions, then recycle it afterwards.
238
 
239
  ---
240
 
241
+ ## 🧠 Inference Server – Session Lifecycle
242
 
243
  1. **Create session**
244
  `POST /api/sessions` with JSON:
245
  ```jsonc
246
  {
247
+ "session_id": "pick_place_demo",
248
+ "policy_path": "LaetusH/act_so101_beyond",
249
+ "camera_names": ["front", "wrist"],
250
+ "transport_server_url": "http://localhost:8000",
251
+ "workspace_id": "<existing-or-new>" // optional
252
  }
253
  ```
254
+ 2. **Receive response**
255
  ```jsonc
256
  {
257
+ "workspace_id": "ws-uuid",
258
+ "camera_room_ids": { "front": "room-id-a", "wrist": "room-id-b" },
259
+ "joint_input_room_id": "room-id-c",
260
+ "joint_output_room_id": "room-id-d"
261
  }
262
  ```
263
  3. **Wire connections**
264
+ - Camera PC joins `front` / `wrist` rooms as **producer** (WebRTC).
265
+ - Robot joins `joint_input_room_id` as **producer** (joint states).
266
+ - Robot (or simulator) joins `joint_output_room_id` as **consumer** (commands).
267
  4. **Start inference**
268
  `POST /api/sessions/{id}/start` – server loads the model and begins publishing commands.
269
+ 5. **Stop / delete** as needed. Stats & health are available via `GET /api/sessions`.
270
 
271
+ The Frontend automates steps 1-4 via the _AI Session_ modal – you only click buttons.
272
 
273
  ---
274
 
275
  ## 🌐 Hosted Demo End-Points
276
 
277
+ | Service | URL | Status |
278
+ | ---------------------------- | -------------------------------------------------------- | ---------------------- |
279
+ | Transport Server | <https://blanchon-robothub-transportserver.hf.space/api> | Public & healthy |
280
+ | Inference Server | <https://blanchon-robothub-inferenceserver.hf.space/api> | `{"status":"healthy"}` |
281
+ | Frontend (read-only preview) | <https://blanchon-robothub-frontend.hf.space> | latest `main` |
282
 
283
+ Point the _Settings → Server Configuration_ panel to these URLs and you can play without any local backend.
284
 
285
  ---
286
 
287
  ## 🎯 Main Use-Cases
288
 
289
+ Below are typical connection patterns you can set-up **entirely from the UI**. Each example lists the raw data-flow (→ = producer to consumer/AI) plus a video placeholder you can swap for a screen-capture.
290
 
291
  ### Direct Tele-Operation (Leader ➜ Follower)
292
+
293
+ _Leader PC_ `USB` ➜ **Robot A** ➜ `Remote producer` → **Transport room** → `Remote consumer` ➜ **Robot B** (`USB`)
294
 
295
  > One human moves Robot A, Robot B mirrors the motion in real-time. Works with any number of followers – just add more consumers to the same room.
296
  >
297
+ > 📺 _demo-teleop-1.mp4_
298
 
299
  ### Web-UI Manual Control
300
+
301
  **Browser sliders** (`ManualControlSheet`) → `Remote producer` → **Robot (USB)**
302
 
303
  > No physical master arm needed – drive joints from any device.
304
  >
305
+ > 📺 _demo-webui.mp4_
306
 
307
  ### AI Inference Loop
308
+
309
  **Robot (USB)** ➜ `Remote producer` → **joint-input room**
310
  **Camera PC** ➜ `Video producer` → **camera room(s)**
311
  **Inference Server** (consumer) → processes → publishes to **joint-output room** → `Remote consumer` ➜ **Robot**
312
 
313
  > Lets a low-power robot PC stream data while a beefy GPU node does the heavy lifting.
314
  >
315
+ > 📺 _demo-inference.mp4_
316
 
317
  ### Hybrid Classroom (Multi-Follower AI)
318
+
319
+ _Same as AI Inference Loop_ with additional **Robot C, D…** subscribing to `joint_output_room_id` to run the same policy in parallel.
320
 
321
  > Useful for swarm behaviours or classroom demonstrations.
322
  >
323
+ > 📺 _demo-classroom.mp4_
324
 
325
  ### Split Video / Robot Across Machines
326
+
327
  **Laptop A** (near cameras) → streams video → Transport
328
+ **Laptop B** (near robot) → joins joint rooms
329
+ **Browser** anywhere → watches video consumer & sends manual overrides
330
 
331
  > Ideal when the camera PC stays close to sensors and you want minimal upstream bandwidth.
332
  >
333
+ > 📺 _demo-splitio.mp4_
src/app.css CHANGED
@@ -6,117 +6,117 @@
6
  @custom-variant dark (&:is(.dark *));
7
 
8
  :root {
9
- --radius: 0.625rem;
10
- --background: oklch(1 0 0);
11
- --foreground: oklch(0.147 0.004 49.25);
12
- --card: oklch(1 0 0);
13
- --card-foreground: oklch(0.147 0.004 49.25);
14
- --popover: oklch(1 0 0);
15
- --popover-foreground: oklch(0.147 0.004 49.25);
16
- --primary: oklch(0.216 0.006 56.043);
17
- --primary-foreground: oklch(0.985 0.001 106.423);
18
- --secondary: oklch(0.97 0.001 106.424);
19
- --secondary-foreground: oklch(0.216 0.006 56.043);
20
- --muted: oklch(0.97 0.001 106.424);
21
- --muted-foreground: oklch(0.553 0.013 58.071);
22
- --accent: oklch(0.97 0.001 106.424);
23
- --accent-foreground: oklch(0.216 0.006 56.043);
24
- --destructive: oklch(0.577 0.245 27.325);
25
- --border: oklch(0.923 0.003 48.717);
26
- --input: oklch(0.923 0.003 48.717);
27
- --ring: oklch(0.709 0.01 56.259);
28
- --chart-1: oklch(0.646 0.222 41.116);
29
- --chart-2: oklch(0.6 0.118 184.704);
30
- --chart-3: oklch(0.398 0.07 227.392);
31
- --chart-4: oklch(0.828 0.189 84.429);
32
- --chart-5: oklch(0.769 0.188 70.08);
33
- --sidebar: oklch(0.985 0.001 106.423);
34
- --sidebar-foreground: oklch(0.147 0.004 49.25);
35
- --sidebar-primary: oklch(0.216 0.006 56.043);
36
- --sidebar-primary-foreground: oklch(0.985 0.001 106.423);
37
- --sidebar-accent: oklch(0.97 0.001 106.424);
38
- --sidebar-accent-foreground: oklch(0.216 0.006 56.043);
39
- --sidebar-border: oklch(0.923 0.003 48.717);
40
- --sidebar-ring: oklch(0.709 0.01 56.259);
41
  }
42
 
43
  .dark {
44
- --background: oklch(0.147 0.004 49.25);
45
- --foreground: oklch(0.985 0.001 106.423);
46
- --card: oklch(0.216 0.006 56.043);
47
- --card-foreground: oklch(0.985 0.001 106.423);
48
- --popover: oklch(0.216 0.006 56.043);
49
- --popover-foreground: oklch(0.985 0.001 106.423);
50
- --primary: oklch(0.923 0.003 48.717);
51
- --primary-foreground: oklch(0.216 0.006 56.043);
52
- --secondary: oklch(0.268 0.007 34.298);
53
- --secondary-foreground: oklch(0.985 0.001 106.423);
54
- --muted: oklch(0.268 0.007 34.298);
55
- --muted-foreground: oklch(0.709 0.01 56.259);
56
- --accent: oklch(0.268 0.007 34.298);
57
- --accent-foreground: oklch(0.985 0.001 106.423);
58
- --destructive: oklch(0.704 0.191 22.216);
59
- --border: oklch(1 0 0 / 10%);
60
- --input: oklch(1 0 0 / 15%);
61
- --ring: oklch(0.553 0.013 58.071);
62
- --chart-1: oklch(0.488 0.243 264.376);
63
- --chart-2: oklch(0.696 0.17 162.48);
64
- --chart-3: oklch(0.769 0.188 70.08);
65
- --chart-4: oklch(0.627 0.265 303.9);
66
- --chart-5: oklch(0.645 0.246 16.439);
67
- --sidebar: oklch(0.216 0.006 56.043);
68
- --sidebar-foreground: oklch(0.985 0.001 106.423);
69
- --sidebar-primary: oklch(0.488 0.243 264.376);
70
- --sidebar-primary-foreground: oklch(0.985 0.001 106.423);
71
- --sidebar-accent: oklch(0.268 0.007 34.298);
72
- --sidebar-accent-foreground: oklch(0.985 0.001 106.423);
73
- --sidebar-border: oklch(1 0 0 / 10%);
74
- --sidebar-ring: oklch(0.553 0.013 58.071);
75
  }
76
 
77
  @theme inline {
78
- --radius-sm: calc(var(--radius) - 4px);
79
- --radius-md: calc(var(--radius) - 2px);
80
- --radius-lg: var(--radius);
81
- --radius-xl: calc(var(--radius) + 4px);
82
- --color-background: var(--background);
83
- --color-foreground: var(--foreground);
84
- --color-card: var(--card);
85
- --color-card-foreground: var(--card-foreground);
86
- --color-popover: var(--popover);
87
- --color-popover-foreground: var(--popover-foreground);
88
- --color-primary: var(--primary);
89
- --color-primary-foreground: var(--primary-foreground);
90
- --color-secondary: var(--secondary);
91
- --color-secondary-foreground: var(--secondary-foreground);
92
- --color-muted: var(--muted);
93
- --color-muted-foreground: var(--muted-foreground);
94
- --color-accent: var(--accent);
95
- --color-accent-foreground: var(--accent-foreground);
96
- --color-destructive: var(--destructive);
97
- --color-border: var(--border);
98
- --color-input: var(--input);
99
- --color-ring: var(--ring);
100
- --color-chart-1: var(--chart-1);
101
- --color-chart-2: var(--chart-2);
102
- --color-chart-3: var(--chart-3);
103
- --color-chart-4: var(--chart-4);
104
- --color-chart-5: var(--chart-5);
105
- --color-sidebar: var(--sidebar);
106
- --color-sidebar-foreground: var(--sidebar-foreground);
107
- --color-sidebar-primary: var(--sidebar-primary);
108
- --color-sidebar-primary-foreground: var(--sidebar-primary-foreground);
109
- --color-sidebar-accent: var(--sidebar-accent);
110
- --color-sidebar-accent-foreground: var(--sidebar-accent-foreground);
111
- --color-sidebar-border: var(--sidebar-border);
112
- --color-sidebar-ring: var(--sidebar-ring);
113
  }
114
 
115
  @layer base {
116
- * {
117
- @apply border-border outline-ring/50;
118
- }
119
- body {
120
- @apply bg-background text-foreground;
121
- }
122
- }
 
6
  @custom-variant dark (&:is(.dark *));
7
 
8
  :root {
9
+ --radius: 0.625rem;
10
+ --background: oklch(1 0 0);
11
+ --foreground: oklch(0.147 0.004 49.25);
12
+ --card: oklch(1 0 0);
13
+ --card-foreground: oklch(0.147 0.004 49.25);
14
+ --popover: oklch(1 0 0);
15
+ --popover-foreground: oklch(0.147 0.004 49.25);
16
+ --primary: oklch(0.216 0.006 56.043);
17
+ --primary-foreground: oklch(0.985 0.001 106.423);
18
+ --secondary: oklch(0.97 0.001 106.424);
19
+ --secondary-foreground: oklch(0.216 0.006 56.043);
20
+ --muted: oklch(0.97 0.001 106.424);
21
+ --muted-foreground: oklch(0.553 0.013 58.071);
22
+ --accent: oklch(0.97 0.001 106.424);
23
+ --accent-foreground: oklch(0.216 0.006 56.043);
24
+ --destructive: oklch(0.577 0.245 27.325);
25
+ --border: oklch(0.923 0.003 48.717);
26
+ --input: oklch(0.923 0.003 48.717);
27
+ --ring: oklch(0.709 0.01 56.259);
28
+ --chart-1: oklch(0.646 0.222 41.116);
29
+ --chart-2: oklch(0.6 0.118 184.704);
30
+ --chart-3: oklch(0.398 0.07 227.392);
31
+ --chart-4: oklch(0.828 0.189 84.429);
32
+ --chart-5: oklch(0.769 0.188 70.08);
33
+ --sidebar: oklch(0.985 0.001 106.423);
34
+ --sidebar-foreground: oklch(0.147 0.004 49.25);
35
+ --sidebar-primary: oklch(0.216 0.006 56.043);
36
+ --sidebar-primary-foreground: oklch(0.985 0.001 106.423);
37
+ --sidebar-accent: oklch(0.97 0.001 106.424);
38
+ --sidebar-accent-foreground: oklch(0.216 0.006 56.043);
39
+ --sidebar-border: oklch(0.923 0.003 48.717);
40
+ --sidebar-ring: oklch(0.709 0.01 56.259);
41
  }
42
 
43
  .dark {
44
+ --background: oklch(0.147 0.004 49.25);
45
+ --foreground: oklch(0.985 0.001 106.423);
46
+ --card: oklch(0.216 0.006 56.043);
47
+ --card-foreground: oklch(0.985 0.001 106.423);
48
+ --popover: oklch(0.216 0.006 56.043);
49
+ --popover-foreground: oklch(0.985 0.001 106.423);
50
+ --primary: oklch(0.923 0.003 48.717);
51
+ --primary-foreground: oklch(0.216 0.006 56.043);
52
+ --secondary: oklch(0.268 0.007 34.298);
53
+ --secondary-foreground: oklch(0.985 0.001 106.423);
54
+ --muted: oklch(0.268 0.007 34.298);
55
+ --muted-foreground: oklch(0.709 0.01 56.259);
56
+ --accent: oklch(0.268 0.007 34.298);
57
+ --accent-foreground: oklch(0.985 0.001 106.423);
58
+ --destructive: oklch(0.704 0.191 22.216);
59
+ --border: oklch(1 0 0 / 10%);
60
+ --input: oklch(1 0 0 / 15%);
61
+ --ring: oklch(0.553 0.013 58.071);
62
+ --chart-1: oklch(0.488 0.243 264.376);
63
+ --chart-2: oklch(0.696 0.17 162.48);
64
+ --chart-3: oklch(0.769 0.188 70.08);
65
+ --chart-4: oklch(0.627 0.265 303.9);
66
+ --chart-5: oklch(0.645 0.246 16.439);
67
+ --sidebar: oklch(0.216 0.006 56.043);
68
+ --sidebar-foreground: oklch(0.985 0.001 106.423);
69
+ --sidebar-primary: oklch(0.488 0.243 264.376);
70
+ --sidebar-primary-foreground: oklch(0.985 0.001 106.423);
71
+ --sidebar-accent: oklch(0.268 0.007 34.298);
72
+ --sidebar-accent-foreground: oklch(0.985 0.001 106.423);
73
+ --sidebar-border: oklch(1 0 0 / 10%);
74
+ --sidebar-ring: oklch(0.553 0.013 58.071);
75
  }
76
 
77
  @theme inline {
78
+ --radius-sm: calc(var(--radius) - 4px);
79
+ --radius-md: calc(var(--radius) - 2px);
80
+ --radius-lg: var(--radius);
81
+ --radius-xl: calc(var(--radius) + 4px);
82
+ --color-background: var(--background);
83
+ --color-foreground: var(--foreground);
84
+ --color-card: var(--card);
85
+ --color-card-foreground: var(--card-foreground);
86
+ --color-popover: var(--popover);
87
+ --color-popover-foreground: var(--popover-foreground);
88
+ --color-primary: var(--primary);
89
+ --color-primary-foreground: var(--primary-foreground);
90
+ --color-secondary: var(--secondary);
91
+ --color-secondary-foreground: var(--secondary-foreground);
92
+ --color-muted: var(--muted);
93
+ --color-muted-foreground: var(--muted-foreground);
94
+ --color-accent: var(--accent);
95
+ --color-accent-foreground: var(--accent-foreground);
96
+ --color-destructive: var(--destructive);
97
+ --color-border: var(--border);
98
+ --color-input: var(--input);
99
+ --color-ring: var(--ring);
100
+ --color-chart-1: var(--chart-1);
101
+ --color-chart-2: var(--chart-2);
102
+ --color-chart-3: var(--chart-3);
103
+ --color-chart-4: var(--chart-4);
104
+ --color-chart-5: var(--chart-5);
105
+ --color-sidebar: var(--sidebar);
106
+ --color-sidebar-foreground: var(--sidebar-foreground);
107
+ --color-sidebar-primary: var(--sidebar-primary);
108
+ --color-sidebar-primary-foreground: var(--sidebar-primary-foreground);
109
+ --color-sidebar-accent: var(--sidebar-accent);
110
+ --color-sidebar-accent-foreground: var(--sidebar-accent-foreground);
111
+ --color-sidebar-border: var(--sidebar-border);
112
+ --color-sidebar-ring: var(--sidebar-ring);
113
  }
114
 
115
  @layer base {
116
+ * {
117
+ @apply border-border outline-ring/50;
118
+ }
119
+ body {
120
+ @apply bg-background text-foreground;
121
+ }
122
+ }
src/app.d.ts CHANGED
@@ -1,4 +1,4 @@
1
- import type { InteractivityProps } from '@threlte/extras'
2
 
3
  // See https://svelte.dev/docs/kit/types#app.d.ts
4
  // for information about these interfaces
 
1
+ import type { InteractivityProps } from "@threlte/extras";
2
 
3
  // See https://svelte.dev/docs/kit/types#app.d.ts
4
  // for information about these interfaces
src/lib/components/3d/elements/compute/ComputeGridItem.svelte CHANGED
@@ -23,7 +23,6 @@
23
  event.stopPropagation();
24
  isToggled = !isToggled;
25
  }
26
-
27
  </script>
28
 
29
  <T.Group
 
23
  event.stopPropagation();
24
  isToggled = !isToggled;
25
  }
 
26
  </script>
27
 
28
  <T.Group
src/lib/components/3d/elements/compute/Computes.svelte CHANGED
@@ -85,14 +85,30 @@
85
 
86
  {#if selectedCompute}
87
  <!-- Inference Session Configuration Modal (for existing computes without sessions) -->
88
- <AISessionConnectionModal bind:open={isAISessionModalOpen} compute={selectedCompute} {workspaceId} />
 
 
 
 
89
  <!-- Video Input Connection Modal -->
90
- <VideoInputConnectionModal bind:open={isVideoInputModalOpen} compute={selectedCompute} {workspaceId} />
 
 
 
 
91
  <!-- Robot Input Connection Modal -->
92
- <RobotInputConnectionModal bind:open={isRobotInputModalOpen} compute={selectedCompute} {workspaceId} />
 
 
 
 
93
  <!-- Robot Output Connection Modal -->
94
- <RobotOutputConnectionModal bind:open={isRobotOutputModalOpen} compute={selectedCompute} {workspaceId} />
 
 
 
 
95
  {/if}
96
 
97
  <!-- AI Model Configuration Modal (for creating new models) -->
98
- <AIModelConfigurationModal bind:open={isAIConfigModalOpen} {workspaceId} />
 
85
 
86
  {#if selectedCompute}
87
  <!-- Inference Session Configuration Modal (for existing computes without sessions) -->
88
+ <AISessionConnectionModal
89
+ bind:open={isAISessionModalOpen}
90
+ compute={selectedCompute}
91
+ {workspaceId}
92
+ />
93
  <!-- Video Input Connection Modal -->
94
+ <VideoInputConnectionModal
95
+ bind:open={isVideoInputModalOpen}
96
+ compute={selectedCompute}
97
+ {workspaceId}
98
+ />
99
  <!-- Robot Input Connection Modal -->
100
+ <RobotInputConnectionModal
101
+ bind:open={isRobotInputModalOpen}
102
+ compute={selectedCompute}
103
+ {workspaceId}
104
+ />
105
  <!-- Robot Output Connection Modal -->
106
+ <RobotOutputConnectionModal
107
+ bind:open={isRobotOutputModalOpen}
108
+ compute={selectedCompute}
109
+ {workspaceId}
110
+ />
111
  {/if}
112
 
113
  <!-- AI Model Configuration Modal (for creating new models) -->
114
+ <AIModelConfigurationModal bind:open={isAIConfigModalOpen} {workspaceId} />
src/lib/components/3d/elements/compute/GPU.svelte CHANGED
@@ -15,7 +15,12 @@
15
  }
16
 
17
  // Props with defaults
18
- let { position = [0, 0, 0], rotation = [0, 0, 0], scale = [1, 1, 1], rotating = false }: Props = $props();
 
 
 
 
 
19
 
20
  // Create the TV frame geometry (outer rounded rectangle)
21
  function createTVFrame(
@@ -87,7 +92,7 @@
87
 
88
  let fan_rotation = $state(0);
89
  let rotationPerSeconds = $state(1); // 1 rotation per second by default
90
-
91
  onMount(() => {
92
  const interval = setInterval(() => {
93
  // Calculate angle increment per frame for desired rotations per second
@@ -95,24 +100,16 @@
95
  const angleIncrement = (Math.PI * 2 * rotationPerSeconds) / 60;
96
  fan_rotation = fan_rotation + angleIncrement;
97
  }
98
- }, 1000/60); // Run at ~60fps
99
 
100
  return () => {
101
  clearInterval(interval);
102
  };
103
  });
104
-
105
-
106
  </script>
107
 
108
- <T.Group
109
- {position}
110
- {rotation}
111
- {scale}
112
- >
113
- <T.Group
114
- scale={[1, 1, 1]}
115
- >
116
- <Model fan_rotation={fan_rotation} />
117
  </T.Group>
118
  </T.Group>
 
15
  }
16
 
17
  // Props with defaults
18
+ let {
19
+ position = [0, 0, 0],
20
+ rotation = [0, 0, 0],
21
+ scale = [1, 1, 1],
22
+ rotating = false
23
+ }: Props = $props();
24
 
25
  // Create the TV frame geometry (outer rounded rectangle)
26
  function createTVFrame(
 
92
 
93
  let fan_rotation = $state(0);
94
  let rotationPerSeconds = $state(1); // 1 rotation per second by default
95
+
96
  onMount(() => {
97
  const interval = setInterval(() => {
98
  // Calculate angle increment per frame for desired rotations per second
 
100
  const angleIncrement = (Math.PI * 2 * rotationPerSeconds) / 60;
101
  fan_rotation = fan_rotation + angleIncrement;
102
  }
103
+ }, 1000 / 60); // Run at ~60fps
104
 
105
  return () => {
106
  clearInterval(interval);
107
  };
108
  });
 
 
109
  </script>
110
 
111
+ <T.Group {position} {rotation} {scale}>
112
+ <T.Group scale={[1, 1, 1]}>
113
+ <Model {fan_rotation} />
 
 
 
 
 
 
114
  </T.Group>
115
  </T.Group>
src/lib/components/3d/elements/compute/GPUModel.svelte CHANGED
@@ -8,193 +8,178 @@ Title: Nvidia GeForce RTX 3090
8
  -->
9
 
10
  <script lang="ts">
11
- import type * as THREE from 'three'
12
 
13
- import type { Snippet } from 'svelte'
14
- import { T, type Props } from '@threlte/core'
15
- import { useGltf } from '@threlte/extras'
16
 
17
- let {
18
- fan_rotation = 0,
19
- fallback,
20
- error,
21
- children,
22
- ref = $bindable(),
23
- ...props
24
- }: Props<THREE.Group<THREE.Object3DEventMap>> & {
25
- fan_rotation?: number
26
- ref?: THREE.Group<THREE.Object3DEventMap> | undefined
27
- children?: Snippet<[{ ref: THREE.Group<THREE.Object3DEventMap> | undefined }]>
28
- fallback?: Snippet
29
- error?: Snippet<[{ error: Error }]>
30
- } = $props()
31
 
32
- type GLTFResult = {
33
- nodes: {
34
- Metal_Frame_Metal_0: THREE.Mesh
35
- Front_Cover_Black_0: THREE.Mesh
36
- Fan_Circle_Black_Fan_0: THREE.Mesh
37
- Fan_F_Black_Fan_0: THREE.Mesh
38
- Fan_F_Slot1_0: THREE.Mesh
39
- Front_Cover_U_Black_0: THREE.Mesh
40
- Front_Cover_T_Black_0: THREE.Mesh
41
- Fan_Circle_B_Black_Fan_0: THREE.Mesh
42
- Grills_U_Metal_Black_0: THREE.Mesh
43
- Grills_T_Metal_Black_0: THREE.Mesh
44
- Plane010_Black001_0: THREE.Mesh
45
- Socket_Slot_0: THREE.Mesh
46
- Side_Metal_Part_Metal_S_0: THREE.Mesh
47
- Grills_F003_Metal_Black_0: THREE.Mesh
48
- Grills_F002_Metal_Black_0: THREE.Mesh
49
- Fan_B_Black_Fan_0: THREE.Mesh
50
- Fan_B_Slot1_0: THREE.Mesh
51
- }
52
- materials: {
53
- Metal: THREE.MeshStandardMaterial
54
- Black: THREE.MeshStandardMaterial
55
- Black_Fan: THREE.MeshStandardMaterial
56
- ['Slot.1']: THREE.MeshStandardMaterial
57
- Metal_Black: THREE.MeshStandardMaterial
58
- ['Black.001']: THREE.MeshStandardMaterial
59
- Slot: THREE.MeshStandardMaterial
60
- Metal_S: THREE.MeshStandardMaterial
61
- }
62
- }
63
 
64
- const gltf = useGltf<GLTFResult>('/gpu/scene.gltf')
65
  </script>
66
 
67
- <T.Group
68
- bind:ref
69
- dispose={false}
70
- {...props as any}
71
- >
72
- {#await gltf}
73
- {@render fallback?.()}
74
- {:then gltf}
75
- <T.Group scale={0.01}>
76
- <T.Group
77
- position={[127.5, 88.51, 10.29]}
78
- rotation={[Math.PI / 2, 0.05, 0]}
79
- scale={0.3}
80
- >
81
- <T.Mesh
82
- geometry={gltf.nodes.Fan_F_Black_Fan_0.geometry}
83
- material={gltf.materials.Black_Fan}
84
- rotation={[0, fan_rotation, 0]}
85
- />
86
- <T.Mesh
87
- geometry={gltf.nodes.Fan_F_Slot1_0.geometry}
88
- material={gltf.materials['Slot.1']}
89
- />
90
- </T.Group>
91
- <T.Group
92
- position={[-123.9, 88.51, -37.82]}
93
- rotation={[Math.PI / 2, -0.05, Math.PI]}
94
- scale={0.3}
95
- >
96
- <T.Mesh
97
- geometry={gltf.nodes.Fan_B_Black_Fan_0.geometry}
98
- material={gltf.materials.Black_Fan}
99
- rotation={[0, fan_rotation, 0]}
100
- />
101
- <T.Mesh
102
- geometry={gltf.nodes.Fan_B_Slot1_0.geometry}
103
- material={gltf.materials['Slot.1']}
104
-
105
- />
106
- </T.Group>
107
- <T.Mesh
108
- geometry={gltf.nodes.Metal_Frame_Metal_0.geometry}
109
- material={gltf.materials.Metal}
110
- position={[0, 88.3, -8.47]}
111
- rotation={[Math.PI / 2, 0, 0]}
112
- />
113
- <T.Mesh
114
- geometry={gltf.nodes.Front_Cover_Black_0.geometry}
115
- material={gltf.materials.Black}
116
- position={[-122.3, 89.69, 12.11]}
117
- rotation={[Math.PI / 2, 0, 0]}
118
- scale={[1, 1, 0.84]}
119
- />
120
- <T.Mesh
121
- geometry={gltf.nodes.Fan_Circle_Black_Fan_0.geometry}
122
- material={gltf.materials.Black_Fan}
123
- position={[127.5, 88.51, 10.29]}
124
- rotation={[Math.PI / 2, 0, 0]}
125
- scale={0.79}
126
- />
127
- <T.Mesh
128
- geometry={gltf.nodes.Front_Cover_U_Black_0.geometry}
129
- material={gltf.materials.Black}
130
- position={[0.02, 26.08, 14.09]}
131
- rotation={[Math.PI / 2, 0, 0]}
132
- />
133
- <T.Mesh
134
- geometry={gltf.nodes.Front_Cover_T_Black_0.geometry}
135
- material={gltf.materials.Black}
136
- position={[-4.75, 163.4, 14.09]}
137
- rotation={[-Math.PI / 2 , 0, -Math.PI]}
138
- />
139
- <T.Mesh
140
- geometry={gltf.nodes.Fan_Circle_B_Black_Fan_0.geometry}
141
- material={gltf.materials.Black_Fan}
142
- position={[-124.15, 88.51, -40.18]}
143
- rotation={[Math.PI / 2, 0, Math.PI]}
144
- scale={0.79}
145
- />
146
- <T.Mesh
147
- geometry={gltf.nodes.Grills_U_Metal_Black_0.geometry}
148
- material={gltf.materials.Metal_Black}
149
- position={[-0.12, 3.16, 3.09]}
150
- rotation={[Math.PI / 2, -Math.PI / 4, 0]}
151
- scale={[0.55, 11.75, 0.55]}
152
- />
153
- <T.Mesh
154
- geometry={gltf.nodes.Grills_T_Metal_Black_0.geometry}
155
- material={gltf.materials.Metal_Black}
156
- position={[0.8, 174.49, 3.09]}
157
- rotation={[-Math.PI / 2, Math.PI / 4, -Math.PI]}
158
- scale={[0.55, 11.75, 0.55]}
159
- />
160
- <T.Mesh
161
- geometry={gltf.nodes.Plane010_Black001_0.geometry}
162
- material={gltf.materials['Black.001']}
163
- position={[121.84, 88.42, -34.24]}
164
- rotation={[-Math.PI / 2, 0, -Math.PI]}
165
- scale={[1, 1, 0.84]}
166
- />
167
- <T.Mesh
168
- geometry={gltf.nodes.Socket_Slot_0.geometry}
169
- material={gltf.materials.Slot}
170
- position={[-149.71, 187.47, -39.01]}
171
- rotation={[Math.PI / 2, 0, 0]}
172
- scale={[1, 1.93, 1]}
173
- />
174
- <T.Mesh
175
- geometry={gltf.nodes.Side_Metal_Part_Metal_S_0.geometry}
176
- material={gltf.materials.Metal_S}
177
- position={[-225.87, 118.09, -12.54]}
178
- rotation={[Math.PI / 2, 0, 0]}
179
- />
180
- <T.Mesh
181
- geometry={gltf.nodes.Grills_F003_Metal_Black_0.geometry}
182
- material={gltf.materials.Metal_Black}
183
- position={[131.49, 88.84, -23.02]}
184
- rotation={[Math.PI / 2, 0, 0]}
185
- scale={[1, 1, 1.02]}
186
- />
187
- <T.Mesh
188
- geometry={gltf.nodes.Grills_F002_Metal_Black_0.geometry}
189
- material={gltf.materials.Metal_Black}
190
- position={[-128.18, 88.84, -4.17]}
191
- rotation={[Math.PI / 2, 0, Math.PI]}
192
- scale={[1, 0.97, 1.02]}
193
- />
194
- </T.Group>
195
- {:catch err}
196
- {@render error?.({ error: err })}
197
- {/await}
198
 
199
- {@render children?.({ ref })}
200
  </T.Group>
 
8
  -->
9
 
10
  <script lang="ts">
11
+ import type * as THREE from "three";
12
 
13
+ import type { Snippet } from "svelte";
14
+ import { T, type Props } from "@threlte/core";
15
+ import { useGltf } from "@threlte/extras";
16
 
17
+ let {
18
+ fan_rotation = 0,
19
+ fallback,
20
+ error,
21
+ children,
22
+ ref = $bindable(),
23
+ ...props
24
+ }: Props<THREE.Group<THREE.Object3DEventMap>> & {
25
+ fan_rotation?: number;
26
+ ref?: THREE.Group<THREE.Object3DEventMap> | undefined;
27
+ children?: Snippet<[{ ref: THREE.Group<THREE.Object3DEventMap> | undefined }]>;
28
+ fallback?: Snippet;
29
+ error?: Snippet<[{ error: Error }]>;
30
+ } = $props();
31
 
32
+ type GLTFResult = {
33
+ nodes: {
34
+ Metal_Frame_Metal_0: THREE.Mesh;
35
+ Front_Cover_Black_0: THREE.Mesh;
36
+ Fan_Circle_Black_Fan_0: THREE.Mesh;
37
+ Fan_F_Black_Fan_0: THREE.Mesh;
38
+ Fan_F_Slot1_0: THREE.Mesh;
39
+ Front_Cover_U_Black_0: THREE.Mesh;
40
+ Front_Cover_T_Black_0: THREE.Mesh;
41
+ Fan_Circle_B_Black_Fan_0: THREE.Mesh;
42
+ Grills_U_Metal_Black_0: THREE.Mesh;
43
+ Grills_T_Metal_Black_0: THREE.Mesh;
44
+ Plane010_Black001_0: THREE.Mesh;
45
+ Socket_Slot_0: THREE.Mesh;
46
+ Side_Metal_Part_Metal_S_0: THREE.Mesh;
47
+ Grills_F003_Metal_Black_0: THREE.Mesh;
48
+ Grills_F002_Metal_Black_0: THREE.Mesh;
49
+ Fan_B_Black_Fan_0: THREE.Mesh;
50
+ Fan_B_Slot1_0: THREE.Mesh;
51
+ };
52
+ materials: {
53
+ Metal: THREE.MeshStandardMaterial;
54
+ Black: THREE.MeshStandardMaterial;
55
+ Black_Fan: THREE.MeshStandardMaterial;
56
+ ["Slot.1"]: THREE.MeshStandardMaterial;
57
+ Metal_Black: THREE.MeshStandardMaterial;
58
+ ["Black.001"]: THREE.MeshStandardMaterial;
59
+ Slot: THREE.MeshStandardMaterial;
60
+ Metal_S: THREE.MeshStandardMaterial;
61
+ };
62
+ };
63
 
64
+ const gltf = useGltf<GLTFResult>("/gpu/scene.gltf");
65
  </script>
66
 
67
+ <T.Group bind:ref dispose={false} {...props as any}>
68
+ {#await gltf}
69
+ {@render fallback?.()}
70
+ {:then gltf}
71
+ <T.Group scale={0.01}>
72
+ <T.Group position={[127.5, 88.51, 10.29]} rotation={[Math.PI / 2, 0.05, 0]} scale={0.3}>
73
+ <T.Mesh
74
+ geometry={gltf.nodes.Fan_F_Black_Fan_0.geometry}
75
+ material={gltf.materials.Black_Fan}
76
+ rotation={[0, fan_rotation, 0]}
77
+ />
78
+ <T.Mesh geometry={gltf.nodes.Fan_F_Slot1_0.geometry} material={gltf.materials["Slot.1"]} />
79
+ </T.Group>
80
+ <T.Group
81
+ position={[-123.9, 88.51, -37.82]}
82
+ rotation={[Math.PI / 2, -0.05, Math.PI]}
83
+ scale={0.3}
84
+ >
85
+ <T.Mesh
86
+ geometry={gltf.nodes.Fan_B_Black_Fan_0.geometry}
87
+ material={gltf.materials.Black_Fan}
88
+ rotation={[0, fan_rotation, 0]}
89
+ />
90
+ <T.Mesh geometry={gltf.nodes.Fan_B_Slot1_0.geometry} material={gltf.materials["Slot.1"]} />
91
+ </T.Group>
92
+ <T.Mesh
93
+ geometry={gltf.nodes.Metal_Frame_Metal_0.geometry}
94
+ material={gltf.materials.Metal}
95
+ position={[0, 88.3, -8.47]}
96
+ rotation={[Math.PI / 2, 0, 0]}
97
+ />
98
+ <T.Mesh
99
+ geometry={gltf.nodes.Front_Cover_Black_0.geometry}
100
+ material={gltf.materials.Black}
101
+ position={[-122.3, 89.69, 12.11]}
102
+ rotation={[Math.PI / 2, 0, 0]}
103
+ scale={[1, 1, 0.84]}
104
+ />
105
+ <T.Mesh
106
+ geometry={gltf.nodes.Fan_Circle_Black_Fan_0.geometry}
107
+ material={gltf.materials.Black_Fan}
108
+ position={[127.5, 88.51, 10.29]}
109
+ rotation={[Math.PI / 2, 0, 0]}
110
+ scale={0.79}
111
+ />
112
+ <T.Mesh
113
+ geometry={gltf.nodes.Front_Cover_U_Black_0.geometry}
114
+ material={gltf.materials.Black}
115
+ position={[0.02, 26.08, 14.09]}
116
+ rotation={[Math.PI / 2, 0, 0]}
117
+ />
118
+ <T.Mesh
119
+ geometry={gltf.nodes.Front_Cover_T_Black_0.geometry}
120
+ material={gltf.materials.Black}
121
+ position={[-4.75, 163.4, 14.09]}
122
+ rotation={[-Math.PI / 2, 0, -Math.PI]}
123
+ />
124
+ <T.Mesh
125
+ geometry={gltf.nodes.Fan_Circle_B_Black_Fan_0.geometry}
126
+ material={gltf.materials.Black_Fan}
127
+ position={[-124.15, 88.51, -40.18]}
128
+ rotation={[Math.PI / 2, 0, Math.PI]}
129
+ scale={0.79}
130
+ />
131
+ <T.Mesh
132
+ geometry={gltf.nodes.Grills_U_Metal_Black_0.geometry}
133
+ material={gltf.materials.Metal_Black}
134
+ position={[-0.12, 3.16, 3.09]}
135
+ rotation={[Math.PI / 2, -Math.PI / 4, 0]}
136
+ scale={[0.55, 11.75, 0.55]}
137
+ />
138
+ <T.Mesh
139
+ geometry={gltf.nodes.Grills_T_Metal_Black_0.geometry}
140
+ material={gltf.materials.Metal_Black}
141
+ position={[0.8, 174.49, 3.09]}
142
+ rotation={[-Math.PI / 2, Math.PI / 4, -Math.PI]}
143
+ scale={[0.55, 11.75, 0.55]}
144
+ />
145
+ <T.Mesh
146
+ geometry={gltf.nodes.Plane010_Black001_0.geometry}
147
+ material={gltf.materials["Black.001"]}
148
+ position={[121.84, 88.42, -34.24]}
149
+ rotation={[-Math.PI / 2, 0, -Math.PI]}
150
+ scale={[1, 1, 0.84]}
151
+ />
152
+ <T.Mesh
153
+ geometry={gltf.nodes.Socket_Slot_0.geometry}
154
+ material={gltf.materials.Slot}
155
+ position={[-149.71, 187.47, -39.01]}
156
+ rotation={[Math.PI / 2, 0, 0]}
157
+ scale={[1, 1.93, 1]}
158
+ />
159
+ <T.Mesh
160
+ geometry={gltf.nodes.Side_Metal_Part_Metal_S_0.geometry}
161
+ material={gltf.materials.Metal_S}
162
+ position={[-225.87, 118.09, -12.54]}
163
+ rotation={[Math.PI / 2, 0, 0]}
164
+ />
165
+ <T.Mesh
166
+ geometry={gltf.nodes.Grills_F003_Metal_Black_0.geometry}
167
+ material={gltf.materials.Metal_Black}
168
+ position={[131.49, 88.84, -23.02]}
169
+ rotation={[Math.PI / 2, 0, 0]}
170
+ scale={[1, 1, 1.02]}
171
+ />
172
+ <T.Mesh
173
+ geometry={gltf.nodes.Grills_F002_Metal_Black_0.geometry}
174
+ material={gltf.materials.Metal_Black}
175
+ position={[-128.18, 88.84, -4.17]}
176
+ rotation={[Math.PI / 2, 0, Math.PI]}
177
+ scale={[1, 0.97, 1.02]}
178
+ />
179
+ </T.Group>
180
+ {:catch err}
181
+ {@render error?.({ error: err })}
182
+ {/await}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
183
 
184
+ {@render children?.({ ref })}
185
  </T.Group>
src/lib/components/3d/elements/compute/modal/AISessionConnectionModal.svelte CHANGED
@@ -380,9 +380,9 @@
380
  placeholder="front, wrist, overhead"
381
  class="border-slate-300 bg-slate-50 text-slate-900 dark:border-slate-600 dark:bg-slate-800 dark:text-slate-100"
382
  />
383
- <p class="text-xs text-slate-600 dark:text-slate-400">
384
- Comma-separated camera names
385
- </p>
386
  </div>
387
 
388
  {#if modelConfig.requiresLanguageInstruction}
@@ -403,7 +403,9 @@
403
  </div>
404
  {/if}
405
 
406
- <div class="rounded-lg border border-green-300/30 bg-green-100/20 p-3 dark:border-green-500/30 dark:bg-green-900/20">
 
 
407
  <div class="flex items-center gap-2">
408
  <span class="icon-[mdi--folder] size-4 text-green-600 dark:text-green-400"></span>
409
  <span class="text-sm font-medium text-green-700 dark:text-green-300">
@@ -420,9 +422,9 @@
420
  >
421
  <div class="text-xs text-slate-600 dark:text-slate-400">
422
  <span class="icon-[mdi--lightbulb] size-3"></span>
423
- <strong>Tip:</strong> This will create a new {modelConfig.label} inference session with dedicated rooms for camera
424
- inputs, joint inputs, and joint outputs in the inference server communication
425
- system.
426
  </div>
427
  </div>
428
 
@@ -450,8 +452,8 @@
450
  class="rounded border border-slate-300 bg-slate-100/30 p-2 text-xs text-slate-600 dark:border-slate-700 dark:bg-slate-800/30 dark:text-slate-500"
451
  >
452
  <span class="icon-[mdi--information] mr-1 size-3"></span>
453
- Inference Sessions require a trained {modelConfig.label} and create dedicated communication rooms for video
454
- inputs, robot joint states, and control outputs in the inference server system.
455
  </div>
456
  </div>
457
  </Dialog.Content>
 
380
  placeholder="front, wrist, overhead"
381
  class="border-slate-300 bg-slate-50 text-slate-900 dark:border-slate-600 dark:bg-slate-800 dark:text-slate-100"
382
  />
383
+ <p class="text-xs text-slate-600 dark:text-slate-400">
384
+ Comma-separated camera names
385
+ </p>
386
  </div>
387
 
388
  {#if modelConfig.requiresLanguageInstruction}
 
403
  </div>
404
  {/if}
405
 
406
+ <div
407
+ class="rounded-lg border border-green-300/30 bg-green-100/20 p-3 dark:border-green-500/30 dark:bg-green-900/20"
408
+ >
409
  <div class="flex items-center gap-2">
410
  <span class="icon-[mdi--folder] size-4 text-green-600 dark:text-green-400"></span>
411
  <span class="text-sm font-medium text-green-700 dark:text-green-300">
 
422
  >
423
  <div class="text-xs text-slate-600 dark:text-slate-400">
424
  <span class="icon-[mdi--lightbulb] size-3"></span>
425
+ <strong>Tip:</strong> This will create a new {modelConfig.label} inference session
426
+ with dedicated rooms for camera inputs, joint inputs, and joint outputs in the inference
427
+ server communication system.
428
  </div>
429
  </div>
430
 
 
452
  class="rounded border border-slate-300 bg-slate-100/30 p-2 text-xs text-slate-600 dark:border-slate-700 dark:bg-slate-800/30 dark:text-slate-500"
453
  >
454
  <span class="icon-[mdi--information] mr-1 size-3"></span>
455
+ Inference Sessions require a trained {modelConfig.label} and create dedicated communication rooms
456
+ for video inputs, robot joint states, and control outputs in the inference server system.
457
  </div>
458
  </div>
459
  </Dialog.Content>
src/lib/components/3d/elements/compute/status/ComputeBoxUIKit.svelte CHANGED
@@ -1,11 +1,7 @@
1
  <script lang="ts">
2
  import type { RemoteCompute } from "$lib/elements/compute//RemoteCompute.svelte";
3
  import { ICON } from "$lib/utils/icon";
4
- import {
5
- BaseStatusBox,
6
- StatusHeader,
7
- StatusContent
8
- } from "$lib/components/3d/ui";
9
 
10
  interface Props {
11
  compute: RemoteCompute;
 
1
  <script lang="ts">
2
  import type { RemoteCompute } from "$lib/elements/compute//RemoteCompute.svelte";
3
  import { ICON } from "$lib/utils/icon";
4
+ import { BaseStatusBox, StatusHeader, StatusContent } from "$lib/components/3d/ui";
 
 
 
 
5
 
6
  interface Props {
7
  compute: RemoteCompute;
src/lib/components/3d/elements/compute/status/ComputeStatusBillboard.svelte CHANGED
@@ -5,7 +5,7 @@
5
  import ComputeConnectionFlowBoxUIKit from "./ComputeConnectionFlowBoxUIKit.svelte";
6
  import type { RemoteCompute } from "$lib/elements/compute//RemoteCompute.svelte";
7
  import { Tween } from "svelte/motion";
8
- import { cubicOut } from "svelte/easing";
9
 
10
  interface Props {
11
  compute: RemoteCompute;
@@ -29,12 +29,18 @@
29
 
30
  interactivity();
31
 
32
- const tweenedScale = Tween.of(() => {
33
- return visible ? 1 : 0;
34
- }, { duration: duration, easing: cubicOut, delay: delay });
35
- const tweenedOpacity = Tween.of(() => {
36
- return visible ? 1 : 0;
37
- }, { duration: duration, easing: cubicOut, delay: delay });
 
 
 
 
 
 
38
  </script>
39
 
40
  <T.Group
 
5
  import ComputeConnectionFlowBoxUIKit from "./ComputeConnectionFlowBoxUIKit.svelte";
6
  import type { RemoteCompute } from "$lib/elements/compute//RemoteCompute.svelte";
7
  import { Tween } from "svelte/motion";
8
+ import { cubicOut } from "svelte/easing";
9
 
10
  interface Props {
11
  compute: RemoteCompute;
 
29
 
30
  interactivity();
31
 
32
+ const tweenedScale = Tween.of(
33
+ () => {
34
+ return visible ? 1 : 0;
35
+ },
36
+ { duration: duration, easing: cubicOut, delay: delay }
37
+ );
38
+ const tweenedOpacity = Tween.of(
39
+ () => {
40
+ return visible ? 1 : 0;
41
+ },
42
+ { duration: duration, easing: cubicOut, delay: delay }
43
+ );
44
  </script>
45
 
46
  <T.Group
src/lib/components/3d/elements/compute/status/RobotOutputBoxUIKit.svelte CHANGED
@@ -1,7 +1,13 @@
1
  <script lang="ts">
2
  import type { RemoteCompute } from "$lib/elements/compute//RemoteCompute.svelte";
3
  import { ICON } from "$lib/utils/icon";
4
- import { BaseStatusBox, StatusHeader, StatusContent, StatusIndicator, StatusButton } from "$lib/components/3d/ui";
 
 
 
 
 
 
5
 
6
  interface Props {
7
  compute: RemoteCompute;
@@ -13,7 +19,7 @@
13
  const outputColor = "rgb(59, 130, 246)";
14
  </script>
15
 
16
- <BaseStatusBox
17
  color={outputColor}
18
  borderOpacity={0.6}
19
  backgroundOpacity={0.2}
@@ -22,14 +28,14 @@
22
  >
23
  {#if compute.hasSession && compute.outputConnections}
24
  <!-- Active Robot Output State -->
25
- <StatusHeader
26
- icon={ICON["icon-[ix--robotic-arm]"].svg}
27
- text="COMMANDS"
28
  color={outputColor}
29
  opacity={0.9}
30
  />
31
 
32
- <StatusContent
33
  title={compute.isRunning ? "AI Commands Active" : "Session Ready"}
34
  subtitle="Motor Control"
35
  color="rgb(37, 99, 235)"
@@ -46,25 +52,25 @@
46
  {/if}
47
  {:else}
48
  <!-- No Session State -->
49
- <StatusHeader
50
- icon={ICON["icon-[ix--robotic-arm]"].svg}
51
- text="NO OUTPUT"
52
  color={outputColor}
53
  opacity={0.7}
54
  />
55
 
56
- <StatusContent
57
- title={!compute.hasSession ? 'Need Session' : 'Click to Configure'}
58
  color="rgb(59, 130, 246)"
59
  variant="secondary"
60
  />
61
 
62
- <StatusButton
63
- icon={ICON["icon-[ix--robotic-arm]"].svg}
64
  text="Setup Output"
65
  color={outputColor}
66
  backgroundOpacity={0.1}
67
  textOpacity={0.7}
68
  />
69
  {/if}
70
- </BaseStatusBox>
 
1
  <script lang="ts">
2
  import type { RemoteCompute } from "$lib/elements/compute//RemoteCompute.svelte";
3
  import { ICON } from "$lib/utils/icon";
4
+ import {
5
+ BaseStatusBox,
6
+ StatusHeader,
7
+ StatusContent,
8
+ StatusIndicator,
9
+ StatusButton
10
+ } from "$lib/components/3d/ui";
11
 
12
  interface Props {
13
  compute: RemoteCompute;
 
19
  const outputColor = "rgb(59, 130, 246)";
20
  </script>
21
 
22
+ <BaseStatusBox
23
  color={outputColor}
24
  borderOpacity={0.6}
25
  backgroundOpacity={0.2}
 
28
  >
29
  {#if compute.hasSession && compute.outputConnections}
30
  <!-- Active Robot Output State -->
31
+ <StatusHeader
32
+ icon={ICON["icon-[ix--robotic-arm]"].svg}
33
+ text="COMMANDS"
34
  color={outputColor}
35
  opacity={0.9}
36
  />
37
 
38
+ <StatusContent
39
  title={compute.isRunning ? "AI Commands Active" : "Session Ready"}
40
  subtitle="Motor Control"
41
  color="rgb(37, 99, 235)"
 
52
  {/if}
53
  {:else}
54
  <!-- No Session State -->
55
+ <StatusHeader
56
+ icon={ICON["icon-[ix--robotic-arm]"].svg}
57
+ text="NO OUTPUT"
58
  color={outputColor}
59
  opacity={0.7}
60
  />
61
 
62
+ <StatusContent
63
+ title={!compute.hasSession ? "Need Session" : "Click to Configure"}
64
  color="rgb(59, 130, 246)"
65
  variant="secondary"
66
  />
67
 
68
+ <StatusButton
69
+ icon={ICON["icon-[ix--robotic-arm]"].svg}
70
  text="Setup Output"
71
  color={outputColor}
72
  backgroundOpacity={0.1}
73
  textOpacity={0.7}
74
  />
75
  {/if}
76
+ </BaseStatusBox>
src/lib/components/3d/elements/compute/status/VideoInputBoxUIKit.svelte CHANGED
@@ -1,6 +1,12 @@
1
  <script lang="ts">
2
  import type { RemoteCompute } from "$lib/elements/compute//RemoteCompute.svelte";
3
- import { BaseStatusBox, StatusHeader, StatusContent, StatusIndicator, StatusButton } from "$lib/components/3d/ui";
 
 
 
 
 
 
4
  import { ICON } from "@/utils/icon";
5
 
6
  interface Props {
@@ -13,7 +19,7 @@
13
  const inputColor = "rgb(34, 197, 94)";
14
  </script>
15
 
16
- <BaseStatusBox
17
  minWidth={100}
18
  minHeight={65}
19
  color={inputColor}
@@ -24,16 +30,16 @@
24
  >
25
  {#if compute.hasSession && compute.inputConnections}
26
  <!-- Active Video Input State -->
27
- <StatusHeader
28
- icon={ICON["icon-[mdi--video]"].svg}
29
- text="VIDEO"
30
  color={inputColor}
31
  opacity={0.9}
32
  fontSize={11}
33
  />
34
 
35
  <!-- Camera Streams -->
36
- <StatusContent
37
  title={`${Object.keys(compute.inputConnections.cameras).length} Cameras`}
38
  color="rgb(21, 128, 61)"
39
  variant="primary"
@@ -43,21 +49,17 @@
43
  <StatusIndicator color={inputColor} />
44
  {:else}
45
  <!-- No Session State -->
46
- <StatusHeader
47
- icon={ICON["icon-[mdi--video-off]"].svg}
48
- text="NO VIDEO"
49
  color={inputColor}
50
  opacity={0.7}
51
  fontSize={11}
52
  />
53
 
54
- <StatusContent
55
- title="Setup Video"
56
- color="rgb(34, 197, 94)"
57
- variant="secondary"
58
- />
59
 
60
- <StatusButton
61
  icon={ICON["icon-[mdi--plus]"].svg}
62
  text="Add"
63
  color={inputColor}
@@ -65,4 +67,4 @@
65
  textOpacity={0.7}
66
  />
67
  {/if}
68
- </BaseStatusBox>
 
1
  <script lang="ts">
2
  import type { RemoteCompute } from "$lib/elements/compute//RemoteCompute.svelte";
3
+ import {
4
+ BaseStatusBox,
5
+ StatusHeader,
6
+ StatusContent,
7
+ StatusIndicator,
8
+ StatusButton
9
+ } from "$lib/components/3d/ui";
10
  import { ICON } from "@/utils/icon";
11
 
12
  interface Props {
 
19
  const inputColor = "rgb(34, 197, 94)";
20
  </script>
21
 
22
+ <BaseStatusBox
23
  minWidth={100}
24
  minHeight={65}
25
  color={inputColor}
 
30
  >
31
  {#if compute.hasSession && compute.inputConnections}
32
  <!-- Active Video Input State -->
33
+ <StatusHeader
34
+ icon={ICON["icon-[mdi--video]"].svg}
35
+ text="VIDEO"
36
  color={inputColor}
37
  opacity={0.9}
38
  fontSize={11}
39
  />
40
 
41
  <!-- Camera Streams -->
42
+ <StatusContent
43
  title={`${Object.keys(compute.inputConnections.cameras).length} Cameras`}
44
  color="rgb(21, 128, 61)"
45
  variant="primary"
 
49
  <StatusIndicator color={inputColor} />
50
  {:else}
51
  <!-- No Session State -->
52
+ <StatusHeader
53
+ icon={ICON["icon-[mdi--video-off]"].svg}
54
+ text="NO VIDEO"
55
  color={inputColor}
56
  opacity={0.7}
57
  fontSize={11}
58
  />
59
 
60
+ <StatusContent title="Setup Video" color="rgb(34, 197, 94)" variant="secondary" />
 
 
 
 
61
 
62
+ <StatusButton
63
  icon={ICON["icon-[mdi--plus]"].svg}
64
  text="Add"
65
  color={inputColor}
 
67
  textOpacity={0.7}
68
  />
69
  {/if}
70
+ </BaseStatusBox>
src/lib/components/3d/elements/robot/RobotGridItem.svelte CHANGED
@@ -106,7 +106,6 @@
106
  }
107
 
108
  const { onPointerEnter, onPointerLeave, hovering } = useCursor();
109
-
110
 
111
  let isToggled = $state(false);
112
 
@@ -124,13 +123,17 @@
124
  scale={[10, 10, 10]}
125
  rotation={[-Math.PI / 2, 0, 0]}
126
  >
127
- <T.Group onpointerenter={(event) => {
128
- event.stopPropagation();
129
- onPointerEnter();
130
- }} onpointerleave={(event) => {
131
- event.stopPropagation();
132
- onPointerLeave();
133
- }} onclick={handleClick}>
 
 
 
 
134
  {#if urdfRobotState}
135
  {#each getRootLinks(urdfRobotState) as link}
136
  <UrdfLink
 
106
  }
107
 
108
  const { onPointerEnter, onPointerLeave, hovering } = useCursor();
 
109
 
110
  let isToggled = $state(false);
111
 
 
123
  scale={[10, 10, 10]}
124
  rotation={[-Math.PI / 2, 0, 0]}
125
  >
126
+ <T.Group
127
+ onpointerenter={(event) => {
128
+ event.stopPropagation();
129
+ onPointerEnter();
130
+ }}
131
+ onpointerleave={(event) => {
132
+ event.stopPropagation();
133
+ onPointerLeave();
134
+ }}
135
+ onclick={handleClick}
136
+ >
137
  {#if urdfRobotState}
138
  {#each getRootLinks(urdfRobotState) as link}
139
  <UrdfLink
src/lib/components/3d/elements/robot/URDF/primitives/UrdfJoint.svelte CHANGED
@@ -81,8 +81,6 @@
81
  // return norm * 200 - 100; // → [-100, 100]
82
  // }
83
 
84
-
85
-
86
  // function denormalizeAngle(normValue: number, name: keyof typeof ANGLE_RANGES): number {
87
  // if (!(name in ANGLE_RANGES)) {
88
  // throw new Error(`Unknown angle name: ${name}`);
@@ -156,11 +154,7 @@
156
 
157
  <T.Mesh rotation={[Math.PI / 2, 0, 0]} {...restProps}>
158
  <T.CylinderGeometry args={[0.004, 0.004, 0.03]} />
159
- <T.MeshBasicMaterial
160
- color={jointColor}
161
- {opacity}
162
- transparent={opacity < 1.0}
163
- />
164
  </T.Mesh>
165
  {/if}
166
  </T.Group>
@@ -174,7 +168,7 @@
174
  renderOrder={999}
175
  frustumCulled={false}
176
  >
177
- <!-- text={joint.name + " " + getJointRotationValue(joint).toFixed(0) + "°" + " (" + normalizeAngle2(getJointRotationValue(joint)).toFixed(0) + ")"} -->
178
  <Text
179
  scale={nameHeight}
180
  color={jointColor}
 
81
  // return norm * 200 - 100; // → [-100, 100]
82
  // }
83
 
 
 
84
  // function denormalizeAngle(normValue: number, name: keyof typeof ANGLE_RANGES): number {
85
  // if (!(name in ANGLE_RANGES)) {
86
  // throw new Error(`Unknown angle name: ${name}`);
 
154
 
155
  <T.Mesh rotation={[Math.PI / 2, 0, 0]} {...restProps}>
156
  <T.CylinderGeometry args={[0.004, 0.004, 0.03]} />
157
+ <T.MeshBasicMaterial color={jointColor} {opacity} transparent={opacity < 1.0} />
 
 
 
 
158
  </T.Mesh>
159
  {/if}
160
  </T.Group>
 
168
  renderOrder={999}
169
  frustumCulled={false}
170
  >
171
+ <!-- text={joint.name + " " + getJointRotationValue(joint).toFixed(0) + "°" + " (" + normalizeAngle2(getJointRotationValue(joint)).toFixed(0) + ")"} -->
172
  <Text
173
  scale={nameHeight}
174
  color={jointColor}
src/lib/components/3d/elements/robot/URDF/primitives/UrdfLink.svelte CHANGED
@@ -47,7 +47,7 @@
47
  jointIndicatorColor = "#000000",
48
  nameHeight = 0.1,
49
  showLine = true,
50
- opacity = 0.7,
51
  }: Props = $props();
52
 
53
  let showPointCloud = false;
 
47
  jointIndicatorColor = "#000000",
48
  nameHeight = 0.1,
49
  showLine = true,
50
+ opacity = 0.7
51
  }: Props = $props();
52
 
53
  let showPointCloud = false;
src/lib/components/3d/elements/robot/URDF/utils/UrdfParser.ts CHANGED
@@ -35,8 +35,6 @@ export function getRootLinks(robot: IUrdfRobot): IUrdfLink[] {
35
  }
36
  }
37
 
38
-
39
-
40
  return links;
41
  }
42
 
 
35
  }
36
  }
37
 
 
 
38
  return links;
39
  }
40
 
src/lib/components/3d/elements/robot/modal/InputConnectionModal.svelte CHANGED
@@ -499,13 +499,15 @@
499
  >
500
  {room.id}
501
  </p>
502
- <div class="flex items-center gap-3 text-xs text-slate-600 dark:text-slate-400">
 
 
503
  <span>{room.has_producer ? "📤 Has Output" : "📥 No Output"}</span>
504
  <span>👥 {room.participants?.total || 0} users</span>
505
  <!-- Monitoring links -->
506
  <div class="flex gap-1">
507
  <a
508
- href={`${settings.transportServerUrl.replace('/api', '')}/${workspaceId}/robotics/consumer?room=${room.id}`}
509
  target="_blank"
510
  rel="noopener noreferrer"
511
  class="inline-flex items-center gap-1 rounded bg-blue-500/10 px-1.5 py-0.5 text-xs text-blue-600 hover:bg-blue-500/20 dark:bg-blue-400/10 dark:text-blue-400 dark:hover:bg-blue-400/20"
@@ -515,7 +517,7 @@
515
  Consumer
516
  </a>
517
  <a
518
- href={`${settings.transportServerUrl.replace('/api', '')}/${workspaceId}/robotics/producer?room=${room.id}`}
519
  target="_blank"
520
  rel="noopener noreferrer"
521
  class="inline-flex items-center gap-1 rounded bg-green-500/10 px-1.5 py-0.5 text-xs text-green-600 hover:bg-green-500/20 dark:bg-green-400/10 dark:text-green-400 dark:hover:bg-green-400/20"
@@ -526,9 +528,8 @@
526
  </a>
527
  </div>
528
  </div>
529
-
530
  </div>
531
-
532
  <Button
533
  variant="secondary"
534
  size="sm"
 
499
  >
500
  {room.id}
501
  </p>
502
+ <div
503
+ class="flex items-center gap-3 text-xs text-slate-600 dark:text-slate-400"
504
+ >
505
  <span>{room.has_producer ? "📤 Has Output" : "📥 No Output"}</span>
506
  <span>👥 {room.participants?.total || 0} users</span>
507
  <!-- Monitoring links -->
508
  <div class="flex gap-1">
509
  <a
510
+ href={`${settings.transportServerUrl.replace("/api", "")}/${workspaceId}/robotics/consumer?room=${room.id}`}
511
  target="_blank"
512
  rel="noopener noreferrer"
513
  class="inline-flex items-center gap-1 rounded bg-blue-500/10 px-1.5 py-0.5 text-xs text-blue-600 hover:bg-blue-500/20 dark:bg-blue-400/10 dark:text-blue-400 dark:hover:bg-blue-400/20"
 
517
  Consumer
518
  </a>
519
  <a
520
+ href={`${settings.transportServerUrl.replace("/api", "")}/${workspaceId}/robotics/producer?room=${room.id}`}
521
  target="_blank"
522
  rel="noopener noreferrer"
523
  class="inline-flex items-center gap-1 rounded bg-green-500/10 px-1.5 py-0.5 text-xs text-green-600 hover:bg-green-500/20 dark:bg-green-400/10 dark:text-green-400 dark:hover:bg-green-400/20"
 
528
  </a>
529
  </div>
530
  </div>
 
531
  </div>
532
+
533
  <Button
534
  variant="secondary"
535
  size="sm"
src/lib/components/3d/elements/robot/modal/OutputConnectionModal.svelte CHANGED
@@ -191,14 +191,14 @@
191
  showUSBCalibration = false;
192
  pendingUSBConnection = null;
193
  isConnecting = false;
194
-
195
  // Clean up the uncalibrated USB producer
196
  const uncalibratedDrivers = robot.getUncalibratedUSBDrivers();
197
  if (uncalibratedDrivers.length > 0) {
198
  // Remove the most recent producer (should be the one we just added)
199
  const lastProducer = robot.producers[robot.producers.length - 1];
200
  if (lastProducer) {
201
- robot.removeProducer(lastProducer.id).catch(err => {
202
  console.error("Failed to clean up USB producer after calibration cancel:", err);
203
  });
204
  }
@@ -277,9 +277,7 @@
277
  onCancel={onCalibrationCancel}
278
  />
279
  {:else}
280
- <div class="text-center text-slate-400">
281
- No USB drivers require calibration
282
- </div>
283
  {/each}
284
  </Card.Content>
285
  </Card.Root>
@@ -340,13 +338,14 @@
340
  <div class="flex items-center justify-between">
341
  <div>
342
  <Card.Title
343
- class="flex items-center gap-2 text-base text-orange-700 dark:text-orange-200 pb-1"
344
  >
345
  <span class="icon-[mdi--cloud-sync] size-4"></span>
346
  Remote Control
347
  </Card.Title>
348
  <Card.Description class="text-xs text-orange-600/70 dark:text-orange-300/70">
349
- Broadcast robot movements to remote robots or AI systems from anywhere in the world
 
350
  </Card.Description>
351
  </div>
352
  <Button
@@ -441,13 +440,15 @@
441
  >
442
  {room.id}
443
  </p>
444
- <div class="flex items-center gap-3 text-xs text-slate-600 dark:text-slate-400">
 
 
445
  <span>{room.has_producer ? "🔴 Occupied" : "🟢 Available"}</span>
446
  <span>👥 {room.participants?.total || 0} users</span>
447
  <!-- Monitoring links -->
448
  <div class="flex gap-1">
449
  <a
450
- href={`${settings.transportServerUrl.replace('/api', '')}/${workspaceId}/robotics/consumer?room=${room.id}`}
451
  target="_blank"
452
  rel="noopener noreferrer"
453
  class="inline-flex items-center gap-1 rounded bg-blue-500/10 px-1.5 py-0.5 text-xs text-blue-600 hover:bg-blue-500/20 dark:bg-blue-400/10 dark:text-blue-400 dark:hover:bg-blue-400/20"
@@ -457,7 +458,7 @@
457
  Consumer
458
  </a>
459
  <a
460
- href={`${settings.transportServerUrl.replace('/api', '')}/${workspaceId}/robotics/producer?room=${room.id}`}
461
  target="_blank"
462
  rel="noopener noreferrer"
463
  class="inline-flex items-center gap-1 rounded bg-green-500/10 px-1.5 py-0.5 text-xs text-green-600 hover:bg-green-500/20 dark:bg-green-400/10 dark:text-green-400 dark:hover:bg-green-400/20"
 
191
  showUSBCalibration = false;
192
  pendingUSBConnection = null;
193
  isConnecting = false;
194
+
195
  // Clean up the uncalibrated USB producer
196
  const uncalibratedDrivers = robot.getUncalibratedUSBDrivers();
197
  if (uncalibratedDrivers.length > 0) {
198
  // Remove the most recent producer (should be the one we just added)
199
  const lastProducer = robot.producers[robot.producers.length - 1];
200
  if (lastProducer) {
201
+ robot.removeProducer(lastProducer.id).catch((err) => {
202
  console.error("Failed to clean up USB producer after calibration cancel:", err);
203
  });
204
  }
 
277
  onCancel={onCalibrationCancel}
278
  />
279
  {:else}
280
+ <div class="text-center text-slate-400">No USB drivers require calibration</div>
 
 
281
  {/each}
282
  </Card.Content>
283
  </Card.Root>
 
338
  <div class="flex items-center justify-between">
339
  <div>
340
  <Card.Title
341
+ class="flex items-center gap-2 pb-1 text-base text-orange-700 dark:text-orange-200"
342
  >
343
  <span class="icon-[mdi--cloud-sync] size-4"></span>
344
  Remote Control
345
  </Card.Title>
346
  <Card.Description class="text-xs text-orange-600/70 dark:text-orange-300/70">
347
+ Broadcast robot movements to remote robots or AI systems from anywhere in the
348
+ world
349
  </Card.Description>
350
  </div>
351
  <Button
 
440
  >
441
  {room.id}
442
  </p>
443
+ <div
444
+ class="flex items-center gap-3 text-xs text-slate-600 dark:text-slate-400"
445
+ >
446
  <span>{room.has_producer ? "🔴 Occupied" : "🟢 Available"}</span>
447
  <span>👥 {room.participants?.total || 0} users</span>
448
  <!-- Monitoring links -->
449
  <div class="flex gap-1">
450
  <a
451
+ href={`${settings.transportServerUrl.replace("/api", "")}/${workspaceId}/robotics/consumer?room=${room.id}`}
452
  target="_blank"
453
  rel="noopener noreferrer"
454
  class="inline-flex items-center gap-1 rounded bg-blue-500/10 px-1.5 py-0.5 text-xs text-blue-600 hover:bg-blue-500/20 dark:bg-blue-400/10 dark:text-blue-400 dark:hover:bg-blue-400/20"
 
458
  Consumer
459
  </a>
460
  <a
461
+ href={`${settings.transportServerUrl.replace("/api", "")}/${workspaceId}/robotics/producer?room=${room.id}`}
462
  target="_blank"
463
  rel="noopener noreferrer"
464
  class="inline-flex items-center gap-1 rounded bg-green-500/10 px-1.5 py-0.5 text-xs text-green-600 hover:bg-green-500/20 dark:bg-green-400/10 dark:text-green-400 dark:hover:bg-green-400/20"
src/lib/components/3d/elements/robot/status/ConnectionFlowBoxUIkit.svelte CHANGED
@@ -18,17 +18,31 @@
18
  delay?: number;
19
  }
20
 
21
- let { visible, robot, onInputBoxClick, onRobotBoxClick, onOutputBoxClick, duration = 100, delay = 0 }: Props = $props();
 
 
 
 
 
 
 
 
22
 
23
  const inputColor = "rgb(34, 197, 94)";
24
  const outputColor = "rgb(59, 130, 246)";
25
 
26
- const tweenedScale = Tween.of(() => {
27
- return visible ? 1 : 0;
28
- }, { duration: duration, easing: cubicOut, delay: delay });
29
- const tweenedOpacity = Tween.of(() => {
30
- return visible ? 1 : 0;
31
- }, { duration: duration, easing: cubicOut, delay: delay });
 
 
 
 
 
 
32
  </script>
33
 
34
  <Container
 
18
  delay?: number;
19
  }
20
 
21
+ let {
22
+ visible,
23
+ robot,
24
+ onInputBoxClick,
25
+ onRobotBoxClick,
26
+ onOutputBoxClick,
27
+ duration = 100,
28
+ delay = 0
29
+ }: Props = $props();
30
 
31
  const inputColor = "rgb(34, 197, 94)";
32
  const outputColor = "rgb(59, 130, 246)";
33
 
34
+ const tweenedScale = Tween.of(
35
+ () => {
36
+ return visible ? 1 : 0;
37
+ },
38
+ { duration: duration, easing: cubicOut, delay: delay }
39
+ );
40
+ const tweenedOpacity = Tween.of(
41
+ () => {
42
+ return visible ? 1 : 0;
43
+ },
44
+ { duration: duration, easing: cubicOut, delay: delay }
45
+ );
46
  </script>
47
 
48
  <Container
src/lib/components/3d/elements/robot/status/RobotBoxUIKit.svelte CHANGED
@@ -1,12 +1,7 @@
1
  <script lang="ts">
2
  import type { Robot } from "$lib/elements/robot/Robot.svelte";
3
  import { ICON } from "$lib/utils/icon";
4
- import {
5
- BaseStatusBox,
6
- StatusHeader,
7
- StatusContent,
8
- StatusButton
9
- } from "$lib/components/3d/ui";
10
 
11
  interface Props {
12
  robot: Robot;
@@ -18,34 +13,28 @@
18
  const robotColor = "rgb(245, 158, 11)";
19
  </script>
20
 
21
-
22
  <BaseStatusBox
23
  color={robotColor}
24
  borderOpacity={0.6}
25
  backgroundOpacity={0.2}
26
  onclick={() => onRobotBoxClick(robot)}
27
  >
28
- <!-- Robot Header -->
29
- <StatusHeader
30
- icon={ICON["icon-[ix--robotic-arm]"].svg}
31
- text="ROBOT"
32
- color={robotColor}
33
- opacity={0.9}
34
- />
35
 
36
- <!-- Robot Info -->
37
- <StatusContent
38
- title={robot.id}
39
- subtitle="Active"
40
- color={robotColor}
41
- variant="primary"
42
- />
43
 
44
- <!-- Status Button -->
45
- <StatusButton
46
- text="Active"
47
- icon={ICON["icon-[iconamoon--lightning-1-duotone]"].svg}
48
- color={robotColor}
49
- textOpacity={0.8}
50
- />
51
  </BaseStatusBox>
 
1
  <script lang="ts">
2
  import type { Robot } from "$lib/elements/robot/Robot.svelte";
3
  import { ICON } from "$lib/utils/icon";
4
+ import { BaseStatusBox, StatusHeader, StatusContent, StatusButton } from "$lib/components/3d/ui";
 
 
 
 
 
5
 
6
  interface Props {
7
  robot: Robot;
 
13
  const robotColor = "rgb(245, 158, 11)";
14
  </script>
15
 
 
16
  <BaseStatusBox
17
  color={robotColor}
18
  borderOpacity={0.6}
19
  backgroundOpacity={0.2}
20
  onclick={() => onRobotBoxClick(robot)}
21
  >
22
+ <!-- Robot Header -->
23
+ <StatusHeader
24
+ icon={ICON["icon-[ix--robotic-arm]"].svg}
25
+ text="ROBOT"
26
+ color={robotColor}
27
+ opacity={0.9}
28
+ />
29
 
30
+ <!-- Robot Info -->
31
+ <StatusContent title={robot.id} subtitle="Active" color={robotColor} variant="primary" />
 
 
 
 
 
32
 
33
+ <!-- Status Button -->
34
+ <StatusButton
35
+ text="Active"
36
+ icon={ICON["icon-[iconamoon--lightning-1-duotone]"].svg}
37
+ color={robotColor}
38
+ textOpacity={0.8}
39
+ />
40
  </BaseStatusBox>
src/lib/components/3d/elements/robot/status/RobotStatusBillboard.svelte CHANGED
@@ -27,34 +27,34 @@
27
  </script>
28
 
29
  <!-- {#if visible} -->
30
- <T.Group
31
- position.z={0.35}
32
- rotation={[Math.PI / 2, 0, 0]}
33
- scale={[0.12, 0.12, 0.12]}
34
- padding={10}
35
- pointerEvents="listener"
36
- >
37
- <Billboard>
38
- <Root name={`robot-status-billboard-${robot.id}`}>
39
- <Container
40
- width="100%"
41
- height="100%"
42
- alignItems="center"
43
- justifyContent="center"
44
- padding={20}
45
- >
46
- <ConnectionFlowBoxUIkit
47
- {visible}
48
- {robot}
49
- {onInputBoxClick}
50
- {onRobotBoxClick}
51
- {onOutputBoxClick}
52
- />
53
- </Container>
54
- </Root>
55
- </Billboard>
56
 
57
- <!-- <Billboard>
58
  <HTML
59
  transform
60
  autoRender={true}
@@ -86,5 +86,5 @@
86
  </div>
87
  </HTML>
88
  </Billboard> -->
89
- </T.Group>
90
  <!-- {/if} -->
 
27
  </script>
28
 
29
  <!-- {#if visible} -->
30
+ <T.Group
31
+ position.z={0.35}
32
+ rotation={[Math.PI / 2, 0, 0]}
33
+ scale={[0.12, 0.12, 0.12]}
34
+ padding={10}
35
+ pointerEvents="listener"
36
+ >
37
+ <Billboard>
38
+ <Root name={`robot-status-billboard-${robot.id}`}>
39
+ <Container
40
+ width="100%"
41
+ height="100%"
42
+ alignItems="center"
43
+ justifyContent="center"
44
+ padding={20}
45
+ >
46
+ <ConnectionFlowBoxUIkit
47
+ {visible}
48
+ {robot}
49
+ {onInputBoxClick}
50
+ {onRobotBoxClick}
51
+ {onOutputBoxClick}
52
+ />
53
+ </Container>
54
+ </Root>
55
+ </Billboard>
56
 
57
+ <!-- <Billboard>
58
  <HTML
59
  transform
60
  autoRender={true}
 
86
  </div>
87
  </HTML>
88
  </Billboard> -->
89
+ </T.Group>
90
  <!-- {/if} -->
src/lib/components/3d/elements/video/modal/VideoInputConnectionModal.svelte CHANGED
@@ -356,7 +356,7 @@
356
  <div class="flex items-center justify-between">
357
  <div>
358
  <Card.Title
359
- class="flex items-center gap-2 text-base text-purple-700 dark:text-purple-200 pb-1"
360
  >
361
  <span class="icon-[mdi--cloud-download] size-4"></span>
362
  Remote Control
@@ -484,7 +484,9 @@
484
  >
485
  {room.id}
486
  </p>
487
- <div class="flex items-center gap-3 text-xs text-slate-600 dark:text-slate-400">
 
 
488
  <span
489
  >{room.participants?.producer
490
  ? "📹 Has Output"
@@ -494,7 +496,7 @@
494
  <!-- Monitoring links -->
495
  <div class="flex gap-1">
496
  <a
497
- href={`${settings.transportServerUrl.replace('/api', '')}/${workspaceId}/video/consumer?room=${room.id}`}
498
  target="_blank"
499
  rel="noopener noreferrer"
500
  class="inline-flex items-center gap-1 rounded bg-blue-500/10 px-1.5 py-0.5 text-xs text-blue-600 hover:bg-blue-500/20 dark:bg-blue-400/10 dark:text-blue-400 dark:hover:bg-blue-400/20"
@@ -504,7 +506,7 @@
504
  Consumer
505
  </a>
506
  <a
507
- href={`${settings.transportServerUrl.replace('/api', '')}/${workspaceId}/video/producer?room=${room.id}`}
508
  target="_blank"
509
  rel="noopener noreferrer"
510
  class="inline-flex items-center gap-1 rounded bg-green-500/10 px-1.5 py-0.5 text-xs text-green-600 hover:bg-green-500/20 dark:bg-green-400/10 dark:text-green-400 dark:hover:bg-green-400/20"
@@ -552,7 +554,6 @@
552
  {/if}
553
  </Card.Content>
554
  </Card.Root>
555
-
556
  </div>
557
  </div>
558
  </Dialog.Content>
 
356
  <div class="flex items-center justify-between">
357
  <div>
358
  <Card.Title
359
+ class="flex items-center gap-2 pb-1 text-base text-purple-700 dark:text-purple-200"
360
  >
361
  <span class="icon-[mdi--cloud-download] size-4"></span>
362
  Remote Control
 
484
  >
485
  {room.id}
486
  </p>
487
+ <div
488
+ class="flex items-center gap-3 text-xs text-slate-600 dark:text-slate-400"
489
+ >
490
  <span
491
  >{room.participants?.producer
492
  ? "📹 Has Output"
 
496
  <!-- Monitoring links -->
497
  <div class="flex gap-1">
498
  <a
499
+ href={`${settings.transportServerUrl.replace("/api", "")}/${workspaceId}/video/consumer?room=${room.id}`}
500
  target="_blank"
501
  rel="noopener noreferrer"
502
  class="inline-flex items-center gap-1 rounded bg-blue-500/10 px-1.5 py-0.5 text-xs text-blue-600 hover:bg-blue-500/20 dark:bg-blue-400/10 dark:text-blue-400 dark:hover:bg-blue-400/20"
 
506
  Consumer
507
  </a>
508
  <a
509
+ href={`${settings.transportServerUrl.replace("/api", "")}/${workspaceId}/video/producer?room=${room.id}`}
510
  target="_blank"
511
  rel="noopener noreferrer"
512
  class="inline-flex items-center gap-1 rounded bg-green-500/10 px-1.5 py-0.5 text-xs text-green-600 hover:bg-green-500/20 dark:bg-green-400/10 dark:text-green-400 dark:hover:bg-green-400/20"
 
554
  {/if}
555
  </Card.Content>
556
  </Card.Root>
 
557
  </div>
558
  </div>
559
  </Dialog.Content>
src/lib/components/3d/elements/video/modal/VideoOutputConnectionModal.svelte CHANGED
@@ -366,7 +366,7 @@
366
  <div class="flex items-center justify-between">
367
  <div>
368
  <Card.Title
369
- class="flex items-center gap-2 text-base text-purple-700 dark:text-purple-200 pb-1"
370
  >
371
  <span class="icon-[mdi--cloud-upload] size-4"></span>
372
  Remote Control
@@ -494,7 +494,9 @@
494
  >
495
  {room.id}
496
  </p>
497
- <div class="flex items-center gap-3 text-xs text-slate-600 dark:text-slate-400">
 
 
498
  <span
499
  >{room.participants?.producer
500
  ? "🔴 Has Output"
@@ -504,7 +506,7 @@
504
  <!-- Monitoring links -->
505
  <div class="flex gap-1">
506
  <a
507
- href={`${settings.transportServerUrl.replace('/api', '')}/${workspaceId}/video/consumer?room=${room.id}`}
508
  target="_blank"
509
  rel="noopener noreferrer"
510
  class="inline-flex items-center gap-1 rounded bg-blue-500/10 px-1.5 py-0.5 text-xs text-blue-600 hover:bg-blue-500/20 dark:bg-blue-400/10 dark:text-blue-400 dark:hover:bg-blue-400/20"
@@ -514,7 +516,7 @@
514
  Consumer
515
  </a>
516
  <a
517
- href={`${settings.transportServerUrl.replace('/api', '')}/${workspaceId}/video/producer?room=${room.id}`}
518
  target="_blank"
519
  rel="noopener noreferrer"
520
  class="inline-flex items-center gap-1 rounded bg-green-500/10 px-1.5 py-0.5 text-xs text-green-600 hover:bg-green-500/20 dark:bg-green-400/10 dark:text-green-400 dark:hover:bg-green-400/20"
@@ -562,8 +564,6 @@
562
  {/if}
563
  </Card.Content>
564
  </Card.Root>
565
-
566
-
567
  </div>
568
  </div>
569
  </Dialog.Content>
 
366
  <div class="flex items-center justify-between">
367
  <div>
368
  <Card.Title
369
+ class="flex items-center gap-2 pb-1 text-base text-purple-700 dark:text-purple-200"
370
  >
371
  <span class="icon-[mdi--cloud-upload] size-4"></span>
372
  Remote Control
 
494
  >
495
  {room.id}
496
  </p>
497
+ <div
498
+ class="flex items-center gap-3 text-xs text-slate-600 dark:text-slate-400"
499
+ >
500
  <span
501
  >{room.participants?.producer
502
  ? "🔴 Has Output"
 
506
  <!-- Monitoring links -->
507
  <div class="flex gap-1">
508
  <a
509
+ href={`${settings.transportServerUrl.replace("/api", "")}/${workspaceId}/video/consumer?room=${room.id}`}
510
  target="_blank"
511
  rel="noopener noreferrer"
512
  class="inline-flex items-center gap-1 rounded bg-blue-500/10 px-1.5 py-0.5 text-xs text-blue-600 hover:bg-blue-500/20 dark:bg-blue-400/10 dark:text-blue-400 dark:hover:bg-blue-400/20"
 
516
  Consumer
517
  </a>
518
  <a
519
+ href={`${settings.transportServerUrl.replace("/api", "")}/${workspaceId}/video/producer?room=${room.id}`}
520
  target="_blank"
521
  rel="noopener noreferrer"
522
  class="inline-flex items-center gap-1 rounded bg-green-500/10 px-1.5 py-0.5 text-xs text-green-600 hover:bg-green-500/20 dark:bg-green-400/10 dark:text-green-400 dark:hover:bg-green-400/20"
 
564
  {/if}
565
  </Card.Content>
566
  </Card.Root>
 
 
567
  </div>
568
  </div>
569
  </Dialog.Content>
src/lib/components/3d/elements/video/status/VideoBoxUIKit.svelte CHANGED
@@ -1,11 +1,7 @@
1
  <script lang="ts">
2
  import { ICON } from "$lib/utils/icon";
3
  import type { VideoInstance } from "$lib/elements/video/VideoManager.svelte";
4
- import {
5
- BaseStatusBox,
6
- StatusHeader,
7
- StatusContent
8
- } from "$lib/components/3d/ui";
9
 
10
  interface Props {
11
  video: VideoInstance;
@@ -16,26 +12,20 @@
16
  const videoColor = "rgb(217, 119, 6)";
17
  </script>
18
 
 
 
 
 
 
 
 
 
19
 
20
- <BaseStatusBox
21
- color={videoColor}
22
- borderOpacity={0.6}
23
- backgroundOpacity={0.2}
24
- clickable={false}
25
- >
26
- <!-- Video Header -->
27
- <StatusHeader
28
- icon={ICON["icon-[mdi--video]"].svg}
29
- text="VIDEO"
30
- color={videoColor}
31
- opacity={0.9}
32
- />
33
-
34
- <!-- Video Info -->
35
- <StatusContent
36
- title={video.name}
37
- subtitle={video.id.slice(0, 8)}
38
- color="rgb(253, 230, 138)"
39
- variant="primary"
40
- />
41
- </BaseStatusBox>
 
1
  <script lang="ts">
2
  import { ICON } from "$lib/utils/icon";
3
  import type { VideoInstance } from "$lib/elements/video/VideoManager.svelte";
4
+ import { BaseStatusBox, StatusHeader, StatusContent } from "$lib/components/3d/ui";
 
 
 
 
5
 
6
  interface Props {
7
  video: VideoInstance;
 
12
  const videoColor = "rgb(217, 119, 6)";
13
  </script>
14
 
15
+ <BaseStatusBox color={videoColor} borderOpacity={0.6} backgroundOpacity={0.2} clickable={false}>
16
+ <!-- Video Header -->
17
+ <StatusHeader
18
+ icon={ICON["icon-[mdi--video]"].svg}
19
+ text="VIDEO"
20
+ color={videoColor}
21
+ opacity={0.9}
22
+ />
23
 
24
+ <!-- Video Info -->
25
+ <StatusContent
26
+ title={video.name}
27
+ subtitle={video.id.slice(0, 8)}
28
+ color="rgb(253, 230, 138)"
29
+ variant="primary"
30
+ />
31
+ </BaseStatusBox>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
src/lib/components/3d/elements/video/status/VideoConnectionFlowBoxUIKit.svelte CHANGED
@@ -17,17 +17,30 @@
17
  delay?: number;
18
  }
19
 
20
- let { visible, video, onInputBoxClick, onOutputBoxClick, duration = 100, delay = 0 }: Props = $props();
 
 
 
 
 
 
 
21
 
22
  const inputColor = "rgb(34, 197, 94)";
23
  const outputColor = "rgb(59, 130, 246)";
24
 
25
- const tweenedScale = Tween.of(() => {
26
- return visible ? 1 : 0;
27
- }, { duration: duration, easing: cubicOut, delay: delay });
28
- const tweenedOpacity = Tween.of(() => {
29
- return visible ? 1 : 0;
30
- }, { duration: duration, easing: cubicOut, delay: delay });
 
 
 
 
 
 
31
  </script>
32
 
33
  <Container
 
17
  delay?: number;
18
  }
19
 
20
+ let {
21
+ visible,
22
+ video,
23
+ onInputBoxClick,
24
+ onOutputBoxClick,
25
+ duration = 100,
26
+ delay = 0
27
+ }: Props = $props();
28
 
29
  const inputColor = "rgb(34, 197, 94)";
30
  const outputColor = "rgb(59, 130, 246)";
31
 
32
+ const tweenedScale = Tween.of(
33
+ () => {
34
+ return visible ? 1 : 0;
35
+ },
36
+ { duration: duration, easing: cubicOut, delay: delay }
37
+ );
38
+ const tweenedOpacity = Tween.of(
39
+ () => {
40
+ return visible ? 1 : 0;
41
+ },
42
+ { duration: duration, easing: cubicOut, delay: delay }
43
+ );
44
  </script>
45
 
46
  <Container
src/lib/components/3d/elements/video/status/VideoStatusBillboard.svelte CHANGED
@@ -19,32 +19,32 @@
19
  </script>
20
 
21
  <!-- {#if visible} -->
22
- <T.Group
23
- onpointerdown={(e) => e.stopPropagation()}
24
- onpointerup={(e) => e.stopPropagation()}
25
- onpointermove={(e) => e.stopPropagation()}
26
- onclick={(e) => e.stopPropagation()}
27
- position.z={0.22}
28
- padding={10}
29
- rotation={[Math.PI / 2, 0, 0]}
30
- scale={[0.12, 0.12, 0.12]}
31
- pointerEvents="listener"
32
- >
33
- <Billboard>
34
- <Root name={`video-status-billboard-${video.id}`}>
35
- <Container
36
- width="100%"
37
- height="100%"
38
- alignItems="center"
39
- justifyContent="center"
40
- padding={20}
41
- >
42
- <VideoConnectionFlowBoxUIKit {visible} {video} {onInputBoxClick} {onOutputBoxClick} />
43
- </Container>
44
- </Root>
45
- </Billboard>
46
 
47
- <!-- <Billboard>
48
  <HTML
49
  transform
50
  autoRender={true}
@@ -72,5 +72,5 @@
72
  </div>
73
  </HTML>
74
  </Billboard> -->
75
- </T.Group>
76
  <!-- {/if} -->
 
19
  </script>
20
 
21
  <!-- {#if visible} -->
22
+ <T.Group
23
+ onpointerdown={(e) => e.stopPropagation()}
24
+ onpointerup={(e) => e.stopPropagation()}
25
+ onpointermove={(e) => e.stopPropagation()}
26
+ onclick={(e) => e.stopPropagation()}
27
+ position.z={0.22}
28
+ padding={10}
29
+ rotation={[Math.PI / 2, 0, 0]}
30
+ scale={[0.12, 0.12, 0.12]}
31
+ pointerEvents="listener"
32
+ >
33
+ <Billboard>
34
+ <Root name={`video-status-billboard-${video.id}`}>
35
+ <Container
36
+ width="100%"
37
+ height="100%"
38
+ alignItems="center"
39
+ justifyContent="center"
40
+ padding={20}
41
+ >
42
+ <VideoConnectionFlowBoxUIKit {visible} {video} {onInputBoxClick} {onOutputBoxClick} />
43
+ </Container>
44
+ </Root>
45
+ </Billboard>
46
 
47
+ <!-- <Billboard>
48
  <HTML
49
  transform
50
  autoRender={true}
 
72
  </div>
73
  </HTML>
74
  </Billboard> -->
75
+ </T.Group>
76
  <!-- {/if} -->
src/lib/components/3d/ui/StatusArrow.svelte CHANGED
@@ -6,7 +6,7 @@
6
  color?: string;
7
  opacity?: number;
8
  size?: number;
9
- direction?: 'right' | 'down' | 'left' | 'up';
10
  minWidth?: number;
11
  minHeight?: number;
12
  }
@@ -15,7 +15,7 @@
15
  color = "rgb(139, 69, 219)",
16
  opacity = 1,
17
  size = 12,
18
- direction = 'right',
19
  minWidth = 20,
20
  minHeight = 12
21
  }: Props = $props();
@@ -29,7 +29,7 @@
29
  {minWidth}
30
  {minHeight}
31
  >
32
- {#if direction === 'right'}
33
  <SVG
34
  width={size}
35
  height={size}
@@ -37,29 +37,11 @@
37
  {opacity}
38
  src={ICON[`icon-[formkit--arrowright]`].svg}
39
  />
40
- {:else if direction === 'down'}
41
- <SVG
42
- width={size}
43
- height={size}
44
- {color}
45
- {opacity}
46
- src={ICON[`icon-[formkit--arrowdown]`].svg}
47
- />
48
- {:else if direction === 'left'}
49
- <SVG
50
- width={size}
51
- height={size}
52
- {color}
53
- {opacity}
54
- src={ICON[`icon-[formkit--arrowleft]`].svg}
55
- />
56
- {:else if direction === 'up'}
57
- <SVG
58
- width={size}
59
- height={size}
60
- {color}
61
- {opacity}
62
- src={ICON[`icon-[formkit--arrowup]`].svg}
63
- />
64
  {/if}
65
- </Container>
 
6
  color?: string;
7
  opacity?: number;
8
  size?: number;
9
+ direction?: "right" | "down" | "left" | "up";
10
  minWidth?: number;
11
  minHeight?: number;
12
  }
 
15
  color = "rgb(139, 69, 219)",
16
  opacity = 1,
17
  size = 12,
18
+ direction = "right",
19
  minWidth = 20,
20
  minHeight = 12
21
  }: Props = $props();
 
29
  {minWidth}
30
  {minHeight}
31
  >
32
+ {#if direction === "right"}
33
  <SVG
34
  width={size}
35
  height={size}
 
37
  {opacity}
38
  src={ICON[`icon-[formkit--arrowright]`].svg}
39
  />
40
+ {:else if direction === "down"}
41
+ <SVG width={size} height={size} {color} {opacity} src={ICON[`icon-[formkit--arrowdown]`].svg} />
42
+ {:else if direction === "left"}
43
+ <SVG width={size} height={size} {color} {opacity} src={ICON[`icon-[formkit--arrowleft]`].svg} />
44
+ {:else if direction === "up"}
45
+ <SVG width={size} height={size} {color} {opacity} src={ICON[`icon-[formkit--arrowup]`].svg} />
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
46
  {/if}
47
+ </Container>
src/lib/components/3d/ui/StatusButton.svelte CHANGED
@@ -47,15 +47,9 @@
47
  {onclick}
48
  >
49
  {#if icon}
50
- <SVG
51
- width={iconSize}
52
- height={iconSize}
53
- {color}
54
- opacity={textOpacity}
55
- src={icon}
56
- />
57
  {/if}
58
-
59
  <Text
60
  {text}
61
  fontSize={textSize}
@@ -64,4 +58,4 @@
64
  opacity={textOpacity}
65
  textAlign="center"
66
  />
67
- </Container>
 
47
  {onclick}
48
  >
49
  {#if icon}
50
+ <SVG width={iconSize} height={iconSize} {color} opacity={textOpacity} src={icon} />
 
 
 
 
 
 
51
  {/if}
52
+
53
  <Text
54
  {text}
55
  fontSize={textSize}
 
58
  opacity={textOpacity}
59
  textAlign="center"
60
  />
61
+ </Container>
src/lib/components/3d/ui/StatusContent.svelte CHANGED
@@ -6,10 +6,10 @@
6
  subtitle?: string;
7
  description?: string;
8
  color?: string;
9
- variant?: 'primary' | 'secondary' | 'tertiary';
10
- size?: 'sm' | 'md' | 'lg';
11
- align?: 'left' | 'center' | 'right';
12
- children?: import('svelte').Snippet;
13
  }
14
 
15
  let {
@@ -17,9 +17,9 @@
17
  subtitle,
18
  description,
19
  color = "rgb(221, 214, 254)",
20
- variant = 'primary',
21
- size = 'md',
22
- align = 'center',
23
  children
24
  }: Props = $props();
25
 
@@ -40,19 +40,15 @@
40
  const config = sizeConfigs[size];
41
  const opacities = opacityLevels[variant];
42
 
43
- const flexAlign = align === 'left' ? 'flex-start' : align === 'right' ? 'flex-end' : 'center';
44
  </script>
45
 
46
- <Container
47
- padding={config.padding}
48
- marginBottom={4}
49
- width="100%"
50
- >
51
  {#if children}
52
  {@render children()}
53
  {:else}
54
- <Container
55
- flexDirection="column"
56
  alignItems={flexAlign}
57
  justifyContent="center"
58
  gap={config.gap}
@@ -68,7 +64,7 @@
68
  width="100%"
69
  />
70
  {/if}
71
-
72
  {#if subtitle}
73
  <Text
74
  text={subtitle}
@@ -80,7 +76,7 @@
80
  width="100%"
81
  />
82
  {/if}
83
-
84
  {#if description}
85
  <Text
86
  text={description}
@@ -94,4 +90,4 @@
94
  {/if}
95
  </Container>
96
  {/if}
97
- </Container>
 
6
  subtitle?: string;
7
  description?: string;
8
  color?: string;
9
+ variant?: "primary" | "secondary" | "tertiary";
10
+ size?: "sm" | "md" | "lg";
11
+ align?: "left" | "center" | "right";
12
+ children?: import("svelte").Snippet;
13
  }
14
 
15
  let {
 
17
  subtitle,
18
  description,
19
  color = "rgb(221, 214, 254)",
20
+ variant = "primary",
21
+ size = "md",
22
+ align = "center",
23
  children
24
  }: Props = $props();
25
 
 
40
  const config = sizeConfigs[size];
41
  const opacities = opacityLevels[variant];
42
 
43
+ const flexAlign = align === "left" ? "flex-start" : align === "right" ? "flex-end" : "center";
44
  </script>
45
 
46
+ <Container padding={config.padding} marginBottom={4} width="100%">
 
 
 
 
47
  {#if children}
48
  {@render children()}
49
  {:else}
50
+ <Container
51
+ flexDirection="column"
52
  alignItems={flexAlign}
53
  justifyContent="center"
54
  gap={config.gap}
 
64
  width="100%"
65
  />
66
  {/if}
67
+
68
  {#if subtitle}
69
  <Text
70
  text={subtitle}
 
76
  width="100%"
77
  />
78
  {/if}
79
+
80
  {#if description}
81
  <Text
82
  text={description}
 
90
  {/if}
91
  </Container>
92
  {/if}
93
+ </Container>
src/lib/components/3d/ui/icons.ts CHANGED
@@ -1,7 +1,8 @@
1
  // Common generic SVG icons encoded as base64 data URLs
2
  export const icons = {
3
  plus: "data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIyNCIgaGVpZ2h0PSIyNCIgdmlld0JveD0iMCAwIDI0IDI0Ij48cGF0aCBmaWxsPSIjMDAwIiBkPSJNMTkgMTNoLTZ2NmgtMnYtNkg1di0yaDZWNWgydjZoNnoiLz48L3N2Zz4=",
4
- settings: "data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIyNCIgaGVpZ2h0PSIyNCIgdmlld0JveD0iMCAwIDI0IDI0Ij48cGF0aCBmaWxsPSIjMDAwIiBkPSJtOS4yNSAyMmwtLjQtMy4ycS0uMzI1LS4xMjUtLjYxMi0uM3QtLjU2My0uMzc1TDQuNyAxOS4zNzVsLTIuNzUtNC43NWwyLjU3NS0xLjk1UTQuNSAxMi41IDQuNSAxMi4zMzh2LS42NzVxMC0uMTYzLjAyNS0uMzM4TDEuOTUgOS4zNzVsMi43NS00Ljc1bDIuOTc5IDEuMjVxLjI3NS0uMi41NzUtLjM3NXQuNi0uM2wuNC0zLjJoNS41bC40IDMuMnEuMzI1LjEyNS42MTMuM3QuNTYyLjM3NWwyLjk3NS0xLjI1bDIuNzUgNC43NWwtMi41NzUgMS45NXEuMDI1LjE3NS4wMjUuMzM4di42NzRxMCAuMTYzLS4wNS4zMzhsMi41NzUgMS45NWwtMi43NSA0Ljc1bC0yLjk1LTEuMjVxLS4yNzUuMi0uNTc1LjM3NXQtLjYuM2wtLjQgMy4yem0yLjgtNi41cTEuNDUgMCAyLjQ3NS0xLjAyNVQxNS41NSAxMnQtMS4wMjUtMi40NzVUMTIuMDUgOC41cS0xLjQ3NSAwLTIuNDg4IDEuMDI1VDguNTUgMTJ0MS4wMTMgMi40NzVUMTIuMDUgMTUuNSIvPjwvc3ZnPg=="
 
5
  } as const;
6
-
7
- export type IconName = keyof typeof icons;
 
1
  // Common generic SVG icons encoded as base64 data URLs
2
  export const icons = {
3
  plus: "data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIyNCIgaGVpZ2h0PSIyNCIgdmlld0JveD0iMCAwIDI0IDI0Ij48cGF0aCBmaWxsPSIjMDAwIiBkPSJNMTkgMTNoLTZ2NmgtMnYtNkg1di0yaDZWNWgydjZoNnoiLz48L3N2Zz4=",
4
+ settings:
5
+ "data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIyNCIgaGVpZ2h0PSIyNCIgdmlld0JveD0iMCAwIDI0IDI0Ij48cGF0aCBmaWxsPSIjMDAwIiBkPSJtOS4yNSAyMmwtLjQtMy4ycS0uMzI1LS4xMjUtLjYxMi0uM3QtLjU2My0uMzc1TDQuNyAxOS4zNzVsLTIuNzUtNC43NWwyLjU3NS0xLjk1UTQuNSAxMi41IDQuNSAxMi4zMzh2LS42NzVxMC0uMTYzLjAyNS0uMzM4TDEuOTUgOS4zNzVsMi43NS00Ljc1bDIuOTc5IDEuMjVxLjI3NS0uMi41NzUtLjM3NXQuNi0uM2wuNC0zLjJoNS41bC40IDMuMnEuMzI1LjEyNS42MTMuM3QuNTYyLjM3NWwyLjk3NS0xLjI1bDIuNzUgNC43NWwtMi41NzUgMS45NXEuMDI1LjE3NS4wMjUuMzM4di42NzRxMCAuMTYzLS4wNS4zMzhsMi41NzUgMS45NWwtMi43NSA0Ljc1bC0yLjk1LTEuMjVxLS4yNzUuMi0uNTc1LjM3NXQtLjYuM2wtLjQgMy4yem0yLjgtNi41cTEuNDUgMCAyLjQ3NS0xLjAyNVQxNS41NSAxMnQtMS4wMjUtMi40NzVUMTIuMDUgOC41cS0xLjQ3NSAwLTIuNDg4IDEuMDI1VDguNTUgMTJ0MS4wMTMgMi40NzVUMTIuMDUgMTUuNSIvPjwvc3ZnPg=="
6
  } as const;
7
+
8
+ export type IconName = keyof typeof icons;
src/lib/components/3d/ui/index.ts CHANGED
@@ -1,6 +1,6 @@
1
- export { default as BaseStatusBox } from './BaseStatusBox.svelte';
2
- export { default as StatusHeader } from './StatusHeader.svelte';
3
- export { default as StatusContent } from './StatusContent.svelte';
4
- export { default as StatusIndicator } from './StatusIndicator.svelte';
5
- export { default as StatusButton } from './StatusButton.svelte';
6
- export { default as StatusArrow } from './StatusArrow.svelte';
 
1
+ export { default as BaseStatusBox } from "./BaseStatusBox.svelte";
2
+ export { default as StatusHeader } from "./StatusHeader.svelte";
3
+ export { default as StatusContent } from "./StatusContent.svelte";
4
+ export { default as StatusIndicator } from "./StatusIndicator.svelte";
5
+ export { default as StatusButton } from "./StatusButton.svelte";
6
+ export { default as StatusArrow } from "./StatusArrow.svelte";
src/lib/components/interface/overlay/AddAIButton.svelte CHANGED
@@ -13,10 +13,10 @@
13
  let { open = $bindable(), workspaceId }: Props = $props();
14
 
15
  let isConfigModalOpen = $state(false);
16
- let selectedModelType = $state<ModelType>('act');
17
 
18
  // Get available model types
19
- const availableModels = Object.values(MODEL_TYPES).filter(model => model.enabled);
20
 
21
  function openConfigModal(modelType: ModelType) {
22
  selectedModelType = modelType;
@@ -25,7 +25,7 @@
25
  }
26
 
27
  function quickAddACT() {
28
- openConfigModal('act');
29
  }
30
 
31
  function formatModelType(modelType: string): string {
@@ -87,10 +87,12 @@
87
  {/snippet}
88
  </DropdownMenu.Trigger>
89
 
90
- <DropdownMenu.Content class="w-64 bg-slate-100 border-slate-300 dark:bg-slate-900 dark:border-slate-600">
 
 
91
  {#each availableModels as model}
92
  <DropdownMenu.Item
93
- class="flex items-center gap-3 p-3 cursor-pointer hover:bg-purple-100 dark:hover:bg-purple-900/30"
94
  onclick={() => openConfigModal(model.id)}
95
  >
96
  <span class="{model.icon} size-5 text-purple-500 dark:text-purple-400"></span>
@@ -108,8 +110,8 @@
108
  </DropdownMenu.Root>
109
 
110
  <!-- Configuration Modal -->
111
- <AIModelConfigurationModal
112
- bind:open={isConfigModalOpen}
113
  {workspaceId}
114
  initialModelType={selectedModelType}
115
  />
 
13
  let { open = $bindable(), workspaceId }: Props = $props();
14
 
15
  let isConfigModalOpen = $state(false);
16
+ let selectedModelType = $state<ModelType>("act");
17
 
18
  // Get available model types
19
+ const availableModels = Object.values(MODEL_TYPES).filter((model) => model.enabled);
20
 
21
  function openConfigModal(modelType: ModelType) {
22
  selectedModelType = modelType;
 
25
  }
26
 
27
  function quickAddACT() {
28
+ openConfigModal("act");
29
  }
30
 
31
  function formatModelType(modelType: string): string {
 
87
  {/snippet}
88
  </DropdownMenu.Trigger>
89
 
90
+ <DropdownMenu.Content
91
+ class="w-64 border-slate-300 bg-slate-100 dark:border-slate-600 dark:bg-slate-900"
92
+ >
93
  {#each availableModels as model}
94
  <DropdownMenu.Item
95
+ class="flex cursor-pointer items-center gap-3 p-3 hover:bg-purple-100 dark:hover:bg-purple-900/30"
96
  onclick={() => openConfigModal(model.id)}
97
  >
98
  <span class="{model.icon} size-5 text-purple-500 dark:text-purple-400"></span>
 
110
  </DropdownMenu.Root>
111
 
112
  <!-- Configuration Modal -->
113
+ <AIModelConfigurationModal
114
+ bind:open={isConfigModalOpen}
115
  {workspaceId}
116
  initialModelType={selectedModelType}
117
  />
src/lib/components/interface/overlay/AddRobotButton.svelte CHANGED
@@ -45,7 +45,7 @@
45
  }
46
 
47
  async function quickAddDefault() {
48
- const defaultRobotType = robotTypes.find(type => robotUrdfConfigMap[type].isDefault);
49
  if (defaultRobotType) {
50
  await addRobot(defaultRobotType);
51
  } else {
@@ -53,7 +53,6 @@
53
  await addRobot(robotTypes[0]);
54
  }
55
  }
56
-
57
  </script>
58
 
59
  <!-- Main Add Button (Default Robot) -->
@@ -127,7 +126,9 @@
127
  <span class="font-medium text-white transition-colors duration-200"
128
  >{urdfConfig.displayName || robotType.replace(/-/g, " ").toUpperCase()}</span
129
  >
130
- <span class="text-xs text-emerald-100 transition-colors duration-200 dark:text-emerald-200">
 
 
131
  {urdfConfig.description || "Robot"}
132
  </span>
133
  </div>
 
45
  }
46
 
47
  async function quickAddDefault() {
48
+ const defaultRobotType = robotTypes.find((type) => robotUrdfConfigMap[type].isDefault);
49
  if (defaultRobotType) {
50
  await addRobot(defaultRobotType);
51
  } else {
 
53
  await addRobot(robotTypes[0]);
54
  }
55
  }
 
56
  </script>
57
 
58
  <!-- Main Add Button (Default Robot) -->
 
126
  <span class="font-medium text-white transition-colors duration-200"
127
  >{urdfConfig.displayName || robotType.replace(/-/g, " ").toUpperCase()}</span
128
  >
129
+ <span
130
+ class="text-xs text-emerald-100 transition-colors duration-200 dark:text-emerald-200"
131
+ >
132
  {urdfConfig.description || "Robot"}
133
  </span>
134
  </div>
src/lib/components/interface/overlay/AddSensorButton.svelte CHANGED
@@ -21,34 +21,34 @@
21
  }
22
 
23
  const sensorConfigs: SensorConfig[] = [
24
- {
25
- id: 'camera',
26
- label: 'Camera',
27
- description: 'Video Camera Sensor',
28
- icon: 'icon-[mdi--camera]',
29
  enabled: true,
30
  isDefault: true
31
  },
32
- {
33
- id: 'depth-camera',
34
- label: 'Depth Camera',
35
- description: 'Depth Camera Sensor',
36
- icon: 'icon-[mdi--camera]',
37
- enabled: false
38
  },
39
- {
40
- id: 'lidar',
41
- label: 'Lidar',
42
- description: 'Distance Sensor',
43
- icon: 'icon-[mdi--radar]',
44
- enabled: false
45
  },
46
- {
47
- id: 'imu',
48
- label: 'IMU',
49
- description: 'Motion Sensor',
50
- icon: 'icon-[mdi--radar]',
51
- enabled: false
52
  }
53
  ];
54
 
@@ -58,7 +58,7 @@
58
  if (!sensorType) return;
59
 
60
  const sensorId = `${sensorType}_${Date.now()}`;
61
-
62
  if (sensorType === "camera") {
63
  // Create video camera
64
  const video = videoManager.createVideo(sensorId);
@@ -67,7 +67,7 @@
67
  });
68
  } else {
69
  // Placeholder for other sensor types
70
- const config = sensorConfigs.find(c => c.id === sensorType);
71
  toast.success("Sensor Added", {
72
  description: `${config?.label || sensorType} sensor ${sensorId.slice(0, 12)}... created successfully.`
73
  });
@@ -105,7 +105,7 @@
105
 
106
  <!-- Dropdown Menu Button -->
107
  <DropdownMenu.Root bind:open>
108
- <DropdownMenu.Trigger >
109
  {#snippet child({ props })}
110
  <Button
111
  {...props}
@@ -155,8 +155,7 @@
155
  ]}
156
  ></span>
157
  <div class="flex flex-1 flex-col">
158
- <span class="font-medium text-white transition-colors duration-200"
159
- >{sensor.label}</span
160
  >
161
  <span class="text-xs text-blue-100 transition-colors duration-200 dark:text-blue-200">
162
  {sensor.description}
@@ -174,4 +173,4 @@
174
  {/each}
175
  </DropdownMenu.Group>
176
  </DropdownMenu.Content>
177
- </DropdownMenu.Root>
 
21
  }
22
 
23
  const sensorConfigs: SensorConfig[] = [
24
+ {
25
+ id: "camera",
26
+ label: "Camera",
27
+ description: "Video Camera Sensor",
28
+ icon: "icon-[mdi--camera]",
29
  enabled: true,
30
  isDefault: true
31
  },
32
+ {
33
+ id: "depth-camera",
34
+ label: "Depth Camera",
35
+ description: "Depth Camera Sensor",
36
+ icon: "icon-[mdi--camera]",
37
+ enabled: false
38
  },
39
+ {
40
+ id: "lidar",
41
+ label: "Lidar",
42
+ description: "Distance Sensor",
43
+ icon: "icon-[mdi--radar]",
44
+ enabled: false
45
  },
46
+ {
47
+ id: "imu",
48
+ label: "IMU",
49
+ description: "Motion Sensor",
50
+ icon: "icon-[mdi--radar]",
51
+ enabled: false
52
  }
53
  ];
54
 
 
58
  if (!sensorType) return;
59
 
60
  const sensorId = `${sensorType}_${Date.now()}`;
61
+
62
  if (sensorType === "camera") {
63
  // Create video camera
64
  const video = videoManager.createVideo(sensorId);
 
67
  });
68
  } else {
69
  // Placeholder for other sensor types
70
+ const config = sensorConfigs.find((c) => c.id === sensorType);
71
  toast.success("Sensor Added", {
72
  description: `${config?.label || sensorType} sensor ${sensorId.slice(0, 12)}... created successfully.`
73
  });
 
105
 
106
  <!-- Dropdown Menu Button -->
107
  <DropdownMenu.Root bind:open>
108
+ <DropdownMenu.Trigger>
109
  {#snippet child({ props })}
110
  <Button
111
  {...props}
 
155
  ]}
156
  ></span>
157
  <div class="flex flex-1 flex-col">
158
+ <span class="font-medium text-white transition-colors duration-200">{sensor.label}</span
 
159
  >
160
  <span class="text-xs text-blue-100 transition-colors duration-200 dark:text-blue-200">
161
  {sensor.description}
 
173
  {/each}
174
  </DropdownMenu.Group>
175
  </DropdownMenu.Content>
176
+ </DropdownMenu.Root>
src/lib/components/interface/overlay/Overlay.svelte CHANGED
@@ -27,32 +27,34 @@
27
 
28
  <div class="select-none">
29
  <!-- Responsive Button Bar Container -->
30
- <div class="fixed top-2 left-2 right-2 z-50 flex flex-wrap items-center justify-between gap-1 select-none md:top-4 md:left-4 md:right-4 md:gap-2">
 
 
31
  <!-- Left Group: Logo + Add Buttons -->
32
- <div class="flex items-center gap-1 flex-wrap md:gap-2">
33
  <!-- Logo/Favicon -->
34
  <div class="flex items-center justify-center">
35
  <img
36
  src="/favicon_1024.png"
37
  alt="Logo"
38
  draggable="false"
39
- class="h-8 w-8 invert-0 filter dark:invert md:h-10 md:w-10"
40
  />
41
  </div>
42
-
43
  <!-- Add Robot Button Group -->
44
  <div class="flex items-center justify-center overflow-hidden rounded-lg">
45
  <AddRobotButton bind:open={addRobotDropdownMenuOpen} />
46
  </div>
47
 
48
  <!-- Add Sensor Button Group - Hidden on very small screens -->
49
- <div class="hidden min-[480px]:flex items-center justify-center overflow-hidden rounded-lg">
50
  <AddSensorButton bind:open={addSensorDropdownMenuOpen} />
51
  </div>
52
 
53
  <!-- Add AI Button Group - Hidden on small screens -->
54
- <div class="hidden min-[560px]:flex items-center justify-center overflow-hidden rounded-lg">
55
- <AddAIButton bind:open={addAIDropdownMenuOpen} workspaceId={workspaceId} />
56
  </div>
57
  </div>
58
 
 
27
 
28
  <div class="select-none">
29
  <!-- Responsive Button Bar Container -->
30
+ <div
31
+ class="fixed top-2 right-2 left-2 z-50 flex flex-wrap items-center justify-between gap-1 select-none md:top-4 md:right-4 md:left-4 md:gap-2"
32
+ >
33
  <!-- Left Group: Logo + Add Buttons -->
34
+ <div class="flex flex-wrap items-center gap-1 md:gap-2">
35
  <!-- Logo/Favicon -->
36
  <div class="flex items-center justify-center">
37
  <img
38
  src="/favicon_1024.png"
39
  alt="Logo"
40
  draggable="false"
41
+ class="h-8 w-8 invert-0 filter md:h-10 md:w-10 dark:invert"
42
  />
43
  </div>
44
+
45
  <!-- Add Robot Button Group -->
46
  <div class="flex items-center justify-center overflow-hidden rounded-lg">
47
  <AddRobotButton bind:open={addRobotDropdownMenuOpen} />
48
  </div>
49
 
50
  <!-- Add Sensor Button Group - Hidden on very small screens -->
51
+ <div class="hidden items-center justify-center overflow-hidden rounded-lg min-[480px]:flex">
52
  <AddSensorButton bind:open={addSensorDropdownMenuOpen} />
53
  </div>
54
 
55
  <!-- Add AI Button Group - Hidden on small screens -->
56
+ <div class="hidden items-center justify-center overflow-hidden rounded-lg min-[560px]:flex">
57
+ <AddAIButton bind:open={addAIDropdownMenuOpen} {workspaceId} />
58
  </div>
59
  </div>
60
 
src/lib/components/interface/overlay/SettingsSheet.svelte CHANGED
@@ -130,7 +130,8 @@
130
  >Inference Server URL</Label
131
  >
132
  <p class="text-xs text-slate-600 dark:text-slate-400">
133
- URL for the remote AI inference server to run policies using remote compute resources
 
134
  </p>
135
  </div>
136
  <div class="flex gap-2">
 
130
  >Inference Server URL</Label
131
  >
132
  <p class="text-xs text-slate-600 dark:text-slate-400">
133
+ URL for the remote AI inference server to run policies using remote compute
134
+ resources
135
  </p>
136
  </div>
137
  <div class="flex gap-2">
src/lib/configs/robotUrdfConfig.ts CHANGED
@@ -13,7 +13,7 @@ export const robotUrdfConfigMap: { [key: string]: RobotUrdfConfig } = {
13
  Elbow: 3,
14
  Wrist_Pitch: 4,
15
  Wrist_Roll: 5,
16
- Jaw: 6,
17
  // camera_mount: 7
18
  },
19
  // Rest position - robot in neutral/calibration pose (all joints at 0 degrees)
@@ -23,7 +23,7 @@ export const robotUrdfConfigMap: { [key: string]: RobotUrdfConfig } = {
23
  Elbow: 0,
24
  Wrist_Pitch: 0,
25
  Wrist_Roll: 0,
26
- Jaw: 0,
27
  // camera_mount: 0
28
  },
29
  compoundMovements: [
 
13
  Elbow: 3,
14
  Wrist_Pitch: 4,
15
  Wrist_Roll: 5,
16
+ Jaw: 6
17
  // camera_mount: 7
18
  },
19
  // Rest position - robot in neutral/calibration pose (all joints at 0 degrees)
 
23
  Elbow: 0,
24
  Wrist_Pitch: 0,
25
  Wrist_Roll: 0,
26
+ Jaw: 0
27
  // camera_mount: 0
28
  },
29
  compoundMovements: [
src/lib/elements/compute/RemoteCompute.svelte.ts CHANGED
@@ -1,147 +1,153 @@
1
- import type { Positionable, Position3D } from '$lib/types/positionable.js';
2
- import type { AISessionConfig, AISessionResponse, ModelType } from './RemoteComputeManager.svelte';
3
 
4
- export type ComputeStatus = 'disconnected' | 'ready' | 'running' | 'stopped' | 'initializing';
5
 
6
  export class RemoteCompute implements Positionable {
7
- readonly id: string;
8
-
9
- // Reactive state using Svelte 5 runes
10
- position = $state<Position3D>({ x: 0, y: 0, z: 0 });
11
- name = $state<string>('');
12
- status = $state<ComputeStatus>('disconnected');
13
- modelType = $state<ModelType>('act'); // Default to ACT model
14
-
15
- // Session data
16
- sessionId = $state<string | null>(null);
17
- sessionConfig = $state<AISessionConfig | null>(null);
18
- sessionData = $state<AISessionResponse | null>(null);
19
-
20
- // Derived reactive values
21
- hasSession = $derived(this.sessionId !== null);
22
- isRunning = $derived(this.status === 'running');
23
- canStart = $derived(this.status === 'ready' || this.status === 'stopped');
24
- canStop = $derived(this.status === 'running');
25
-
26
- constructor(id: string, name?: string) {
27
- this.id = id;
28
- this.name = name || `Compute ${id}`;
29
- }
30
-
31
- /**
32
- * Get input connections (camera and joint inputs)
33
- */
34
- get inputConnections() {
35
- if (!this.sessionData) return null;
36
-
37
- return {
38
- cameras: this.sessionData.camera_room_ids,
39
- jointInput: this.sessionData.joint_input_room_id,
40
- workspaceId: this.sessionData.workspace_id
41
- };
42
- }
43
-
44
- /**
45
- * Get output connections (joint output)
46
- */
47
- get outputConnections() {
48
- if (!this.sessionData) return null;
49
-
50
- return {
51
- jointOutput: this.sessionData.joint_output_room_id,
52
- workspaceId: this.sessionData.workspace_id
53
- };
54
- }
55
-
56
- /**
57
- * Get display information for UI
58
- */
59
- get displayInfo() {
60
- return {
61
- id: this.id,
62
- name: this.name,
63
- status: this.status,
64
- sessionId: this.sessionId,
65
- policyPath: this.sessionConfig?.policyPath,
66
- cameraNames: this.sessionConfig?.cameraNames || [],
67
- hasSession: this.hasSession,
68
- isRunning: this.isRunning,
69
- canStart: this.canStart,
70
- canStop: this.canStop
71
- };
72
- }
73
-
74
- /**
75
- * Get status for billboard display
76
- */
77
- get statusInfo() {
78
- const status = this.status;
79
- let statusText = '';
80
- let statusColor = '';
81
-
82
- switch (status) {
83
- case 'disconnected':
84
- statusText = 'Disconnected';
85
- statusColor = 'rgb(107, 114, 128)'; // gray
86
- break;
87
- case 'ready':
88
- statusText = 'Ready';
89
- statusColor = 'rgb(245, 158, 11)'; // yellow
90
- break;
91
- case 'running':
92
- statusText = 'Running';
93
- statusColor = 'rgb(34, 197, 94)'; // green
94
- break;
95
- case 'stopped':
96
- statusText = 'Stopped';
97
- statusColor = 'rgb(239, 68, 68)'; // red
98
- break;
99
- case 'initializing':
100
- statusText = 'Initializing';
101
- statusColor = 'rgb(59, 130, 246)'; // blue
102
- break;
103
- }
104
-
105
- return {
106
- status,
107
- statusText,
108
- statusColor,
109
- emoji: this.getStatusEmoji()
110
- };
111
- }
112
-
113
- private getStatusEmoji(): string {
114
- switch (this.status) {
115
- case 'disconnected': return '⚪';
116
- case 'ready': return '🟡';
117
- case 'running': return '🟢';
118
- case 'stopped': return '🔴';
119
- case 'initializing': return '🟠';
120
- default: return '⚪';
121
- }
122
- }
123
-
124
- /**
125
- * Reset session data
126
- */
127
- resetSession(): void {
128
- this.sessionId = null;
129
- this.sessionConfig = null;
130
- this.sessionData = null;
131
- this.status = 'disconnected';
132
- }
133
-
134
- /**
135
- * Update position
136
- */
137
- updatePosition(newPosition: Position3D): void {
138
- this.position = { ...newPosition };
139
- }
140
-
141
- /**
142
- * Update name
143
- */
144
- updateName(newName: string): void {
145
- this.name = newName;
146
- }
147
- }
 
 
 
 
 
 
 
1
+ import type { Positionable, Position3D } from "$lib/types/positionable.js";
2
+ import type { AISessionConfig, AISessionResponse, ModelType } from "./RemoteComputeManager.svelte";
3
 
4
+ export type ComputeStatus = "disconnected" | "ready" | "running" | "stopped" | "initializing";
5
 
6
  export class RemoteCompute implements Positionable {
7
+ readonly id: string;
8
+
9
+ // Reactive state using Svelte 5 runes
10
+ position = $state<Position3D>({ x: 0, y: 0, z: 0 });
11
+ name = $state<string>("");
12
+ status = $state<ComputeStatus>("disconnected");
13
+ modelType = $state<ModelType>("act"); // Default to ACT model
14
+
15
+ // Session data
16
+ sessionId = $state<string | null>(null);
17
+ sessionConfig = $state<AISessionConfig | null>(null);
18
+ sessionData = $state<AISessionResponse | null>(null);
19
+
20
+ // Derived reactive values
21
+ hasSession = $derived(this.sessionId !== null);
22
+ isRunning = $derived(this.status === "running");
23
+ canStart = $derived(this.status === "ready" || this.status === "stopped");
24
+ canStop = $derived(this.status === "running");
25
+
26
+ constructor(id: string, name?: string) {
27
+ this.id = id;
28
+ this.name = name || `Compute ${id}`;
29
+ }
30
+
31
+ /**
32
+ * Get input connections (camera and joint inputs)
33
+ */
34
+ get inputConnections() {
35
+ if (!this.sessionData) return null;
36
+
37
+ return {
38
+ cameras: this.sessionData.camera_room_ids,
39
+ jointInput: this.sessionData.joint_input_room_id,
40
+ workspaceId: this.sessionData.workspace_id
41
+ };
42
+ }
43
+
44
+ /**
45
+ * Get output connections (joint output)
46
+ */
47
+ get outputConnections() {
48
+ if (!this.sessionData) return null;
49
+
50
+ return {
51
+ jointOutput: this.sessionData.joint_output_room_id,
52
+ workspaceId: this.sessionData.workspace_id
53
+ };
54
+ }
55
+
56
+ /**
57
+ * Get display information for UI
58
+ */
59
+ get displayInfo() {
60
+ return {
61
+ id: this.id,
62
+ name: this.name,
63
+ status: this.status,
64
+ sessionId: this.sessionId,
65
+ policyPath: this.sessionConfig?.policyPath,
66
+ cameraNames: this.sessionConfig?.cameraNames || [],
67
+ hasSession: this.hasSession,
68
+ isRunning: this.isRunning,
69
+ canStart: this.canStart,
70
+ canStop: this.canStop
71
+ };
72
+ }
73
+
74
+ /**
75
+ * Get status for billboard display
76
+ */
77
+ get statusInfo() {
78
+ const status = this.status;
79
+ let statusText = "";
80
+ let statusColor = "";
81
+
82
+ switch (status) {
83
+ case "disconnected":
84
+ statusText = "Disconnected";
85
+ statusColor = "rgb(107, 114, 128)"; // gray
86
+ break;
87
+ case "ready":
88
+ statusText = "Ready";
89
+ statusColor = "rgb(245, 158, 11)"; // yellow
90
+ break;
91
+ case "running":
92
+ statusText = "Running";
93
+ statusColor = "rgb(34, 197, 94)"; // green
94
+ break;
95
+ case "stopped":
96
+ statusText = "Stopped";
97
+ statusColor = "rgb(239, 68, 68)"; // red
98
+ break;
99
+ case "initializing":
100
+ statusText = "Initializing";
101
+ statusColor = "rgb(59, 130, 246)"; // blue
102
+ break;
103
+ }
104
+
105
+ return {
106
+ status,
107
+ statusText,
108
+ statusColor,
109
+ emoji: this.getStatusEmoji()
110
+ };
111
+ }
112
+
113
+ private getStatusEmoji(): string {
114
+ switch (this.status) {
115
+ case "disconnected":
116
+ return "⚪";
117
+ case "ready":
118
+ return "🟡";
119
+ case "running":
120
+ return "🟢";
121
+ case "stopped":
122
+ return "🔴";
123
+ case "initializing":
124
+ return "🟠";
125
+ default:
126
+ return "⚪";
127
+ }
128
+ }
129
+
130
+ /**
131
+ * Reset session data
132
+ */
133
+ resetSession(): void {
134
+ this.sessionId = null;
135
+ this.sessionConfig = null;
136
+ this.sessionData = null;
137
+ this.status = "disconnected";
138
+ }
139
+
140
+ /**
141
+ * Update position
142
+ */
143
+ updatePosition(newPosition: Position3D): void {
144
+ this.position = { ...newPosition };
145
+ }
146
+
147
+ /**
148
+ * Update name
149
+ */
150
+ updateName(newName: string): void {
151
+ this.name = newName;
152
+ }
153
+ }
src/lib/elements/compute/RemoteComputeManager.svelte.ts CHANGED
@@ -1,497 +1,506 @@
1
- import { RemoteCompute } from './RemoteCompute.svelte';
2
- import type { Position3D } from '$lib/types/positionable.js';
3
- import { generateName } from '$lib/utils/generateName.js';
4
- import { positionManager } from '$lib/utils/positionManager.js';
5
- import {
6
- rootGet,
7
- healthCheckHealthGet,
8
- listSessionsSessionsGet,
9
- createSessionSessionsPost,
10
- startInferenceSessionsSessionIdStartPost,
11
- stopInferenceSessionsSessionIdStopPost,
12
- deleteSessionSessionsSessionIdDelete
13
- } from '@robothub/inference-server-client';
14
- import { settings } from '$lib/runes/settings.svelte';
15
- import type {
16
- CreateSessionRequest,
17
- CreateSessionResponse
18
- } from '@robothub/inference-server-client';
19
-
20
- export type ModelType = 'act' | 'diffusion' | 'smolvla' | 'pi0' | 'groot' | 'custom';
21
 
22
  export interface ModelTypeConfig {
23
- id: ModelType;
24
- label: string;
25
- icon: string;
26
- description: string;
27
- defaultPolicyPath: string;
28
- defaultCameraNames: string[];
29
- requiresLanguageInstruction?: boolean;
30
- enabled: boolean;
31
  }
32
 
33
  export const MODEL_TYPES: Record<ModelType, ModelTypeConfig> = {
34
- act: {
35
- id: 'act',
36
- label: 'ACT Model',
37
- icon: 'icon-[mdi--brain]',
38
- description: 'Action Chunking with Transformers',
39
- defaultPolicyPath: 'LaetusH/act_so101_beyond',
40
- defaultCameraNames: ['front'],
41
- enabled: true
42
- },
43
- diffusion: {
44
- id: 'diffusion',
45
- label: 'Diffusion Policy',
46
- icon: 'icon-[mdi--creation]',
47
- description: 'Diffusion-based robot control',
48
- defaultPolicyPath: 'diffusion_policy/default',
49
- defaultCameraNames: ['front', 'wrist'],
50
- enabled: true
51
- },
52
- smolvla: {
53
- id: 'smolvla',
54
- label: 'SmolVLA',
55
- icon: 'icon-[mdi--eye-outline]',
56
- description: 'Small Vision-Language-Action model',
57
- defaultPolicyPath: 'smolvla/latest',
58
- defaultCameraNames: ['front'],
59
- requiresLanguageInstruction: true,
60
- enabled: true
61
- },
62
- pi0: {
63
- id: 'pi0',
64
- label: 'Pi0',
65
- icon: 'icon-[mdi--pi]',
66
- description: 'Lightweight robotics model',
67
- defaultPolicyPath: 'pi0/base',
68
- defaultCameraNames: ['front'],
69
- enabled: true
70
- },
71
- groot: {
72
- id: 'groot',
73
- label: 'NVIDIA Groot',
74
- icon: 'icon-[mdi--robot-outline]',
75
- description: 'Humanoid robotics foundation model',
76
- defaultPolicyPath: 'nvidia/groot',
77
- defaultCameraNames: ['front', 'left', 'right'],
78
- requiresLanguageInstruction: true,
79
- enabled: false // Not yet implemented
80
- },
81
- custom: {
82
- id: 'custom',
83
- label: 'Custom Model',
84
- icon: 'icon-[mdi--cog]',
85
- description: 'Custom model configuration',
86
- defaultPolicyPath: '',
87
- defaultCameraNames: ['front'],
88
- enabled: true
89
- }
90
  };
91
 
92
  export interface AISessionConfig {
93
- sessionId: string;
94
- modelType: ModelType;
95
- policyPath: string;
96
- cameraNames: string[];
97
- transportServerUrl: string;
98
- workspaceId?: string;
99
- languageInstruction?: string;
100
  }
101
 
102
  export interface AISessionResponse {
103
- workspace_id: string;
104
- camera_room_ids: Record<string, string>;
105
- joint_input_room_id: string;
106
- joint_output_room_id: string;
107
  }
108
 
109
  export interface AISessionStatus {
110
- session_id: string;
111
- status: 'initializing' | 'ready' | 'running' | 'stopped';
112
- policy_path: string;
113
- camera_names: string[];
114
- workspace_id: string;
115
- rooms: {
116
- workspace_id: string;
117
- camera_room_ids: Record<string, string>;
118
- joint_input_room_id: string;
119
- joint_output_room_id: string;
120
- };
121
- stats: {
122
- inference_count: number;
123
- commands_sent: number;
124
- joints_received: number;
125
- images_received: Record<string, number>;
126
- errors: number;
127
- actions_in_queue: number;
128
- };
129
- inference_stats?: {
130
- inference_count: number;
131
- total_inference_time: number;
132
- average_inference_time: number;
133
- average_fps: number;
134
- is_loaded: boolean;
135
- device: string;
136
- };
137
- error_message?: string;
138
  }
139
 
140
  export class RemoteComputeManager {
141
- private _computes = $state<RemoteCompute[]>([]);
142
-
143
- constructor() {
144
- // No client initialization needed anymore
145
- }
146
-
147
- // Reactive getters
148
- get computes(): RemoteCompute[] {
149
- return this._computes;
150
- }
151
-
152
- get computeCount(): number {
153
- return this._computes.length;
154
- }
155
-
156
- get runningComputes(): RemoteCompute[] {
157
- return this._computes.filter(compute => compute.status === 'running');
158
- }
159
-
160
- /**
161
- * Get available model types
162
- */
163
- get availableModelTypes(): ModelTypeConfig[] {
164
- return Object.values(MODEL_TYPES).filter(model => model.enabled);
165
- }
166
-
167
- /**
168
- * Get model type configuration
169
- */
170
- getModelTypeConfig(modelType: ModelType): ModelTypeConfig | undefined {
171
- return MODEL_TYPES[modelType];
172
- }
173
-
174
- /**
175
- * Create a new AI compute instance with full configuration
176
- */
177
- async createComputeWithSession(
178
- config: AISessionConfig,
179
- computeId?: string,
180
- computeName?: string,
181
- position?: Position3D
182
- ): Promise<{ success: boolean; error?: string; compute?: RemoteCompute }> {
183
- const finalComputeId = computeId || generateName();
184
-
185
- // Check if compute already exists
186
- if (this._computes.find(c => c.id === finalComputeId)) {
187
- return { success: false, error: `Compute with ID ${finalComputeId} already exists` };
188
- }
189
-
190
- try {
191
- // Create compute instance
192
- const compute = new RemoteCompute(finalComputeId, computeName);
193
- compute.modelType = config.modelType;
194
-
195
- // Set position (from position manager if not provided)
196
- compute.position = position || positionManager.getNextPosition();
197
-
198
- // Add to reactive array
199
- this._computes.push(compute);
200
-
201
- // Create the session immediately
202
- const sessionResult = await this.createSession(compute.id, config);
203
- if (!sessionResult.success) {
204
- // Remove compute if session creation failed
205
- await this.removeCompute(compute.id);
206
- return { success: false, error: sessionResult.error };
207
- }
208
-
209
- console.log(`Created compute ${finalComputeId} with ${config.modelType} model at position (${compute.position.x.toFixed(1)}, ${compute.position.y.toFixed(1)}, ${compute.position.z.toFixed(1)}). Total computes: ${this._computes.length}`);
210
-
211
- return { success: true, compute };
212
- } catch (error) {
213
- console.error('Failed to create compute with session:', error);
214
- return {
215
- success: false,
216
- error: error instanceof Error ? error.message : String(error)
217
- };
218
- }
219
- }
220
-
221
- /**
222
- * Create a new AI compute instance (legacy method)
223
- */
224
- createCompute(id?: string, name?: string, position?: Position3D): RemoteCompute {
225
- const computeId = id || generateName();
226
-
227
- // Check if compute already exists
228
- if (this._computes.find(c => c.id === computeId)) {
229
- throw new Error(`Compute with ID ${computeId} already exists`);
230
- }
231
-
232
- // Create compute instance
233
- const compute = new RemoteCompute(computeId, name);
234
-
235
- // Set position (from position manager if not provided)
236
- compute.position = position || positionManager.getNextPosition();
237
-
238
- // Add to reactive array
239
- this._computes.push(compute);
240
-
241
- console.log(`Created compute ${computeId} at position (${compute.position.x.toFixed(1)}, ${compute.position.y.toFixed(1)}, ${compute.position.z.toFixed(1)}). Total computes: ${this._computes.length}`);
242
-
243
- return compute;
244
- }
245
-
246
- /**
247
- * Get compute by ID
248
- */
249
- getCompute(id: string): RemoteCompute | undefined {
250
- return this._computes.find(c => c.id === id);
251
- }
252
-
253
- /**
254
- * Remove a compute instance
255
- */
256
- async removeCompute(id: string): Promise<void> {
257
- const computeIndex = this._computes.findIndex(c => c.id === id);
258
- if (computeIndex === -1) return;
259
-
260
- const compute = this._computes[computeIndex];
261
-
262
- // Clean up compute resources
263
- await this.stopSession(id);
264
- await this.deleteSession(id);
265
-
266
- // Remove from reactive array
267
- this._computes.splice(computeIndex, 1);
268
-
269
- console.log(`Removed compute ${id}. Remaining computes: ${this._computes.length}`);
270
- }
271
-
272
- /**
273
- * Create an Inference Session
274
- */
275
- async createSession(computeId: string, config: AISessionConfig): Promise<{ success: boolean; error?: string; data?: AISessionResponse }> {
276
- const compute = this.getCompute(computeId);
277
- if (!compute) {
278
- return { success: false, error: `Compute ${computeId} not found` };
279
- }
280
-
281
- try {
282
- const request: CreateSessionRequest = {
283
- session_id: config.sessionId,
284
- policy_path: config.policyPath,
285
- camera_names: config.cameraNames,
286
- transport_server_url: config.transportServerUrl,
287
- workspace_id: config.workspaceId || undefined,
288
- policy_type: config.modelType, // Use model type as policy type
289
- language_instruction: config.languageInstruction || undefined,
290
- };
291
-
292
- const response = await createSessionSessionsPost({
293
- body: request,
294
- baseUrl: settings.inferenceServerUrl
295
- });
296
-
297
- if (!response.data) {
298
- throw new Error('Failed to create session - no data returned');
299
- }
300
-
301
- const data: CreateSessionResponse = response.data;
302
-
303
- // Update compute with session info
304
- compute.sessionId = config.sessionId;
305
- compute.status = 'ready';
306
- compute.sessionConfig = config;
307
- compute.sessionData = {
308
- workspace_id: data.workspace_id,
309
- camera_room_ids: data.camera_room_ids,
310
- joint_input_room_id: data.joint_input_room_id,
311
- joint_output_room_id: data.joint_output_room_id
312
- };
313
-
314
- return { success: true, data: compute.sessionData };
315
- } catch (error) {
316
- console.error(`Failed to create session for compute ${computeId}:`, error);
317
- return {
318
- success: false,
319
- error: error instanceof Error ? error.message : String(error)
320
- };
321
- }
322
- }
323
-
324
- /**
325
- * Start inference for a session
326
- */
327
- async startSession(computeId: string): Promise<{ success: boolean; error?: string }> {
328
- const compute = this.getCompute(computeId);
329
- if (!compute || !compute.sessionId) {
330
- return { success: false, error: 'No session to start' };
331
- }
332
-
333
- try {
334
- await startInferenceSessionsSessionIdStartPost({
335
- path: { session_id: compute.sessionId },
336
- baseUrl: settings.inferenceServerUrl
337
- });
338
- compute.status = 'running';
339
- return { success: true };
340
- } catch (error) {
341
- console.error(`Failed to start session for compute ${computeId}:`, error);
342
- return {
343
- success: false,
344
- error: error instanceof Error ? error.message : String(error)
345
- };
346
- }
347
- }
348
-
349
- /**
350
- * Stop inference for a session
351
- */
352
- async stopSession(computeId: string): Promise<{ success: boolean; error?: string }> {
353
- const compute = this.getCompute(computeId);
354
- if (!compute || !compute.sessionId) {
355
- return { success: false, error: 'No session to stop' };
356
- }
357
-
358
- try {
359
- await stopInferenceSessionsSessionIdStopPost({
360
- path: { session_id: compute.sessionId },
361
- baseUrl: settings.inferenceServerUrl
362
- });
363
- compute.status = 'stopped';
364
- return { success: true };
365
- } catch (error) {
366
- console.error(`Failed to stop session for compute ${computeId}:`, error);
367
- return {
368
- success: false,
369
- error: error instanceof Error ? error.message : String(error)
370
- };
371
- }
372
- }
373
-
374
- /**
375
- * Delete a session
376
- */
377
- async deleteSession(computeId: string): Promise<{ success: boolean; error?: string }> {
378
- const compute = this.getCompute(computeId);
379
- if (!compute || !compute.sessionId) {
380
- return { success: true }; // Already deleted
381
- }
382
-
383
- try {
384
- await deleteSessionSessionsSessionIdDelete({
385
- path: { session_id: compute.sessionId },
386
- baseUrl: settings.inferenceServerUrl
387
- });
388
-
389
- // Reset compute session info
390
- compute.sessionId = null;
391
- compute.status = 'disconnected';
392
- compute.sessionConfig = null;
393
- compute.sessionData = null;
394
-
395
- return { success: true };
396
- } catch (error) {
397
- console.error(`Failed to delete session for compute ${computeId}:`, error);
398
- return {
399
- success: false,
400
- error: error instanceof Error ? error.message : String(error)
401
- };
402
- }
403
- }
404
-
405
- /**
406
- * Get session status
407
- */
408
- async getSessionStatus(computeId: string): Promise<{ success: boolean; data?: AISessionStatus; error?: string }> {
409
- const compute = this.getCompute(computeId);
410
- if (!compute || !compute.sessionId) {
411
- return { success: false, error: 'No session found' };
412
- }
413
-
414
- try {
415
- // Get all sessions and find the one we want
416
- const response = await listSessionsSessionsGet({
417
- baseUrl: settings.inferenceServerUrl
418
- });
419
-
420
- if (!response.data) {
421
- throw new Error('Failed to get sessions list');
422
- }
423
-
424
- const session = response.data.find(s => s.session_id === compute.sessionId);
425
- if (!session) {
426
- throw new Error(`Session ${compute.sessionId} not found`);
427
- }
428
-
429
- // Update compute status
430
- compute.status = session.status as 'initializing' | 'ready' | 'running' | 'stopped';
431
-
432
- // Convert to AISessionStatus format
433
- const sessionStatus: AISessionStatus = {
434
- session_id: session.session_id,
435
- status: session.status as 'initializing' | 'ready' | 'running' | 'stopped',
436
- policy_path: session.policy_path,
437
- camera_names: session.camera_names,
438
- workspace_id: session.workspace_id,
439
- rooms: session.rooms as any,
440
- stats: session.stats as any,
441
- inference_stats: session.inference_stats as any,
442
- error_message: session.error_message || undefined
443
- };
444
-
445
- return { success: true, data: sessionStatus };
446
- } catch (error) {
447
- console.error(`Failed to get session status for compute ${computeId}:`, error);
448
- return {
449
- success: false,
450
- error: error instanceof Error ? error.message : String(error)
451
- };
452
- }
453
- }
454
-
455
- /**
456
- * Check AI server health
457
- */
458
- async checkServerHealth(): Promise<{ success: boolean; data?: any; error?: string }> {
459
- try {
460
- const healthResponse = await rootGet({
461
- baseUrl: settings.inferenceServerUrl
462
- });
463
-
464
- if (!healthResponse.data) {
465
- return { success: false, error: 'Server is not healthy' };
466
- }
467
-
468
- // Get detailed health info
469
- const detailedHealthResponse = await healthCheckHealthGet({
470
- baseUrl: settings.inferenceServerUrl
471
- });
472
-
473
- return { success: true, data: detailedHealthResponse.data };
474
- } catch (error) {
475
- console.error('Failed to check AI server health:', error);
476
- return {
477
- success: false,
478
- error: error instanceof Error ? error.message : String(error)
479
- };
480
- }
481
- }
482
-
483
- /**
484
- * Clean up all computes
485
- */
486
- async destroy(): Promise<void> {
487
- const cleanupPromises = this._computes.map(async (compute) => {
488
- await this.stopSession(compute.id);
489
- await this.deleteSession(compute.id);
490
- });
491
- await Promise.allSettled(cleanupPromises);
492
- this._computes.length = 0;
493
- }
 
 
 
 
 
 
 
 
 
494
  }
495
 
496
  // Global compute manager instance
497
- export const remoteComputeManager = new RemoteComputeManager();
 
1
+ import { RemoteCompute } from "./RemoteCompute.svelte";
2
+ import type { Position3D } from "$lib/types/positionable.js";
3
+ import { generateName } from "$lib/utils/generateName.js";
4
+ import { positionManager } from "$lib/utils/positionManager.js";
5
+ import {
6
+ rootGet,
7
+ healthCheckHealthGet,
8
+ listSessionsSessionsGet,
9
+ createSessionSessionsPost,
10
+ startInferenceSessionsSessionIdStartPost,
11
+ stopInferenceSessionsSessionIdStopPost,
12
+ deleteSessionSessionsSessionIdDelete
13
+ } from "@robothub/inference-server-client";
14
+ import { settings } from "$lib/runes/settings.svelte";
15
+ import type {
16
+ CreateSessionRequest,
17
+ CreateSessionResponse
18
+ } from "@robothub/inference-server-client";
19
+
20
+ export type ModelType = "act" | "diffusion" | "smolvla" | "pi0" | "groot" | "custom";
21
 
22
  export interface ModelTypeConfig {
23
+ id: ModelType;
24
+ label: string;
25
+ icon: string;
26
+ description: string;
27
+ defaultPolicyPath: string;
28
+ defaultCameraNames: string[];
29
+ requiresLanguageInstruction?: boolean;
30
+ enabled: boolean;
31
  }
32
 
33
  export const MODEL_TYPES: Record<ModelType, ModelTypeConfig> = {
34
+ act: {
35
+ id: "act",
36
+ label: "ACT Model",
37
+ icon: "icon-[mdi--brain]",
38
+ description: "Action Chunking with Transformers",
39
+ defaultPolicyPath: "LaetusH/act_so101_beyond",
40
+ defaultCameraNames: ["front"],
41
+ enabled: true
42
+ },
43
+ diffusion: {
44
+ id: "diffusion",
45
+ label: "Diffusion Policy",
46
+ icon: "icon-[mdi--creation]",
47
+ description: "Diffusion-based robot control",
48
+ defaultPolicyPath: "diffusion_policy/default",
49
+ defaultCameraNames: ["front", "wrist"],
50
+ enabled: true
51
+ },
52
+ smolvla: {
53
+ id: "smolvla",
54
+ label: "SmolVLA",
55
+ icon: "icon-[mdi--eye-outline]",
56
+ description: "Small Vision-Language-Action model",
57
+ defaultPolicyPath: "smolvla/latest",
58
+ defaultCameraNames: ["front"],
59
+ requiresLanguageInstruction: true,
60
+ enabled: true
61
+ },
62
+ pi0: {
63
+ id: "pi0",
64
+ label: "Pi0",
65
+ icon: "icon-[mdi--pi]",
66
+ description: "Lightweight robotics model",
67
+ defaultPolicyPath: "pi0/base",
68
+ defaultCameraNames: ["front"],
69
+ enabled: true
70
+ },
71
+ groot: {
72
+ id: "groot",
73
+ label: "NVIDIA Groot",
74
+ icon: "icon-[mdi--robot-outline]",
75
+ description: "Humanoid robotics foundation model",
76
+ defaultPolicyPath: "nvidia/groot",
77
+ defaultCameraNames: ["front", "left", "right"],
78
+ requiresLanguageInstruction: true,
79
+ enabled: false // Not yet implemented
80
+ },
81
+ custom: {
82
+ id: "custom",
83
+ label: "Custom Model",
84
+ icon: "icon-[mdi--cog]",
85
+ description: "Custom model configuration",
86
+ defaultPolicyPath: "",
87
+ defaultCameraNames: ["front"],
88
+ enabled: true
89
+ }
90
  };
91
 
92
  export interface AISessionConfig {
93
+ sessionId: string;
94
+ modelType: ModelType;
95
+ policyPath: string;
96
+ cameraNames: string[];
97
+ transportServerUrl: string;
98
+ workspaceId?: string;
99
+ languageInstruction?: string;
100
  }
101
 
102
  export interface AISessionResponse {
103
+ workspace_id: string;
104
+ camera_room_ids: Record<string, string>;
105
+ joint_input_room_id: string;
106
+ joint_output_room_id: string;
107
  }
108
 
109
  export interface AISessionStatus {
110
+ session_id: string;
111
+ status: "initializing" | "ready" | "running" | "stopped";
112
+ policy_path: string;
113
+ camera_names: string[];
114
+ workspace_id: string;
115
+ rooms: {
116
+ workspace_id: string;
117
+ camera_room_ids: Record<string, string>;
118
+ joint_input_room_id: string;
119
+ joint_output_room_id: string;
120
+ };
121
+ stats: {
122
+ inference_count: number;
123
+ commands_sent: number;
124
+ joints_received: number;
125
+ images_received: Record<string, number>;
126
+ errors: number;
127
+ actions_in_queue: number;
128
+ };
129
+ inference_stats?: {
130
+ inference_count: number;
131
+ total_inference_time: number;
132
+ average_inference_time: number;
133
+ average_fps: number;
134
+ is_loaded: boolean;
135
+ device: string;
136
+ };
137
+ error_message?: string;
138
  }
139
 
140
  export class RemoteComputeManager {
141
+ private _computes = $state<RemoteCompute[]>([]);
142
+
143
+ constructor() {
144
+ // No client initialization needed anymore
145
+ }
146
+
147
+ // Reactive getters
148
+ get computes(): RemoteCompute[] {
149
+ return this._computes;
150
+ }
151
+
152
+ get computeCount(): number {
153
+ return this._computes.length;
154
+ }
155
+
156
+ get runningComputes(): RemoteCompute[] {
157
+ return this._computes.filter((compute) => compute.status === "running");
158
+ }
159
+
160
+ /**
161
+ * Get available model types
162
+ */
163
+ get availableModelTypes(): ModelTypeConfig[] {
164
+ return Object.values(MODEL_TYPES).filter((model) => model.enabled);
165
+ }
166
+
167
+ /**
168
+ * Get model type configuration
169
+ */
170
+ getModelTypeConfig(modelType: ModelType): ModelTypeConfig | undefined {
171
+ return MODEL_TYPES[modelType];
172
+ }
173
+
174
+ /**
175
+ * Create a new AI compute instance with full configuration
176
+ */
177
+ async createComputeWithSession(
178
+ config: AISessionConfig,
179
+ computeId?: string,
180
+ computeName?: string,
181
+ position?: Position3D
182
+ ): Promise<{ success: boolean; error?: string; compute?: RemoteCompute }> {
183
+ const finalComputeId = computeId || generateName();
184
+
185
+ // Check if compute already exists
186
+ if (this._computes.find((c) => c.id === finalComputeId)) {
187
+ return { success: false, error: `Compute with ID ${finalComputeId} already exists` };
188
+ }
189
+
190
+ try {
191
+ // Create compute instance
192
+ const compute = new RemoteCompute(finalComputeId, computeName);
193
+ compute.modelType = config.modelType;
194
+
195
+ // Set position (from position manager if not provided)
196
+ compute.position = position || positionManager.getNextPosition();
197
+
198
+ // Add to reactive array
199
+ this._computes.push(compute);
200
+
201
+ // Create the session immediately
202
+ const sessionResult = await this.createSession(compute.id, config);
203
+ if (!sessionResult.success) {
204
+ // Remove compute if session creation failed
205
+ await this.removeCompute(compute.id);
206
+ return { success: false, error: sessionResult.error };
207
+ }
208
+
209
+ console.log(
210
+ `Created compute ${finalComputeId} with ${config.modelType} model at position (${compute.position.x.toFixed(1)}, ${compute.position.y.toFixed(1)}, ${compute.position.z.toFixed(1)}). Total computes: ${this._computes.length}`
211
+ );
212
+
213
+ return { success: true, compute };
214
+ } catch (error) {
215
+ console.error("Failed to create compute with session:", error);
216
+ return {
217
+ success: false,
218
+ error: error instanceof Error ? error.message : String(error)
219
+ };
220
+ }
221
+ }
222
+
223
+ /**
224
+ * Create a new AI compute instance (legacy method)
225
+ */
226
+ createCompute(id?: string, name?: string, position?: Position3D): RemoteCompute {
227
+ const computeId = id || generateName();
228
+
229
+ // Check if compute already exists
230
+ if (this._computes.find((c) => c.id === computeId)) {
231
+ throw new Error(`Compute with ID ${computeId} already exists`);
232
+ }
233
+
234
+ // Create compute instance
235
+ const compute = new RemoteCompute(computeId, name);
236
+
237
+ // Set position (from position manager if not provided)
238
+ compute.position = position || positionManager.getNextPosition();
239
+
240
+ // Add to reactive array
241
+ this._computes.push(compute);
242
+
243
+ console.log(
244
+ `Created compute ${computeId} at position (${compute.position.x.toFixed(1)}, ${compute.position.y.toFixed(1)}, ${compute.position.z.toFixed(1)}). Total computes: ${this._computes.length}`
245
+ );
246
+
247
+ return compute;
248
+ }
249
+
250
+ /**
251
+ * Get compute by ID
252
+ */
253
+ getCompute(id: string): RemoteCompute | undefined {
254
+ return this._computes.find((c) => c.id === id);
255
+ }
256
+
257
+ /**
258
+ * Remove a compute instance
259
+ */
260
+ async removeCompute(id: string): Promise<void> {
261
+ const computeIndex = this._computes.findIndex((c) => c.id === id);
262
+ if (computeIndex === -1) return;
263
+
264
+ const compute = this._computes[computeIndex];
265
+
266
+ // Clean up compute resources
267
+ await this.stopSession(id);
268
+ await this.deleteSession(id);
269
+
270
+ // Remove from reactive array
271
+ this._computes.splice(computeIndex, 1);
272
+
273
+ console.log(`Removed compute ${id}. Remaining computes: ${this._computes.length}`);
274
+ }
275
+
276
+ /**
277
+ * Create an Inference Session
278
+ */
279
+ async createSession(
280
+ computeId: string,
281
+ config: AISessionConfig
282
+ ): Promise<{ success: boolean; error?: string; data?: AISessionResponse }> {
283
+ const compute = this.getCompute(computeId);
284
+ if (!compute) {
285
+ return { success: false, error: `Compute ${computeId} not found` };
286
+ }
287
+
288
+ try {
289
+ const request: CreateSessionRequest = {
290
+ session_id: config.sessionId,
291
+ policy_path: config.policyPath,
292
+ camera_names: config.cameraNames,
293
+ transport_server_url: config.transportServerUrl,
294
+ workspace_id: config.workspaceId || undefined,
295
+ policy_type: config.modelType, // Use model type as policy type
296
+ language_instruction: config.languageInstruction || undefined
297
+ };
298
+
299
+ const response = await createSessionSessionsPost({
300
+ body: request,
301
+ baseUrl: settings.inferenceServerUrl
302
+ });
303
+
304
+ if (!response.data) {
305
+ throw new Error("Failed to create session - no data returned");
306
+ }
307
+
308
+ const data: CreateSessionResponse = response.data;
309
+
310
+ // Update compute with session info
311
+ compute.sessionId = config.sessionId;
312
+ compute.status = "ready";
313
+ compute.sessionConfig = config;
314
+ compute.sessionData = {
315
+ workspace_id: data.workspace_id,
316
+ camera_room_ids: data.camera_room_ids,
317
+ joint_input_room_id: data.joint_input_room_id,
318
+ joint_output_room_id: data.joint_output_room_id
319
+ };
320
+
321
+ return { success: true, data: compute.sessionData };
322
+ } catch (error) {
323
+ console.error(`Failed to create session for compute ${computeId}:`, error);
324
+ return {
325
+ success: false,
326
+ error: error instanceof Error ? error.message : String(error)
327
+ };
328
+ }
329
+ }
330
+
331
+ /**
332
+ * Start inference for a session
333
+ */
334
+ async startSession(computeId: string): Promise<{ success: boolean; error?: string }> {
335
+ const compute = this.getCompute(computeId);
336
+ if (!compute || !compute.sessionId) {
337
+ return { success: false, error: "No session to start" };
338
+ }
339
+
340
+ try {
341
+ await startInferenceSessionsSessionIdStartPost({
342
+ path: { session_id: compute.sessionId },
343
+ baseUrl: settings.inferenceServerUrl
344
+ });
345
+ compute.status = "running";
346
+ return { success: true };
347
+ } catch (error) {
348
+ console.error(`Failed to start session for compute ${computeId}:`, error);
349
+ return {
350
+ success: false,
351
+ error: error instanceof Error ? error.message : String(error)
352
+ };
353
+ }
354
+ }
355
+
356
+ /**
357
+ * Stop inference for a session
358
+ */
359
+ async stopSession(computeId: string): Promise<{ success: boolean; error?: string }> {
360
+ const compute = this.getCompute(computeId);
361
+ if (!compute || !compute.sessionId) {
362
+ return { success: false, error: "No session to stop" };
363
+ }
364
+
365
+ try {
366
+ await stopInferenceSessionsSessionIdStopPost({
367
+ path: { session_id: compute.sessionId },
368
+ baseUrl: settings.inferenceServerUrl
369
+ });
370
+ compute.status = "stopped";
371
+ return { success: true };
372
+ } catch (error) {
373
+ console.error(`Failed to stop session for compute ${computeId}:`, error);
374
+ return {
375
+ success: false,
376
+ error: error instanceof Error ? error.message : String(error)
377
+ };
378
+ }
379
+ }
380
+
381
+ /**
382
+ * Delete a session
383
+ */
384
+ async deleteSession(computeId: string): Promise<{ success: boolean; error?: string }> {
385
+ const compute = this.getCompute(computeId);
386
+ if (!compute || !compute.sessionId) {
387
+ return { success: true }; // Already deleted
388
+ }
389
+
390
+ try {
391
+ await deleteSessionSessionsSessionIdDelete({
392
+ path: { session_id: compute.sessionId },
393
+ baseUrl: settings.inferenceServerUrl
394
+ });
395
+
396
+ // Reset compute session info
397
+ compute.sessionId = null;
398
+ compute.status = "disconnected";
399
+ compute.sessionConfig = null;
400
+ compute.sessionData = null;
401
+
402
+ return { success: true };
403
+ } catch (error) {
404
+ console.error(`Failed to delete session for compute ${computeId}:`, error);
405
+ return {
406
+ success: false,
407
+ error: error instanceof Error ? error.message : String(error)
408
+ };
409
+ }
410
+ }
411
+
412
+ /**
413
+ * Get session status
414
+ */
415
+ async getSessionStatus(
416
+ computeId: string
417
+ ): Promise<{ success: boolean; data?: AISessionStatus; error?: string }> {
418
+ const compute = this.getCompute(computeId);
419
+ if (!compute || !compute.sessionId) {
420
+ return { success: false, error: "No session found" };
421
+ }
422
+
423
+ try {
424
+ // Get all sessions and find the one we want
425
+ const response = await listSessionsSessionsGet({
426
+ baseUrl: settings.inferenceServerUrl
427
+ });
428
+
429
+ if (!response.data) {
430
+ throw new Error("Failed to get sessions list");
431
+ }
432
+
433
+ const session = response.data.find((s) => s.session_id === compute.sessionId);
434
+ if (!session) {
435
+ throw new Error(`Session ${compute.sessionId} not found`);
436
+ }
437
+
438
+ // Update compute status
439
+ compute.status = session.status as "initializing" | "ready" | "running" | "stopped";
440
+
441
+ // Convert to AISessionStatus format
442
+ const sessionStatus: AISessionStatus = {
443
+ session_id: session.session_id,
444
+ status: session.status as "initializing" | "ready" | "running" | "stopped",
445
+ policy_path: session.policy_path,
446
+ camera_names: session.camera_names,
447
+ workspace_id: session.workspace_id,
448
+ rooms: session.rooms as any,
449
+ stats: session.stats as any,
450
+ inference_stats: session.inference_stats as any,
451
+ error_message: session.error_message || undefined
452
+ };
453
+
454
+ return { success: true, data: sessionStatus };
455
+ } catch (error) {
456
+ console.error(`Failed to get session status for compute ${computeId}:`, error);
457
+ return {
458
+ success: false,
459
+ error: error instanceof Error ? error.message : String(error)
460
+ };
461
+ }
462
+ }
463
+
464
+ /**
465
+ * Check AI server health
466
+ */
467
+ async checkServerHealth(): Promise<{ success: boolean; data?: any; error?: string }> {
468
+ try {
469
+ const healthResponse = await rootGet({
470
+ baseUrl: settings.inferenceServerUrl
471
+ });
472
+
473
+ if (!healthResponse.data) {
474
+ return { success: false, error: "Server is not healthy" };
475
+ }
476
+
477
+ // Get detailed health info
478
+ const detailedHealthResponse = await healthCheckHealthGet({
479
+ baseUrl: settings.inferenceServerUrl
480
+ });
481
+
482
+ return { success: true, data: detailedHealthResponse.data };
483
+ } catch (error) {
484
+ console.error("Failed to check AI server health:", error);
485
+ return {
486
+ success: false,
487
+ error: error instanceof Error ? error.message : String(error)
488
+ };
489
+ }
490
+ }
491
+
492
+ /**
493
+ * Clean up all computes
494
+ */
495
+ async destroy(): Promise<void> {
496
+ const cleanupPromises = this._computes.map(async (compute) => {
497
+ await this.stopSession(compute.id);
498
+ await this.deleteSession(compute.id);
499
+ });
500
+ await Promise.allSettled(cleanupPromises);
501
+ this._computes.length = 0;
502
+ }
503
  }
504
 
505
  // Global compute manager instance
506
+ export const remoteComputeManager = new RemoteComputeManager();
src/lib/elements/compute/index.ts CHANGED
@@ -1,5 +1,11 @@
1
- export { RemoteComputeManager, remoteComputeManager } from './RemoteComputeManager.svelte.js';
2
- export { RemoteCompute } from './RemoteCompute.svelte.js';
3
- export type { AISessionConfig, AISessionResponse, AISessionStatus, ModelType, ModelTypeConfig } from './RemoteComputeManager.svelte.js';
4
- export { MODEL_TYPES } from './RemoteComputeManager.svelte.js';
5
- export type { ComputeStatus } from './RemoteCompute.svelte.js';
 
 
 
 
 
 
 
1
+ export { RemoteComputeManager, remoteComputeManager } from "./RemoteComputeManager.svelte.js";
2
+ export { RemoteCompute } from "./RemoteCompute.svelte.js";
3
+ export type {
4
+ AISessionConfig,
5
+ AISessionResponse,
6
+ AISessionStatus,
7
+ ModelType,
8
+ ModelTypeConfig
9
+ } from "./RemoteComputeManager.svelte.js";
10
+ export { MODEL_TYPES } from "./RemoteComputeManager.svelte.js";
11
+ export type { ComputeStatus } from "./RemoteCompute.svelte.js";
src/lib/elements/robot/Robot.svelte.ts CHANGED
@@ -1,496 +1,537 @@
1
- import type {
2
- JointState,
3
- RobotCommand,
4
- ConnectionStatus,
5
- USBDriverConfig,
6
- RemoteDriverConfig,
7
- Consumer,
8
- Producer
9
- } from './models.js';
10
- import type { Positionable, Position3D } from '$lib/types/positionable.js';
11
- import { USBConsumer } from './drivers/USBConsumer.js';
12
- import { USBProducer } from './drivers/USBProducer.js';
13
- import { RemoteConsumer } from './drivers/RemoteConsumer.js';
14
- import { RemoteProducer } from './drivers/RemoteProducer.js';
15
- import { USBServoDriver } from './drivers/USBServoDriver.js';
16
-
17
- import { ROBOT_CONFIG } from './config.js';
18
- import type IUrdfRobot from '@/components/3d/elements/robot/URDF/interfaces/IUrdfRobot.js';
19
 
20
  export class Robot implements Positionable {
21
- // Core robot data
22
- readonly id: string;
23
- private unsubscribeFns: (() => void)[] = [];
24
-
25
- // Command synchronization to prevent state conflicts
26
- private commandMutex = $state(false);
27
- private pendingCommands: RobotCommand[] = [];
28
-
29
- // Command deduplication to prevent rapid duplicate commands
30
- private lastCommandTime = 0;
31
- private lastCommandValues: Record<string, number> = {};
32
-
33
- // Memory management
34
- private lastCleanup = 0;
35
-
36
- // Single consumer and multiple producers using Svelte 5 runes - PUBLIC for reactive access
37
- consumer = $state<Consumer | null>(null);
38
- producers = $state<Producer[]>([]);
39
-
40
- // Reactive state using Svelte 5 runes - PUBLIC for reactive access
41
- joints = $state<Record<string, JointState>>({});
42
- position = $state<Position3D>({ x: 0, y: 0, z: 0 });
43
- isManualControlEnabled = $state(true);
44
- connectionStatus = $state<ConnectionStatus>({ isConnected: false });
45
-
46
- // URDF robot state for 3D visualization - PUBLIC for reactive access
47
- urdfRobotState = $state<IUrdfRobot | null>(null);
48
-
49
- // Derived reactive values for components
50
- jointArray = $derived(Object.values(this.joints));
51
- hasProducers = $derived(this.producers.length > 0);
52
- hasConsumer = $derived(this.consumer !== null && this.consumer.status.isConnected);
53
- outputDriverCount = $derived(this.producers.filter(d => d.status.isConnected).length);
54
-
55
- constructor(id: string, initialJoints: JointState[], urdfRobotState?: IUrdfRobot) {
56
- this.id = id;
57
-
58
- // Store URDF robot state if provided
59
- this.urdfRobotState = urdfRobotState || null;
60
-
61
- // Initialize joints with normalized values
62
- initialJoints.forEach(joint => {
63
- const isGripper = joint.name.toLowerCase() === 'jaw' || joint.name.toLowerCase() === 'gripper';
64
- this.joints[joint.name] = {
65
- ...joint,
66
- value: isGripper ? 0 : 0 // Start at neutral position
67
- };
68
- });
69
- }
70
-
71
- // Method to set URDF robot state after creation (for async loading)
72
- setUrdfRobotState(urdfRobotState: any): void {
73
- this.urdfRobotState = urdfRobotState;
74
- }
75
-
76
- /**
77
- * Update position (implements Positionable interface)
78
- */
79
- updatePosition(newPosition: Position3D): void {
80
- this.position = { ...newPosition };
81
- }
82
-
83
- // Get all USB drivers (both consumer and producers) for calibration
84
- getUSBDrivers(): USBServoDriver[] {
85
- const usbDrivers: USBServoDriver[] = [];
86
-
87
- // Check consumer
88
- if (this.consumer && USBServoDriver.isUSBDriver(this.consumer)) {
89
- usbDrivers.push(this.consumer);
90
- }
91
-
92
- // Check producers
93
- this.producers.forEach(producer => {
94
- if (USBServoDriver.isUSBDriver(producer)) {
95
- usbDrivers.push(producer);
96
- }
97
- });
98
-
99
- return usbDrivers;
100
- }
101
-
102
- // Get uncalibrated USB drivers that need calibration
103
- getUncalibratedUSBDrivers(): USBServoDriver[] {
104
- return this.getUSBDrivers().filter(driver => driver.needsCalibration);
105
- }
106
-
107
- // Check if robot has any USB drivers
108
- hasUSBDrivers(): boolean {
109
- return this.getUSBDrivers().length > 0;
110
- }
111
-
112
- // Check if all USB drivers are calibrated
113
- areAllUSBDriversCalibrated(): boolean {
114
- const usbDrivers = this.getUSBDrivers();
115
- return usbDrivers.length > 0 && usbDrivers.every(driver => driver.isCalibrated);
116
- }
117
-
118
- // Joint value updates (normalized) - for manual control
119
- updateJoint(name: string, normalizedValue: number): void {
120
- if (!this.isManualControlEnabled) {
121
- console.warn('Manual control is disabled');
122
- return;
123
- }
124
-
125
- this.updateJointValue(name, normalizedValue, true);
126
- }
127
-
128
- // Internal joint value update (used by both manual control and USB calibration sync)
129
- updateJointValue(name: string, normalizedValue: number, sendToProducers: boolean = false): void {
130
- const joint = this.joints[name];
131
- if (!joint) {
132
- console.warn(`Joint ${name} not found`);
133
- return;
134
- }
135
-
136
- // Clamp to appropriate normalized range based on joint type
137
- if (name.toLowerCase() === 'jaw' || name.toLowerCase() === 'gripper') {
138
- normalizedValue = Math.max(0, Math.min(100, normalizedValue));
139
- } else {
140
- normalizedValue = Math.max(-100, Math.min(100, normalizedValue));
141
- }
142
-
143
- console.debug(`[Robot ${this.id}] Update joint ${name} to ${normalizedValue} (normalized, sendToProducers: ${sendToProducers})`);
144
-
145
- // Create a new joint object to ensure reactivity
146
- this.joints[name] = { ...joint, value: normalizedValue };
147
-
148
- // Send normalized command to producers if requested
149
- if (sendToProducers) {
150
- this.sendToProducers({ joints: [{ name, value: normalizedValue }] });
151
- }
152
- }
153
-
154
- executeCommand(command: RobotCommand): void {
155
- // Command deduplication - skip if same values sent within dedup window
156
- const now = Date.now();
157
- if (now - this.lastCommandTime < ROBOT_CONFIG.commands.dedupWindow) {
158
- const hasChanges = command.joints.some(joint =>
159
- Math.abs((this.lastCommandValues[joint.name] || 0) - joint.value) > 0.5
160
- );
161
- if (!hasChanges) {
162
- console.debug(`[Robot ${this.id}] 🔄 Skipping duplicate command within ${ROBOT_CONFIG.commands.dedupWindow}ms window`);
163
- return;
164
- }
165
- }
166
-
167
- // Update deduplication tracking
168
- this.lastCommandTime = now;
169
- command.joints.forEach(joint => {
170
- this.lastCommandValues[joint.name] = joint.value;
171
- });
172
-
173
- // Queue command if mutex is locked to prevent race conditions
174
- if (this.commandMutex) {
175
- if (this.pendingCommands.length >= ROBOT_CONFIG.commands.maxQueueSize) {
176
- console.warn(`[Robot ${this.id}] Command queue full, dropping oldest command`);
177
- this.pendingCommands.shift();
178
- }
179
- this.pendingCommands.push(command);
180
- return;
181
- }
182
-
183
- this.commandMutex = true;
184
-
185
- try {
186
- console.debug(`[Robot ${this.id}] Executing command with ${command.joints.length} joints:`,
187
- command.joints.map(j => `${j.name}=${j.value}`).join(', '));
188
-
189
- // Update virtual robot joints with normalized values
190
- command.joints.forEach(jointCmd => {
191
- const joint = this.joints[jointCmd.name];
192
- if (joint) {
193
- // Clamp to appropriate normalized range based on joint type
194
- let normalizedValue: number;
195
- if (jointCmd.name.toLowerCase() === 'jaw' || jointCmd.name.toLowerCase() === 'gripper') {
196
- normalizedValue = Math.max(0, Math.min(100, jointCmd.value));
197
- } else {
198
- normalizedValue = Math.max(-100, Math.min(100, jointCmd.value));
199
- }
200
-
201
- console.debug(`[Robot ${this.id}] Joint ${jointCmd.name}: ${jointCmd.value} -> ${normalizedValue} (normalized)`);
202
-
203
- // Create a new joint object to ensure reactivity
204
- this.joints[jointCmd.name] = { ...joint, value: normalizedValue };
205
- } else {
206
- console.warn(`[Robot ${this.id}] Joint ${jointCmd.name} not found`);
207
- }
208
- });
209
-
210
- // Send normalized command to producers
211
- this.sendToProducers(command);
212
- } finally {
213
- this.commandMutex = false;
214
-
215
- // Periodic cleanup to prevent memory leaks
216
- const now = Date.now();
217
- if (now - this.lastCleanup > ROBOT_CONFIG.performance.memoryCleanupInterval) {
218
- // Clear old command values that haven't been updated recently
219
- Object.keys(this.lastCommandValues).forEach(jointName => {
220
- if (now - this.lastCommandTime > ROBOT_CONFIG.performance.memoryCleanupInterval) {
221
- delete this.lastCommandValues[jointName];
222
- }
223
- });
224
- this.lastCleanup = now;
225
- }
226
-
227
- // Process any pending commands
228
- if (this.pendingCommands.length > 0) {
229
- const nextCommand = this.pendingCommands.shift();
230
- if (nextCommand) {
231
- // Use setTimeout to prevent stack overflow with rapid commands
232
- setTimeout(() => this.executeCommand(nextCommand), 0);
233
- }
234
- }
235
- }
236
- }
237
-
238
- // Consumer management (input driver) - SINGLE consumer only
239
- async setConsumer(config: USBDriverConfig | RemoteDriverConfig): Promise<string> {
240
- return this._setConsumer(config, false);
241
- }
242
-
243
- // Join existing room as consumer (for Inference Session integration)
244
- async joinAsConsumer(config: RemoteDriverConfig): Promise<string> {
245
- if (config.type !== 'remote') {
246
- throw new Error('joinAsConsumer only supports remote drivers');
247
- }
248
- return this._setConsumer(config, true);
249
- }
250
-
251
- private async _setConsumer(config: USBDriverConfig | RemoteDriverConfig, joinExistingRoom: boolean): Promise<string> {
252
- // Remove existing consumer if any
253
- if (this.consumer) {
254
- await this.removeConsumer();
255
- }
256
-
257
- const consumer = this.createConsumer(config);
258
-
259
- // Set up calibration completion callback for USB drivers
260
- if (USBServoDriver.isUSBDriver(consumer)) {
261
- const calibrationUnsubscribe = consumer.onCalibrationCompleteWithPositions(async (finalPositions: Record<string, number>) => {
262
- console.log(`[Robot ${this.id}] Calibration complete, syncing robot to final positions`);
263
- consumer.syncRobotPositions(finalPositions, (jointName: string, normalizedValue: number) => {
264
- this.updateJointValue(jointName, normalizedValue, false); // Don't send to producers to avoid feedback loop
265
- });
266
-
267
- // Start listening now that calibration is complete
268
- if ('startListening' in consumer && consumer.startListening) {
269
- try {
270
- await consumer.startListening();
271
- console.log(`[Robot ${this.id}] Started listening after calibration completion`);
272
- } catch (error) {
273
- console.error(`[Robot ${this.id}] Failed to start listening after calibration:`, error);
274
- }
275
- }
276
- });
277
- this.unsubscribeFns.push(calibrationUnsubscribe);
278
- }
279
-
280
- // Only pass joinExistingRoom to remote drivers
281
- if (config.type === 'remote') {
282
- await (consumer as RemoteConsumer).connect(joinExistingRoom);
283
- } else {
284
- await consumer.connect();
285
- }
286
-
287
- // Set up command listening
288
- const commandUnsubscribe = consumer.onCommand((command: RobotCommand) => {
289
- this.executeCommand(command);
290
- });
291
- this.unsubscribeFns.push(commandUnsubscribe);
292
-
293
- // Monitor status changes
294
- const statusUnsubscribe = consumer.onStatusChange(() => {
295
- this.updateStates();
296
- });
297
- this.unsubscribeFns.push(statusUnsubscribe);
298
-
299
- // Start listening for consumers with this capability (only if calibrated for USB)
300
- if ('startListening' in consumer && consumer.startListening) {
301
- // For USB consumers, only start listening if calibrated
302
- if (USBServoDriver.isUSBDriver(consumer)) {
303
- if (consumer.isCalibrated) {
304
- await consumer.startListening();
305
- }
306
- // If not calibrated, startListening will be called after calibration completion
307
- } else {
308
- // For non-USB consumers, start listening immediately
309
- await consumer.startListening();
310
- }
311
- }
312
-
313
- this.consumer = consumer;
314
- this.updateStates();
315
-
316
- return consumer.id;
317
- }
318
-
319
- // Producer management (output drivers) - MULTIPLE allowed
320
- async addProducer(config: USBDriverConfig | RemoteDriverConfig): Promise<string> {
321
- return this._addProducer(config, false);
322
- }
323
-
324
- // Join existing room as producer (for Inference Session integration)
325
- async joinAsProducer(config: RemoteDriverConfig): Promise<string> {
326
- if (config.type !== 'remote') {
327
- throw new Error('joinAsProducer only supports remote drivers');
328
- }
329
- return this._addProducer(config, true);
330
- }
331
-
332
- private async _addProducer(config: USBDriverConfig | RemoteDriverConfig, joinExistingRoom: boolean): Promise<string> {
333
- const producer = this.createProducer(config);
334
-
335
- // Set up calibration completion callback for USB drivers
336
- if (USBServoDriver.isUSBDriver(producer)) {
337
- const calibrationUnsubscribe = producer.onCalibrationCompleteWithPositions(async (finalPositions: Record<string, number>) => {
338
- console.log(`[Robot ${this.id}] Calibration complete, syncing robot to final positions`);
339
- producer.syncRobotPositions(finalPositions, (jointName: string, normalizedValue: number) => {
340
- this.updateJointValue(jointName, normalizedValue, false); // Don't send to producers to avoid feedback loop
341
- });
342
-
343
- console.log(`[Robot ${this.id}] USB Producer calibration completed and ready for commands`);
344
- });
345
- this.unsubscribeFns.push(calibrationUnsubscribe);
346
- }
347
-
348
- // Only pass joinExistingRoom to remote drivers
349
- if (config.type === 'remote') {
350
- await (producer as RemoteProducer).connect(joinExistingRoom);
351
- } else {
352
- await producer.connect();
353
- }
354
-
355
- // Monitor status changes
356
- const statusUnsubscribe = producer.onStatusChange(() => {
357
- this.updateStates();
358
- });
359
- this.unsubscribeFns.push(statusUnsubscribe);
360
-
361
- this.producers.push(producer);
362
- this.updateStates();
363
-
364
- return producer.id;
365
- }
366
-
367
- async removeConsumer(): Promise<void> {
368
- if (this.consumer) {
369
- // Stop listening for consumers with this capability
370
- if ('stopListening' in this.consumer && this.consumer.stopListening) {
371
- await this.consumer.stopListening();
372
- }
373
- await this.consumer.disconnect();
374
-
375
- this.consumer = null;
376
- this.updateStates();
377
- }
378
- }
379
-
380
- async removeProducer(driverId: string): Promise<void> {
381
- const driverIndex = this.producers.findIndex(d => d.id === driverId);
382
- if (driverIndex >= 0) {
383
- const driver = this.producers[driverIndex];
384
- await driver.disconnect();
385
-
386
- this.producers.splice(driverIndex, 1);
387
- this.updateStates();
388
- }
389
- }
390
-
391
- // Private methods
392
- private createConsumer(config: USBDriverConfig | RemoteDriverConfig): Consumer {
393
- switch (config.type) {
394
- case 'usb':
395
- return new USBConsumer(config);
396
- case 'remote':
397
- return new RemoteConsumer(config);
398
- default:
399
- const _exhaustive: never = config;
400
- throw new Error(`Unknown consumer type: ${JSON.stringify(_exhaustive)}`);
401
- }
402
- }
403
-
404
- private createProducer(config: USBDriverConfig | RemoteDriverConfig): Producer {
405
- switch (config.type) {
406
- case 'usb':
407
- return new USBProducer(config);
408
- case 'remote':
409
- return new RemoteProducer(config);
410
- default:
411
- const _exhaustive: never = config;
412
- throw new Error(`Unknown producer type: ${JSON.stringify(_exhaustive)}`);
413
- }
414
- }
415
-
416
- // Convert normalized values to URDF radians for 3D visualization
417
- convertNormalizedToUrdfRadians(jointName: string, normalizedValue: number): number {
418
- const joint = this.joints[jointName];
419
- if (!joint?.limits || joint.limits.lower === undefined || joint.limits.upper === undefined) {
420
- // Default ranges
421
- if (jointName.toLowerCase() === 'jaw' || jointName.toLowerCase() === 'gripper') {
422
- return (normalizedValue / 100) * Math.PI;
423
- } else {
424
- return (normalizedValue / 100) * Math.PI;
425
- }
426
- }
427
-
428
- const { lower, upper } = joint.limits;
429
-
430
- // Map normalized value to URDF range
431
- let normalizedRatio: number;
432
- if (jointName.toLowerCase() === 'jaw' || jointName.toLowerCase() === 'gripper') {
433
- normalizedRatio = normalizedValue / 100; // 0-100 -> 0-1
434
- } else {
435
- normalizedRatio = (normalizedValue + 100) / 200; // -100-+100 -> 0-1
436
- }
437
-
438
- const urdfRadians = lower + normalizedRatio * (upper - lower);
439
-
440
- console.debug(`[Robot ${this.id}] Joint ${jointName}: ${normalizedValue} (norm) -> ${urdfRadians.toFixed(3)} (rad)`);
441
-
442
- return urdfRadians;
443
- }
444
-
445
- private async sendToProducers(command: RobotCommand): Promise<void> {
446
- const connectedProducers = this.producers.filter(d => d.status.isConnected);
447
-
448
- console.debug(`[Robot ${this.id}] Sending command to ${connectedProducers.length} producers:`, command);
449
-
450
- // Send to all connected producers
451
- await Promise.all(
452
- connectedProducers.map(async (producer) => {
453
- try {
454
- await producer.sendCommand(command);
455
- } catch (error) {
456
- console.error(`[Robot ${this.id}] Failed to send command to producer ${producer.id}:`, error);
457
- }
458
- })
459
- );
460
- }
461
-
462
- private updateStates(): void {
463
- // Update connection status
464
- const hasConnectedDrivers = (this.consumer?.status.isConnected) ||
465
- this.producers.some(d => d.status.isConnected);
466
-
467
- this.connectionStatus = {
468
- isConnected: hasConnectedDrivers,
469
- lastConnected: hasConnectedDrivers ? new Date() : this.connectionStatus.lastConnected
470
- };
471
-
472
- // Manual control is enabled when no connected consumer
473
- this.isManualControlEnabled = !this.consumer?.status.isConnected;
474
- }
475
-
476
- // Cleanup
477
- async destroy(): Promise<void> {
478
- // Unsubscribe from all callbacks
479
- this.unsubscribeFns.forEach(fn => fn());
480
- this.unsubscribeFns = [];
481
-
482
- // Disconnect all drivers
483
- const allDrivers = [this.consumer, ...this.producers].filter(Boolean) as (Consumer | Producer)[];
484
- await Promise.allSettled(
485
- allDrivers.map(async (driver) => {
486
- try {
487
- await driver.disconnect();
488
- } catch (error) {
489
- console.error(`Error disconnecting driver ${driver.id}:`, error);
490
- }
491
- })
492
- );
493
-
494
- // Calibration cleanup is handled by individual USB drivers
495
- }
496
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import type {
2
+ JointState,
3
+ RobotCommand,
4
+ ConnectionStatus,
5
+ USBDriverConfig,
6
+ RemoteDriverConfig,
7
+ Consumer,
8
+ Producer
9
+ } from "./models.js";
10
+ import type { Positionable, Position3D } from "$lib/types/positionable.js";
11
+ import { USBConsumer } from "./drivers/USBConsumer.js";
12
+ import { USBProducer } from "./drivers/USBProducer.js";
13
+ import { RemoteConsumer } from "./drivers/RemoteConsumer.js";
14
+ import { RemoteProducer } from "./drivers/RemoteProducer.js";
15
+ import { USBServoDriver } from "./drivers/USBServoDriver.js";
16
+
17
+ import { ROBOT_CONFIG } from "./config.js";
18
+ import type IUrdfRobot from "@/components/3d/elements/robot/URDF/interfaces/IUrdfRobot.js";
19
 
20
  export class Robot implements Positionable {
21
+ // Core robot data
22
+ readonly id: string;
23
+ private unsubscribeFns: (() => void)[] = [];
24
+
25
+ // Command synchronization to prevent state conflicts
26
+ private commandMutex = $state(false);
27
+ private pendingCommands: RobotCommand[] = [];
28
+
29
+ // Command deduplication to prevent rapid duplicate commands
30
+ private lastCommandTime = 0;
31
+ private lastCommandValues: Record<string, number> = {};
32
+
33
+ // Memory management
34
+ private lastCleanup = 0;
35
+
36
+ // Single consumer and multiple producers using Svelte 5 runes - PUBLIC for reactive access
37
+ consumer = $state<Consumer | null>(null);
38
+ producers = $state<Producer[]>([]);
39
+
40
+ // Reactive state using Svelte 5 runes - PUBLIC for reactive access
41
+ joints = $state<Record<string, JointState>>({});
42
+ position = $state<Position3D>({ x: 0, y: 0, z: 0 });
43
+ isManualControlEnabled = $state(true);
44
+ connectionStatus = $state<ConnectionStatus>({ isConnected: false });
45
+
46
+ // URDF robot state for 3D visualization - PUBLIC for reactive access
47
+ urdfRobotState = $state<IUrdfRobot | null>(null);
48
+
49
+ // Derived reactive values for components
50
+ jointArray = $derived(Object.values(this.joints));
51
+ hasProducers = $derived(this.producers.length > 0);
52
+ hasConsumer = $derived(this.consumer !== null && this.consumer.status.isConnected);
53
+ outputDriverCount = $derived(this.producers.filter((d) => d.status.isConnected).length);
54
+
55
+ constructor(id: string, initialJoints: JointState[], urdfRobotState?: IUrdfRobot) {
56
+ this.id = id;
57
+
58
+ // Store URDF robot state if provided
59
+ this.urdfRobotState = urdfRobotState || null;
60
+
61
+ // Initialize joints with normalized values
62
+ initialJoints.forEach((joint) => {
63
+ const isGripper =
64
+ joint.name.toLowerCase() === "jaw" || joint.name.toLowerCase() === "gripper";
65
+ this.joints[joint.name] = {
66
+ ...joint,
67
+ value: isGripper ? 0 : 0 // Start at neutral position
68
+ };
69
+ });
70
+ }
71
+
72
+ // Method to set URDF robot state after creation (for async loading)
73
+ setUrdfRobotState(urdfRobotState: any): void {
74
+ this.urdfRobotState = urdfRobotState;
75
+ }
76
+
77
+ /**
78
+ * Update position (implements Positionable interface)
79
+ */
80
+ updatePosition(newPosition: Position3D): void {
81
+ this.position = { ...newPosition };
82
+ }
83
+
84
+ // Get all USB drivers (both consumer and producers) for calibration
85
+ getUSBDrivers(): USBServoDriver[] {
86
+ const usbDrivers: USBServoDriver[] = [];
87
+
88
+ // Check consumer
89
+ if (this.consumer && USBServoDriver.isUSBDriver(this.consumer)) {
90
+ usbDrivers.push(this.consumer);
91
+ }
92
+
93
+ // Check producers
94
+ this.producers.forEach((producer) => {
95
+ if (USBServoDriver.isUSBDriver(producer)) {
96
+ usbDrivers.push(producer);
97
+ }
98
+ });
99
+
100
+ return usbDrivers;
101
+ }
102
+
103
+ // Get uncalibrated USB drivers that need calibration
104
+ getUncalibratedUSBDrivers(): USBServoDriver[] {
105
+ return this.getUSBDrivers().filter((driver) => driver.needsCalibration);
106
+ }
107
+
108
+ // Check if robot has any USB drivers
109
+ hasUSBDrivers(): boolean {
110
+ return this.getUSBDrivers().length > 0;
111
+ }
112
+
113
+ // Check if all USB drivers are calibrated
114
+ areAllUSBDriversCalibrated(): boolean {
115
+ const usbDrivers = this.getUSBDrivers();
116
+ return usbDrivers.length > 0 && usbDrivers.every((driver) => driver.isCalibrated);
117
+ }
118
+
119
+ // Joint value updates (normalized) - for manual control
120
+ updateJoint(name: string, normalizedValue: number): void {
121
+ if (!this.isManualControlEnabled) {
122
+ console.warn("Manual control is disabled");
123
+ return;
124
+ }
125
+
126
+ this.updateJointValue(name, normalizedValue, true);
127
+ }
128
+
129
+ // Internal joint value update (used by both manual control and USB calibration sync)
130
+ updateJointValue(name: string, normalizedValue: number, sendToProducers: boolean = false): void {
131
+ const joint = this.joints[name];
132
+ if (!joint) {
133
+ console.warn(`Joint ${name} not found`);
134
+ return;
135
+ }
136
+
137
+ // Clamp to appropriate normalized range based on joint type
138
+ if (name.toLowerCase() === "jaw" || name.toLowerCase() === "gripper") {
139
+ normalizedValue = Math.max(0, Math.min(100, normalizedValue));
140
+ } else {
141
+ normalizedValue = Math.max(-100, Math.min(100, normalizedValue));
142
+ }
143
+
144
+ console.debug(
145
+ `[Robot ${this.id}] Update joint ${name} to ${normalizedValue} (normalized, sendToProducers: ${sendToProducers})`
146
+ );
147
+
148
+ // Create a new joint object to ensure reactivity
149
+ this.joints[name] = { ...joint, value: normalizedValue };
150
+
151
+ // Send normalized command to producers if requested
152
+ if (sendToProducers) {
153
+ this.sendToProducers({ joints: [{ name, value: normalizedValue }] });
154
+ }
155
+ }
156
+
157
+ executeCommand(command: RobotCommand): void {
158
+ // Command deduplication - skip if same values sent within dedup window
159
+ const now = Date.now();
160
+ if (now - this.lastCommandTime < ROBOT_CONFIG.commands.dedupWindow) {
161
+ const hasChanges = command.joints.some(
162
+ (joint) => Math.abs((this.lastCommandValues[joint.name] || 0) - joint.value) > 0.5
163
+ );
164
+ if (!hasChanges) {
165
+ console.debug(
166
+ `[Robot ${this.id}] 🔄 Skipping duplicate command within ${ROBOT_CONFIG.commands.dedupWindow}ms window`
167
+ );
168
+ return;
169
+ }
170
+ }
171
+
172
+ // Update deduplication tracking
173
+ this.lastCommandTime = now;
174
+ command.joints.forEach((joint) => {
175
+ this.lastCommandValues[joint.name] = joint.value;
176
+ });
177
+
178
+ // Queue command if mutex is locked to prevent race conditions
179
+ if (this.commandMutex) {
180
+ if (this.pendingCommands.length >= ROBOT_CONFIG.commands.maxQueueSize) {
181
+ console.warn(`[Robot ${this.id}] Command queue full, dropping oldest command`);
182
+ this.pendingCommands.shift();
183
+ }
184
+ this.pendingCommands.push(command);
185
+ return;
186
+ }
187
+
188
+ this.commandMutex = true;
189
+
190
+ try {
191
+ console.debug(
192
+ `[Robot ${this.id}] Executing command with ${command.joints.length} joints:`,
193
+ command.joints.map((j) => `${j.name}=${j.value}`).join(", ")
194
+ );
195
+
196
+ // Update virtual robot joints with normalized values
197
+ command.joints.forEach((jointCmd) => {
198
+ const joint = this.joints[jointCmd.name];
199
+ if (joint) {
200
+ // Clamp to appropriate normalized range based on joint type
201
+ let normalizedValue: number;
202
+ if (jointCmd.name.toLowerCase() === "jaw" || jointCmd.name.toLowerCase() === "gripper") {
203
+ normalizedValue = Math.max(0, Math.min(100, jointCmd.value));
204
+ } else {
205
+ normalizedValue = Math.max(-100, Math.min(100, jointCmd.value));
206
+ }
207
+
208
+ console.debug(
209
+ `[Robot ${this.id}] Joint ${jointCmd.name}: ${jointCmd.value} -> ${normalizedValue} (normalized)`
210
+ );
211
+
212
+ // Create a new joint object to ensure reactivity
213
+ this.joints[jointCmd.name] = { ...joint, value: normalizedValue };
214
+ } else {
215
+ console.warn(`[Robot ${this.id}] Joint ${jointCmd.name} not found`);
216
+ }
217
+ });
218
+
219
+ // Send normalized command to producers
220
+ this.sendToProducers(command);
221
+ } finally {
222
+ this.commandMutex = false;
223
+
224
+ // Periodic cleanup to prevent memory leaks
225
+ const now = Date.now();
226
+ if (now - this.lastCleanup > ROBOT_CONFIG.performance.memoryCleanupInterval) {
227
+ // Clear old command values that haven't been updated recently
228
+ Object.keys(this.lastCommandValues).forEach((jointName) => {
229
+ if (now - this.lastCommandTime > ROBOT_CONFIG.performance.memoryCleanupInterval) {
230
+ delete this.lastCommandValues[jointName];
231
+ }
232
+ });
233
+ this.lastCleanup = now;
234
+ }
235
+
236
+ // Process any pending commands
237
+ if (this.pendingCommands.length > 0) {
238
+ const nextCommand = this.pendingCommands.shift();
239
+ if (nextCommand) {
240
+ // Use setTimeout to prevent stack overflow with rapid commands
241
+ setTimeout(() => this.executeCommand(nextCommand), 0);
242
+ }
243
+ }
244
+ }
245
+ }
246
+
247
+ // Consumer management (input driver) - SINGLE consumer only
248
+ async setConsumer(config: USBDriverConfig | RemoteDriverConfig): Promise<string> {
249
+ return this._setConsumer(config, false);
250
+ }
251
+
252
+ // Join existing room as consumer (for Inference Session integration)
253
+ async joinAsConsumer(config: RemoteDriverConfig): Promise<string> {
254
+ if (config.type !== "remote") {
255
+ throw new Error("joinAsConsumer only supports remote drivers");
256
+ }
257
+ return this._setConsumer(config, true);
258
+ }
259
+
260
+ private async _setConsumer(
261
+ config: USBDriverConfig | RemoteDriverConfig,
262
+ joinExistingRoom: boolean
263
+ ): Promise<string> {
264
+ // Remove existing consumer if any
265
+ if (this.consumer) {
266
+ await this.removeConsumer();
267
+ }
268
+
269
+ const consumer = this.createConsumer(config);
270
+
271
+ // Set up calibration completion callback for USB drivers
272
+ if (USBServoDriver.isUSBDriver(consumer)) {
273
+ const calibrationUnsubscribe = consumer.onCalibrationCompleteWithPositions(
274
+ async (finalPositions: Record<string, number>) => {
275
+ console.log(`[Robot ${this.id}] Calibration complete, syncing robot to final positions`);
276
+ consumer.syncRobotPositions(
277
+ finalPositions,
278
+ (jointName: string, normalizedValue: number) => {
279
+ this.updateJointValue(jointName, normalizedValue, false); // Don't send to producers to avoid feedback loop
280
+ }
281
+ );
282
+
283
+ // Start listening now that calibration is complete
284
+ if ("startListening" in consumer && consumer.startListening) {
285
+ try {
286
+ await consumer.startListening();
287
+ console.log(`[Robot ${this.id}] Started listening after calibration completion`);
288
+ } catch (error) {
289
+ console.error(
290
+ `[Robot ${this.id}] Failed to start listening after calibration:`,
291
+ error
292
+ );
293
+ }
294
+ }
295
+ }
296
+ );
297
+ this.unsubscribeFns.push(calibrationUnsubscribe);
298
+ }
299
+
300
+ // Only pass joinExistingRoom to remote drivers
301
+ if (config.type === "remote") {
302
+ await (consumer as RemoteConsumer).connect(joinExistingRoom);
303
+ } else {
304
+ await consumer.connect();
305
+ }
306
+
307
+ // Set up command listening
308
+ const commandUnsubscribe = consumer.onCommand((command: RobotCommand) => {
309
+ this.executeCommand(command);
310
+ });
311
+ this.unsubscribeFns.push(commandUnsubscribe);
312
+
313
+ // Monitor status changes
314
+ const statusUnsubscribe = consumer.onStatusChange(() => {
315
+ this.updateStates();
316
+ });
317
+ this.unsubscribeFns.push(statusUnsubscribe);
318
+
319
+ // Start listening for consumers with this capability (only if calibrated for USB)
320
+ if ("startListening" in consumer && consumer.startListening) {
321
+ // For USB consumers, only start listening if calibrated
322
+ if (USBServoDriver.isUSBDriver(consumer)) {
323
+ if (consumer.isCalibrated) {
324
+ await consumer.startListening();
325
+ }
326
+ // If not calibrated, startListening will be called after calibration completion
327
+ } else {
328
+ // For non-USB consumers, start listening immediately
329
+ await consumer.startListening();
330
+ }
331
+ }
332
+
333
+ this.consumer = consumer;
334
+ this.updateStates();
335
+
336
+ return consumer.id;
337
+ }
338
+
339
+ // Producer management (output drivers) - MULTIPLE allowed
340
+ async addProducer(config: USBDriverConfig | RemoteDriverConfig): Promise<string> {
341
+ return this._addProducer(config, false);
342
+ }
343
+
344
+ // Join existing room as producer (for Inference Session integration)
345
+ async joinAsProducer(config: RemoteDriverConfig): Promise<string> {
346
+ if (config.type !== "remote") {
347
+ throw new Error("joinAsProducer only supports remote drivers");
348
+ }
349
+ return this._addProducer(config, true);
350
+ }
351
+
352
+ private async _addProducer(
353
+ config: USBDriverConfig | RemoteDriverConfig,
354
+ joinExistingRoom: boolean
355
+ ): Promise<string> {
356
+ const producer = this.createProducer(config);
357
+
358
+ // Set up calibration completion callback for USB drivers
359
+ if (USBServoDriver.isUSBDriver(producer)) {
360
+ const calibrationUnsubscribe = producer.onCalibrationCompleteWithPositions(
361
+ async (finalPositions: Record<string, number>) => {
362
+ console.log(`[Robot ${this.id}] Calibration complete, syncing robot to final positions`);
363
+ producer.syncRobotPositions(
364
+ finalPositions,
365
+ (jointName: string, normalizedValue: number) => {
366
+ this.updateJointValue(jointName, normalizedValue, false); // Don't send to producers to avoid feedback loop
367
+ }
368
+ );
369
+
370
+ console.log(
371
+ `[Robot ${this.id}] USB Producer calibration completed and ready for commands`
372
+ );
373
+ }
374
+ );
375
+ this.unsubscribeFns.push(calibrationUnsubscribe);
376
+ }
377
+
378
+ // Only pass joinExistingRoom to remote drivers
379
+ if (config.type === "remote") {
380
+ await (producer as RemoteProducer).connect(joinExistingRoom);
381
+ } else {
382
+ await producer.connect();
383
+ }
384
+
385
+ // Monitor status changes
386
+ const statusUnsubscribe = producer.onStatusChange(() => {
387
+ this.updateStates();
388
+ });
389
+ this.unsubscribeFns.push(statusUnsubscribe);
390
+
391
+ this.producers.push(producer);
392
+ this.updateStates();
393
+
394
+ return producer.id;
395
+ }
396
+
397
+ async removeConsumer(): Promise<void> {
398
+ if (this.consumer) {
399
+ // Stop listening for consumers with this capability
400
+ if ("stopListening" in this.consumer && this.consumer.stopListening) {
401
+ await this.consumer.stopListening();
402
+ }
403
+ await this.consumer.disconnect();
404
+
405
+ this.consumer = null;
406
+ this.updateStates();
407
+ }
408
+ }
409
+
410
+ async removeProducer(driverId: string): Promise<void> {
411
+ const driverIndex = this.producers.findIndex((d) => d.id === driverId);
412
+ if (driverIndex >= 0) {
413
+ const driver = this.producers[driverIndex];
414
+ await driver.disconnect();
415
+
416
+ this.producers.splice(driverIndex, 1);
417
+ this.updateStates();
418
+ }
419
+ }
420
+
421
+ // Private methods
422
+ private createConsumer(config: USBDriverConfig | RemoteDriverConfig): Consumer {
423
+ switch (config.type) {
424
+ case "usb":
425
+ return new USBConsumer(config);
426
+ case "remote":
427
+ return new RemoteConsumer(config);
428
+ default:
429
+ const _exhaustive: never = config;
430
+ throw new Error(`Unknown consumer type: ${JSON.stringify(_exhaustive)}`);
431
+ }
432
+ }
433
+
434
+ private createProducer(config: USBDriverConfig | RemoteDriverConfig): Producer {
435
+ switch (config.type) {
436
+ case "usb":
437
+ return new USBProducer(config);
438
+ case "remote":
439
+ return new RemoteProducer(config);
440
+ default:
441
+ const _exhaustive: never = config;
442
+ throw new Error(`Unknown producer type: ${JSON.stringify(_exhaustive)}`);
443
+ }
444
+ }
445
+
446
+ // Convert normalized values to URDF radians for 3D visualization
447
+ convertNormalizedToUrdfRadians(jointName: string, normalizedValue: number): number {
448
+ const joint = this.joints[jointName];
449
+ if (!joint?.limits || joint.limits.lower === undefined || joint.limits.upper === undefined) {
450
+ // Default ranges
451
+ if (jointName.toLowerCase() === "jaw" || jointName.toLowerCase() === "gripper") {
452
+ return (normalizedValue / 100) * Math.PI;
453
+ } else {
454
+ return (normalizedValue / 100) * Math.PI;
455
+ }
456
+ }
457
+
458
+ const { lower, upper } = joint.limits;
459
+
460
+ // Map normalized value to URDF range
461
+ let normalizedRatio: number;
462
+ if (jointName.toLowerCase() === "jaw" || jointName.toLowerCase() === "gripper") {
463
+ normalizedRatio = normalizedValue / 100; // 0-100 -> 0-1
464
+ } else {
465
+ normalizedRatio = (normalizedValue + 100) / 200; // -100-+100 -> 0-1
466
+ }
467
+
468
+ const urdfRadians = lower + normalizedRatio * (upper - lower);
469
+
470
+ console.debug(
471
+ `[Robot ${this.id}] Joint ${jointName}: ${normalizedValue} (norm) -> ${urdfRadians.toFixed(3)} (rad)`
472
+ );
473
+
474
+ return urdfRadians;
475
+ }
476
+
477
+ private async sendToProducers(command: RobotCommand): Promise<void> {
478
+ const connectedProducers = this.producers.filter((d) => d.status.isConnected);
479
+
480
+ console.debug(
481
+ `[Robot ${this.id}] Sending command to ${connectedProducers.length} producers:`,
482
+ command
483
+ );
484
+
485
+ // Send to all connected producers
486
+ await Promise.all(
487
+ connectedProducers.map(async (producer) => {
488
+ try {
489
+ await producer.sendCommand(command);
490
+ } catch (error) {
491
+ console.error(
492
+ `[Robot ${this.id}] Failed to send command to producer ${producer.id}:`,
493
+ error
494
+ );
495
+ }
496
+ })
497
+ );
498
+ }
499
+
500
+ private updateStates(): void {
501
+ // Update connection status
502
+ const hasConnectedDrivers =
503
+ this.consumer?.status.isConnected || this.producers.some((d) => d.status.isConnected);
504
+
505
+ this.connectionStatus = {
506
+ isConnected: hasConnectedDrivers,
507
+ lastConnected: hasConnectedDrivers ? new Date() : this.connectionStatus.lastConnected
508
+ };
509
+
510
+ // Manual control is enabled when no connected consumer
511
+ this.isManualControlEnabled = !this.consumer?.status.isConnected;
512
+ }
513
+
514
+ // Cleanup
515
+ async destroy(): Promise<void> {
516
+ // Unsubscribe from all callbacks
517
+ this.unsubscribeFns.forEach((fn) => fn());
518
+ this.unsubscribeFns = [];
519
+
520
+ // Disconnect all drivers
521
+ const allDrivers = [this.consumer, ...this.producers].filter(Boolean) as (
522
+ | Consumer
523
+ | Producer
524
+ )[];
525
+ await Promise.allSettled(
526
+ allDrivers.map(async (driver) => {
527
+ try {
528
+ await driver.disconnect();
529
+ } catch (error) {
530
+ console.error(`Error disconnecting driver ${driver.id}:`, error);
531
+ }
532
+ })
533
+ );
534
+
535
+ // Calibration cleanup is handled by individual USB drivers
536
+ }
537
+ }
src/lib/elements/robot/RobotManager.svelte.ts CHANGED
@@ -1,262 +1,275 @@
1
- import { Robot } from './Robot.svelte.js';
2
- import type { JointState, USBDriverConfig, RemoteDriverConfig } from './models.js';
3
- import type { Position3D } from '$lib/types/positionable.js';
4
- import { createUrdfRobot } from '@/elements/robot/createRobot.svelte.js';
5
- import type { RobotUrdfConfig } from '$lib/types/urdf.js';
6
- import { generateName } from '$lib/utils/generateName.js';
7
- import { positionManager } from '$lib/utils/positionManager.js';
8
- import { settings } from '$lib/runes/settings.svelte';
9
- import { robotics } from '@robothub/transport-server-client';
10
- import type { robotics as roboticsTypes } from '@robothub/transport-server-client';
11
  import { robotUrdfConfigMap } from "$lib/configs/robotUrdfConfig";
12
 
13
  export class RobotManager {
14
- private _robots = $state<Robot[]>([]);
15
-
16
- // Room management state - using transport server for communication
17
- rooms = $state<roboticsTypes.RoomInfo[]>([]);
18
- roomsLoading = $state(false);
19
-
20
- // Reactive getters
21
- get robots(): Robot[] {
22
- return this._robots;
23
- }
24
-
25
- get robotCount(): number {
26
- return this._robots.length;
27
- }
28
-
29
- /**
30
- * Room Management Methods
31
- */
32
- async listRooms(workspaceId: string): Promise<roboticsTypes.RoomInfo[]> {
33
- try {
34
- const client = new robotics.RoboticsClientCore(settings.transportServerUrl);
35
- const rooms = await client.listRooms(workspaceId);
36
- this.rooms = rooms;
37
- return rooms;
38
- } catch (error) {
39
- console.error('Failed to list robotics rooms:', error);
40
- return [];
41
- }
42
- }
43
-
44
- async refreshRooms(workspaceId: string): Promise<void> {
45
- this.roomsLoading = true;
46
- try {
47
- await this.listRooms(workspaceId);
48
- } finally {
49
- this.roomsLoading = false;
50
- }
51
- }
52
-
53
- async createRoboticsRoom(workspaceId: string, roomId?: string): Promise<{ success: boolean; roomId?: string; error?: string }> {
54
- try {
55
- const client = new robotics.RoboticsClientCore(settings.transportServerUrl);
56
- const result = await client.createRoom(workspaceId, roomId);
57
- // Refresh rooms list to include the new room
58
- await this.refreshRooms(workspaceId);
59
- return { success: true, roomId: result.roomId };
60
- } catch (error) {
61
- console.error('Failed to create robotics room:', error);
62
- return { success: false, error: error instanceof Error ? error.message : 'Unknown error' };
63
- }
64
- }
65
-
66
- generateRoomId(robotId: string): string {
67
- return `${robotId}-${generateName()}`;
68
- }
69
-
70
- /**
71
- * Connect consumer to an existing robotics room as consumer
72
- * This will receive commands from producers in that room
73
- */
74
- async connectConsumerToRoom(workspaceId: string, robotId: string, roomId: string): Promise<void> {
75
- const robot = this.getRobot(robotId);
76
- if (!robot) {
77
- throw new Error(`Robot ${robotId} not found`);
78
- }
79
-
80
- const config: RemoteDriverConfig = {
81
- type: 'remote',
82
- url: settings.transportServerUrl.replace('http://', 'ws://').replace('https://', 'wss://'),
83
- robotId: roomId,
84
- workspaceId: workspaceId
85
- };
86
-
87
- // Use joinAsConsumer to join existing room
88
- await robot.joinAsConsumer(config);
89
- }
90
-
91
- /**
92
- * Connect producer to an existing robotics room as producer
93
- * This will send commands to consumers in that room
94
- */
95
- async connectProducerToRoom(workspaceId: string, robotId: string, roomId: string): Promise<void> {
96
- const robot = this.getRobot(robotId);
97
- if (!robot) {
98
- throw new Error(`Robot ${robotId} not found`);
99
- }
100
-
101
- const config: RemoteDriverConfig = {
102
- type: 'remote',
103
- url: settings.transportServerUrl.replace('http://', 'ws://').replace('https://', 'wss://'),
104
- robotId: roomId,
105
- workspaceId: workspaceId
106
- };
107
-
108
- // Use joinAsProducer to join existing room
109
- await robot.joinAsProducer(config);
110
- }
111
-
112
- /**
113
- * Create and connect producer as producer to a new room
114
- */
115
- async connectProducerAsProducer(workspaceId: string, robotId: string, roomId?: string): Promise<{ success: boolean; roomId?: string; error?: string }> {
116
- try {
117
- // Create room first if roomId provided, otherwise generate one
118
- const finalRoomId = roomId || this.generateRoomId(robotId);
119
- const createResult = await this.createRoboticsRoom(workspaceId, finalRoomId);
120
-
121
- if (!createResult.success) {
122
- return createResult;
123
- }
124
-
125
- // Connect producer to the new room
126
- await this.connectProducerToRoom(workspaceId, robotId, createResult.roomId!);
127
-
128
- return { success: true, roomId: createResult.roomId };
129
- } catch (error) {
130
- return { success: false, error: error instanceof Error ? error.message : 'Unknown error' };
131
- }
132
- }
133
-
134
- /**
135
- * Create a robot with the default SO-100 arm configuration
136
- */
137
- async createSO100Robot(id?: string, position?: Position3D): Promise<Robot> {
138
- const robotId = id || `so100-${Date.now()}`;
139
- const urdfConfig = robotUrdfConfigMap["so-arm100"];
140
-
141
- return this.createRobotFromUrdf(robotId, urdfConfig, position);
142
- }
143
-
144
- /**
145
- * Create a new robot directly from URDF configuration - automatically extracts joint limits
146
- */
147
- async createRobotFromUrdf(id: string, urdfConfig: RobotUrdfConfig, position?: Position3D): Promise<Robot> {
148
- // Check if robot already exists
149
- if (this._robots.find(r => r.id === id)) {
150
- throw new Error(`Robot with ID ${id} already exists`);
151
- }
152
-
153
- try {
154
- // Load and parse URDF
155
- const robotState = await createUrdfRobot(urdfConfig);
156
-
157
- // Extract joint information from URDF
158
- const joints: JointState[] = [];
159
- let servoId = 1; // Auto-assign servo IDs in order
160
-
161
- for (const urdfJoint of robotState.urdfRobot.joints) {
162
- // Only include revolute joints (movable joints)
163
- if (urdfJoint.type === 'revolute' && urdfJoint.name) {
164
- const jointState: JointState = {
165
- name: urdfJoint.name,
166
- value: 0, // Start at center (0%)
167
- servoId: servoId++
168
- };
169
-
170
- // Extract limits from URDF if available
171
- if (urdfJoint.limit) {
172
- jointState.limits = {
173
- lower: urdfJoint.limit.lower,
174
- upper: urdfJoint.limit.upper
175
- };
176
- }
177
-
178
- joints.push(jointState);
179
- }
180
- }
181
-
182
- console.log(`Extracted ${joints.length} joints from URDF:`, joints.map(j => `${j.name} [${j.limits?.lower?.toFixed(2)}:${j.limits?.upper?.toFixed(2)}]`));
183
-
184
- // Create robot with extracted joints AND URDF robot state
185
- const robot = new Robot(id, joints, robotState.urdfRobot);
186
-
187
- // Set position (from position manager if not provided)
188
- robot.position = position || positionManager.getNextPosition();
189
-
190
- // Add to reactive array
191
- this._robots.push(robot);
192
-
193
- console.log(`Created robot ${id} from URDF. Total robots: ${this._robots.length}`);
194
- return robot;
195
-
196
- } catch (error) {
197
- console.error(`Failed to create robot ${id} from URDF:`, error);
198
- throw error;
199
- }
200
- }
201
-
202
- /**
203
- * Create a new robot with joints defined at initialization (for backwards compatibility)
204
- */
205
- createRobot(id: string, joints: JointState[], position?: Position3D): Robot {
206
- // Check if robot already exists
207
- if (this._robots.find(r => r.id === id)) {
208
- throw new Error(`Robot with ID ${id} already exists`);
209
- }
210
-
211
- // Create robot
212
- const robot = new Robot(id, joints);
213
-
214
- // Set position (from position manager if not provided)
215
- robot.position = position || positionManager.getNextPosition();
216
-
217
- // Add to reactive array
218
- this._robots.push(robot);
219
-
220
- console.log(`Created robot ${id}. Total robots: ${this._robots.length}`);
221
- return robot;
222
- }
223
-
224
- /**
225
- * Remove a robot
226
- */
227
- async removeRobot(id: string): Promise<void> {
228
- const robotIndex = this._robots.findIndex(r => r.id === id);
229
- if (robotIndex === -1) return;
230
-
231
- const robot = this._robots[robotIndex];
232
-
233
- // Clean up robot resources
234
- await robot.destroy();
235
-
236
- // Remove from reactive array
237
- this._robots.splice(robotIndex, 1);
238
-
239
- console.log(`Removed robot ${id}. Remaining robots: ${this._robots.length}`);
240
- }
241
-
242
- /**
243
- * Get robot by ID
244
- */
245
- getRobot(id: string): Robot | undefined {
246
- return this._robots.find(r => r.id === id);
247
- }
248
-
249
-
250
-
251
- /**
252
- * Clean up all robots
253
- */
254
- async destroy(): Promise<void> {
255
- const cleanupPromises = this._robots.map(robot => robot.destroy());
256
- await Promise.allSettled(cleanupPromises);
257
- this._robots.length = 0;
258
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
259
  }
260
 
261
  // Global robot manager instance
262
- export const robotManager = new RobotManager();
 
1
+ import { Robot } from "./Robot.svelte.js";
2
+ import type { JointState, USBDriverConfig, RemoteDriverConfig } from "./models.js";
3
+ import type { Position3D } from "$lib/types/positionable.js";
4
+ import { createUrdfRobot } from "@/elements/robot/createRobot.svelte.js";
5
+ import type { RobotUrdfConfig } from "$lib/types/urdf.js";
6
+ import { generateName } from "$lib/utils/generateName.js";
7
+ import { positionManager } from "$lib/utils/positionManager.js";
8
+ import { settings } from "$lib/runes/settings.svelte";
9
+ import { robotics } from "@robothub/transport-server-client";
10
+ import type { robotics as roboticsTypes } from "@robothub/transport-server-client";
11
  import { robotUrdfConfigMap } from "$lib/configs/robotUrdfConfig";
12
 
13
  export class RobotManager {
14
+ private _robots = $state<Robot[]>([]);
15
+
16
+ // Room management state - using transport server for communication
17
+ rooms = $state<roboticsTypes.RoomInfo[]>([]);
18
+ roomsLoading = $state(false);
19
+
20
+ // Reactive getters
21
+ get robots(): Robot[] {
22
+ return this._robots;
23
+ }
24
+
25
+ get robotCount(): number {
26
+ return this._robots.length;
27
+ }
28
+
29
+ /**
30
+ * Room Management Methods
31
+ */
32
+ async listRooms(workspaceId: string): Promise<roboticsTypes.RoomInfo[]> {
33
+ try {
34
+ const client = new robotics.RoboticsClientCore(settings.transportServerUrl);
35
+ const rooms = await client.listRooms(workspaceId);
36
+ this.rooms = rooms;
37
+ return rooms;
38
+ } catch (error) {
39
+ console.error("Failed to list robotics rooms:", error);
40
+ return [];
41
+ }
42
+ }
43
+
44
+ async refreshRooms(workspaceId: string): Promise<void> {
45
+ this.roomsLoading = true;
46
+ try {
47
+ await this.listRooms(workspaceId);
48
+ } finally {
49
+ this.roomsLoading = false;
50
+ }
51
+ }
52
+
53
+ async createRoboticsRoom(
54
+ workspaceId: string,
55
+ roomId?: string
56
+ ): Promise<{ success: boolean; roomId?: string; error?: string }> {
57
+ try {
58
+ const client = new robotics.RoboticsClientCore(settings.transportServerUrl);
59
+ const result = await client.createRoom(workspaceId, roomId);
60
+ // Refresh rooms list to include the new room
61
+ await this.refreshRooms(workspaceId);
62
+ return { success: true, roomId: result.roomId };
63
+ } catch (error) {
64
+ console.error("Failed to create robotics room:", error);
65
+ return { success: false, error: error instanceof Error ? error.message : "Unknown error" };
66
+ }
67
+ }
68
+
69
+ generateRoomId(robotId: string): string {
70
+ return `${robotId}-${generateName()}`;
71
+ }
72
+
73
+ /**
74
+ * Connect consumer to an existing robotics room as consumer
75
+ * This will receive commands from producers in that room
76
+ */
77
+ async connectConsumerToRoom(workspaceId: string, robotId: string, roomId: string): Promise<void> {
78
+ const robot = this.getRobot(robotId);
79
+ if (!robot) {
80
+ throw new Error(`Robot ${robotId} not found`);
81
+ }
82
+
83
+ const config: RemoteDriverConfig = {
84
+ type: "remote",
85
+ url: settings.transportServerUrl.replace("http://", "ws://").replace("https://", "wss://"),
86
+ robotId: roomId,
87
+ workspaceId: workspaceId
88
+ };
89
+
90
+ // Use joinAsConsumer to join existing room
91
+ await robot.joinAsConsumer(config);
92
+ }
93
+
94
+ /**
95
+ * Connect producer to an existing robotics room as producer
96
+ * This will send commands to consumers in that room
97
+ */
98
+ async connectProducerToRoom(workspaceId: string, robotId: string, roomId: string): Promise<void> {
99
+ const robot = this.getRobot(robotId);
100
+ if (!robot) {
101
+ throw new Error(`Robot ${robotId} not found`);
102
+ }
103
+
104
+ const config: RemoteDriverConfig = {
105
+ type: "remote",
106
+ url: settings.transportServerUrl.replace("http://", "ws://").replace("https://", "wss://"),
107
+ robotId: roomId,
108
+ workspaceId: workspaceId
109
+ };
110
+
111
+ // Use joinAsProducer to join existing room
112
+ await robot.joinAsProducer(config);
113
+ }
114
+
115
+ /**
116
+ * Create and connect producer as producer to a new room
117
+ */
118
+ async connectProducerAsProducer(
119
+ workspaceId: string,
120
+ robotId: string,
121
+ roomId?: string
122
+ ): Promise<{ success: boolean; roomId?: string; error?: string }> {
123
+ try {
124
+ // Create room first if roomId provided, otherwise generate one
125
+ const finalRoomId = roomId || this.generateRoomId(robotId);
126
+ const createResult = await this.createRoboticsRoom(workspaceId, finalRoomId);
127
+
128
+ if (!createResult.success) {
129
+ return createResult;
130
+ }
131
+
132
+ // Connect producer to the new room
133
+ await this.connectProducerToRoom(workspaceId, robotId, createResult.roomId!);
134
+
135
+ return { success: true, roomId: createResult.roomId };
136
+ } catch (error) {
137
+ return { success: false, error: error instanceof Error ? error.message : "Unknown error" };
138
+ }
139
+ }
140
+
141
+ /**
142
+ * Create a robot with the default SO-100 arm configuration
143
+ */
144
+ async createSO100Robot(id?: string, position?: Position3D): Promise<Robot> {
145
+ const robotId = id || `so100-${Date.now()}`;
146
+ const urdfConfig = robotUrdfConfigMap["so-arm100"];
147
+
148
+ return this.createRobotFromUrdf(robotId, urdfConfig, position);
149
+ }
150
+
151
+ /**
152
+ * Create a new robot directly from URDF configuration - automatically extracts joint limits
153
+ */
154
+ async createRobotFromUrdf(
155
+ id: string,
156
+ urdfConfig: RobotUrdfConfig,
157
+ position?: Position3D
158
+ ): Promise<Robot> {
159
+ // Check if robot already exists
160
+ if (this._robots.find((r) => r.id === id)) {
161
+ throw new Error(`Robot with ID ${id} already exists`);
162
+ }
163
+
164
+ try {
165
+ // Load and parse URDF
166
+ const robotState = await createUrdfRobot(urdfConfig);
167
+
168
+ // Extract joint information from URDF
169
+ const joints: JointState[] = [];
170
+ let servoId = 1; // Auto-assign servo IDs in order
171
+
172
+ for (const urdfJoint of robotState.urdfRobot.joints) {
173
+ // Only include revolute joints (movable joints)
174
+ if (urdfJoint.type === "revolute" && urdfJoint.name) {
175
+ const jointState: JointState = {
176
+ name: urdfJoint.name,
177
+ value: 0, // Start at center (0%)
178
+ servoId: servoId++
179
+ };
180
+
181
+ // Extract limits from URDF if available
182
+ if (urdfJoint.limit) {
183
+ jointState.limits = {
184
+ lower: urdfJoint.limit.lower,
185
+ upper: urdfJoint.limit.upper
186
+ };
187
+ }
188
+
189
+ joints.push(jointState);
190
+ }
191
+ }
192
+
193
+ console.log(
194
+ `Extracted ${joints.length} joints from URDF:`,
195
+ joints.map(
196
+ (j) => `${j.name} [${j.limits?.lower?.toFixed(2)}:${j.limits?.upper?.toFixed(2)}]`
197
+ )
198
+ );
199
+
200
+ // Create robot with extracted joints AND URDF robot state
201
+ const robot = new Robot(id, joints, robotState.urdfRobot);
202
+
203
+ // Set position (from position manager if not provided)
204
+ robot.position = position || positionManager.getNextPosition();
205
+
206
+ // Add to reactive array
207
+ this._robots.push(robot);
208
+
209
+ console.log(`Created robot ${id} from URDF. Total robots: ${this._robots.length}`);
210
+ return robot;
211
+ } catch (error) {
212
+ console.error(`Failed to create robot ${id} from URDF:`, error);
213
+ throw error;
214
+ }
215
+ }
216
+
217
+ /**
218
+ * Create a new robot with joints defined at initialization (for backwards compatibility)
219
+ */
220
+ createRobot(id: string, joints: JointState[], position?: Position3D): Robot {
221
+ // Check if robot already exists
222
+ if (this._robots.find((r) => r.id === id)) {
223
+ throw new Error(`Robot with ID ${id} already exists`);
224
+ }
225
+
226
+ // Create robot
227
+ const robot = new Robot(id, joints);
228
+
229
+ // Set position (from position manager if not provided)
230
+ robot.position = position || positionManager.getNextPosition();
231
+
232
+ // Add to reactive array
233
+ this._robots.push(robot);
234
+
235
+ console.log(`Created robot ${id}. Total robots: ${this._robots.length}`);
236
+ return robot;
237
+ }
238
+
239
+ /**
240
+ * Remove a robot
241
+ */
242
+ async removeRobot(id: string): Promise<void> {
243
+ const robotIndex = this._robots.findIndex((r) => r.id === id);
244
+ if (robotIndex === -1) return;
245
+
246
+ const robot = this._robots[robotIndex];
247
+
248
+ // Clean up robot resources
249
+ await robot.destroy();
250
+
251
+ // Remove from reactive array
252
+ this._robots.splice(robotIndex, 1);
253
+
254
+ console.log(`Removed robot ${id}. Remaining robots: ${this._robots.length}`);
255
+ }
256
+
257
+ /**
258
+ * Get robot by ID
259
+ */
260
+ getRobot(id: string): Robot | undefined {
261
+ return this._robots.find((r) => r.id === id);
262
+ }
263
+
264
+ /**
265
+ * Clean up all robots
266
+ */
267
+ async destroy(): Promise<void> {
268
+ const cleanupPromises = this._robots.map((robot) => robot.destroy());
269
+ await Promise.allSettled(cleanupPromises);
270
+ this._robots.length = 0;
271
+ }
272
  }
273
 
274
  // Global robot manager instance
275
+ export const robotManager = new RobotManager();
src/lib/elements/robot/calibration/CalibrationState.svelte.ts CHANGED
@@ -1,271 +1,292 @@
1
- import type { JointCalibration } from '../models.js';
2
- import { ROBOT_CONFIG } from '../config.js';
3
 
4
  export class CalibrationState {
5
- // Reactive calibration state
6
- isCalibrating = $state(false);
7
- progress = $state(0);
8
-
9
- // Joint calibration data
10
- private jointCalibrations = $state<Record<string, JointCalibration>>({});
11
- private currentValues = $state<Record<string, number>>({});
12
-
13
- // Callbacks for completion with final positions
14
- private completionCallbacks: Array<(positions: Record<string, number>) => void> = [];
15
-
16
- constructor() {
17
- // Initialize calibration data for expected joints
18
- const jointNames = ["Rotation", "Pitch", "Elbow", "Wrist_Pitch", "Wrist_Roll", "Jaw"];
19
- jointNames.forEach(name => {
20
- this.jointCalibrations[name] = {
21
- isCalibrated: false,
22
- minServoValue: undefined,
23
- maxServoValue: undefined
24
- };
25
- this.currentValues[name] = 0;
26
- });
27
- }
28
-
29
- // Computed properties
30
- get needsCalibration(): boolean {
31
- return Object.values(this.jointCalibrations).some(cal => !cal.isCalibrated);
32
- }
33
-
34
- get isCalibrated(): boolean {
35
- return Object.values(this.jointCalibrations).every(cal => cal.isCalibrated);
36
- }
37
-
38
- // Update current servo value during calibration
39
- updateCurrentValue(jointName: string, servoValue: number): void {
40
- this.currentValues[jointName] = servoValue;
41
-
42
- // Update calibration range if calibrating
43
- if (this.isCalibrating) {
44
- const calibration = this.jointCalibrations[jointName];
45
- if (calibration) {
46
- // Update min/max values
47
- if (calibration.minServoValue === undefined || servoValue < calibration.minServoValue) {
48
- calibration.minServoValue = servoValue;
49
- }
50
- if (calibration.maxServoValue === undefined || servoValue > calibration.maxServoValue) {
51
- calibration.maxServoValue = servoValue;
52
- }
53
-
54
- // Update progress based on range coverage
55
- this.updateProgress();
56
- }
57
- }
58
- }
59
-
60
- // Get current value for a joint
61
- getCurrentValue(jointName: string): number | undefined {
62
- return this.currentValues[jointName];
63
- }
64
-
65
- // Get calibration data for a joint
66
- getJointCalibration(jointName: string): JointCalibration | undefined {
67
- return this.jointCalibrations[jointName];
68
- }
69
-
70
- // Get formatted range string for display
71
- getJointRange(jointName: string): string {
72
- const calibration = this.jointCalibrations[jointName];
73
- if (!calibration || calibration.minServoValue === undefined || calibration.maxServoValue === undefined) {
74
- return "Not set";
75
- }
76
- return `${calibration.minServoValue}-${calibration.maxServoValue}`;
77
- }
78
-
79
- // Format servo value for display
80
- formatServoValue(value: number | undefined): string {
81
- return value !== undefined ? value.toString() : "---";
82
- }
83
-
84
- // Start calibration process
85
- startCalibration(): void {
86
- console.log("[CalibrationState] Starting calibration...");
87
- this.isCalibrating = true;
88
- this.progress = 0;
89
-
90
- // Reset calibration data
91
- Object.keys(this.jointCalibrations).forEach(jointName => {
92
- this.jointCalibrations[jointName] = {
93
- isCalibrated: false,
94
- minServoValue: undefined,
95
- maxServoValue: undefined
96
- };
97
- });
98
- }
99
-
100
- // Complete calibration and mark joints as calibrated
101
- completeCalibration(): Record<string, number> {
102
- console.log("[CalibrationState] Completing calibration...");
103
-
104
- const finalPositions: Record<string, number> = {};
105
-
106
- // Mark all joints with sufficient range as calibrated
107
- Object.keys(this.jointCalibrations).forEach(jointName => {
108
- const calibration = this.jointCalibrations[jointName];
109
- if (calibration.minServoValue !== undefined && calibration.maxServoValue !== undefined) {
110
- const range = calibration.maxServoValue - calibration.minServoValue;
111
- if (range >= ROBOT_CONFIG.calibration.minRangeThreshold) {
112
- calibration.isCalibrated = true;
113
- finalPositions[jointName] = this.currentValues[jointName] || 0;
114
- console.log(`[CalibrationState] Joint ${jointName} calibrated: ${this.getJointRange(jointName)} (range: ${range})`);
115
- } else {
116
- console.warn(`[CalibrationState] Joint ${jointName} range too small: ${range} < ${ROBOT_CONFIG.calibration.minRangeThreshold}`);
117
- }
118
- }
119
- });
120
-
121
- this.isCalibrating = false;
122
- this.progress = 100;
123
-
124
- // Notify completion callbacks
125
- this.completionCallbacks.forEach(callback => {
126
- try {
127
- callback(finalPositions);
128
- } catch (error) {
129
- console.error("[CalibrationState] Error in completion callback:", error);
130
- }
131
- });
132
-
133
- return finalPositions;
134
- }
135
-
136
- // Cancel calibration
137
- cancelCalibration(): void {
138
- console.log("[CalibrationState] Calibration cancelled");
139
- this.isCalibrating = false;
140
- this.progress = 0;
141
-
142
- // Reset calibration data
143
- Object.keys(this.jointCalibrations).forEach(jointName => {
144
- this.jointCalibrations[jointName] = {
145
- isCalibrated: false,
146
- minServoValue: undefined,
147
- maxServoValue: undefined
148
- };
149
- });
150
- }
151
-
152
- // Skip calibration (use predefined values)
153
- skipCalibration(): void {
154
- console.log("[CalibrationState] Skipping calibration with predefined values");
155
-
156
- // Set predefined calibration values for SO-100 arm
157
- const predefinedCalibrations = {
158
- "Rotation": { minServoValue: 500, maxServoValue: 3500, isCalibrated: true },
159
- "Pitch": { minServoValue: 500, maxServoValue: 3500, isCalibrated: true },
160
- "Elbow": { minServoValue: 500, maxServoValue: 3500, isCalibrated: true },
161
- "Wrist_Pitch": { minServoValue: 500, maxServoValue: 3500, isCalibrated: true },
162
- "Wrist_Roll": { minServoValue: 500, maxServoValue: 3500, isCalibrated: true },
163
- "Jaw": { minServoValue: 1000, maxServoValue: 3000, isCalibrated: true }
164
- };
165
-
166
- Object.entries(predefinedCalibrations).forEach(([jointName, calibration]) => {
167
- this.jointCalibrations[jointName] = calibration;
168
- });
169
-
170
- this.isCalibrating = false;
171
- this.progress = 100;
172
- }
173
-
174
- // Convert raw servo value to normalized percentage (for USB INPUT - reading from servo)
175
- normalizeValue(rawValue: number, jointName: string): number {
176
- const calibration = this.jointCalibrations[jointName];
177
- if (!calibration || !calibration.isCalibrated ||
178
- calibration.minServoValue === undefined || calibration.maxServoValue === undefined) {
179
- // No calibration, use appropriate default conversion
180
- const isGripper = jointName.toLowerCase() === 'jaw' || jointName.toLowerCase() === 'gripper';
181
- if (isGripper) {
182
- return Math.max(0, Math.min(100, (rawValue / 4095) * 100));
183
- } else {
184
- return Math.max(-100, Math.min(100, ((rawValue - 2048) / 2048) * 100));
185
- }
186
- }
187
-
188
- const { minServoValue, maxServoValue } = calibration;
189
- if (maxServoValue === minServoValue) return 0;
190
-
191
- // Bound the input servo value to calibrated range
192
- const bounded = Math.max(minServoValue, Math.min(maxServoValue, rawValue));
193
-
194
- const isGripper = jointName.toLowerCase() === 'jaw' || jointName.toLowerCase() === 'gripper';
195
- if (isGripper) {
196
- // Gripper: 0-100%
197
- return ((bounded - minServoValue) / (maxServoValue - minServoValue)) * 100;
198
- } else {
199
- // Regular joint: -100 to +100%
200
- return (((bounded - minServoValue) / (maxServoValue - minServoValue)) * 200) - 100;
201
- }
202
- }
203
-
204
- // Convert normalized percentage to raw servo value (for USB OUTPUT - writing to servo)
205
- denormalizeValue(normalizedValue: number, jointName: string): number {
206
- const calibration = this.jointCalibrations[jointName];
207
- if (!calibration || !calibration.isCalibrated ||
208
- calibration.minServoValue === undefined || calibration.maxServoValue === undefined) {
209
- // No calibration, use appropriate default conversion
210
- const isGripper = jointName.toLowerCase() === 'jaw' || jointName.toLowerCase() === 'gripper';
211
- if (isGripper) {
212
- return Math.round((normalizedValue / 100) * 4095);
213
- } else {
214
- return Math.round(2048 + (normalizedValue / 100) * 2048);
215
- }
216
- }
217
-
218
- const { minServoValue, maxServoValue } = calibration;
219
- const range = maxServoValue - minServoValue;
220
-
221
- let normalizedRatio: number;
222
- const isGripper = jointName.toLowerCase() === 'jaw' || jointName.toLowerCase() === 'gripper';
223
- if (isGripper) {
224
- // Gripper: 0-100% -> 0-1
225
- normalizedRatio = Math.max(0, Math.min(1, normalizedValue / 100));
226
- } else {
227
- // Regular joint: -100 to +100% -> 0-1
228
- normalizedRatio = Math.max(0, Math.min(1, (normalizedValue + 100) / 200));
229
- }
230
-
231
- return Math.round(minServoValue + normalizedRatio * range);
232
- }
233
-
234
- // Register callback for calibration completion with final positions
235
- onCalibrationCompleteWithPositions(callback: (positions: Record<string, number>) => void): () => void {
236
- this.completionCallbacks.push(callback);
237
-
238
- // Return unsubscribe function
239
- return () => {
240
- const index = this.completionCallbacks.indexOf(callback);
241
- if (index >= 0) {
242
- this.completionCallbacks.splice(index, 1);
243
- }
244
- };
245
- }
246
-
247
- // Update progress based on calibration coverage
248
- private updateProgress(): void {
249
- if (!this.isCalibrating) return;
250
-
251
- let totalProgress = 0;
252
- let jointCount = 0;
253
-
254
- Object.values(this.jointCalibrations).forEach(calibration => {
255
- jointCount++;
256
- if (calibration.minServoValue !== undefined && calibration.maxServoValue !== undefined) {
257
- const range = calibration.maxServoValue - calibration.minServoValue;
258
- // Progress is based on range size (more range = more progress)
259
- const jointProgress = Math.min(100, (range / ROBOT_CONFIG.calibration.minRangeThreshold) * 100);
260
- totalProgress += jointProgress;
261
- }
262
- });
263
-
264
- this.progress = jointCount > 0 ? totalProgress / jointCount : 0;
265
- }
266
-
267
- // Cleanup
268
- destroy(): void {
269
- this.completionCallbacks = [];
270
- }
271
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import type { JointCalibration } from "../models.js";
2
+ import { ROBOT_CONFIG } from "../config.js";
3
 
4
  export class CalibrationState {
5
+ // Reactive calibration state
6
+ isCalibrating = $state(false);
7
+ progress = $state(0);
8
+
9
+ // Joint calibration data
10
+ private jointCalibrations = $state<Record<string, JointCalibration>>({});
11
+ private currentValues = $state<Record<string, number>>({});
12
+
13
+ // Callbacks for completion with final positions
14
+ private completionCallbacks: Array<(positions: Record<string, number>) => void> = [];
15
+
16
+ constructor() {
17
+ // Initialize calibration data for expected joints
18
+ const jointNames = ["Rotation", "Pitch", "Elbow", "Wrist_Pitch", "Wrist_Roll", "Jaw"];
19
+ jointNames.forEach((name) => {
20
+ this.jointCalibrations[name] = {
21
+ isCalibrated: false,
22
+ minServoValue: undefined,
23
+ maxServoValue: undefined
24
+ };
25
+ this.currentValues[name] = 0;
26
+ });
27
+ }
28
+
29
+ // Computed properties
30
+ get needsCalibration(): boolean {
31
+ return Object.values(this.jointCalibrations).some((cal) => !cal.isCalibrated);
32
+ }
33
+
34
+ get isCalibrated(): boolean {
35
+ return Object.values(this.jointCalibrations).every((cal) => cal.isCalibrated);
36
+ }
37
+
38
+ // Update current servo value during calibration
39
+ updateCurrentValue(jointName: string, servoValue: number): void {
40
+ this.currentValues[jointName] = servoValue;
41
+
42
+ // Update calibration range if calibrating
43
+ if (this.isCalibrating) {
44
+ const calibration = this.jointCalibrations[jointName];
45
+ if (calibration) {
46
+ // Update min/max values
47
+ if (calibration.minServoValue === undefined || servoValue < calibration.minServoValue) {
48
+ calibration.minServoValue = servoValue;
49
+ }
50
+ if (calibration.maxServoValue === undefined || servoValue > calibration.maxServoValue) {
51
+ calibration.maxServoValue = servoValue;
52
+ }
53
+
54
+ // Update progress based on range coverage
55
+ this.updateProgress();
56
+ }
57
+ }
58
+ }
59
+
60
+ // Get current value for a joint
61
+ getCurrentValue(jointName: string): number | undefined {
62
+ return this.currentValues[jointName];
63
+ }
64
+
65
+ // Get calibration data for a joint
66
+ getJointCalibration(jointName: string): JointCalibration | undefined {
67
+ return this.jointCalibrations[jointName];
68
+ }
69
+
70
+ // Get formatted range string for display
71
+ getJointRange(jointName: string): string {
72
+ const calibration = this.jointCalibrations[jointName];
73
+ if (
74
+ !calibration ||
75
+ calibration.minServoValue === undefined ||
76
+ calibration.maxServoValue === undefined
77
+ ) {
78
+ return "Not set";
79
+ }
80
+ return `${calibration.minServoValue}-${calibration.maxServoValue}`;
81
+ }
82
+
83
+ // Format servo value for display
84
+ formatServoValue(value: number | undefined): string {
85
+ return value !== undefined ? value.toString() : "---";
86
+ }
87
+
88
+ // Start calibration process
89
+ startCalibration(): void {
90
+ console.log("[CalibrationState] Starting calibration...");
91
+ this.isCalibrating = true;
92
+ this.progress = 0;
93
+
94
+ // Reset calibration data
95
+ Object.keys(this.jointCalibrations).forEach((jointName) => {
96
+ this.jointCalibrations[jointName] = {
97
+ isCalibrated: false,
98
+ minServoValue: undefined,
99
+ maxServoValue: undefined
100
+ };
101
+ });
102
+ }
103
+
104
+ // Complete calibration and mark joints as calibrated
105
+ completeCalibration(): Record<string, number> {
106
+ console.log("[CalibrationState] Completing calibration...");
107
+
108
+ const finalPositions: Record<string, number> = {};
109
+
110
+ // Mark all joints with sufficient range as calibrated
111
+ Object.keys(this.jointCalibrations).forEach((jointName) => {
112
+ const calibration = this.jointCalibrations[jointName];
113
+ if (calibration.minServoValue !== undefined && calibration.maxServoValue !== undefined) {
114
+ const range = calibration.maxServoValue - calibration.minServoValue;
115
+ if (range >= ROBOT_CONFIG.calibration.minRangeThreshold) {
116
+ calibration.isCalibrated = true;
117
+ finalPositions[jointName] = this.currentValues[jointName] || 0;
118
+ console.log(
119
+ `[CalibrationState] Joint ${jointName} calibrated: ${this.getJointRange(jointName)} (range: ${range})`
120
+ );
121
+ } else {
122
+ console.warn(
123
+ `[CalibrationState] Joint ${jointName} range too small: ${range} < ${ROBOT_CONFIG.calibration.minRangeThreshold}`
124
+ );
125
+ }
126
+ }
127
+ });
128
+
129
+ this.isCalibrating = false;
130
+ this.progress = 100;
131
+
132
+ // Notify completion callbacks
133
+ this.completionCallbacks.forEach((callback) => {
134
+ try {
135
+ callback(finalPositions);
136
+ } catch (error) {
137
+ console.error("[CalibrationState] Error in completion callback:", error);
138
+ }
139
+ });
140
+
141
+ return finalPositions;
142
+ }
143
+
144
+ // Cancel calibration
145
+ cancelCalibration(): void {
146
+ console.log("[CalibrationState] Calibration cancelled");
147
+ this.isCalibrating = false;
148
+ this.progress = 0;
149
+
150
+ // Reset calibration data
151
+ Object.keys(this.jointCalibrations).forEach((jointName) => {
152
+ this.jointCalibrations[jointName] = {
153
+ isCalibrated: false,
154
+ minServoValue: undefined,
155
+ maxServoValue: undefined
156
+ };
157
+ });
158
+ }
159
+
160
+ // Skip calibration (use predefined values)
161
+ skipCalibration(): void {
162
+ console.log("[CalibrationState] Skipping calibration with predefined values");
163
+
164
+ // Set predefined calibration values for SO-100 arm
165
+ const predefinedCalibrations = {
166
+ Rotation: { minServoValue: 500, maxServoValue: 3500, isCalibrated: true },
167
+ Pitch: { minServoValue: 500, maxServoValue: 3500, isCalibrated: true },
168
+ Elbow: { minServoValue: 500, maxServoValue: 3500, isCalibrated: true },
169
+ Wrist_Pitch: { minServoValue: 500, maxServoValue: 3500, isCalibrated: true },
170
+ Wrist_Roll: { minServoValue: 500, maxServoValue: 3500, isCalibrated: true },
171
+ Jaw: { minServoValue: 1000, maxServoValue: 3000, isCalibrated: true }
172
+ };
173
+
174
+ Object.entries(predefinedCalibrations).forEach(([jointName, calibration]) => {
175
+ this.jointCalibrations[jointName] = calibration;
176
+ });
177
+
178
+ this.isCalibrating = false;
179
+ this.progress = 100;
180
+ }
181
+
182
+ // Convert raw servo value to normalized percentage (for USB INPUT - reading from servo)
183
+ normalizeValue(rawValue: number, jointName: string): number {
184
+ const calibration = this.jointCalibrations[jointName];
185
+ if (
186
+ !calibration ||
187
+ !calibration.isCalibrated ||
188
+ calibration.minServoValue === undefined ||
189
+ calibration.maxServoValue === undefined
190
+ ) {
191
+ // No calibration, use appropriate default conversion
192
+ const isGripper = jointName.toLowerCase() === "jaw" || jointName.toLowerCase() === "gripper";
193
+ if (isGripper) {
194
+ return Math.max(0, Math.min(100, (rawValue / 4095) * 100));
195
+ } else {
196
+ return Math.max(-100, Math.min(100, ((rawValue - 2048) / 2048) * 100));
197
+ }
198
+ }
199
+
200
+ const { minServoValue, maxServoValue } = calibration;
201
+ if (maxServoValue === minServoValue) return 0;
202
+
203
+ // Bound the input servo value to calibrated range
204
+ const bounded = Math.max(minServoValue, Math.min(maxServoValue, rawValue));
205
+
206
+ const isGripper = jointName.toLowerCase() === "jaw" || jointName.toLowerCase() === "gripper";
207
+ if (isGripper) {
208
+ // Gripper: 0-100%
209
+ return ((bounded - minServoValue) / (maxServoValue - minServoValue)) * 100;
210
+ } else {
211
+ // Regular joint: -100 to +100%
212
+ return ((bounded - minServoValue) / (maxServoValue - minServoValue)) * 200 - 100;
213
+ }
214
+ }
215
+
216
+ // Convert normalized percentage to raw servo value (for USB OUTPUT - writing to servo)
217
+ denormalizeValue(normalizedValue: number, jointName: string): number {
218
+ const calibration = this.jointCalibrations[jointName];
219
+ if (
220
+ !calibration ||
221
+ !calibration.isCalibrated ||
222
+ calibration.minServoValue === undefined ||
223
+ calibration.maxServoValue === undefined
224
+ ) {
225
+ // No calibration, use appropriate default conversion
226
+ const isGripper = jointName.toLowerCase() === "jaw" || jointName.toLowerCase() === "gripper";
227
+ if (isGripper) {
228
+ return Math.round((normalizedValue / 100) * 4095);
229
+ } else {
230
+ return Math.round(2048 + (normalizedValue / 100) * 2048);
231
+ }
232
+ }
233
+
234
+ const { minServoValue, maxServoValue } = calibration;
235
+ const range = maxServoValue - minServoValue;
236
+
237
+ let normalizedRatio: number;
238
+ const isGripper = jointName.toLowerCase() === "jaw" || jointName.toLowerCase() === "gripper";
239
+ if (isGripper) {
240
+ // Gripper: 0-100% -> 0-1
241
+ normalizedRatio = Math.max(0, Math.min(1, normalizedValue / 100));
242
+ } else {
243
+ // Regular joint: -100 to +100% -> 0-1
244
+ normalizedRatio = Math.max(0, Math.min(1, (normalizedValue + 100) / 200));
245
+ }
246
+
247
+ return Math.round(minServoValue + normalizedRatio * range);
248
+ }
249
+
250
+ // Register callback for calibration completion with final positions
251
+ onCalibrationCompleteWithPositions(
252
+ callback: (positions: Record<string, number>) => void
253
+ ): () => void {
254
+ this.completionCallbacks.push(callback);
255
+
256
+ // Return unsubscribe function
257
+ return () => {
258
+ const index = this.completionCallbacks.indexOf(callback);
259
+ if (index >= 0) {
260
+ this.completionCallbacks.splice(index, 1);
261
+ }
262
+ };
263
+ }
264
+
265
+ // Update progress based on calibration coverage
266
+ private updateProgress(): void {
267
+ if (!this.isCalibrating) return;
268
+
269
+ let totalProgress = 0;
270
+ let jointCount = 0;
271
+
272
+ Object.values(this.jointCalibrations).forEach((calibration) => {
273
+ jointCount++;
274
+ if (calibration.minServoValue !== undefined && calibration.maxServoValue !== undefined) {
275
+ const range = calibration.maxServoValue - calibration.minServoValue;
276
+ // Progress is based on range size (more range = more progress)
277
+ const jointProgress = Math.min(
278
+ 100,
279
+ (range / ROBOT_CONFIG.calibration.minRangeThreshold) * 100
280
+ );
281
+ totalProgress += jointProgress;
282
+ }
283
+ });
284
+
285
+ this.progress = jointCount > 0 ? totalProgress / jointCount : 0;
286
+ }
287
+
288
+ // Cleanup
289
+ destroy(): void {
290
+ this.completionCallbacks = [];
291
+ }
292
+ }
src/lib/elements/robot/calibration/USBCalibrationPanel.svelte CHANGED
@@ -39,7 +39,6 @@
39
  }
40
  );
41
 
42
-
43
  async function startCalibration() {
44
  await calibrationManager.startCalibration();
45
  }
@@ -109,7 +108,8 @@
109
  <div class="grid grid-cols-2 gap-2">
110
  {#each jointNames as jointName}
111
  {@const currentValue = calibrationManager.calibrationState.getCurrentValue(jointName)}
112
- {@const calibration = calibrationManager.calibrationState.getJointCalibration(jointName)}
 
113
 
114
  <div class="space-y-1 rounded bg-slate-700/50 p-2">
115
  <div class="flex items-center justify-between">
@@ -120,8 +120,16 @@
120
  </div>
121
 
122
  <div class="flex justify-between text-xs text-slate-500">
123
- <span>Min: {calibrationManager.calibrationState.formatServoValue(calibration?.minServoValue)}</span>
124
- <span>Max: {calibrationManager.calibrationState.formatServoValue(calibration?.maxServoValue)}</span>
 
 
 
 
 
 
 
 
125
  </div>
126
 
127
  {#if calibration?.minServoValue !== undefined && calibration?.maxServoValue !== undefined && currentValue !== undefined}
@@ -167,7 +175,8 @@
167
  <div class="max-h-32 overflow-y-auto">
168
  <div class="grid grid-cols-2 gap-1">
169
  {#each jointNames as jointName}
170
- {@const calibration = calibrationManager.calibrationState.getJointCalibration(jointName)}
 
171
  {@const range = calibrationManager.calibrationState.getJointRange(jointName)}
172
 
173
  <div class="flex items-center justify-between rounded bg-slate-700/30 p-2 text-xs">
 
39
  }
40
  );
41
 
 
42
  async function startCalibration() {
43
  await calibrationManager.startCalibration();
44
  }
 
108
  <div class="grid grid-cols-2 gap-2">
109
  {#each jointNames as jointName}
110
  {@const currentValue = calibrationManager.calibrationState.getCurrentValue(jointName)}
111
+ {@const calibration =
112
+ calibrationManager.calibrationState.getJointCalibration(jointName)}
113
 
114
  <div class="space-y-1 rounded bg-slate-700/50 p-2">
115
  <div class="flex items-center justify-between">
 
120
  </div>
121
 
122
  <div class="flex justify-between text-xs text-slate-500">
123
+ <span
124
+ >Min: {calibrationManager.calibrationState.formatServoValue(
125
+ calibration?.minServoValue
126
+ )}</span
127
+ >
128
+ <span
129
+ >Max: {calibrationManager.calibrationState.formatServoValue(
130
+ calibration?.maxServoValue
131
+ )}</span
132
+ >
133
  </div>
134
 
135
  {#if calibration?.minServoValue !== undefined && calibration?.maxServoValue !== undefined && currentValue !== undefined}
 
175
  <div class="max-h-32 overflow-y-auto">
176
  <div class="grid grid-cols-2 gap-1">
177
  {#each jointNames as jointName}
178
+ {@const calibration =
179
+ calibrationManager.calibrationState.getJointCalibration(jointName)}
180
  {@const range = calibrationManager.calibrationState.getJointRange(jointName)}
181
 
182
  <div class="flex items-center justify-between rounded bg-slate-700/30 p-2 text-xs">
src/lib/elements/robot/calibration/index.ts CHANGED
@@ -1,2 +1,2 @@
1
- export { CalibrationState } from './CalibrationState.svelte.js';
2
- export { default as USBCalibrationPanel } from './USBCalibrationPanel.svelte';
 
1
+ export { CalibrationState } from "./CalibrationState.svelte.js";
2
+ export { default as USBCalibrationPanel } from "./USBCalibrationPanel.svelte";
src/lib/elements/robot/components/ConnectionPanel.svelte CHANGED
@@ -42,11 +42,11 @@
42
  // Find USB driver for calibration (if any)
43
  function getUSBDriver(): any {
44
  // Check consumer first
45
- if (robot.consumer && 'calibrationState' in robot.consumer) {
46
  return robot.consumer;
47
  }
48
  // Then check producers
49
- return robot.producers.find(p => 'calibrationState' in p) || null;
50
  }
51
 
52
  async function connectUSBConsumer() {
@@ -62,7 +62,7 @@
62
  } catch (err) {
63
  console.error("Failed to connect USB consumer:", err);
64
  // Check if it's a calibration error
65
- if (err instanceof Error && err.message.includes('calibration')) {
66
  pendingUSBConnection = "consumer";
67
  showUSBCalibration = true;
68
  return;
@@ -104,7 +104,7 @@
104
  } catch (err) {
105
  console.error("Failed to connect USB producer:", err);
106
  // Check if it's a calibration error
107
- if (err instanceof Error && err.message.includes('calibration')) {
108
  pendingUSBConnection = "producer";
109
  showUSBCalibration = true;
110
  return;
@@ -545,9 +545,7 @@
545
  onCancel={onCalibrationCancel}
546
  />
547
  {:else}
548
- <div class="text-center text-slate-400">
549
- No USB driver available for calibration
550
- </div>
551
  {/if}
552
  </div>
553
  </div>
 
42
  // Find USB driver for calibration (if any)
43
  function getUSBDriver(): any {
44
  // Check consumer first
45
+ if (robot.consumer && "calibrationState" in robot.consumer) {
46
  return robot.consumer;
47
  }
48
  // Then check producers
49
+ return robot.producers.find((p) => "calibrationState" in p) || null;
50
  }
51
 
52
  async function connectUSBConsumer() {
 
62
  } catch (err) {
63
  console.error("Failed to connect USB consumer:", err);
64
  // Check if it's a calibration error
65
+ if (err instanceof Error && err.message.includes("calibration")) {
66
  pendingUSBConnection = "consumer";
67
  showUSBCalibration = true;
68
  return;
 
104
  } catch (err) {
105
  console.error("Failed to connect USB producer:", err);
106
  // Check if it's a calibration error
107
+ if (err instanceof Error && err.message.includes("calibration")) {
108
  pendingUSBConnection = "producer";
109
  showUSBCalibration = true;
110
  return;
 
545
  onCancel={onCalibrationCancel}
546
  />
547
  {:else}
548
+ <div class="text-center text-slate-400">No USB driver available for calibration</div>
 
 
549
  {/if}
550
  </div>
551
  </div>
src/lib/elements/robot/components/RobotGrid.svelte CHANGED
@@ -1,106 +1,111 @@
1
  <script lang="ts">
2
- import { T } from "@threlte/core";
3
- import { robotManager } from '../RobotManager.svelte.js';
4
- import { settings } from '$lib/runes/settings.svelte';
5
- import RobotItem from './RobotItem.svelte';
6
- import type { Robot } from '../Robot.svelte.js';
7
-
8
- let selectedRobot = $state<Robot | null>(null);
9
- let showConnectionModal = $state(false);
10
- let modalType = $state<'consumer' | 'producer' | 'manual'>('consumer');
11
 
12
- function handleRobotClick(robot: Robot, type: 'consumer' | 'producer' | 'manual') {
13
- selectedRobot = robot;
14
- modalType = type;
15
- showConnectionModal = true;
16
- }
17
 
18
- // Access reactive robots
19
- const robots = $derived(robotManager.robots);
 
 
 
 
 
 
20
  </script>
21
 
22
  <T.Group>
23
- {#each robots as robot (robot.id)}
24
- <RobotItem
25
- {robot}
26
- onInteract={handleRobotClick}
27
- />
28
- {/each}
29
  </T.Group>
30
 
31
  <!-- Connection modal will be added here -->
32
  {#if showConnectionModal && selectedRobot}
33
- <div class="fixed inset-0 bg-black/50 flex items-center justify-center z-50">
34
- <div class="bg-slate-800 rounded-lg p-6 max-w-md w-full m-4 space-y-4">
35
- <div class="flex justify-between items-center">
36
- <h2 class="text-lg font-semibold text-white">
37
- {modalType === 'consumer' ? 'Consumer Driver' : modalType === 'producer' ? 'Producer Drivers' : 'Manual Control'}
38
- </h2>
39
- <button
40
- onclick={() => showConnectionModal = false}
41
- class="text-gray-400 hover:text-white"
42
- >
43
-
44
- </button>
45
- </div>
46
-
47
- <div class="space-y-3">
48
- {#if modalType === 'consumer'}
49
- <button
50
- onclick={async () => {
51
- await selectedRobot?.setConsumer({ type: 'usb', baudRate: 1000000 });
52
- showConnectionModal = false;
53
- }}
54
- class="w-full px-4 py-2 bg-blue-600 hover:bg-blue-700 text-white rounded-md"
55
- >
56
- Connect USB Consumer
57
- </button>
58
- <button
59
- onclick={async () => {
60
- await selectedRobot?.setConsumer({
61
- type: 'remote',
62
- url: settings.transportServerUrl.replace('http://', 'ws://').replace('https://', 'wss://'),
63
- robotId: selectedRobot.id
64
- });
65
- showConnectionModal = false;
66
- }}
67
- class="w-full px-4 py-2 bg-purple-600 hover:bg-purple-700 text-white rounded-md"
68
- >
69
- Connect Transport Consumer
70
- </button>
71
- {:else if modalType === 'producer'}
72
- <button
73
- onclick={async () => {
74
- await selectedRobot?.addProducer({ type: 'usb', baudRate: 1000000 });
75
- showConnectionModal = false;
76
- }}
77
- class="w-full px-4 py-2 bg-green-600 hover:bg-green-700 text-white rounded-md"
78
- >
79
- Connect USB Producer
80
- </button>
81
- <button
82
- onclick={async () => {
83
- await selectedRobot?.addProducer({
84
- type: 'remote',
85
- url: settings.transportServerUrl.replace('http://', 'ws://').replace('https://', 'wss://'),
86
- robotId: selectedRobot.id
87
- });
88
- showConnectionModal = false;
89
- }}
90
- class="w-full px-4 py-2 bg-orange-600 hover:bg-orange-700 text-white rounded-md"
91
- >
92
- Connect Transport Producer
93
- </button>
94
- {:else}
95
- <p class="text-gray-300">Manual control interface would go here</p>
96
- {/if}
97
- </div>
98
-
99
- <div class="text-xs text-slate-500 text-center">
100
- {#if modalType !== 'manual'}
101
- Note: USB connections will prompt for calibration if needed
102
- {/if}
103
- </div>
104
- </div>
105
- </div>
106
- {/if}
 
 
 
 
 
 
 
 
 
1
  <script lang="ts">
2
+ import { T } from "@threlte/core";
3
+ import { robotManager } from "../RobotManager.svelte.js";
4
+ import { settings } from "$lib/runes/settings.svelte";
5
+ import RobotItem from "./RobotItem.svelte";
6
+ import type { Robot } from "../Robot.svelte.js";
 
 
 
 
7
 
8
+ let selectedRobot = $state<Robot | null>(null);
9
+ let showConnectionModal = $state(false);
10
+ let modalType = $state<"consumer" | "producer" | "manual">("consumer");
 
 
11
 
12
+ function handleRobotClick(robot: Robot, type: "consumer" | "producer" | "manual") {
13
+ selectedRobot = robot;
14
+ modalType = type;
15
+ showConnectionModal = true;
16
+ }
17
+
18
+ // Access reactive robots
19
+ const robots = $derived(robotManager.robots);
20
  </script>
21
 
22
  <T.Group>
23
+ {#each robots as robot (robot.id)}
24
+ <RobotItem {robot} onInteract={handleRobotClick} />
25
+ {/each}
 
 
 
26
  </T.Group>
27
 
28
  <!-- Connection modal will be added here -->
29
  {#if showConnectionModal && selectedRobot}
30
+ <div class="fixed inset-0 z-50 flex items-center justify-center bg-black/50">
31
+ <div class="m-4 w-full max-w-md space-y-4 rounded-lg bg-slate-800 p-6">
32
+ <div class="flex items-center justify-between">
33
+ <h2 class="text-lg font-semibold text-white">
34
+ {modalType === "consumer"
35
+ ? "Consumer Driver"
36
+ : modalType === "producer"
37
+ ? "Producer Drivers"
38
+ : "Manual Control"}
39
+ </h2>
40
+ <button
41
+ onclick={() => (showConnectionModal = false)}
42
+ class="text-gray-400 hover:text-white"
43
+ >
44
+
45
+ </button>
46
+ </div>
47
+
48
+ <div class="space-y-3">
49
+ {#if modalType === "consumer"}
50
+ <button
51
+ onclick={async () => {
52
+ await selectedRobot?.setConsumer({ type: "usb", baudRate: 1000000 });
53
+ showConnectionModal = false;
54
+ }}
55
+ class="w-full rounded-md bg-blue-600 px-4 py-2 text-white hover:bg-blue-700"
56
+ >
57
+ Connect USB Consumer
58
+ </button>
59
+ <button
60
+ onclick={async () => {
61
+ await selectedRobot?.setConsumer({
62
+ type: "remote",
63
+ url: settings.transportServerUrl
64
+ .replace("http://", "ws://")
65
+ .replace("https://", "wss://"),
66
+ robotId: selectedRobot.id
67
+ });
68
+ showConnectionModal = false;
69
+ }}
70
+ class="w-full rounded-md bg-purple-600 px-4 py-2 text-white hover:bg-purple-700"
71
+ >
72
+ Connect Transport Consumer
73
+ </button>
74
+ {:else if modalType === "producer"}
75
+ <button
76
+ onclick={async () => {
77
+ await selectedRobot?.addProducer({ type: "usb", baudRate: 1000000 });
78
+ showConnectionModal = false;
79
+ }}
80
+ class="w-full rounded-md bg-green-600 px-4 py-2 text-white hover:bg-green-700"
81
+ >
82
+ Connect USB Producer
83
+ </button>
84
+ <button
85
+ onclick={async () => {
86
+ await selectedRobot?.addProducer({
87
+ type: "remote",
88
+ url: settings.transportServerUrl
89
+ .replace("http://", "ws://")
90
+ .replace("https://", "wss://"),
91
+ robotId: selectedRobot.id
92
+ });
93
+ showConnectionModal = false;
94
+ }}
95
+ class="w-full rounded-md bg-orange-600 px-4 py-2 text-white hover:bg-orange-700"
96
+ >
97
+ Connect Transport Producer
98
+ </button>
99
+ {:else}
100
+ <p class="text-gray-300">Manual control interface would go here</p>
101
+ {/if}
102
+ </div>
103
+
104
+ <div class="text-center text-xs text-slate-500">
105
+ {#if modalType !== "manual"}
106
+ Note: USB connections will prompt for calibration if needed
107
+ {/if}
108
+ </div>
109
+ </div>
110
+ </div>
111
+ {/if}
src/lib/elements/robot/components/RobotItem.svelte CHANGED
@@ -1,205 +1,217 @@
1
  <script lang="ts">
2
- import { T } from "@threlte/core";
3
- import { Billboard } from "@threlte/extras";
4
- import type { Robot } from '../Robot.svelte.js';
5
- import { interactivity, type IntersectionEvent, useCursor } from "@threlte/extras";
6
- import { getRootLinks } from "@/components/3d/elements/robot/URDF/utils/UrdfParser";
7
- import UrdfLink from "@/components/3d/elements/robot/URDF/primitives/UrdfLink.svelte";
8
- import { ROBOT_CONFIG } from '../config.js';
9
-
10
- interface Props {
11
- robot: Robot;
12
- onInteract: (robot: Robot, type: 'consumer' | 'producer' | 'manual') => void;
13
- }
14
-
15
- let { robot, onInteract }: Props = $props();
16
-
17
- // Reactive values
18
- const position = $derived(robot.position);
19
- const hasConsumer = $derived(robot.hasConsumer);
20
- const outputDriverCount = $derived(robot.outputDriverCount);
21
- const isManualControl = $derived(robot.isManualControlEnabled);
22
- const connectionStatus = $derived(robot.connectionStatus);
23
- const jointArray = $derived(robot.jointArray);
24
-
25
- // Use the robot's stored URDF state (loaded once when robot was created)
26
- const urdfRobotState = $derived(robot.urdfRobotState);
27
-
28
- let isHovered = $state(false);
29
- let isSelected = $state(false);
30
- let lastJointValues = $state<Record<string, number>>({});
31
-
32
- // Sync joint values from simplified Robot to URDF joints with optimized updates
33
- $effect(() => {
34
- if (!urdfRobotState || jointArray.length === 0) return;
35
-
36
- // Check if this is the initial sync (no previous values recorded)
37
- const isInitialSync = Object.keys(lastJointValues).length === 0;
38
-
39
- // Check if any joint values have actually changed (using config threshold)
40
- const threshold = isInitialSync ? 0 : ROBOT_CONFIG.performance.uiUpdateThreshold;
41
- const hasSignificantChanges = isInitialSync || jointArray.some(joint =>
42
- Math.abs((lastJointValues[joint.name] || 0) - joint.value) > threshold
43
- );
44
- if (!hasSignificantChanges) return;
45
-
46
- // Batch update all joints that have changed (or all joints on initial sync)
47
- let updatedCount = 0;
48
- jointArray.forEach(joint => {
49
- if (isInitialSync || Math.abs((lastJointValues[joint.name] || 0) - joint.value) > threshold) {
50
- lastJointValues[joint.name] = joint.value;
51
- const urdfJoint = findUrdfJoint(urdfRobotState, joint.name);
52
- if (urdfJoint) {
53
- // Initialize rotation array if it doesn't exist
54
- if (!urdfJoint.rotation) {
55
- urdfJoint.rotation = [0, 0, 0];
56
- }
57
-
58
- // Use the Robot's conversion method for proper coordinate mapping
59
- const radians = robot.convertNormalizedToUrdfRadians(joint.name, joint.value);
60
- const axis = urdfJoint.axis_xyz || [0, 0, 1];
61
-
62
- // Reset rotation and apply to the appropriate axis
63
- urdfJoint.rotation = [0, 0, 0];
64
- for (let i = 0; i < 3; i++) {
65
- if (Math.abs(axis[i]) > 0.001) {
66
- urdfJoint.rotation[i] = radians * axis[i];
67
- }
68
- }
69
- updatedCount++;
70
- }
71
- }
72
- });
73
-
74
- if (updatedCount > 0) {
75
- console.debug(`${isInitialSync ? 'Initial sync: ' : ''}Updated ${updatedCount} URDF joints for robot ${robot.id}`);
76
- }
77
- });
78
-
79
- function findUrdfJoint(robot: any, jointName: string): any {
80
- // Search through the robot's joints array
81
- if (robot.joints && Array.isArray(robot.joints)) {
82
- for (const joint of robot.joints) {
83
- if (joint.name === jointName) {
84
- return joint;
85
- }
86
- }
87
- }
88
- return null;
89
- }
90
-
91
- const { onPointerEnter, onPointerLeave } = useCursor();
92
- interactivity();
 
 
 
 
93
  </script>
94
 
95
  <T.Group
96
- position.x={position.x}
97
- position.y={position.y}
98
- position.z={position.z}
99
- scale={[10, 10, 10]}
100
- rotation={[-Math.PI / 2, 0, 0]}
101
  >
102
- {#if urdfRobotState}
103
- <!-- URDF Robot representation -->
104
- <T.Group
105
- onclick={(event: IntersectionEvent<MouseEvent>) => {
106
- event.stopPropagation();
107
- isSelected = true;
108
- onInteract(robot, 'manual');
109
- }}
110
- onpointerenter={(event: IntersectionEvent<PointerEvent>) => {
111
- event.stopPropagation();
112
- onPointerEnter();
113
- isHovered = true;
114
- }}
115
- onpointerleave={(event: IntersectionEvent<PointerEvent>) => {
116
- event.stopPropagation();
117
- onPointerLeave();
118
- isHovered = false;
119
- }}
120
- >
121
- {#each getRootLinks(urdfRobotState) as link}
122
- <UrdfLink
123
- robot={urdfRobotState}
124
- {link}
125
- textScale={0.2}
126
- showName={isHovered || isSelected}
127
- showVisual={true}
128
- showCollision={false}
129
- visualColor={connectionStatus.isConnected ? "#10b981" : "#6b7280"}
130
- visualOpacity={isHovered || isSelected ? 0.4 : 1.0}
131
- collisionOpacity={1.0}
132
- collisionColor="#813d9c"
133
- jointNames={isHovered}
134
- joints={isHovered}
135
- jointColor="#62a0ea"
136
- jointIndicatorColor="#f66151"
137
- nameHeight={0.1}
138
- showLine={isHovered || isSelected}
139
- opacity={1}
140
- />
141
- {/each}
142
- </T.Group>
143
- {:else}
144
- <!-- Fallback simple representation while URDF loads -->
145
- <T.Mesh
146
- onpointerenter={() => isHovered = true}
147
- onpointerleave={() => isHovered = false}
148
- onclick={() => onInteract(robot, 'manual')}
149
- >
150
- <T.BoxGeometry args={[0.1, 0.1, 0.1]} />
151
- <T.MeshStandardMaterial
152
- color={connectionStatus.isConnected ? "#10b981" : "#6b7280"}
153
- opacity={isHovered ? 0.8 : 1.0}
154
- transparent
155
- />
156
- </T.Mesh>
157
- {/if}
158
-
159
- <!-- Status billboard when hovered -->
160
- {#if isHovered}
161
- <Billboard>
162
- <T.Group position.y={1.5}>
163
- <div class="bg-slate-800/90 rounded-lg p-3 text-white text-sm min-w-48 backdrop-blur">
164
- <div class="font-semibold mb-2">{robot.id}</div>
165
-
166
- <!-- Connection status boxes -->
167
- <div class="flex gap-2 mb-2">
168
- <!-- Consumer status -->
169
- <button
170
- onclick={() => onInteract(robot, 'consumer')}
171
- class="flex-1 p-2 rounded border transition-colors {hasConsumer ? 'bg-green-600 border-green-500' : 'bg-gray-600 border-gray-500 hover:bg-gray-500'}"
172
- >
173
- <div class="text-xs">Consumer</div>
174
- <div class="text-[10px] opacity-75">
175
- {hasConsumer ? 'Connected' : 'None'}
176
- </div>
177
- </button>
178
-
179
- <!-- Robot status -->
180
- <div class="flex-1 p-2 rounded border border-yellow-500 bg-yellow-600">
181
- <div class="text-xs">Robot</div>
182
- <div class="text-[10px] opacity-75">{robot.jointArray.length} joints</div>
183
- </div>
184
-
185
- <!-- Producer status -->
186
- <button
187
- onclick={() => onInteract(robot, 'producer')}
188
- class="flex-1 p-2 rounded border transition-colors {outputDriverCount > 0 ? 'bg-blue-600 border-blue-500' : 'bg-gray-600 border-gray-500 hover:bg-gray-500'}"
189
- >
190
- <div class="text-xs">Producer</div>
191
- <div class="text-[10px] opacity-75">
192
- {outputDriverCount} driver{outputDriverCount !== 1 ? 's' : ''}
193
- </div>
194
- </button>
195
- </div>
196
-
197
- <!-- Control status -->
198
- <div class="text-xs text-center px-2 py-1 rounded {isManualControl ? 'bg-purple-600' : 'bg-orange-600'}">
199
- {isManualControl ? 'Manual Control' : 'External Control'}
200
- </div>
201
- </div>
202
- </T.Group>
203
- </Billboard>
204
- {/if}
205
- </T.Group>
 
 
 
 
 
 
 
 
 
1
  <script lang="ts">
2
+ import { T } from "@threlte/core";
3
+ import { Billboard } from "@threlte/extras";
4
+ import type { Robot } from "../Robot.svelte.js";
5
+ import { interactivity, type IntersectionEvent, useCursor } from "@threlte/extras";
6
+ import { getRootLinks } from "@/components/3d/elements/robot/URDF/utils/UrdfParser";
7
+ import UrdfLink from "@/components/3d/elements/robot/URDF/primitives/UrdfLink.svelte";
8
+ import { ROBOT_CONFIG } from "../config.js";
9
+
10
+ interface Props {
11
+ robot: Robot;
12
+ onInteract: (robot: Robot, type: "consumer" | "producer" | "manual") => void;
13
+ }
14
+
15
+ let { robot, onInteract }: Props = $props();
16
+
17
+ // Reactive values
18
+ const position = $derived(robot.position);
19
+ const hasConsumer = $derived(robot.hasConsumer);
20
+ const outputDriverCount = $derived(robot.outputDriverCount);
21
+ const isManualControl = $derived(robot.isManualControlEnabled);
22
+ const connectionStatus = $derived(robot.connectionStatus);
23
+ const jointArray = $derived(robot.jointArray);
24
+
25
+ // Use the robot's stored URDF state (loaded once when robot was created)
26
+ const urdfRobotState = $derived(robot.urdfRobotState);
27
+
28
+ let isHovered = $state(false);
29
+ let isSelected = $state(false);
30
+ let lastJointValues = $state<Record<string, number>>({});
31
+
32
+ // Sync joint values from simplified Robot to URDF joints with optimized updates
33
+ $effect(() => {
34
+ if (!urdfRobotState || jointArray.length === 0) return;
35
+
36
+ // Check if this is the initial sync (no previous values recorded)
37
+ const isInitialSync = Object.keys(lastJointValues).length === 0;
38
+
39
+ // Check if any joint values have actually changed (using config threshold)
40
+ const threshold = isInitialSync ? 0 : ROBOT_CONFIG.performance.uiUpdateThreshold;
41
+ const hasSignificantChanges =
42
+ isInitialSync ||
43
+ jointArray.some(
44
+ (joint) => Math.abs((lastJointValues[joint.name] || 0) - joint.value) > threshold
45
+ );
46
+ if (!hasSignificantChanges) return;
47
+
48
+ // Batch update all joints that have changed (or all joints on initial sync)
49
+ let updatedCount = 0;
50
+ jointArray.forEach((joint) => {
51
+ if (isInitialSync || Math.abs((lastJointValues[joint.name] || 0) - joint.value) > threshold) {
52
+ lastJointValues[joint.name] = joint.value;
53
+ const urdfJoint = findUrdfJoint(urdfRobotState, joint.name);
54
+ if (urdfJoint) {
55
+ // Initialize rotation array if it doesn't exist
56
+ if (!urdfJoint.rotation) {
57
+ urdfJoint.rotation = [0, 0, 0];
58
+ }
59
+
60
+ // Use the Robot's conversion method for proper coordinate mapping
61
+ const radians = robot.convertNormalizedToUrdfRadians(joint.name, joint.value);
62
+ const axis = urdfJoint.axis_xyz || [0, 0, 1];
63
+
64
+ // Reset rotation and apply to the appropriate axis
65
+ urdfJoint.rotation = [0, 0, 0];
66
+ for (let i = 0; i < 3; i++) {
67
+ if (Math.abs(axis[i]) > 0.001) {
68
+ urdfJoint.rotation[i] = radians * axis[i];
69
+ }
70
+ }
71
+ updatedCount++;
72
+ }
73
+ }
74
+ });
75
+
76
+ if (updatedCount > 0) {
77
+ console.debug(
78
+ `${isInitialSync ? "Initial sync: " : ""}Updated ${updatedCount} URDF joints for robot ${robot.id}`
79
+ );
80
+ }
81
+ });
82
+
83
+ function findUrdfJoint(robot: any, jointName: string): any {
84
+ // Search through the robot's joints array
85
+ if (robot.joints && Array.isArray(robot.joints)) {
86
+ for (const joint of robot.joints) {
87
+ if (joint.name === jointName) {
88
+ return joint;
89
+ }
90
+ }
91
+ }
92
+ return null;
93
+ }
94
+
95
+ const { onPointerEnter, onPointerLeave } = useCursor();
96
+ interactivity();
97
  </script>
98
 
99
  <T.Group
100
+ position.x={position.x}
101
+ position.y={position.y}
102
+ position.z={position.z}
103
+ scale={[10, 10, 10]}
104
+ rotation={[-Math.PI / 2, 0, 0]}
105
  >
106
+ {#if urdfRobotState}
107
+ <!-- URDF Robot representation -->
108
+ <T.Group
109
+ onclick={(event: IntersectionEvent<MouseEvent>) => {
110
+ event.stopPropagation();
111
+ isSelected = true;
112
+ onInteract(robot, "manual");
113
+ }}
114
+ onpointerenter={(event: IntersectionEvent<PointerEvent>) => {
115
+ event.stopPropagation();
116
+ onPointerEnter();
117
+ isHovered = true;
118
+ }}
119
+ onpointerleave={(event: IntersectionEvent<PointerEvent>) => {
120
+ event.stopPropagation();
121
+ onPointerLeave();
122
+ isHovered = false;
123
+ }}
124
+ >
125
+ {#each getRootLinks(urdfRobotState) as link}
126
+ <UrdfLink
127
+ robot={urdfRobotState}
128
+ {link}
129
+ textScale={0.2}
130
+ showName={isHovered || isSelected}
131
+ showVisual={true}
132
+ showCollision={false}
133
+ visualColor={connectionStatus.isConnected ? "#10b981" : "#6b7280"}
134
+ visualOpacity={isHovered || isSelected ? 0.4 : 1.0}
135
+ collisionOpacity={1.0}
136
+ collisionColor="#813d9c"
137
+ jointNames={isHovered}
138
+ joints={isHovered}
139
+ jointColor="#62a0ea"
140
+ jointIndicatorColor="#f66151"
141
+ nameHeight={0.1}
142
+ showLine={isHovered || isSelected}
143
+ opacity={1}
144
+ />
145
+ {/each}
146
+ </T.Group>
147
+ {:else}
148
+ <!-- Fallback simple representation while URDF loads -->
149
+ <T.Mesh
150
+ onpointerenter={() => (isHovered = true)}
151
+ onpointerleave={() => (isHovered = false)}
152
+ onclick={() => onInteract(robot, "manual")}
153
+ >
154
+ <T.BoxGeometry args={[0.1, 0.1, 0.1]} />
155
+ <T.MeshStandardMaterial
156
+ color={connectionStatus.isConnected ? "#10b981" : "#6b7280"}
157
+ opacity={isHovered ? 0.8 : 1.0}
158
+ transparent
159
+ />
160
+ </T.Mesh>
161
+ {/if}
162
+
163
+ <!-- Status billboard when hovered -->
164
+ {#if isHovered}
165
+ <Billboard>
166
+ <T.Group position.y={1.5}>
167
+ <div class="min-w-48 rounded-lg bg-slate-800/90 p-3 text-sm text-white backdrop-blur">
168
+ <div class="mb-2 font-semibold">{robot.id}</div>
169
+
170
+ <!-- Connection status boxes -->
171
+ <div class="mb-2 flex gap-2">
172
+ <!-- Consumer status -->
173
+ <button
174
+ onclick={() => onInteract(robot, "consumer")}
175
+ class="flex-1 rounded border p-2 transition-colors {hasConsumer
176
+ ? 'border-green-500 bg-green-600'
177
+ : 'border-gray-500 bg-gray-600 hover:bg-gray-500'}"
178
+ >
179
+ <div class="text-xs">Consumer</div>
180
+ <div class="text-[10px] opacity-75">
181
+ {hasConsumer ? "Connected" : "None"}
182
+ </div>
183
+ </button>
184
+
185
+ <!-- Robot status -->
186
+ <div class="flex-1 rounded border border-yellow-500 bg-yellow-600 p-2">
187
+ <div class="text-xs">Robot</div>
188
+ <div class="text-[10px] opacity-75">{robot.jointArray.length} joints</div>
189
+ </div>
190
+
191
+ <!-- Producer status -->
192
+ <button
193
+ onclick={() => onInteract(robot, "producer")}
194
+ class="flex-1 rounded border p-2 transition-colors {outputDriverCount > 0
195
+ ? 'border-blue-500 bg-blue-600'
196
+ : 'border-gray-500 bg-gray-600 hover:bg-gray-500'}"
197
+ >
198
+ <div class="text-xs">Producer</div>
199
+ <div class="text-[10px] opacity-75">
200
+ {outputDriverCount} driver{outputDriverCount !== 1 ? "s" : ""}
201
+ </div>
202
+ </button>
203
+ </div>
204
+
205
+ <!-- Control status -->
206
+ <div
207
+ class="rounded px-2 py-1 text-center text-xs {isManualControl
208
+ ? 'bg-purple-600'
209
+ : 'bg-orange-600'}"
210
+ >
211
+ {isManualControl ? "Manual Control" : "External Control"}
212
+ </div>
213
+ </div>
214
+ </T.Group>
215
+ </Billboard>
216
+ {/if}
217
+ </T.Group>
src/lib/elements/robot/components/index.ts CHANGED
@@ -1,5 +1,5 @@
1
  // Robot components exports
2
- export { default as RobotItem } from './RobotItem.svelte';
3
- export { default as RobotGrid } from './RobotGrid.svelte';
4
- export { default as ConnectionPanel } from './ConnectionPanel.svelte';
5
- export { default as RobotControls } from './RobotControls.svelte';
 
1
  // Robot components exports
2
+ export { default as RobotItem } from "./RobotItem.svelte";
3
+ export { default as RobotGrid } from "./RobotGrid.svelte";
4
+ export { default as ConnectionPanel } from "./ConnectionPanel.svelte";
5
+ export { default as RobotControls } from "./RobotControls.svelte";