koodilabran v0.1
This commit is contained in:
@@ -1,75 +1,129 @@
|
||||
# Kipinä Agentic Network PoC (WebGPU Edition)
|
||||
# Kipinä Agentic Network PoC
|
||||
|
||||
Tämä on hajautetun tekoälylaskennan (Agentic Compute) kokeilulaboratorio. Projekti koostuu Rust-pohjaisesta keskuksesta (Hub) ja selainpohjaisista työntekijöistä (Nodet), jotka suorittavat tekoälytensoreiden matriisilaskentaa **WebGPU**-rajapintaa ja **Burn AI** -koneoppimiskirjastoa hyödyntäen.
|
||||
Hajautettu AI-laskentaverkko selaimessa ja natiivina. Käyttäjät tarjoavat GPU/CPU-laskentatehoa avaamalla verkkosivun tai ajamalla natiivi-noden.
|
||||
|
||||
Normaalin keskitetyn palvelimen sijaan tämä kokeilu hyödyntää selaimeen kytkettyjen lukemattomien laitteiden vapaana olevaa tehokapasiteettia hajautetusti P2P-tyylillä.
|
||||
**Tuotanto:** https://kipina.studio | **Admin:** https://kipina.studio/admin
|
||||
|
||||
## Kuinka käynnistää projekti paikallisesti
|
||||
## Arkkitehtuuri
|
||||
|
||||
1. **Rakenna solmun WebAssembly-binääri**
|
||||
Paketoi Rust WebAssemblyksi (vaatii `wasm-pack`-työkalun):
|
||||
```bash
|
||||
cd node
|
||||
wasm-pack build --target web --out-dir ../static/pkg
|
||||
```
|
||||
┌─────────────────┐
|
||||
│ Hub (Axum) │
|
||||
│ :3000 / Caddy │
|
||||
│ SQLite, WS BC │
|
||||
└────────┬────────┘
|
||||
WebSocket │ WebSocket
|
||||
┌────────────────────┼────────────────────┐
|
||||
▼ ▼ ▼
|
||||
┌────────────────┐ ┌────────────────┐ ┌─────────────────┐
|
||||
│ Selainsolmu │ │ Selainsolmu │ │ Native Node │
|
||||
│ Wasm + Burn │ │ Wasm + Candle │ │ Rust + Candle │
|
||||
│ WebGPU/NdArray │ │ SmolLM/Qwen │ │ CPU/CUDA │
|
||||
└────────────────┘ └────────────────┘ └─────────────────┘
|
||||
```
|
||||
|
||||
2. **Käynnistä Hub-Keskuspalvelin**
|
||||
```bash
|
||||
cd hub
|
||||
cargo run
|
||||
```
|
||||
Palvelin lähtee pyörimään ja tarjoamaan sekä WebSocket-reititintä että staattista Dashboard-sivustoa lokaalisti portissa `3000`.
|
||||
**Hub** broadcastaa tehtäviä (tokenisointiparit, LLM-promptit) kaikille solmuille WebSocketin kautta. Solmut käsittelevät vain oman tehtävätyyppinsä mukaiset viestit.
|
||||
|
||||
---
|
||||
## Cratet
|
||||
|
||||
## ⚠️ WebGPU Ota-Käyttöön -ohjeet (Linux / Mac / Win)
|
||||
| Crate | Polku | Kuvaus |
|
||||
|---|---|---|
|
||||
| `hub` | `hub/` | Axum WebSocket -palvelin, tehtävien jakelu, admin-API, SQLite |
|
||||
| `node` | `node/` | Wasm-selainsolmu: Burn (tensorit), Candle (LLM), tokenizer |
|
||||
| `native-node` | `native-node/` | Natiivi Rust-solmu: Candle LLM, NVML/wgpu GPU-tunnistus, sysinfo |
|
||||
|
||||
Selainvalmistajat rajoittavat tällä hetkellä uuden WebGPU-rajapinnan hardware-yhteyttä (fyysiseen näytönohjaimeen) turvallisuus- ja vakaussyistä, erityisesti Linuxin Wayland-ympäristöissä (kuten Pop!_OS, Ubuntu).
|
||||
### Hub (`hub/src/`)
|
||||
|
||||
Päästäksesi hyödyntämään solmun laskentatehoa selaimesi ja tietokoneesi näytönohjaimen läpi, joudut todennäköisesti pakottamaan sen käyntiin.
|
||||
- `main.rs` — WebSocket-reititin, tehtäväjakelu (10s intervalli), origin-tarkistus, IP-rajoitus, admin HTML
|
||||
- `db.rs` — SQLite: `node_sessions` + `pair_results` taulut, skeemaversiointi
|
||||
|
||||
### Chromium-pohjaiset selaimet (Google Chrome, Brave, Chromium)
|
||||
### Node (`node/src/`)
|
||||
|
||||
**Vaihtoehto 1: Käynnistys lipuilla (Suositeltu Linuxille ja Waylandille)**
|
||||
Jos Chromesi tuottaa Wasm-kaatumisia tai väittää ettei adapteria löydy, laitteesi Wayland-palvelin estää Vulkan-rajapinnan oletuksena. Käynnistä selaimesi komentoriviltä pakottamalla vanha X11-ikkunointi ja Vulkan:
|
||||
- `lib.rs` — Wasm-entrypoint, tehtävävalinta (`SELECTED_TASK`), WebSocket-handler, GPU/CPU-valinta
|
||||
- `storage.rs` — IndexedDB read/write (tokenizer, mallin painot)
|
||||
- `smollm.rs` — SmolLM 135M Candle-inferenssi (Llama-arkkitehtuuri)
|
||||
- `qwen.rs` — Qwen2.5 0.5B Candle-inferenssi (Qwen2-arkkitehtuuri)
|
||||
- `phi3.rs` — Phi-3 placeholder (liian iso selaimelle)
|
||||
|
||||
### Native Node (`native-node/src/`)
|
||||
|
||||
- `main.rs` — GPU-tunnistus (wgpu + NVML + sysfs + Apple), HF Hub -lataus, WS-yhteys
|
||||
- `inference.rs` — Qwen2.5-0.5B Candle-inferenssi, KV-cache reset per prompti, mmap-lataus
|
||||
|
||||
## Kehitysympäristö
|
||||
|
||||
```bash
|
||||
# Google Chrome
|
||||
google-chrome --enable-unsafe-webgpu --enable-features=Vulkan --ignore-gpu-blocklist --use-angle=vulkan --ozone-platform=x11
|
||||
# Vaatimukset
|
||||
rustup target add wasm32-unknown-unknown
|
||||
cargo install wasm-pack
|
||||
|
||||
# Brave Browser
|
||||
brave-browser --enable-unsafe-webgpu --enable-features=Vulkan --ignore-gpu-blocklist --use-angle=vulkan --ozone-platform=x11
|
||||
# Kehitys (Docker — Wasm buildataan automaattisesti)
|
||||
docker compose up
|
||||
|
||||
# Chromium
|
||||
chromium-browser --enable-unsafe-webgpu --enable-features=Vulkan --ignore-gpu-blocklist --use-angle=vulkan --ozone-platform=x11
|
||||
# Kehitys (ilman Dockeria)
|
||||
cd node && wasm-pack build --dev --target web --out-dir ../static/pkg && cd ..
|
||||
cargo run -p hub
|
||||
# → http://localhost:3000
|
||||
|
||||
# Native node (erillinen terminaali)
|
||||
CARGO_TARGET_DIR=target-native HUB_URL=ws://localhost:3000/ws cargo run --release -p native-node
|
||||
```
|
||||
|
||||
*(Voit halutessasi testata puhdasta testi-ikkunaa erillisen profiilin kera, lisäämällä perään `--user-data-dir=/tmp/kipin-webgpu-test` jottei asetus sotke tai ohjaudu vanhaan auki olevaan sessioosi).*
|
||||
## Viestityyypit (WebSocket JSON)
|
||||
|
||||
**Vaihtoehto 2: Sisäänrakennetun Flagin kääntö (Windows / Mac / Osittain Linux)**
|
||||
1. Kirjoita selaimen osoiteriville `chrome://flags` (tai `brave://flags`)
|
||||
2. Etsi hakusanalla **WebGPU** (Unsafe WebGPU / WebGPU Developer Features) ja vaihda tilaksi `Enabled`
|
||||
3. Etsi hakusanalla **Vulkan** ja vaihda tilaan `Enabled`
|
||||
4. Uudelleenkäynnistä selain pienen napin kautta.
|
||||
Hub → solmut:
|
||||
| Tyyppi | Kuvaus |
|
||||
|---|---|
|
||||
| `pair_task` | `{en, fi}` — tokenisointipari |
|
||||
| `llm_prompt` | `{prompt, model}` — LLM-tehtävä |
|
||||
| `stats` | `{nodes, vram_gb, tasks, version}` |
|
||||
| `node_joined` | `{node_id}` |
|
||||
|
||||
---
|
||||
Solmu → hub:
|
||||
| Tyyppi | Kuvaus |
|
||||
|---|---|
|
||||
| `auth` | Laitetiedot, `selected_task`, `allocated_gb` |
|
||||
| `pair_done` | Tokenisointitulos: `{en, fi, overhead_pct, duration_ms}` |
|
||||
| `llm_done` | LLM-tulos: `{response, tokens_generated, tokens_per_sec}` |
|
||||
| `llm_chunk` | Streaming-token |
|
||||
| `download_progress` | Mallin latauksen edistyminen |
|
||||
|
||||
### Mozilla Firefox
|
||||
## API-endpointit
|
||||
|
||||
Firefox tukee WebGPU:ta toistaiseksi vahvasti vain Nightly-versioissa, mutta sitä voi yrittää aktivoida Config-asetuksista.
|
||||
1. Kirjoita osoiteriville `about:config` ja ymmärrä riskit.
|
||||
2. Etsi `dom.webgpu.enabled` ja tuplaklikkaa arvoksi `true`.
|
||||
3. Etsi `gfx.webrender.all` ja aseta se `true`.
|
||||
4. Uudelleenkäynnistä Firefox.
|
||||
| Polku | Kuvaus |
|
||||
|---|---|
|
||||
| `GET /` | Dashboard (staattinen HTML) |
|
||||
| `GET /ws` | WebSocket-yhteys |
|
||||
| `GET /admin` | Admin-dashboard |
|
||||
| `GET /api/sessions` | Node-sessiot (JSON) |
|
||||
| `GET /api/pairs` | Tokenisointitulokset (JSON) |
|
||||
| `GET /api/stats` | Yhteenvetotilastot (JSON) |
|
||||
|
||||
*(Huomio Linux-käyttäjille: Firefox saattaa edellyttää MOZ_ENABLE_WAYLAND ympäristömuuttujaa).*
|
||||
## Tietoturva
|
||||
|
||||
---
|
||||
- **Origin-tarkistus** — vain `https://kipina.studio` ja `localhost:3000`
|
||||
- **IP-rajoitus** — max 4 WS-yhteyttä per IP, X-Forwarded-For -tuki
|
||||
- **Viestivalidointi** — pakollinen `type`, sallitut tyypit, kenttäkohtaiset rajat
|
||||
- **Viestikoko** — max 16 KB per WebSocket-viesti
|
||||
- **Caddy** — automaattinen TLS (Let's Encrypt)
|
||||
|
||||
### Apple Safari (Mac)
|
||||
## Tuotanto-deploy
|
||||
|
||||
Apple käyttää konepellin alla vahvaa omaa Metal-rajapintaansa ja tukee WebGPU:ta uudemmissa Safari-versioissa kehittäjäasetusten takaa:
|
||||
1. Varmista ensin Safarin asetuksista (Preferences -> Advanced) , että ruutu on ruksittu kohdasta `"Show Develop menu in menu bar"`.
|
||||
2. Valitse yläpalkista avautuva **Develop**-valikko -> **Feature Flags**.
|
||||
3. Etsi listalta **WebGPU** ja laita siihen täppä pelastamaan tilanne.
|
||||
4. Päivitä Dashboard-sivu.
|
||||
```bash
|
||||
# Buildaa lokaalisti, siirrä palvelimelle, käynnistä
|
||||
./deploy.sh
|
||||
|
||||
# Manuaalisesti palvelimella
|
||||
docker compose -f docker-compose.prod.yml down && docker compose -f docker-compose.prod.yml up -d
|
||||
```
|
||||
|
||||
## Tiedossa olevat rajoitukset
|
||||
|
||||
- LLM-inferenssi on **greedy** (argmax) — ei temperature/top-p samplingia Wasmissa (Candlen `SoftmaxLastDim` bugi)
|
||||
- Qwen selaimessa: ~0.4 tok/s CPU — käyttökelpoinen demona mutta ei tuotantoon
|
||||
- Hub broadcastaa kaikki viestit kaikille — ei kohdennettu reititystä
|
||||
- CUDA-tuki vaatii `nvidia-cuda-toolkit` asennuksen + Cargo.toml featuren
|
||||
|
||||
## Lisenssi
|
||||
|
||||
Kipinä Technologies Oy — sisäinen projekti.
|
||||
|
||||
@@ -20,7 +20,7 @@ Kipinä Agentic Network on hajautettu tekoälylaskentaverkko, jossa selaimet ja
|
||||
## Kaksi tapaa osallistua verkkoon
|
||||
|
||||
### 1. Selainsolmu (Wasm + WebGPU)
|
||||
- Avaa `http://localhost:3000` selaimessa ja klikkaa "Liity laskentaverkkoon"
|
||||
- Avaa `http://localhost:3000` | `https://kipina.studio` selaimessa ja klikkaa "Liity laskentaverkkoon"
|
||||
- Selain tunnistaa automaattisesti WebGPU-tuen — jos ei löydy, käytetään CPU-fallbackia
|
||||
- Tokenizer ladataan HuggingFacesta ensimmäisellä kerralla ja tallennetaan IndexedDB:hen
|
||||
- GPU-kuormitusta voi säätää sliderilla (0–75 %)
|
||||
@@ -42,7 +42,7 @@ docker compose up
|
||||
docker compose --profile native up
|
||||
```
|
||||
|
||||
Dashboard avautuu osoitteessa http://localhost:3000
|
||||
Dashboard avautuu osoitteessa http://localhost:3000 | https://kipina.studio
|
||||
|
||||
### Ilman Dockeria
|
||||
|
||||
@@ -53,13 +53,14 @@ cd node && wasm-pack build --target web --out-dir ../static/pkg && cd ..
|
||||
# 2. Käynnistä hub (terminaali 1)
|
||||
cargo run -p hub
|
||||
|
||||
# 3. Avaa selain: http://localhost:3000
|
||||
# 3. Avaa selain: http://localhost:3000 | https://kipina.studio
|
||||
|
||||
# 4. Valinnainen: natiivi-node LLM-inferenssillä (terminaali 2)
|
||||
# Lataa Qwen2.5-0.5B automaattisesti HuggingFacesta (~990 MB, cachetetaan)
|
||||
# Release-moodissa ~11 tok/s CPU:lla (32 ydintä)
|
||||
CARGO_TARGET_DIR=target-native HUB_URL=ws://localhost:3000/ws ALLOCATED_GB=4 cargo run --release -p native-node
|
||||
|
||||
|
||||
# Tai yhdistä tuotantopalvelimeen:
|
||||
CARGO_TARGET_DIR=target-native HUB_URL=wss://kipina.studio/ws ALLOCATED_GB=4 cargo run --release -p native-node
|
||||
```
|
||||
@@ -77,15 +78,26 @@ sudo apt install nvidia-cuda-toolkit
|
||||
|
||||
# Aja — malli käyttää automaattisesti GPU:ta
|
||||
CARGO_TARGET_DIR=target-native HUB_URL=ws://localhost:3000/ws cargo run --release -p native-node
|
||||
|
||||
CARGO_TARGET_DIR=target-native HUB_URL=ws://kipina.studio/ws cargo run --release -p native-node
|
||||
```
|
||||
|
||||
## WebGPU-asetukset selaimessa
|
||||
|
||||
WebGPU ei ole oletuksena päällä kaikissa selaimissa. Jos "Liity laskentaverkkoon" -nappi käynnistää CPU-fallbackin vaikka koneessa on näytönohjain:
|
||||
|
||||
**Chrome / Brave (Linux + Wayland):**
|
||||
**Chrome / Brave (Linux APT/DEB):**
|
||||
```bash
|
||||
google-chrome --enable-unsafe-webgpu --enable-features=Vulkan --ignore-gpu-blocklist --use-angle=vulkan --ozone-platform=x11
|
||||
google-chrome --enable-unsafe-webgpu --enable-features=Vulkan --ignore-gpu-blocklist --use-angle=vulkan --ozone-platform=x11 https://kipina.studio
|
||||
|
||||
brave-browser --enable-unsafe-webgpu --enable-features=Vulkan --ignore-gpu-blocklist --use-angle=vulkan --ozone-platform=x11 https://kipina.studio
|
||||
```
|
||||
|
||||
**Chrome / Brave (Linux Flatpak):**
|
||||
```bash
|
||||
flatpak run com.google.Chrome --enable-unsafe-webgpu --enable-features=Vulkan --ignore-gpu-blocklist --use-angle=vulkan --ozone-platform=x11 https://kipina.studio
|
||||
|
||||
flatpak run com.brave.Browser --enable-unsafe-webgpu --enable-features=Vulkan --ignore-gpu-blocklist --use-angle=vulkan --ozone-platform=x11 https://kipina.studio
|
||||
```
|
||||
|
||||
**Chrome / Brave (Windows / Mac):**
|
||||
|
||||
@@ -25,7 +25,7 @@ const ALLOWED_ORIGINS: &[&str] = &[
|
||||
];
|
||||
|
||||
// Sallitut viestityyypit clientilta
|
||||
const ALLOWED_MSG_TYPES: &[&str] = &["auth", "result", "pair_done", "llm_chunk", "llm_done", "download_progress"];
|
||||
const ALLOWED_MSG_TYPES: &[&str] = &["auth", "result", "pair_done", "llm_chunk", "llm_done", "download_progress", "user_text"];
|
||||
|
||||
struct AppState {
|
||||
next_node_id: Mutex<u64>,
|
||||
@@ -304,7 +304,28 @@ async fn main() {
|
||||
});
|
||||
let _ = state_for_task.stats_tx.send(phi3_msg.to_string());
|
||||
|
||||
tracing::debug!("Tehtävät lähetetty: pair + smollm + qwen + phi3");
|
||||
// Coder-promptit — pieniä Python-tehtäviä
|
||||
let code_prompts = vec![
|
||||
"Write a Python function that checks if a number is prime.",
|
||||
"Write a Python function that reverses a string without using slicing.",
|
||||
"Write a Python function to find the factorial of a number using recursion.",
|
||||
"Write a Python function that returns the Fibonacci sequence up to n numbers.",
|
||||
"Write a Python function to check if a string is a palindrome.",
|
||||
"Write a Python function that sorts a list using bubble sort.",
|
||||
"Write a Python function to count the occurrences of each character in a string.",
|
||||
"Write a Python function that flattens a nested list.",
|
||||
"Write a Python function to find the greatest common divisor of two numbers.",
|
||||
"Write a Python function that converts Celsius to Fahrenheit.",
|
||||
];
|
||||
let code_idx = (rng_state as usize / 13) % code_prompts.len();
|
||||
let coder_msg = serde_json::json!({
|
||||
"type": "llm_prompt",
|
||||
"prompt": code_prompts[code_idx],
|
||||
"model": "qwen-coder",
|
||||
});
|
||||
let _ = state_for_task.stats_tx.send(coder_msg.to_string());
|
||||
|
||||
tracing::debug!("Tehtävät lähetetty: pair + smollm + qwen + phi3 + coder");
|
||||
}
|
||||
});
|
||||
|
||||
@@ -678,6 +699,36 @@ async fn handle_socket(socket: WebSocket, state: Arc<AppState>, ip: IpAddr) {
|
||||
}
|
||||
broadcast_stats(&state).await;
|
||||
}
|
||||
} else if msg_type == "user_text" {
|
||||
// Käyttäjän lähettämä teksti — broadcastataan pair_taskina ja llm_promptina
|
||||
let text = json.get("text").and_then(|v| v.as_str()).unwrap_or("").to_string();
|
||||
let task_type = json.get("task_type").and_then(|v| v.as_str()).unwrap_or("tokenize");
|
||||
if !text.is_empty() {
|
||||
tracing::info!("Solmu {} lähetti oman tekstin ({}): \"{}\"", node_id, task_type, &text[..text.len().min(80)]);
|
||||
match task_type {
|
||||
"tokenize" => {
|
||||
// Tokenisoidaan käyttäjän teksti EN-puolella, FI jätetään tyhjäksi
|
||||
let pair = serde_json::json!({
|
||||
"type": "pair_task",
|
||||
"en": text,
|
||||
"fi": text,
|
||||
"user_submitted": true,
|
||||
});
|
||||
let _ = state.stats_tx.send(pair.to_string());
|
||||
}
|
||||
_ => {
|
||||
// LLM-prompti
|
||||
for model in &["smollm-135m", "qwen-05b", "phi3-mini", "qwen-coder"] {
|
||||
let prompt = serde_json::json!({
|
||||
"type": "llm_prompt",
|
||||
"prompt": text,
|
||||
"model": model,
|
||||
});
|
||||
let _ = state.stats_tx.send(prompt.to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -9,6 +9,7 @@ use burn::backend::{Wgpu, NdArray};
|
||||
pub mod storage;
|
||||
pub mod smollm;
|
||||
pub mod qwen;
|
||||
pub mod qwen_coder;
|
||||
pub mod phi3;
|
||||
|
||||
#[macro_export]
|
||||
@@ -159,7 +160,7 @@ pub async fn start_agent_node(hub_url: String, has_webgpu: bool, device_info_jso
|
||||
HAS_WEBGPU.store(has_webgpu, Ordering::SeqCst);
|
||||
SELECTED_TASK.store(task_id, Ordering::SeqCst);
|
||||
let backend_name = if has_webgpu { "WebGPU" } else { "CPU (NdArray)" };
|
||||
let task_names = ["tokenize", "smollm-135m", "qwen-05b", "phi3-mini"];
|
||||
let task_names = ["tokenize", "smollm-135m", "qwen-05b", "phi3-mini", "qwen-coder-05b", "qwen-coder-3b"];
|
||||
let task_name = task_names.get(task_id as usize).unwrap_or(&"tokenize");
|
||||
console_log!("Kipinä Agent Node käynnistyy — backend: {} | tehtävä: {}", backend_name, task_name);
|
||||
|
||||
@@ -248,6 +249,21 @@ pub async fn start_agent_node(hub_url: String, has_webgpu: bool, device_info_jso
|
||||
});
|
||||
}
|
||||
}
|
||||
} else if msg.contains("llm_prompt") && (current_task == 4 || current_task == 5) {
|
||||
// Qwen2.5-Coder: 4 = 0.5B, 5 = 3B
|
||||
if LLM_BUSY.load(Ordering::SeqCst) {
|
||||
} else if let Ok(task) = serde_json::from_str::<serde_json::Value>(&msg) {
|
||||
let prompt = task.get("prompt").and_then(|v| v.as_str()).unwrap_or("").to_string();
|
||||
if !prompt.is_empty() {
|
||||
let use_3b = current_task == 5;
|
||||
LLM_BUSY.store(true, Ordering::SeqCst);
|
||||
let ws_for_async = ws_clone.clone();
|
||||
wasm_bindgen_futures::spawn_local(async move {
|
||||
qwen_coder::run_coder_inference(prompt, ws_for_async, use_3b).await;
|
||||
LLM_BUSY.store(false, Ordering::SeqCst);
|
||||
});
|
||||
}
|
||||
}
|
||||
} else if msg.contains("ai_task") {
|
||||
console_log!("Hub task vastaanotettu, ajetaan GPU:lla...");
|
||||
let ws_for_async = ws_clone.clone();
|
||||
|
||||
268
network-poc/node/src/qwen_coder.rs
Normal file
268
network-poc/node/src/qwen_coder.rs
Normal file
@@ -0,0 +1,268 @@
|
||||
use candle_core::{Device, Tensor, DType};
|
||||
use candle_nn::VarBuilder;
|
||||
use candle_transformers::models::qwen2::{Config as QwenConfig, ModelForCausalLM as QwenModel};
|
||||
use wasm_bindgen::JsCast;
|
||||
use std::cell::RefCell;
|
||||
use std::rc::Rc;
|
||||
use web_sys::WebSocket;
|
||||
|
||||
use crate::storage;
|
||||
|
||||
macro_rules! console_log {
|
||||
($($t:tt)*) => (web_sys::console::log_1(&format_args!($($t)*).to_string().into()))
|
||||
}
|
||||
|
||||
// 0.5B — nopea, sopii kaikille laitteille
|
||||
const MODEL_05B_URL: &str = "https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B-Instruct/resolve/main/model.safetensors";
|
||||
const TOKENIZER_05B_URL: &str = "https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B-Instruct/resolve/main/tokenizer.json";
|
||||
|
||||
// 3B — parempi laatu, vaatii enemmän muistia (~6 GB lataus, ~12 GB RAM)
|
||||
const MODEL_3B_PART1_URL: &str = "https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct/resolve/main/model-00001-of-00002.safetensors";
|
||||
const MODEL_3B_PART2_URL: &str = "https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct/resolve/main/model-00002-of-00002.safetensors";
|
||||
const TOKENIZER_3B_URL: &str = "https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct/resolve/main/tokenizer.json";
|
||||
|
||||
async fn ensure_cached(key: &str, url: &str, ws: &Rc<RefCell<WebSocket>>) -> Result<Vec<u8>, String> {
|
||||
if let Ok(Some(bytes)) = storage::load_from_idb(key).await {
|
||||
console_log!("[Coder] {} löytyi välimuistista ({} MB)", key, bytes.len() / 1024 / 1024);
|
||||
return Ok(bytes);
|
||||
}
|
||||
|
||||
console_log!("[Coder] Ladataan {}...", key);
|
||||
|
||||
let window = web_sys::window().unwrap();
|
||||
let resp_val = wasm_bindgen_futures::JsFuture::from(window.fetch_with_str(url))
|
||||
.await.map_err(|e| format!("Fetch: {:?}", e))?;
|
||||
let resp: web_sys::Response = resp_val.dyn_into().map_err(|_| "Ei Response".to_string())?;
|
||||
if !resp.ok() { return Err(format!("HTTP {}", resp.status())); }
|
||||
|
||||
let total_size: usize = resp.headers()
|
||||
.get("content-length").ok().flatten()
|
||||
.and_then(|s| s.parse().ok())
|
||||
.unwrap_or(0);
|
||||
|
||||
let body = resp.body().ok_or("Ei bodyä")?;
|
||||
let reader: web_sys::ReadableStreamDefaultReader = body.get_reader().dyn_into().map_err(|_| "Ei reader".to_string())?;
|
||||
|
||||
let mut data: Vec<u8> = Vec::with_capacity(total_size);
|
||||
let mut last_pct: u32 = 0;
|
||||
|
||||
loop {
|
||||
let chunk = wasm_bindgen_futures::JsFuture::from(reader.read())
|
||||
.await.map_err(|e| format!("Read: {:?}", e))?;
|
||||
let done = js_sys::Reflect::get(&chunk, &"done".into()).ok().and_then(|v| v.as_bool()).unwrap_or(true);
|
||||
if done { break; }
|
||||
let value = js_sys::Reflect::get(&chunk, &"value".into()).map_err(|_| "value puuttuu".to_string())?;
|
||||
let array = js_sys::Uint8Array::new(&value);
|
||||
let mut buf = vec![0u8; array.length() as usize];
|
||||
array.copy_to(&mut buf);
|
||||
data.extend_from_slice(&buf);
|
||||
|
||||
if total_size > 0 {
|
||||
let pct = ((data.len() as f64 / total_size as f64) * 100.0) as u32;
|
||||
if pct >= last_pct + 5 || pct == 100 {
|
||||
last_pct = pct;
|
||||
console_log!("[Coder] {} lataus: {}%", key, pct);
|
||||
let msg = serde_json::json!({ "type": "download_progress", "file": key, "pct": pct, "loaded_mb": data.len()/1024/1024, "total_mb": total_size/1024/1024 });
|
||||
let _ = ws.borrow().send_with_str(&msg.to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
console_log!("[Coder] Tallennetaan {} ({} MB)...", key, data.len() / 1024 / 1024);
|
||||
let _ = storage::save_to_idb(key, &data).await;
|
||||
console_log!("[Coder] {} tallennettu!", key);
|
||||
|
||||
Ok(data)
|
||||
}
|
||||
|
||||
/// use_3b: false = 0.5B (nopea), true = 3B (laadukas)
|
||||
pub async fn run_coder_inference(prompt: String, ws: Rc<RefCell<WebSocket>>, use_3b: bool) {
|
||||
let perf = web_sys::window().unwrap().performance().unwrap();
|
||||
let size_label = if use_3b { "3B" } else { "0.5B" };
|
||||
|
||||
// Tokenizer (sama molemmille)
|
||||
let tok_url = if use_3b { TOKENIZER_3B_URL } else { TOKENIZER_05B_URL };
|
||||
let tok_key = if use_3b { "coder3b-tokenizer.json" } else { "coder05b-tokenizer.json" };
|
||||
let tok_bytes = match ensure_cached(tok_key, tok_url, &ws).await {
|
||||
Ok(b) => b,
|
||||
Err(e) => { console_log!("[Coder] Tokenizer-virhe: {}", e); return; }
|
||||
};
|
||||
let tokenizer = match tokenizers::Tokenizer::from_bytes(&tok_bytes) {
|
||||
Ok(t) => t,
|
||||
Err(e) => { console_log!("[Coder] Tokenizer-parsinta: {}", e); return; }
|
||||
};
|
||||
|
||||
// Mallin painot
|
||||
let device = Device::Cpu;
|
||||
let dtype = DType::F32;
|
||||
|
||||
let tensors = if use_3b {
|
||||
// 3B: kaksi osaa
|
||||
let part1 = match ensure_cached("coder3b-model-part1.safetensors", MODEL_3B_PART1_URL, &ws).await {
|
||||
Ok(b) => b,
|
||||
Err(e) => { console_log!("[Coder] Malli osa 1 virhe: {}", e); return; }
|
||||
};
|
||||
let part2 = match ensure_cached("coder3b-model-part2.safetensors", MODEL_3B_PART2_URL, &ws).await {
|
||||
Ok(b) => b,
|
||||
Err(e) => { console_log!("[Coder] Malli osa 2 virhe: {}", e); return; }
|
||||
};
|
||||
console_log!("[Coder] Rakennetaan 3B-mallia...");
|
||||
let mut all_tensors = candle_core::safetensors::load_buffer(&part1, &device)
|
||||
.map_err(|e| format!("Part1: {}", e)).unwrap();
|
||||
let tensors2 = candle_core::safetensors::load_buffer(&part2, &device)
|
||||
.map_err(|e| format!("Part2: {}", e)).unwrap();
|
||||
all_tensors.extend(tensors2);
|
||||
all_tensors
|
||||
} else {
|
||||
// 0.5B: yksi osa
|
||||
let model_bytes = match ensure_cached("coder05b-model.safetensors", MODEL_05B_URL, &ws).await {
|
||||
Ok(b) => b,
|
||||
Err(e) => { console_log!("[Coder] Malli-virhe: {}", e); return; }
|
||||
};
|
||||
console_log!("[Coder] Rakennetaan 0.5B-mallia...");
|
||||
match candle_core::safetensors::load_buffer(&model_bytes, &device) {
|
||||
Ok(t) => t,
|
||||
Err(e) => { console_log!("[Coder] Safetensors: {}", e); return; }
|
||||
}
|
||||
};
|
||||
|
||||
let start_load = perf.now();
|
||||
let vb = VarBuilder::from_tensors(tensors, dtype, &device);
|
||||
|
||||
let config = if use_3b {
|
||||
QwenConfig {
|
||||
vocab_size: 151936,
|
||||
hidden_size: 2048,
|
||||
intermediate_size: 11008,
|
||||
num_hidden_layers: 36,
|
||||
num_attention_heads: 16,
|
||||
num_key_value_heads: 2,
|
||||
max_position_embeddings: 32768,
|
||||
sliding_window: 32768,
|
||||
max_window_layers: 36,
|
||||
tie_word_embeddings: true,
|
||||
rope_theta: 1000000.0,
|
||||
rms_norm_eps: 1e-6,
|
||||
use_sliding_window: false,
|
||||
hidden_act: candle_nn::Activation::Silu,
|
||||
}
|
||||
} else {
|
||||
QwenConfig {
|
||||
vocab_size: 151936,
|
||||
hidden_size: 896,
|
||||
intermediate_size: 4864,
|
||||
num_hidden_layers: 24,
|
||||
num_attention_heads: 14,
|
||||
num_key_value_heads: 2,
|
||||
max_position_embeddings: 32768,
|
||||
sliding_window: 32768,
|
||||
max_window_layers: 21,
|
||||
tie_word_embeddings: true,
|
||||
rope_theta: 1000000.0,
|
||||
rms_norm_eps: 1e-6,
|
||||
use_sliding_window: false,
|
||||
hidden_act: candle_nn::Activation::Silu,
|
||||
}
|
||||
};
|
||||
|
||||
let mut model = match QwenModel::new(&config, vb) {
|
||||
Ok(m) => m,
|
||||
Err(e) => { console_log!("[Coder] Mallin lataus: {}", e); return; }
|
||||
};
|
||||
|
||||
let load_time = perf.now() - start_load;
|
||||
console_log!("[Coder] Malli ladattu ({:.0}ms). Generoidaan...", load_time);
|
||||
|
||||
// Muotoillaan chat-template
|
||||
let formatted = format!("<|im_start|>system\nYou are a Python coding assistant. Write only code, no explanations.<|im_end|>\n<|im_start|>user\n{}<|im_end|>\n<|im_start|>assistant\n", prompt);
|
||||
|
||||
let encoding = match tokenizer.encode(formatted.as_str(), true) {
|
||||
Ok(e) => e,
|
||||
Err(e) => { console_log!("[Coder] Tokenisointivirhe: {}", e); return; }
|
||||
};
|
||||
let input_ids: Vec<u32> = encoding.get_ids().to_vec();
|
||||
let input_len = input_ids.len();
|
||||
console_log!("[Coder] Syöte: {} tokenia", input_len);
|
||||
|
||||
let start_gen = perf.now();
|
||||
let max_new_tokens = 128; // Koodille enemmän tokeneita
|
||||
let mut generated_text = String::new();
|
||||
let mut tokens_generated: usize = 0;
|
||||
let eos_token = 151645u32;
|
||||
|
||||
// Prefill
|
||||
let input = match Tensor::new(input_ids.as_slice(), &device).and_then(|t| t.unsqueeze(0)) {
|
||||
Ok(t) => t,
|
||||
Err(e) => { console_log!("[Coder] Tensor: {}", e); return; }
|
||||
};
|
||||
let logits = match model.forward(&input, 0) {
|
||||
Ok(l) => l,
|
||||
Err(e) => { console_log!("[Coder] Forward (prefill): {}", e); return; }
|
||||
};
|
||||
|
||||
let logits = logits.squeeze(0).unwrap();
|
||||
let logits = if logits.dims().len() == 2 {
|
||||
logits.get(logits.dim(0).unwrap() - 1).unwrap()
|
||||
} else {
|
||||
logits
|
||||
};
|
||||
let mut next_token = logits.argmax(0).unwrap().to_vec0::<u32>().unwrap();
|
||||
|
||||
if next_token != eos_token {
|
||||
if let Ok(text) = tokenizer.decode(&[next_token], true) {
|
||||
generated_text.push_str(&text);
|
||||
let chunk = serde_json::json!({ "type": "llm_chunk", "token": text, "prompt": prompt, "model": "Qwen2.5-Coder" });
|
||||
let _ = ws.borrow().send_with_str(&chunk.to_string());
|
||||
}
|
||||
tokens_generated += 1;
|
||||
}
|
||||
|
||||
// Autoregressive
|
||||
let mut pos = input_len;
|
||||
for _ in 1..max_new_tokens {
|
||||
if next_token == eos_token { break; }
|
||||
|
||||
let input = match Tensor::new(&[next_token], &device).and_then(|t| t.unsqueeze(0)) {
|
||||
Ok(t) => t,
|
||||
Err(e) => { console_log!("[Coder] Tensor: {}", e); break; }
|
||||
};
|
||||
let logits = match model.forward(&input, pos) {
|
||||
Ok(l) => l,
|
||||
Err(e) => { console_log!("[Coder] Forward pos {}: {}", pos, e); break; }
|
||||
};
|
||||
|
||||
let logits = logits.squeeze(0).unwrap();
|
||||
let logits = if logits.dims().len() == 2 {
|
||||
logits.get(logits.dim(0).unwrap() - 1).unwrap()
|
||||
} else {
|
||||
logits
|
||||
};
|
||||
next_token = logits.argmax(0).unwrap().to_vec0::<u32>().unwrap();
|
||||
pos += 1;
|
||||
|
||||
if next_token == eos_token { break; }
|
||||
|
||||
if let Ok(text) = tokenizer.decode(&[next_token], true) {
|
||||
generated_text.push_str(&text);
|
||||
let chunk = serde_json::json!({ "type": "llm_chunk", "token": text, "prompt": prompt, "model": "Qwen2.5-Coder" });
|
||||
let _ = ws.borrow().send_with_str(&chunk.to_string());
|
||||
}
|
||||
tokens_generated += 1;
|
||||
}
|
||||
|
||||
let gen_time = perf.now() - start_gen;
|
||||
let tokens_per_sec = if gen_time > 0.0 { (tokens_generated as f64 / gen_time) * 1000.0 } else { 0.0 };
|
||||
console_log!("[Coder] {} tokenia | {:.0}ms | {:.1} tok/s", tokens_generated, gen_time, tokens_per_sec);
|
||||
|
||||
let done = serde_json::json!({
|
||||
"type": "llm_done",
|
||||
"prompt": prompt,
|
||||
"model": format!("Qwen2.5-Coder-{}-Instruct", size_label),
|
||||
"response": generated_text,
|
||||
"tokens_generated": tokens_generated,
|
||||
"duration_ms": (gen_time * 100.0).round() / 100.0,
|
||||
"tokens_per_sec": (tokens_per_sec * 10.0).round() / 10.0,
|
||||
"load_time_ms": (load_time * 100.0).round() / 100.0,
|
||||
});
|
||||
let _ = ws.borrow().send_with_str(&done.to_string());
|
||||
}
|
||||
@@ -97,6 +97,69 @@
|
||||
h1 span { color: var(--accent-color); }
|
||||
.sub { color: #8b949e; margin-bottom: 25px; }
|
||||
|
||||
.main-tabs {
|
||||
display: flex;
|
||||
gap: 4px;
|
||||
margin-bottom: 20px;
|
||||
border-bottom: 2px solid var(--border-color);
|
||||
padding-bottom: 0;
|
||||
}
|
||||
.main-tab {
|
||||
padding: 10px 20px;
|
||||
font-size: 15px;
|
||||
font-weight: 500;
|
||||
color: #8b949e;
|
||||
cursor: pointer;
|
||||
border-bottom: 2px solid transparent;
|
||||
margin-bottom: -2px;
|
||||
transition: color 0.2s, border-color 0.2s;
|
||||
}
|
||||
.main-tab:hover { color: var(--text-color); }
|
||||
.main-tab.active { color: var(--accent-color); border-bottom-color: var(--accent-color); }
|
||||
.main-panel { display: none; }
|
||||
.main-panel.active { display: block; }
|
||||
|
||||
.code-output {
|
||||
font-family: 'Courier New', Courier, monospace;
|
||||
background: #010409;
|
||||
border: 1px solid var(--border-color);
|
||||
border-radius: 6px;
|
||||
padding: 14px;
|
||||
font-size: 13px;
|
||||
line-height: 1.6;
|
||||
color: var(--success-color);
|
||||
white-space: pre-wrap;
|
||||
overflow-x: auto;
|
||||
max-height: 400px;
|
||||
overflow-y: auto;
|
||||
}
|
||||
.code-output .keyword { color: #ff7b72; }
|
||||
.code-output .string { color: #a5d6ff; }
|
||||
.code-output .comment { color: #8b949e; }
|
||||
|
||||
.code-task-card {
|
||||
background: #0d1117;
|
||||
border: 1px solid var(--border-color);
|
||||
border-radius: 6px;
|
||||
padding: 14px;
|
||||
margin-bottom: 12px;
|
||||
}
|
||||
.code-task-card .prompt { color: #d29922; font-size: 14px; margin-bottom: 10px; }
|
||||
.code-task-card .meta { color: #8b949e; font-size: 12px; margin-top: 10px; }
|
||||
|
||||
.code-step {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
font-size: 13px;
|
||||
color: #8b949e;
|
||||
padding: 6px 0;
|
||||
}
|
||||
.code-step.active { color: var(--accent-color); }
|
||||
.code-step.done { color: var(--success-color); }
|
||||
.code-step.error { color: #f85149; }
|
||||
.step-icon { font-size: 16px; width: 20px; text-align: center; }
|
||||
|
||||
.status-box {
|
||||
font-family: 'Courier New', Courier, monospace;
|
||||
background-color: #010409;
|
||||
@@ -310,6 +373,15 @@
|
||||
<h1>Kipinä <span>Agent Dashboard</span></h1>
|
||||
<p class="sub">Hajautettu WebGPU Laskentaverkko · <span id="hub-version" style="color:#58a6ff">-</span></p>
|
||||
|
||||
<!-- Päävälilehdet -->
|
||||
<div class="main-tabs">
|
||||
<div class="main-tab active" onclick="switchMainTab('network')">Laskentaverkko</div>
|
||||
<div class="main-tab" onclick="switchMainTab('codelab')">Koodilaboratorio</div>
|
||||
</div>
|
||||
|
||||
<!-- PANEELI 1: Laskentaverkko -->
|
||||
<div id="panel-network" class="main-panel active">
|
||||
|
||||
<!-- Global Cluster Statistics (UI) -->
|
||||
<div class="dashboard-panel">
|
||||
<div class="stat-box" style="border-right: 1px solid #30363d;">
|
||||
@@ -413,7 +485,7 @@
|
||||
<div style="background:#0d1117;border:1px solid var(--border-color);border-radius:6px;padding:16px;margin-bottom:16px">
|
||||
<div style="display:flex;justify-content:space-between;align-items:center;margin-bottom:12px">
|
||||
<span style="font-weight:600;font-size:15px">Resurssien hallinta</span>
|
||||
<span id="node-status" style="font-size:12px;color:var(--success-color)">Aktiivinen</span>
|
||||
<span id="node-status" style="font-size:12px;color:#8b949e">Ei yhdistetty</span>
|
||||
</div>
|
||||
|
||||
<!-- Kuormitussäädin -->
|
||||
@@ -449,6 +521,14 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="user-input-box" class="hidden" style="background:#0d1117;border:1px solid var(--border-color);border-radius:6px;padding:12px;margin-bottom:12px">
|
||||
<div style="font-size:13px;color:#8b949e;margin-bottom:8px">Kokeile omaa tekstiä:</div>
|
||||
<div style="display:flex;gap:8px">
|
||||
<input type="text" id="user-text" placeholder="Kirjoita teksti tokenisoitavaksi tai promptiksi..." style="flex:1;background:var(--panel-bg);border:1px solid var(--border-color);border-radius:4px;padding:8px 12px;color:var(--text-color);font-size:14px;outline:none">
|
||||
<button id="send-btn" style="background:#238636;color:#fff;border:1px solid rgba(240,246,252,0.1);border-radius:4px;padding:8px 16px;font-size:14px;cursor:pointer;white-space:nowrap">Tokenisoi</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="chat-box" class="chat-box hidden">
|
||||
<div style="color: #8b949e; text-align: center; margin-top: 80px;">Odotetaan Generointitehtäviä Hubilta...</div>
|
||||
</div>
|
||||
@@ -457,11 +537,127 @@
|
||||
<p>> Odotetaan uusia tehtäviä Hubulta...</p>
|
||||
</div>
|
||||
</div>
|
||||
</div><!-- /panel-network -->
|
||||
|
||||
<!-- PANEELI 2: Koodilaboratorio -->
|
||||
<div id="panel-codelab" class="main-panel">
|
||||
<div style="background:#0d1117;border:1px solid var(--border-color);border-radius:6px;padding:16px;margin-bottom:16px">
|
||||
<div style="display:flex;justify-content:space-between;align-items:center;margin-bottom:8px">
|
||||
<span style="font-weight:600;font-size:15px">Qwen2.5-Coder-0.5B-Instruct</span>
|
||||
<span id="coder-status" style="font-size:12px;color:#8b949e">Ei yhdistetty</span>
|
||||
</div>
|
||||
<p style="font-size:12px;color:#8b949e;line-height:1.5;margin-bottom:12px">
|
||||
Code-specialized language model trained on 5.5T tokens of source code.
|
||||
Generates Python code in your browser via WebAssembly. Choose model size and write your own prompt.
|
||||
</p>
|
||||
<!-- Model size selector -->
|
||||
<div style="display:flex;gap:8px;margin-bottom:10px">
|
||||
<label style="flex:1;display:flex;align-items:center;gap:6px;background:var(--panel-bg);border:2px solid var(--accent-color);border-radius:4px;padding:8px 12px;cursor:pointer;font-size:13px" id="coder-opt-05b">
|
||||
<input type="radio" name="coder-size" value="05b" checked style="accent-color:var(--accent-color)">
|
||||
<div>
|
||||
<strong style="color:var(--text-color)">0.5B</strong>
|
||||
<span style="color:#8b949e"> — 990 MB, ~0.4 tok/s</span>
|
||||
</div>
|
||||
</label>
|
||||
<label style="flex:1;display:flex;align-items:center;gap:6px;background:var(--panel-bg);border:2px solid var(--border-color);border-radius:4px;padding:8px 12px;cursor:pointer;font-size:13px" id="coder-opt-3b">
|
||||
<input type="radio" name="coder-size" value="3b" style="accent-color:var(--accent-color)">
|
||||
<div>
|
||||
<strong style="color:var(--text-color)">3B</strong>
|
||||
<span style="color:#8b949e"> — 6.2 GB, better quality, slower</span>
|
||||
</div>
|
||||
</label>
|
||||
</div>
|
||||
<div style="display:flex;gap:8px">
|
||||
<input type="text" id="code-input" placeholder="e.g. Write a Python function that checks if a number is prime" style="flex:1;background:var(--panel-bg);border:1px solid var(--border-color);border-radius:4px;padding:8px 12px;color:var(--text-color);font-size:14px;outline:none">
|
||||
<button id="code-send-btn" style="background:#238636;color:#fff;border:1px solid rgba(240,246,252,0.1);border-radius:4px;padding:8px 16px;font-size:14px;cursor:pointer">Generate</button>
|
||||
</div>
|
||||
<div id="code-loading" style="display:none;margin-top:8px;font-size:12px;color:#d29922">Starting Coder model...</div>
|
||||
</div>
|
||||
|
||||
<!-- Koodilaboratorion metriikat -->
|
||||
<div style="display:grid;grid-template-columns:repeat(3,1fr);gap:8px;margin-bottom:16px">
|
||||
<div class="metric-card">
|
||||
<div class="metric-val" id="code-m-tasks">0</div>
|
||||
<div class="metric-label">Tehtäviä</div>
|
||||
</div>
|
||||
<div class="metric-card">
|
||||
<div class="metric-val" id="code-m-tokens">0</div>
|
||||
<div class="metric-label">Tokeneita</div>
|
||||
</div>
|
||||
<div class="metric-card">
|
||||
<div class="metric-val" id="code-m-speed">-</div>
|
||||
<div class="metric-label">tok/s</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Latausvaiheet -->
|
||||
<div id="code-pipeline" style="background:#0d1117;border:1px solid var(--border-color);border-radius:6px;padding:16px;margin-bottom:16px;display:none">
|
||||
<div style="font-size:13px;font-weight:600;margin-bottom:12px">Valmistautuminen</div>
|
||||
<div id="code-steps" style="display:flex;flex-direction:column;gap:8px">
|
||||
<div class="code-step" id="step-wasm">
|
||||
<span class="step-icon">◯</span>
|
||||
<span>WebAssembly-ytimen lataus</span>
|
||||
</div>
|
||||
<div class="code-step" id="step-tokenizer">
|
||||
<span class="step-icon">◯</span>
|
||||
<span>Tokenizer (7 MB)</span>
|
||||
</div>
|
||||
<div class="code-step" id="step-model">
|
||||
<span class="step-icon">◯</span>
|
||||
<span>Qwen2.5-Coder-0.5B painot (990 MB)</span>
|
||||
<span id="step-model-pct" style="color:var(--accent-color);margin-left:auto;font-size:12px"></span>
|
||||
</div>
|
||||
<div class="code-step" id="step-build">
|
||||
<span class="step-icon">◯</span>
|
||||
<span>Mallin rakentaminen muistiin</span>
|
||||
</div>
|
||||
<div class="code-step" id="step-ready">
|
||||
<span class="step-icon">◯</span>
|
||||
<span>Valmis generoimaan</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Kooditulokset -->
|
||||
<div id="code-results" style="display:flex;flex-direction:column;gap:12px">
|
||||
<div data-placeholder style="color:#8b949e;text-align:center;padding:40px">Kirjoita ohjelmointitehtävä ja paina Koodaa</div>
|
||||
</div>
|
||||
</div><!-- /panel-codelab -->
|
||||
|
||||
</div>
|
||||
|
||||
<script type="module">
|
||||
import init, { start_agent_node, set_gpu_load } from './pkg/node.js';
|
||||
|
||||
// Päävälilehtien vaihto
|
||||
window.switchMainTab = function(tab) {
|
||||
document.querySelectorAll('.main-panel').forEach(p => p.classList.remove('active'));
|
||||
document.querySelectorAll('.main-tab').forEach(t => t.classList.remove('active'));
|
||||
document.getElementById('panel-' + tab).classList.add('active');
|
||||
event.target.classList.add('active');
|
||||
};
|
||||
|
||||
// Koodilaboratorion tila
|
||||
const codeMetrics = { tasks: 0, tokens: 0, lastSpeed: 0 };
|
||||
let coderJoined = false;
|
||||
let coderSize = '05b'; // '05b' tai '3b'
|
||||
|
||||
// Mallivalinnan radio-napit
|
||||
document.querySelectorAll('input[name="coder-size"]').forEach(radio => {
|
||||
radio.addEventListener('change', (e) => {
|
||||
coderSize = e.target.value;
|
||||
// Visuaalinen korostus
|
||||
document.getElementById('coder-opt-05b').style.borderColor = coderSize === '05b' ? 'var(--accent-color)' : 'var(--border-color)';
|
||||
document.getElementById('coder-opt-3b').style.borderColor = coderSize === '3b' ? 'var(--accent-color)' : 'var(--border-color)';
|
||||
// Jos jo liittynyt, pitää liittyä uudelleen toisella mallilla
|
||||
if (coderJoined) {
|
||||
coderJoined = false;
|
||||
document.getElementById('coder-status').textContent = 'Model changed — rejoin on next generate';
|
||||
document.getElementById('coder-status').style.color = '#d29922';
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
const btn = document.getElementById('start-btn');
|
||||
const logBox = document.getElementById('log-box');
|
||||
const loadSlider = document.getElementById('gpu-load');
|
||||
@@ -506,6 +702,20 @@
|
||||
}
|
||||
setInterval(updateMetrics, 1000);
|
||||
|
||||
// Laskentaverkko: status Connected (keltainen) ↔ Computing (vihreä)
|
||||
let computingTimer = null;
|
||||
function flashComputing() {
|
||||
const el = document.getElementById('node-status');
|
||||
if (!el || !window.wasm_active) return;
|
||||
el.textContent = 'Computing';
|
||||
el.style.color = 'var(--success-color)';
|
||||
clearTimeout(computingTimer);
|
||||
computingTimer = setTimeout(() => {
|
||||
el.textContent = 'Connected';
|
||||
el.style.color = '#d29922';
|
||||
}, 3000);
|
||||
}
|
||||
|
||||
// Ylikirjoitetaan console.log uppoamaan lokilaatikkoon
|
||||
const originalLog = console.log;
|
||||
console.log = function(...args) {
|
||||
@@ -546,6 +756,28 @@
|
||||
}
|
||||
});
|
||||
|
||||
// Käyttäjän oma tekstisyöte
|
||||
const userInput = document.getElementById('user-text');
|
||||
const sendBtn = document.getElementById('send-btn');
|
||||
|
||||
function sendUserText() {
|
||||
const text = userInput.value.trim();
|
||||
if (!text || !uiSocket || uiSocket.readyState !== 1) return;
|
||||
const msg = JSON.stringify({
|
||||
type: 'user_text',
|
||||
text: text,
|
||||
task_type: selectedTask,
|
||||
});
|
||||
uiSocket.send(msg);
|
||||
userInput.value = '';
|
||||
console.log(`Lähetetty: "${text}" (${selectedTask})`);
|
||||
}
|
||||
|
||||
sendBtn?.addEventListener('click', sendUserText);
|
||||
userInput?.addEventListener('keydown', (e) => {
|
||||
if (e.key === 'Enter') sendUserText();
|
||||
});
|
||||
|
||||
// Kytkemme sivuston UI-puolen (JS) omaan passiiviseen WebSocket-kuuntelijaan.
|
||||
const uiSocket = new WebSocket(`${window.location.protocol === 'https:' ? 'wss:' : 'ws:'}//${window.location.host}/ws`);
|
||||
uiSocket.onmessage = (event) => {
|
||||
@@ -608,6 +840,7 @@
|
||||
metrics.totalTokens += (en.token_count || 0) + (fi.token_count || 0);
|
||||
metrics.totalTimeMs += ms;
|
||||
updateMetrics();
|
||||
flashComputing();
|
||||
|
||||
// Lokiboksiin yhteenveto
|
||||
console.log(`EN: ${en.token_count} tokenia (${(en.chars_per_token||0).toFixed(2)} m/t) vs FI: ${fi.token_count} tokenia (${(fi.chars_per_token||0).toFixed(2)} m/t) | ylikustannus: ${overhead}% | ${typeof ms === 'number' ? ms.toFixed(2) : ms}ms`);
|
||||
@@ -688,8 +921,8 @@
|
||||
<div style="font-size:13px;color:#8b949e;margin-bottom:6px">
|
||||
Prompt: <span style="color:#d29922">"${data.prompt || ''}"</span>
|
||||
</div>
|
||||
<div style="font-size:14px;color:var(--text-color);line-height:1.5">
|
||||
${data.response || '<em>tyhjä vastaus</em>'}
|
||||
<div style="font-size:14px;color:var(--text-color);line-height:1.5;${(model.includes('Coder') || (data.response||'').includes('def ')) ? 'font-family:Courier New,monospace;background:#010409;padding:10px;border-radius:4px;white-space:pre-wrap;font-size:12px' : ''}">
|
||||
${(data.response || '<em>tyhjä vastaus</em>').replace(/</g, '<').replace(/>/g, '>')}
|
||||
</div>
|
||||
<div style="margin-top:8px;font-size:12px;color:#8b949e">
|
||||
${tokGen} tokenia generoitu | malli ladattu: ${typeof loadMs === 'number' ? loadMs.toFixed(0) : loadMs}ms
|
||||
@@ -701,6 +934,7 @@
|
||||
metrics.tasks++;
|
||||
metrics.totalTokens += tokGen;
|
||||
metrics.totalTimeMs += durMs;
|
||||
flashComputing();
|
||||
updateMetrics();
|
||||
|
||||
console.log(`[${model}] ${tokGen} tokenia | ${typeof durMs === 'number' ? durMs.toFixed(0) : durMs}ms | ${tokS} tok/s | "${(data.response || '').substring(0, 60)}..."`);
|
||||
@@ -812,26 +1046,210 @@
|
||||
|
||||
document.getElementById('initial-state').classList.add('hidden');
|
||||
document.getElementById('active-state').classList.remove('hidden');
|
||||
document.getElementById('user-input-box').classList.remove('hidden');
|
||||
btn.style.display = 'none';
|
||||
|
||||
// Nappin teksti ja placeholder tehtävän mukaan
|
||||
const sendBtnEl = document.getElementById('send-btn');
|
||||
if (selectedTask === 'tokenize') {
|
||||
sendBtnEl.textContent = 'Tokenisoi';
|
||||
document.getElementById('user-text').placeholder = 'Kirjoita teksti tokenisoitavaksi...';
|
||||
} else if (selectedTask === 'qwen-coder') {
|
||||
sendBtnEl.textContent = 'Koodaa';
|
||||
document.getElementById('user-text').placeholder = 'Kuvaile Python-ohjelmointitehtävä...';
|
||||
} else {
|
||||
sendBtnEl.textContent = 'Generoi';
|
||||
document.getElementById('user-text').placeholder = 'Kirjoita prompti kielimallille...';
|
||||
}
|
||||
|
||||
try {
|
||||
console.log("Ladataan Burn Wasm -binääriä...");
|
||||
await init();
|
||||
window.wasm_active = true;
|
||||
metrics.startTime = Date.now();
|
||||
|
||||
// Asetetaan Connected-tila (keltainen) — vihreäksi vasta kun laskentaa tapahtuu
|
||||
const nodeStatusEl = document.getElementById('node-status');
|
||||
nodeStatusEl.textContent = 'Connected';
|
||||
nodeStatusEl.style.color = '#d29922';
|
||||
|
||||
// Varmistetaan, että Wasm saa nykyisen sliderin arvon heti kärkeen
|
||||
set_gpu_load(parseInt(loadSlider.value));
|
||||
|
||||
// WebAssembly yhdistää oikeaksi Agent Nodeksi
|
||||
const wsUrl = `${window.location.protocol === 'https:' ? 'wss:' : 'ws:'}//${window.location.host}/ws`;
|
||||
const taskIds = {'tokenize': 0, 'smollm-135m': 1, 'qwen-05b': 2, 'phi3-mini': 3};
|
||||
const taskIds = {'tokenize': 0, 'smollm-135m': 1, 'qwen-05b': 2, 'phi3-mini': 3, 'qwen-coder-05b': 4, 'qwen-coder-3b': 5};
|
||||
const taskId = taskIds[selectedTask] || 0;
|
||||
await start_agent_node(wsUrl, hasWebGPU, JSON.stringify(deviceInfo), taskId);
|
||||
} catch(e) {
|
||||
console.log("Virhe GPU-käynnistyksessä: " + e);
|
||||
}
|
||||
});
|
||||
|
||||
// === Koodilaboratorio ===
|
||||
const codeInput = document.getElementById('code-input');
|
||||
const codeSendBtn = document.getElementById('code-send-btn');
|
||||
const codeResults = document.getElementById('code-results');
|
||||
const codeLoading = document.getElementById('code-loading');
|
||||
let coderWsReady = false;
|
||||
let coderWs = null; // Erillinen WS coder-nodelle
|
||||
let pendingCodePrompt = null;
|
||||
|
||||
function addCodeResult(data) {
|
||||
const model = data.model || 'Coder';
|
||||
const tokGen = data.tokens_generated || 0;
|
||||
const durMs = data.duration_ms || 0;
|
||||
const tokS = data.tokens_per_sec || 0;
|
||||
const response = (data.response || '').replace(/</g, '<').replace(/>/g, '>');
|
||||
|
||||
codeMetrics.tasks++;
|
||||
codeMetrics.tokens += tokGen;
|
||||
codeMetrics.lastSpeed = tokS;
|
||||
document.getElementById('code-m-tasks').textContent = codeMetrics.tasks;
|
||||
document.getElementById('code-m-tokens').textContent = codeMetrics.tokens.toLocaleString('fi-FI');
|
||||
document.getElementById('code-m-speed').textContent = tokS + ' tok/s';
|
||||
|
||||
if (codeResults.querySelector('[data-placeholder]')) {
|
||||
codeResults.innerHTML = '';
|
||||
}
|
||||
codeLoading.style.display = 'none';
|
||||
codeSendBtn.disabled = false;
|
||||
codeSendBtn.textContent = 'Generate';
|
||||
|
||||
const card = document.createElement('div');
|
||||
card.className = 'code-task-card';
|
||||
card.innerHTML = `
|
||||
<div class="prompt">${data.prompt || ''}</div>
|
||||
<div class="code-output">${response}</div>
|
||||
<div class="meta">
|
||||
${model} · ${tokGen} tokenia · ${typeof durMs === 'number' ? durMs.toFixed(0) : durMs}ms · ${tokS} tok/s
|
||||
</div>`;
|
||||
codeResults.insertBefore(card, codeResults.firstChild);
|
||||
if (codeResults.children.length > 10) codeResults.removeChild(codeResults.lastChild);
|
||||
}
|
||||
|
||||
// Kuuntele coder-tuloksia UI WebSocketista
|
||||
uiSocket.addEventListener('message', (event) => {
|
||||
try {
|
||||
const data = JSON.parse(event.data);
|
||||
if (data.type === 'llm_done' && (data.model || '').includes('Coder')) {
|
||||
addCodeResult(data);
|
||||
}
|
||||
} catch(e) {}
|
||||
});
|
||||
|
||||
// Pipeline-vaiheiden päivitys
|
||||
function setStep(id, state, extra) {
|
||||
const el = document.getElementById(id);
|
||||
if (!el) return;
|
||||
el.className = 'code-step ' + state;
|
||||
const icon = el.querySelector('.step-icon');
|
||||
if (state === 'active') icon.textContent = '\u25F7'; // spinning
|
||||
else if (state === 'done') icon.textContent = '\u2713';
|
||||
else if (state === 'error') icon.textContent = '\u2717';
|
||||
if (extra) {
|
||||
const pct = document.getElementById(id + '-pct');
|
||||
if (pct) pct.textContent = extra;
|
||||
}
|
||||
}
|
||||
|
||||
// Kuuntele console.log-viestejä pipeline-vaiheiden seuraamiseksi
|
||||
const origCodeLog = console.log;
|
||||
const codeLogListener = (...args) => {
|
||||
const msg = args.join(' ');
|
||||
if (msg.includes('[Coder]') || msg.includes('Burn Wasm') || msg.includes('Kipinä Agent Node')) {
|
||||
if (msg.includes('Burn Wasm')) setStep('step-wasm', 'active');
|
||||
if (msg.includes('Agent Node käynnistyy')) { setStep('step-wasm', 'done'); }
|
||||
if (msg.includes('[Coder]') && msg.includes('tokenizer') && msg.includes('löytyi')) { setStep('step-tokenizer', 'done'); }
|
||||
if (msg.includes('[Coder]') && msg.includes('Ladataan') && msg.includes('tokenizer')) { setStep('step-tokenizer', 'active'); }
|
||||
if (msg.includes('[Coder]') && msg.includes('tokenizer') && msg.includes('tallennettu')) { setStep('step-tokenizer', 'done'); }
|
||||
if (msg.includes('[Coder]') && msg.includes('model') && msg.includes('lataus:')) {
|
||||
setStep('step-model', 'active');
|
||||
const match = msg.match(/lataus: (\d+)%/);
|
||||
if (match) setStep('step-model', 'active', match[1] + '%');
|
||||
}
|
||||
if (msg.includes('[Coder]') && msg.includes('model') && msg.includes('löytyi')) { setStep('step-model', 'done', 'cache'); }
|
||||
if (msg.includes('[Coder]') && msg.includes('model') && msg.includes('tallennettu')) { setStep('step-model', 'done', '100%'); }
|
||||
if (msg.includes('[Coder]') && msg.includes('Rakennetaan')) { setStep('step-build', 'active'); }
|
||||
if (msg.includes('[Coder]') && msg.includes('Malli ladattu')) { setStep('step-build', 'done'); setStep('step-ready', 'done'); }
|
||||
if (msg.includes('[Coder]') && msg.includes('Syöte:')) {
|
||||
// Pipeline piiloon kun generointi alkaa
|
||||
setTimeout(() => { document.getElementById('code-pipeline').style.display = 'none'; }, 1000);
|
||||
}
|
||||
}
|
||||
};
|
||||
// Lisätään kuuntelija alkuperäisen console.log ylikirjoituksen päälle
|
||||
const _prevConsoleLog = console.log;
|
||||
console.log = function(...args) { _prevConsoleLog.apply(console, args); codeLogListener(...args); };
|
||||
|
||||
// Käynnistä Coder-node automaattisesti ensimmäisellä kerralla
|
||||
async function ensureCoderNode() {
|
||||
if (coderJoined) return;
|
||||
coderJoined = true;
|
||||
document.getElementById('coder-status').textContent = 'Käynnistyy...';
|
||||
document.getElementById('coder-status').style.color = '#d29922';
|
||||
document.getElementById('code-pipeline').style.display = 'block';
|
||||
setStep('step-wasm', 'active');
|
||||
|
||||
try {
|
||||
await init();
|
||||
setStep('step-wasm', 'done');
|
||||
setStep('step-tokenizer', 'active');
|
||||
|
||||
const wsUrl = `${window.location.protocol === 'https:' ? 'wss:' : 'ws:'}//${window.location.host}/ws`;
|
||||
const deviceInfo = {
|
||||
allocated_gb: 4,
|
||||
cpu_cores: navigator.hardwareConcurrency || 0,
|
||||
device_memory_gb: navigator.deviceMemory || 0,
|
||||
platform: navigator.platform || "",
|
||||
gpu: null,
|
||||
selected_task: coderSize === '3b' ? 'qwen-coder-3b' : 'qwen-coder-05b'
|
||||
};
|
||||
const taskId = coderSize === '3b' ? 5 : 4;
|
||||
await start_agent_node(wsUrl, false, JSON.stringify(deviceInfo), taskId);
|
||||
document.getElementById('coder-status').textContent = 'Connected';
|
||||
document.getElementById('coder-status').style.color = '#d29922';
|
||||
coderWsReady = true;
|
||||
|
||||
if (pendingCodePrompt) {
|
||||
sendCodeToHub(pendingCodePrompt);
|
||||
pendingCodePrompt = null;
|
||||
}
|
||||
} catch(e) {
|
||||
console.log("Coder-virhe: " + e);
|
||||
document.getElementById('coder-status').textContent = 'Virhe';
|
||||
document.getElementById('coder-status').style.color = '#f85149';
|
||||
coderJoined = false;
|
||||
}
|
||||
}
|
||||
|
||||
function sendCodeToHub(text) {
|
||||
if (uiSocket && uiSocket.readyState === 1) {
|
||||
uiSocket.send(JSON.stringify({ type: 'user_text', text: text, task_type: 'qwen-coder' }));
|
||||
}
|
||||
}
|
||||
|
||||
async function handleCodeSubmit() {
|
||||
const text = codeInput.value.trim();
|
||||
if (!text) return;
|
||||
codeInput.value = '';
|
||||
codeSendBtn.disabled = true;
|
||||
codeSendBtn.textContent = 'Generating...';
|
||||
codeLoading.style.display = 'block';
|
||||
|
||||
if (!coderJoined) {
|
||||
pendingCodePrompt = text;
|
||||
const dlSize = coderSize === '3b' ? '~6.2 GB' : '~990 MB';
|
||||
codeLoading.textContent = `Loading Qwen2.5-Coder-${coderSize === '3b' ? '3B' : '0.5B'} (${dlSize} on first run)...`;
|
||||
await ensureCoderNode();
|
||||
} else {
|
||||
codeLoading.textContent = 'Generating code...';
|
||||
sendCodeToHub(text);
|
||||
}
|
||||
}
|
||||
|
||||
codeSendBtn?.addEventListener('click', handleCodeSubmit);
|
||||
codeInput?.addEventListener('keydown', (e) => { if (e.key === 'Enter') handleCodeSubmit(); });
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
Reference in New Issue
Block a user