jaakko/agentic-studio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Benchmark: pytest ajetaan Docker-kontissa (kipina-pytest)

Browse Source 
Kontti hoitaa uv init + uv add + pytest eristetyssä ympäristössä.
Python 3.14, ei VIRTUAL_ENV-ongelmia, täysi toistettavuus.
Image: docker build -t kipina-pytest -f tests/Dockerfile.pytest tests/
This commit is contained in:
jaakko
2026-04-14 07:39:23 +03:00
parent 42ee959781
commit 16f40a7536
2 changed files with 16 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
network-poc/tests/Dockerfile.pytest Normal file
View File

@@ -0,0 +1,5 @@
FROM python:3.14-slim
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
WORKDIR /work
ENV PYTHONPATH=/work
ENTRYPOINT ["sh", "-c", "uv init --no-readme --python '>=3.14' 2>/dev/null && rm -f hello.py main.py && uv add fastapi 'uvicorn[standard]' sqlalchemy pytest httpx 2>/dev/null && cp /src/*.py . && rm -f app.db test.db && uv run pytest test_main.py -v --tb=short 2>&1"]

20
network-poc/tests/model-benchmark.mjs
View File

@@ -335,20 +335,18 @@ async function runPipeline(model, scenario) {
result.validationIssues = issues.length; result.validationIssues = issues.length;
result.fixRounds = fixRound; result.fixRounds = fixRound;
// 5. Projektin alustus (uv init) + kirjoita tiedostot + pytest // Kirjoita LLM:n generoimat Python-tiedostot
console.log(` [5/5] Pytest...`);
try {
const uvPath = process.env.HOME + '/.local/bin/uv';
const uv = existsSync(uvPath) ? uvPath : 'uv';
execSync(`cd "${dir}" && ${uv} init --no-readme --python ">=3.14" 2>/dev/null && rm -f hello.py main.py`, { timeout: 30000, stdio: 'pipe' });
execSync(`cd "${dir}" && ${uv} add fastapi "uvicorn[standard]" sqlalchemy pytest httpx 2>/dev/null`, { timeout: 60000, stdio: 'pipe' });
// Kirjoita LLM:n generoimat Python-tiedostot (uv initin jälkeen)
for (const [fn, content] of Object.entries(files)) { for (const [fn, content] of Object.entries(files)) {
if (fn.endsWith('.py')) writeFileSync(`${dir}/${fn}`, content); if (fn.endsWith('.py')) writeFileSync(`${dir}/${fn}`, content);
} }
execSync(`cd "${dir}" && rm -f app.db test.db`, { stdio: 'pipe' });
const pytestOut = execSync(`cd "${dir}" && ${uv} run pytest test_main.py -v --tb=short 2>&1`, { timeout: 60000, encoding: 'utf-8' }); // 5. Pytest Docker-kontissa (kipina-pytest image)
console.log(` [5/5] Pytest (Docker)...`);
try {
const pytestOut = execSync(
`docker run --rm -v "${dir}:/src:ro" kipina-pytest 2>&1`,
{ timeout: 120000, encoding: 'utf-8' }
);
writeFileSync(`${dir}/_pytest.txt`, pytestOut); writeFileSync(`${dir}/_pytest.txt`, pytestOut);
const passedMatch = pytestOut.match(/(\d+) passed/); const passedMatch = pytestOut.match(/(\d+) passed/);