Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/workflows/artist-intent-experimental.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ on:
push:
branches:
- integration/artist-intent-core-2026-10-10
- integration/live-texture-2026-10-10
- integration/artist-intent-propagation-2026-10-10
- integration/modly-next-full-2026-10-10
workflow_dispatch:

permissions:
Expand Down Expand Up @@ -73,6 +76,10 @@ jobs:
run: node --test --experimental-strip-types --experimental-loader ./scripts/node-ts-extensionless-loader.mjs electron/main/storage-directory-guard.test.ts
- name: Windows artist-intent regression
run: node --test src/areas/workflows/nodes/intent-travel/processor.test.mjs
- name: Windows intent propagation safety tests
run: node --experimental-strip-types --test src/areas/workflows/intentPropagation.test.mjs
- name: Windows texture watcher, IPC and Three.js helper regressions
run: node --test --experimental-strip-types --experimental-loader ./scripts/node-ts-extensionless-loader.mjs electron/main/texture-watch-service.test.ts electron/preload/texture-watch-preload.test.ts src/areas/generate/components/liveTexture.test.ts
- name: Windows main-process typecheck
run: npx --no-install tsc --noEmit -p tsconfig.node.json
- name: Windows built-ins typecheck
Expand Down
63 changes: 63 additions & 0 deletions docs/experimental-artist-intent-propagation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Artist Context — Explicit Start, Automatic Stage Carry

Source: [OmarB97's Modly #252](https://github.com/lightningpixel/modly/pull/252).
This experimental slice stacks on the standalone
[Artist Context core](https://github.com/JayceeB1/modly/pull/6).

## What changes

Only a workflow with the **Remember Artist Context** (intent-travel/start)
node is opted in. Its output is a staged image containing a new
\`modly.intent.v1\` sidecar. A processing stage receiving that image or a
derived image/mesh automatically runs intent-travel/carry before publishing
its output, appending a stage event and retaining the immutable request ID,
artist words, style and project.

The goal is to preserve the original brief through:

Image → Remember Artist Context → Style → Generator → Rig/Optimize → GLB Export

The **Carry Artist Context** node also remains available for an explicit
source/destination transfer when a workflow cannot infer the connection.

## Fail-closed rules

- Workflows without an explicit start node remain unchanged.
- A generated stage never invents an original request or ancestry.
- Multiple input asset paths must **all** carry the same intent marker.
Mixed tagged/untagged files or different roots produce an explicit error
rather than silently attributing the output to the wrong source.
- Text-only edge inputs do not invalidate one tagged image/mesh.
- Processors can return the original path unchanged (e.g. Full detail);
no copy is needed then.
- If the processor fails to carry the record, the workflow returns an
error rather than declaring completion with missing metadata.
- The renderer cannot choose arbitrary artist-intent sidecar paths;
the underlying process extension rechecks physical workspace location
and sidecar size limits.
- A drag/drop image in \`selectedImageData\` can initialize the record,
with the same image size cap as the source processor.

## Tests and evidence

- Seven pure Node regression cases validate the decision logic for opt-in,
common origins, conflicting sources, orphaned/unmarked sources and text.
- The helper is wired into the existing Node test command and runs in
Windows and Linux checks.
- Integration compilation (Electron, TypeScript) ensures the runner and
extension signatures remain compatible.

## Remaining acceptance work

- [ ] Real Windows desktop run with a reference PNG and installed generators
(Style → 3D model → optimizer → exporter)
- [ ] Inspect the exported \`.tags.json\`: stable \`intent_id\`, original
sentences, stage events, project, \`derived_from\` unchanged
- [ ] Verify restart/continue and interrupted stage scenarios, including
source-output races and large inputs
- [ ] Review storage retention: staged source image copies currently remain
in \`Workflows/intent-sources\` until cleaned intentionally
- [ ] Validate compatibility with multi-input texture and scene generators

Not an automatic change to all existing Modly workflows; **not approved for
release** until runtime smoke tests and storage retention policy are reviewed.
46 changes: 46 additions & 0 deletions docs/experimental-live-texture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Experimental Live Texture Preview — upstream #252 (second slice)

Source: [OmarB97's PR #252](https://github.com/lightningpixel/modly/pull/252).
Stacked after the explicit Artist Context experiment in [PR #6](https://github.com/JayceeB1/modly/pull/6).

This is a **candidate**, not an approved release or a replacement for the
original upstream implementation.

## Behavior

An explicit "Watch texture" action opens an OS-owned PNG picker. The main
process owns the filesystem watcher per renderer and sends revisioned, bounded
PNG snapshots to the preview. Only a loaded model with a *single painted
texture* is supported; multi-texture assets are rejected with an explanation.
The texture must retain its original dimensions.

Saving the chosen file emits a coalesced change notification; incomplete
saves are retried. Good changes are applied to the viewer, and invalid saves
show an error without discarding the previous appearance.

## Additional guardrails relative to source PR

- No arbitrary watch-path argument from renderer code. The OS picker chooses
the file; only the owning window receives updates. Watchers are cleaned up
on navigation/destruction and app quit.
- Maximum 32 MiB encoded PNG per read, with bounded file-handle reads and a
check for file-size/mtime changes during save. Reject direct symlink files,
oversized dimensions and non-PNG paths.
- Do not forcibly change texture filters or color space: reuse settings from
the model's original map. Restore original maps before freeing live preview
GPU textures on panel unmount; discard stale asynchronous loads.
- Unit regressions cover truncated saves, recovery, per-owner stop,
symlinks, size caps, invalid dimensions, IPC visibility, single/multiple
texture detection, map swaps, and cleanup.

## Validation

- [ ] Linux GitHub Actions: full Node/Python, build, lint and TypeScript gates
- [ ] Windows GitHub Actions: watcher/IPC/Three.js tests and desktop bundle
- [ ] Real interactive Electron test: preview texture edit in external editor,
incomplete save, recovery, restart, project change, GPU usage, undo/redo
- [ ] Manual large/animated/UV-transformed GLB cases and image decode failures

This feature is an **opt-in viewer preview**, not a background image editor or
a guarantee of texture persistence into saved GLBs. It deliberately does not
automatically rewrite any user files, nor import the old global UI-zoom changes.
29 changes: 29 additions & 0 deletions docs/experimental-modly-next-combined.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Modly Next — combined experimental preview (2026-10-10)

This branch is the **working integration snapshot** of
[Modly Next's earlier PRs](https://github.com/JayceeB1/modly/pulls) and
the two independent parts of upstream
[OmarB97's Modly #252](https://github.com/lightningpixel/modly/pull/252):

- **Artist Context:** opt-in \`modly.intent.v1\` record and automatic carrying
of verified intent only when the workflow contains "Remember Artist Context"
([PR #8](https://github.com/JayceeB1/modly/pull/8)).
- **Live Texture:** OS-picked PNG watcher with bounded reads, recoverable
saves and viewer preview cleanup
([PR #7](https://github.com/JayceeB1/modly/pull/7)).

A two-parent Git commit combines those separately reviewed changes without
rewriting either original branch or flattening attribution.

## Current gates

- [ ] Run the **combined** Linux full Node, Python, lint, typecheck, build
- [ ] Run combined Windows watcher, intent, filesystem and build checks
- [ ] Real Windows desktop pilot in a disposable workspace
- [ ] Exercise a real image→model→optimizer→export workflow carrying intent
- [ ] Watch a real texture file edited in another program and verify cleanup
- [ ] Verify migration/retention of staged images and \`.tags.json\` files
- [ ] Review existing storage deletion policies before opening real user data

This is **not** a published release. Do not merge into \`main\` or distribute
an installer without the manual gates above.
44 changes: 44 additions & 0 deletions electron/main/ipc-handlers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,8 @@ import {
import { registerWorkspaceAssetLibraryIpcHandlers } from './artifact-registry-service'
import { updatesSupported } from './updater'
import { ModelWeightOperations } from './model-weight-operations'
import { TextureWatchService } from './texture-watch-service'
import type { TextureWatchChoice } from '../../src/shared/types/liveTexture'
import { readLocalFileBase64 } from './bounded-file-reader'
import { encryptSecret, decryptSecret } from './secure-store'
import { getHfToken, initHfToken, setHfToken } from './hf-token'
Expand Down Expand Up @@ -172,6 +174,10 @@ const renameWithRetry = (from: string, to: string, label: string) =>
renameExtensionWithRetry(from, to, label, logger)

export function setupIpcHandlers(pythonBridge: PythonBridge, getWindow: WindowGetter): void {
const textureWatchService = new TextureWatchService()
const textureWatchOwners = new Set<number>()
app.once('before-quit', () => textureWatchService.stopAll())

type ActiveDownload = {
progress: DownloadProgress & { variantId?: string }
done: Promise<void>
Expand Down Expand Up @@ -475,6 +481,44 @@ export function setupIpcHandlers(pythonBridge: PythonBridge, getWindow: WindowGe
}
})

// Only a user-selected PNG may be monitored; the renderer cannot supply
// filesystem paths directly. One watcher is owned by each Electron renderer.
ipcMain.handle('texture:chooseAndWatch', async (event): Promise<TextureWatchChoice> => {
const win = getWindow()
if (!win || win.webContents.isDestroyed() || event.sender.id !== win.webContents.id) {
return { cancelled: true }
}

const result = await dialog.showOpenDialog(win, {
title: 'Choose a texture to watch',
filters: [{ name: 'PNG image', extensions: ['png'] }],
properties: ['openFile'],
})
if (result.canceled || !result.filePaths[0]) return { cancelled: true }

const ownerId = event.sender.id
if (!textureWatchOwners.has(ownerId)) {
textureWatchOwners.add(ownerId)
event.sender.once('destroyed', () => {
textureWatchOwners.delete(ownerId)
textureWatchService.stop(ownerId)
})
}

try {
const update = await textureWatchService.start(ownerId, result.filePaths[0], (next) => {
if (!event.sender.isDestroyed()) event.sender.send('texture:changed', next)
})
return { cancelled: false, update }
} catch (err) {
return { cancelled: true, error: String(err) }
}
})

ipcMain.handle('texture:stopWatching', (event) => {
textureWatchService.stop(event.sender.id)
})

// Read local file → base64 (bypasses file:// restrictions in the renderer)
ipcMain.handle('fs:readFileBase64', (_, filePath: string) => readLocalFileBase64(filePath))

Expand Down
7 changes: 6 additions & 1 deletion electron/main/model-weight-ipc.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -49,12 +49,17 @@ function fixture(t) {
}
const stub = new Proxy({}, { get: () => () => {} })
const deps = (name) => {
if (name === 'electron') return { ipcMain: { handle: (id, fn) => handlers.set(id, fn), on: () => {} }, app: { getPath: () => root, on: () => {} } }
if (name === 'electron') return { ipcMain: { handle: (id, fn) => handlers.set(id, fn), on: () => {} }, app: { getPath: () => root, on: () => {}, once: () => {} } }
if (name === 'axios') return {
post: (...args) => hooks.unload(...args),
get: (...args) => hooks.listModels(...args),
isAxiosError: (err) => Boolean(err?.isAxiosError),
}
// The storage/texture watcher is unrelated to weight removal. Do not start
// real OS watchers in this mocked Electron IPC fixture.
if (name === './texture-watch-service') return {
TextureWatchService: class { stopAll() {} stop() {} },
}
if (name === './model-sources') return realSources
if (name === './model-download-plan') return realPlan
if (name === './extension-path-guard') return realGuard
Expand Down
127 changes: 127 additions & 0 deletions electron/main/texture-watch-service.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
import assert from 'node:assert/strict'
import { mkdtemp, rm, writeFile, truncate, symlink } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import test from 'node:test'

import type { TextureWatchUpdate } from '../../src/shared/types/liveTexture.ts'
import { MAX_TEXTURE_BYTES, readPngInfo, TextureWatchService } from './texture-watch-service.ts'

const ONE_PIXEL_PNG = Buffer.from(
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=',
'base64',
)

async function waitFor(
updates: TextureWatchUpdate[],
predicate: (update: TextureWatchUpdate) => boolean,
timeoutMs = 3_000,
): Promise<TextureWatchUpdate> {
const started = Date.now()
while (Date.now() - started < timeoutMs) {
const match = updates.find(predicate)
if (match) return match
await new Promise((resolve) => setTimeout(resolve, 20))
}
throw new Error(`Timed out waiting for texture update. Saw: ${JSON.stringify(updates)}`)
}

test('reads PNG dimensions and rejects a partial save', () => {
assert.deepEqual(readPngInfo(ONE_PIXEL_PNG), { width: 1, height: 1 })
assert.throws(
() => readPngInfo(ONE_PIXEL_PNG.subarray(0, 24)),
/invalid PNG header|incomplete PNG file/,
)
})

test('reports a malformed save, keeps watching, and recovers on the next valid save', async () => {
const dir = await mkdtemp(join(tmpdir(), 'modly-texture-watch-'))
const filePath = join(dir, 'paint.png')
const updates: TextureWatchUpdate[] = []
const service = new TextureWatchService()

try {
await writeFile(filePath, ONE_PIXEL_PNG)
const initial = await service.start(7, filePath, (update) => updates.push(update))
assert.equal(initial.state, 'ready')
assert.equal(initial.width, 1)
assert.equal(initial.height, 1)

await writeFile(filePath, Buffer.from('not an image'))
await waitFor(updates, (update) => update.state === 'reading')
const failed = await waitFor(updates, (update) => update.state === 'error')
assert.match(failed.message ?? '', /could not be read/)

await writeFile(filePath, ONE_PIXEL_PNG)
const recovered = await waitFor(
updates,
(update) => update.state === 'ready' && update.revision > failed.revision,
)
assert.equal(recovered.width, 1)
assert.equal(recovered.height, 1)
assert.match(recovered.dataUrl ?? '', /^data:image\/png;base64,/)
} finally {
service.stopAll()
await rm(dir, { recursive: true, force: true })
}
})


test('rejects oversized PNGs before reading their contents into memory', async () => {
const dir = await mkdtemp(join(tmpdir(), 'modly-texture-cap-'))
const file = join(dir, 'large.png')
const service = new TextureWatchService()
try {
await writeFile(file, ONE_PIXEL_PNG)
await truncate(file, MAX_TEXTURE_BYTES + 1)
const update = await service.start(1, file, () => {})
assert.equal(update.state, 'error')
assert.match(update.message ?? '', /32 MiB/)
} finally {
service.stopAll()
await rm(dir, { recursive: true, force: true })
}
})

test('rejects invalid dimensions before previewing a PNG', () => {
const invalid = Buffer.from(ONE_PIXEL_PNG)
invalid.writeUInt32BE(8193, 16)
assert.throws(() => readPngInfo(invalid), /incomplete PNG file/)
})

test('does not read a symlink substituted for an OS-selected texture', {
skip: process.platform === 'win32',
}, async () => {
const dir = await mkdtemp(join(tmpdir(), 'modly-texture-link-'))
const real = join(dir, 'real.png')
const alias = join(dir, 'alias.png')
const service = new TextureWatchService()
try {
await writeFile(real, ONE_PIXEL_PNG)
await symlink(real, alias)
const update = await service.start(2, alias, () => {})
assert.equal(update.state, 'error')
} finally {
service.stopAll()
await rm(dir, { recursive: true, force: true })
}
})

test('rejects non-PNG selections and releases sessions when stopped', async () => {
const dir = await mkdtemp(join(tmpdir(), 'modly-texture-stop-'))
const file = join(dir, 'asset.png')
const service = new TextureWatchService()
try {
await writeFile(file, ONE_PIXEL_PNG)
await assert.rejects(() => service.start(8, join(dir, 'asset.jpg'), () => {}), /PNG files/)
const updates: TextureWatchUpdate[] = []
assert.equal((await service.start(8, file, (x) => updates.push(x))).state, 'ready')
service.stop(8)
await writeFile(file, Buffer.from('invalid'))
await new Promise((resolve) => setTimeout(resolve, 300))
assert.equal(updates.length, 0)
} finally {
service.stopAll()
await rm(dir, { recursive: true, force: true })
}
})
Loading
Loading