Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@

# Compiled binaries
flux
iris
iris.exe

# Model weights (large files)
flux-klein-*/
Expand Down
76 changes: 75 additions & 1 deletion AGENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,79 @@ main.c - CLI entry point

This project implements three targets:
- MPS: Apple Silicon GPU path.
- BLAS: optimized CPU inference via BLAS/OpenBLAS.
- BLAS: optimized CPU inference via BLAS/OpenBLAS. On Windows (MSYS2
UCRT64) this is the only supported target; it builds without the
interactive REPL.
- generic: pure C fallback, very slow.

# Windows Notes

Build from an MSYS2 UCRT64 shell:

pacman -S mingw-w64-ucrt-x86_64-gcc mingw-w64-ucrt-x86_64-openblas make
make blas

Platform shims, all guarded by `#ifdef _WIN32`:
- `iris_safetensors.c`: file mapping via CreateFileMapping/MapViewOfFile
instead of mmap. The view keeps its own references, so the file and
mapping handles are closed right after mapping, mirroring the POSIX
code that closes the fd after mmap. Release goes through the
`iris_unmap()` helper.
- `iris_transformer_flux.c`: GetSystemInfo instead of sysconf for the CPU
count. MinGW ships unistd.h but not sysconf.
- `terminals.c`: the iTerm2 temp-file path returns -1. MinGW has no
mkstemps(), and iTerm2 is never detected on Windows anyway.
- Config, tokenizer and index files are opened in binary mode ("rb"):
text mode would translate CRLF and break sized reads.

Not built on Windows: the interactive REPL (`iris_cli.c`, `linenoise.c`,
`embcache.c` need termios). Requesting interactive mode prints an error
and exits. Terminal image previews do not render.

Note that `make pngtest` still fails on its first image pair: upstream
compares `woman_with_sunglasses.png` (512x512) against
`woman_with_sunglasses_compressed2.png` (256x256). The `cat_*` pair is
correctly matched and passes.

## Windows performance

Measured on a Xeon W-2125 (4 cores / 8 threads, 32 GB), flux-klein-4b,
512x512 at 4 steps. **Use the mmap default and cap BLAS at one thread per
physical core** -- that combination is 2.7x faster than the opposite one:

./iris.exe -d flux-klein-4b --blas-threads 4 -p "..." -o out.png

| Config | 512x512 | 256x256 |
|---------------------------|---------|---------|
| mmap + 4 threads | 219s | 138s |
| --no-mmap + 4 threads | | 238s |
| --no-mmap + 8 threads | 592s | 293s |

Two results worth keeping:
- **`--no-mmap` is slower here**, unlike what the README suggests for
machines with spare RAM. Copying ~15 GB of weights into RAM costs 26s
for the transformer alone and doubles text encoding; a single
generation never earns that back. It may still pay off in a long
interactive session, which this build does not have.
- **8 threads lose to 4.** The 4 physical cores already saturate the
vector units, so hyperthreading only adds contention with the
head-parallel attention threads.

Neither switch changes the output: mmap vs --no-mmap and 4 vs 8 threads
produce bit-identical images (mean_diff 0.000000).

## Windows test results

`run_test.py` cannot be used as-is on hardware this slow: it kills each
test after 300s, and the 512x512 case needs more. The three cases were
run by hand instead, all passing well inside the threshold of 20:

| Test | mean_diff | upstream expects |
|-------------------------------|-----------|------------------|
| 64x64, 2 steps, seed 42 | 1.14 | ~3.4 |
| 512x512, 4 steps, seed 123 | 2.12 | ~1.7 |
| img2img 256x256, seed 456 | 8.29 | 6-17 |

# Development Rules

- No additional project dependencies. Acceptable external deps are BLAS/OpenBLAS and Metal/MPS from macOS.
Expand All @@ -91,6 +161,10 @@ Z-Image example:

./iris -d zimage-turbo -p "a fish" -o /tmp/zimage.png

On Windows the binary is `iris.exe`:

./iris.exe -d flux-klein-4b -p "a cat and a dog playing" -o test.png

If model weights are missing, use the download script only after user approval.

# Python Reference Implementations
Expand Down
19 changes: 17 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,12 @@ UNAME_M := $(shell uname -m)
# Source files
SRCS = iris.c iris_kernels.c iris_tokenizer.c iris_vae.c iris_transformer_flux.c iris_transformer_zimage.c iris_sample.c iris_image.c jpeg.c iris_safetensors.c iris_qwen3.c iris_qwen3_tokenizer.c terminals.c
OBJS = $(SRCS:.c=.o)
# The interactive REPL needs termios/linenoise: not available on Windows.
ifneq (,$(filter MINGW% MSYS%,$(UNAME_S)))
CLI_SRCS =
else
CLI_SRCS = iris_cli.c linenoise.c embcache.c
endif
CLI_OBJS = $(CLI_SRCS:.c=.o)
MAIN = main.c
TARGET = iris
Expand Down Expand Up @@ -65,9 +70,15 @@ ifeq ($(UNAME_S),Darwin)
blas: CFLAGS = $(CFLAGS_BASE) -DUSE_BLAS -DACCELERATE_NEW_LAPACK
blas: LDFLAGS += -framework Accelerate
else
ifneq (,$(filter MINGW% MSYS%,$(UNAME_S)))
MINGW_PREFIX ?= /ucrt64
blas: CFLAGS = $(CFLAGS_BASE) -DUSE_BLAS -DUSE_OPENBLAS -DIRIS_NO_REPL -I$(MINGW_PREFIX)/include/openblas
blas: LDFLAGS += -lopenblas
else
blas: CFLAGS = $(CFLAGS_BASE) -DUSE_BLAS -DUSE_OPENBLAS -I/usr/include/openblas
blas: LDFLAGS += -lopenblas
endif
endif
blas: clean $(TARGET)
@echo ""
@echo "Built with BLAS backend (~30x faster than generic)"
Expand Down Expand Up @@ -139,10 +150,10 @@ test-quick:

pngtest:
@echo "Running PNG compression compare test..."
@$(CC) $(CFLAGS_BASE) -I. png_compare.c iris_image.c -lm -o /tmp/iris_png_compare
@$(CC) $(CFLAGS_BASE) -I. png_compare.c iris_image.c jpeg.c -lm -o /tmp/iris_png_compare
@/tmp/iris_png_compare images/woman_with_sunglasses.png images/woman_with_sunglasses_compressed2.png
@/tmp/iris_png_compare images/cat_uncompressed.png images/cat_compressed.png
@rm -f /tmp/iris_png_compare
@rm -f /tmp/iris_png_compare /tmp/iris_png_compare.exe
@echo "PNG TEST PASSED"

install: $(TARGET) $(LIB)
Expand All @@ -169,9 +180,13 @@ ifeq ($(UNAME_S),Darwin)
ifeq ($(UNAME_M),arm64)
@echo " mps - Metal GPU (recommended)"
endif
else
ifneq (,$(filter MINGW% MSYS%,$(UNAME_S)))
@echo " blas - OpenBLAS (MSYS2 UCRT64; REPL not built)"
else
@echo " blas - OpenBLAS (requires libopenblas-dev)"
endif
endif

# =============================================================================
# Dependencies
Expand Down
14 changes: 13 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Supported model families:
```bash
# Build (choose your backend)
make mps # Apple Silicon (fastest)
# or: make blas # Intel Mac / Linux with OpenBLAS
# or: make blas # Intel Mac / Linux / Windows (MSYS2) with OpenBLAS
# or: make generic # Pure C, no dependencies

# Download a model (~16GB) - pick one:
Expand Down Expand Up @@ -281,6 +281,7 @@ make mps # Apple Silicon Metal GPU (fastest, macOS only)
- macOS Intel: `make blas`
- Linux with OpenBLAS: `make blas`
- Linux without OpenBLAS: `make generic`
- Windows: `make blas` from an MSYS2 UCRT64 shell

For `make blas` on Linux, install OpenBLAS first:
```bash
Expand All @@ -291,6 +292,17 @@ sudo apt install libopenblas-dev
sudo dnf install openblas-devel
```

On Windows, build from an [MSYS2](https://www.msys2.org/) **UCRT64** shell:
```bash
pacman -S mingw-w64-ucrt-x86_64-gcc mingw-w64-ucrt-x86_64-openblas make
make blas
```

The resulting `iris.exe` needs the UCRT64 DLLs, so run it from that shell
or copy the DLLs next to the binary. The interactive REPL is not built on
Windows (linenoise needs termios), and terminal image previews do not
render there.

Other targets:
```bash
make clean # Clean build artifacts
Expand Down
6 changes: 3 additions & 3 deletions iris.c
Original file line number Diff line number Diff line change
Expand Up @@ -260,7 +260,7 @@ iris_ctx *iris_load_dir(const char *model_dir) {
ctx->is_zimage = 0;
snprintf(path, sizeof(path), "%s/model_index.json", model_dir);
if (file_exists(path)) {
FILE *f = fopen(path, "r");
FILE *f = fopen(path, "rb");
if (f) {
char buf[4096];
size_t n = fread(buf, 1, sizeof(buf) - 1, f);
Expand All @@ -284,7 +284,7 @@ iris_ctx *iris_load_dir(const char *model_dir) {
ctx->text_dim = 7680; /* default 4B: 3 * 2560 */
snprintf(path, sizeof(path), "%s/transformer/config.json", model_dir);
if (file_exists(path)) {
FILE *f = fopen(path, "r");
FILE *f = fopen(path, "rb");
if (f) {
char buf[8192];
size_t n = fread(buf, 1, sizeof(buf) - 1, f);
Expand Down Expand Up @@ -390,7 +390,7 @@ iris_ctx *iris_load_dir(const char *model_dir) {
ctx->vae_shift = 0.0f;
snprintf(path, sizeof(path), "%s/vae/config.json", model_dir);
if (file_exists(path)) {
FILE *f = fopen(path, "r");
FILE *f = fopen(path, "rb");
if (f) {
char buf[4096];
size_t n = fread(buf, 1, sizeof(buf) - 1, f);
Expand Down
4 changes: 2 additions & 2 deletions iris_qwen3.c
Original file line number Diff line number Diff line change
Expand Up @@ -1379,7 +1379,7 @@ static int parse_qwen3_config(const char *model_dir, qwen3_model_t *model) {
char path[1024];
snprintf(path, sizeof(path), "%s/config.json", model_dir);

FILE *f = fopen(path, "r");
FILE *f = fopen(path, "rb");
if (!f) return -1;

char buf[8192];
Expand Down Expand Up @@ -1453,7 +1453,7 @@ static int open_safetensors_shards(const char *model_dir,

/* First try: read the index JSON to discover shard filenames */
snprintf(path, sizeof(path), "%s/model.safetensors.index.json", model_dir);
FILE *f = fopen(path, "r");
FILE *f = fopen(path, "rb");
if (f) {
/* Read the whole index file */
fseek(f, 0, SEEK_END);
Expand Down
66 changes: 63 additions & 3 deletions iris_safetensors.c
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,26 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#ifdef _WIN32
#define WIN32_LEAN_AND_MEAN
#define NOMINMAX
#include <windows.h>
#else
#include <fcntl.h>
#include <unistd.h>
#include <sys/mman.h>
#include <sys/stat.h>
#endif

/* Release a read-only file mapping created in safetensors_open(). */
static void iris_unmap(void *data, size_t size) {
#ifdef _WIN32
(void)size; /* Windows tracks the view size internally */
UnmapViewOfFile(data);
#else
munmap(data, size);
#endif
}

/* Minimal JSON parser for safetensors header */

Expand Down Expand Up @@ -205,6 +221,49 @@ static int parse_header(safetensors_file_t *sf) {
* OS page in tensor data on demand, avoiding upfront reads of multi-GB model
* files -- only the weights actually used get loaded into RAM. */
safetensors_file_t *safetensors_open(const char *path) {
#ifdef _WIN32
HANDLE fh = CreateFileA(path, GENERIC_READ, FILE_SHARE_READ, NULL,
OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
if (fh == INVALID_HANDLE_VALUE) {
fprintf(stderr, "safetensors_open: cannot open %s (error %lu)\n",
path, (unsigned long)GetLastError());
return NULL;
}

LARGE_INTEGER fsize;
if (!GetFileSizeEx(fh, &fsize)) {
fprintf(stderr, "safetensors_open: cannot stat %s (error %lu)\n",
path, (unsigned long)GetLastError());
CloseHandle(fh);
return NULL;
}

size_t file_size = (size_t)fsize.QuadPart;
if (file_size < 8) {
fprintf(stderr, "safetensors_open: file too small\n");
CloseHandle(fh);
return NULL;
}

HANDLE mapping = CreateFileMappingA(fh, NULL, PAGE_READONLY, 0, 0, NULL);
if (!mapping) {
fprintf(stderr, "safetensors_open: CreateFileMapping failed (error %lu)\n",
(unsigned long)GetLastError());
CloseHandle(fh);
return NULL;
}

void *data = MapViewOfFile(mapping, FILE_MAP_READ, 0, 0, 0);
/* The view holds its own references, so both handles can go now. */
CloseHandle(mapping);
CloseHandle(fh);

if (!data) {
fprintf(stderr, "safetensors_open: MapViewOfFile failed (error %lu)\n",
(unsigned long)GetLastError());
return NULL;
}
#else
int fd = open(path, O_RDONLY);
if (fd < 0) {
perror("safetensors_open: open failed");
Expand Down Expand Up @@ -232,20 +291,21 @@ safetensors_file_t *safetensors_open(const char *path) {
perror("safetensors_open: mmap failed");
return NULL;
}
#endif

/* Read header size (8-byte little-endian) */
uint64_t header_size = 0;
memcpy(&header_size, data, 8);

if (header_size > file_size - 8) {
fprintf(stderr, "safetensors_open: invalid header size\n");
munmap(data, file_size);
iris_unmap(data, file_size);
return NULL;
}

safetensors_file_t *sf = calloc(1, sizeof(safetensors_file_t));
if (!sf) {
munmap(data, file_size);
iris_unmap(data, file_size);
return NULL;
}

Expand Down Expand Up @@ -294,7 +354,7 @@ safetensors_file_t *safetensors_open(const char *path) {

void safetensors_close(safetensors_file_t *sf) {
if (!sf) return;
if (sf->data) munmap(sf->data, sf->file_size);
if (sf->data) iris_unmap(sf->data, sf->file_size);
free(sf->path);
free(sf->header_json);
free(sf);
Expand Down
15 changes: 13 additions & 2 deletions iris_transformer_flux.c
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,11 @@ static double tf_get_time_ms(void) {
#endif
#include <pthread.h>
#include <unistd.h>
#ifdef _WIN32
#define WIN32_LEAN_AND_MEAN
#define NOMINMAX
#include <windows.h> /* GetSystemInfo: MinGW has unistd.h but not sysconf */
#endif
#endif

/* Use Metal for GPU acceleration when available */
Expand Down Expand Up @@ -337,7 +342,7 @@ static int parse_transformer_config(const char *model_dir, iris_transformer_flux
char path[1024];
snprintf(path, sizeof(path), "%s/transformer/config.json", model_dir);

FILE *f = fopen(path, "r");
FILE *f = fopen(path, "rb");
if (!f) return -1;

char buf[4096];
Expand Down Expand Up @@ -407,7 +412,7 @@ static int open_transformer_shards(const char *model_dir,
/* Try to read index JSON for sharded models */
snprintf(path, sizeof(path), "%s/transformer/diffusion_pytorch_model.safetensors.index.json",
model_dir);
FILE *fp = fopen(path, "r");
FILE *fp = fopen(path, "rb");
if (fp) {
fseek(fp, 0, SEEK_END);
long len = ftell(fp);
Expand Down Expand Up @@ -1562,7 +1567,13 @@ static void *joint_attn_thread_worker(void *arg) {
static int get_attn_num_threads(int heads) {
static int cached = 0;
if (cached) return cached;
#ifdef _WIN32
SYSTEM_INFO si;
GetSystemInfo(&si);
int ncpu = (int)si.dwNumberOfProcessors;
#else
int ncpu = (int)sysconf(_SC_NPROCESSORS_ONLN);
#endif
if (ncpu < 2) { cached = 1; return 1; }
if (ncpu > heads) ncpu = heads;
/* Round down to divide heads evenly */
Expand Down
2 changes: 1 addition & 1 deletion iris_transformer_zimage.c
Original file line number Diff line number Diff line change
Expand Up @@ -2193,7 +2193,7 @@ zi_transformer_t *zi_transformer_load_safetensors(const char *model_dir,

/* Try index file first for sharded models */
snprintf(path, sizeof(path), "%s/transformer/diffusion_pytorch_model.safetensors.index.json", model_dir);
FILE *idx_f = fopen(path, "r");
FILE *idx_f = fopen(path, "rb");

safetensors_file_t *files[ZI_MAX_SHARDS] = {0};
int n_files = 0;
Expand Down
Loading