dofus-sqlite

Up to date Dofus 3 data, automatically extracted and packaged into convenient SQLite databases — updated every hour.

Release assets

Each release contains the following files:

File	Description
dofus.sqlite	Full game database — quests, achievements, items, map positions, i18n, and more
maps.sqlite	Map interactions database — interactable elements and their positions per map
dofus.proto	Obfuscated Protobuf definition extracted from the game binary
*.json	Raw JSON files for every data class, one file per type
images-<category>.zip	Game images as PNG, one zip per category (item, monster, spell, emblem, worldmap, ...)

How to use

1. Go to the Releases page
2. Download the latest dofus.sqlite (and optionally maps.sqlite) file
3. That's it! The database is ready to use

Note: This repo contains the extraction code that creates these releases. If you just want the data, you don't need to clone this repository.

Usage with kysely

1. Install dependencies

pnpm i kysely better-sqlite3
pnpm i -D kysely-codegen @types/better-sqlite3

2. Pull TS types from database

kysely-codegen --dialect sqlite --url /path/to/dofus.sqlite --out-file /path/to/dofus.d.ts

3. Enjoy full type-safety

import SQLite from "better-sqlite3";
import { Kysely, SqliteDialect } from "kysely";
import type { DB } from "/path/to/dofus.d.ts";

const database = new SQLite("/path/to/dofus.sqlite");
const dialect = new SqliteDialect({ database });

const db = new Kysely<DB>({ dialect });

const potionRecipes = await db
  .selectFrom("ItemData")
  .innerJoin("RecipeData", "RecipeData.resultId", "ItemData.id")
  .innerJoin("translations", "ItemData.nameId", "translations.id")
  .where("translations.value", "like", "%potion%")
  .where("translations.lang", "=", "fr")
  .select(["translations.value as name"])
  .selectAll(["RecipeData"])
  .execute();

Tables are named after the game's data classes (ItemData, RecipeData, MonsterData, ...). Every *NameId / *DescriptionId column joins on translations.id (an INTEGER), filtered by translations.lang (fr, en, es, de, pt).

A few classes are shared by several game files and get an extra source column (part of the primary key) telling which file a row comes from, e.g. SocialRightData.source is guildrightgroups or alliancerightgroups.

Images

Each images-<category>.zip holds the PNGs of one category. When the game provides two resolutions, they sit in 1x/ and 2x/ folders. Paths follow the game's own asset paths where it has them, e.g. images-emblem.zip contains big/up/2x/98.png (each emblem layer: up, backcontent, outlineguild, outlinealliance).

Most images can be joined to the database by their file name:

Zip	File name	Database column
images-item.zip	1x/<id>.png, 2x/<id>.png	ItemData.iconId
images-monster.zip	1x/<id>.png, 2x/<id>.png	MonsterData.gfxId
images-spell.zip	1x/sort_<id>.png, 2x/sort_<id>.png	SpellData.iconId

A few bundles only name their textures, with several textures sharing a name. Those get the size, then an index, appended: 10_668x400.png, 10_1024x1024_3.png. World maps are exported as the tiles the game stores (one name per world map), not as assembled maps.

How the pipeline works

Releases are produced by 2 GitHub Actions workflows that chain together automatically:

┌──────────────────────────────────────────┐
│  1 - Check Version & Create Pre-release  │  ← hourly schedule, push to development,
│            ubuntu-latest                 │    or manual dispatch
└─────────────────┬────────────────────────┘
                  │  creates pre-release, passes tag via artifact
                  ▼
┌──────────────────────────────────────────┐
│  2 - Populate Release: setup (windows)   │  downloads game files once, shares them as artifacts
└─────────────────┬────────────────────────┘
       ┌──────────┼───────────┬────────────┐      (run in parallel)
       ▼          ▼           ▼            ▼
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│  data    │ │  proto   │ │  maps    │ │  images  │
│  windows │ │  windows │ │  windows │ │  ubuntu  │
└────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘
     └────────────┴─────┬──────┴────────────┘
                        ▼
              promote to full release

Workflow 1 checks the current Dofus 3 version via cytrus-v6. If the version changed (or the run was triggered manually or by a push to development), it creates a pre-release and publishes a release-tag artifact consumed by workflow 2.

• Scheduled / manual → compares against the latest non-prerelease; creates a standard pre-release
• Push to development → always runs; creates a draft pre-release with a timestamp suffix

Workflow 2 downloads the game files in a setup job, then runs these jobs in parallel:

• data: parses Data/**/*.bundle and I18n/*.bin to JSON (pnpm extract), generates dofus.sqlite (pnpm db), uploads both
• proto: runs Il2CppDumper.exe then protodec.exe on GameAssembly.dll + global-metadata.dat → dofus.proto
• maps: parses Map/Data/**/*.bundle → maps.sqlite (pnpm maps)
• images: exports Picto/**/*.bundle → images-<category>.zip (dotnet cs/... images <picto folder> <output folder>)

Once all four succeed, the release is promoted from pre-release to latest (dev releases stay drafts). Workflow 2 also supports workflow_dispatch with a release_tag input to re-populate an existing pre-release without re-running the version check; uploads overwrite existing assets.

Developer Instructions

Prerequisites

• pnpm
• Node.js v20
• dotnet v8

Setup

pnpm install

Copy the .env.dist file to a .env and fill your Dofus folder path

Available Scripts

1. First executes pnpm extract to convert game files to readable .json files
2. Then runs pnpm db to generate a .sqlite file from .json files

Images are exported by the C# tool directly, without going through JSON:

dotnet build cs -c Release
dotnet cs/bin/Release/net8.0/unity-bundle-unwrap.dll images <folder with Picto bundles> <output folder> [--only item monster]

pnpm maps reads the map bundles (<INPUT_FOLDER>/Dofus_Data/StreamingAssets/Content/Map/Data) and writes the interactive elements of every map to maps.sqlite (or MAP_INTERACTIONS_DB). It doesn't go through JSON: the C# tool's map-interactions command reads only those elements, all bundles in parallel.

Changelog between two releases

pnpm changelog compares two releases and writes, in changelog-out/:

• changelog.json: what changed per entity, for the domains opendofusdb shows (item, set, monster, dungeon, spell, quest, achievement, area, subArea, worldMap, hint), with the same filters (hidden item types, quest-only monsters, class spells...).
• first-seen.json: the release each entity first appeared in, built on top of the old release's first-seen.json when it has one.

# Latest release vs the one before it
pnpm changelog

# Two specific releases
pnpm changelog v6.0_3.5.17.26 v6.0_3.6.12.16

# A release vs freshly generated local data (CI)
pnpm changelog v6.0_3.6.12.16 --new-db dofus.sqlite --new-json json --new-tag v6.0_3.6.13.17

Each entry is { domain, id, change: "added" | "removed" | "changed", name, meta, changes | snapshot }. Names are in every language, and names lists the name of every object a change links to. Each change has a kind that tells how to display it (types in model.ts):

kind	Example
value	{ field: "level", old: 190, new: 200 }
text	a name or description, in every language
ref / refs	a linked object (item, monster...) that changed, or linked objects added and removed
effects	effect lines, added / removed / changed, as raw effect instances to format like the game does
list	entries of a list (monster grades, drops, set bonuses, quest steps...) added, removed or changed, recursively

Effect lines are paired by effect for rolled values (characteristics, damage), so +301 to 350 Vitality becoming +320 to 400 Vitality is one changed line, and by effect and target for the others (a modified spell, a summoned monster). Item effects and set bonuses are read from itemsdata.json and itemsetsdata.json, since the database cannot link effects to their item.

Each release gets both files from the data job of workflow 2. pnpm changelog:backfill [--upload] generates them for every past release: releases from before the current database layout only seed first-seen.json with their ids, the others get a changelog against the previous release.

pnpm changelog:raw still writes the table-by-table Markdown diff of two databases, for debugging.

Running the pipeline locally

Use run-local.ps1 to replicate the full CI pipeline on your machine:

# Full pipeline (download → parse → db → maps → images → proto)
.\run-local.ps1

# Skip download, re-parse and regenerate databases from existing temp/
.\run-local.ps1 -SkipDownload

# Skip download and parse, only regenerate databases from existing json/
.\run-local.ps1 -SkipDownload -SkipParse

# Only re-export images from existing temp/
.\run-local.ps1 -SkipDownload -SkipParse -SkipDatabase -SkipMaps -SkipProto

# Skip everything except proto generation
.\run-local.ps1 -SkipDownload -SkipParse -SkipDatabase -SkipMaps -SkipImages

# Create a GitHub release after the pipeline (requires GH_TOKEN)
.\run-local.ps1 -CreateRelease -ReleaseTag my-test