Skip to content

x86_64: add SMP support - #339

Draft
dgruss wants to merge 25 commits into
mainfrom
x86-apic
Draft

dgruss wants to merge 25 commits into
mainfrom
x86-apic

Conversation

@dgruss

@dgruss dgruss commented Oct 23, 2026

Copy link
Copy Markdown
Member

Based on Florian Kargl's SMP work in #287 (@Woazboat), including ACPI/APIC, CPU-local storage, CPU startup, constructor support and several locking fixes.

One round-robin scheduler per CPU, including its own idle and cleanup threads. __cpu uses thread_local, also for the scheduler object. New processes are distributed across CPUs; a thread stays on its CPU once enqueued.

Replace PIC interrupt delivery with APIC, use local timers on each CPU, stop the other CPUs before printing panic diagnostics, and wait for TLB invalidation before reusing pages. Also fix the shared lock diagnostics, console buffers and ATA request/completion races. VFS operations use one mutex for now, with user buffers copied outside the lock so demand paging can read the executable without deadlocking.

make 1core, make 2core, make 4core and make 8core change the QEMU CPU count while keeping the current architecture and build settings. The Raspberry Pi QEMU boards require four CPUs.

Includes the constructor support from #336, needed for the CPU-local scheduler object. SMP scheduling is currently supported on x86_64.

This still needs a run on a full SWEB with more filesystem interactions before merging.

Woazboat and others added 25 commits October 23, 2026 09:59
Adapt ACPI table discovery and MADT parsing from #287 (754446b and 4019905). Keep the existing direct mapping and fixed ISA IRQs, with explicit bounds for the supported configuration.

Co-authored-by: Daniel Gruss <daniel.gruss@tugraz.at>
Adapt APIC timer initialization and ISA redirection from Florian Kargl's #287 (3789e49 and b40c471). Keep fixed vectors and one I/O APIC, with two uncached MMIO pages just below the framebuffer area.

Co-authored-by: Florian Kargl <f.kargl@posteo.de>
Keep the existing IRQ vectors and handlers, route device interrupts to the boot CPU and use the local APIC timer for scheduling. Leave the legacy PIC masked and remove its driver.

Co-authored-by: Florian Kargl <f.kargl@posteo.de>
Adapt Florian Kargl's __cpu storage from #287 (593c4ea and 6e8ecf4). Set up the boot CPU's TLS before kernel initialization and preserve FS on interrupt entry.

Co-authored-by: Florian Kargl <f.kargl@posteo.de>
Adapt the per-CPU descriptor setup from Florian Kargl's #287 (6edb30b and da4757d). Keep CPU metadata in one explicit structure and reach the current CPU through GCC TLS.

Co-authored-by: Florian Kargl <f.kargl@posteo.de>
Adapt the INIT/SIPI sequence and real-mode entry from Florian Kargl's #287 (2546c25 and 8256d12). Keep the entry code in a naked C++ function, copy it to a reserved low page, and give each CPU its own TLS, stack, GDT and TSS.

Secondary CPUs acknowledge initialization and remain parked with interrupts disabled until SMP scheduling is implemented.

Co-authored-by: Florian Kargl <f.kargl@posteo.de>
Add compiler memory barriers to interrupt enable/disable and make the x86_64 IF query volatile.
Use the existing ArchThreads::atomic_set when releasing mutexes, spinlocks and waiter-list locks.

Adapt the lock-release fix from Florian Kargl's #287 (a6cb42b) without changing the ARM atomic implementations.

Co-authored-by: Florian Kargl <f.kargl@posteo.de>
Disable local interrupts across the sleep/unlock handoff, then yield and restore the previous interrupt state. Waking a thread no longer needs to wait for it to fall asleep. Use acquire/release accesses for thread state.

Adapted from Florian Kargl's #287 (e6f3518).

Co-authored-by: Florian Kargl <f.kargl@posteo.de>
Adapt constructor-array support and startup from Florian Kargl's #287 (1b4eecb and c0df7ba).

Co-authored-by: Florian Kargl <f.kargl@posteo.de>
Use thread_local for __cpu and let the compiler construct one scheduler per CPU. Initialize secondary CPU schedulers before reporting them online. Reserve room for the embedded thread stacks in boot TLS and omit TLS destructor registration.

Build on Florian Kargl's CPU-local storage and earlier per-CPU scheduler in #287 (593c4ea and 20389d1).

Co-authored-by: Florian Kargl <f.kargl@posteo.de>
Assign threads to a scheduler when they are enqueued and require local access to its thread list. Its cleanup thread can only run after the exiting thread has left its stack.

Allow killing the current thread or a thread not yet enqueued. Cross-CPU wakeups still only change the thread state.

Use the per-CPU cleanup approach from Florian Kargl's earlier CPU-local scheduler in #287 (20389d1).

Co-authored-by: Florian Kargl <f.kargl@posteo.de>
Give each CPU its own outstanding EOI count, repeated-page-fault counters and assertion recursion flag.

Adapt the CPU-local assertion recursion flag from Florian Kargl's #287 (7bb3e92).

Co-authored-by: Florian Kargl <f.kargl@posteo.de>
Run local APIC setup on each CPU while keeping discovery and I/O APIC routing on the boot CPU. Secondary CPUs wait until boot allocations are finished, then enable their timer and enter their scheduler. Keep the existing heartbeat display on the boot CPU.
Use an NMI so CPUs also stop with interrupts disabled. Wait for their acknowledgements before setting KPANIC, which bypasses locks. Concurrent panics leave one CPU printing diagnostics.
Flush every active CPU and wait for acknowledgements before reusing mapped pages or page tables. Generation counters allow concurrent requests and coalesced IPIs without a shootdown lock.

Adapt the unmap ordering from Florian Kargl's #287.

Co-authored-by: Florian Kargl <f.kargl@posteo.de>
Adapt the console lock-check removal and debug-output serialization from Florian Kargl's #287 (ed76267 and 42b6345). Protect the deferred-output buffer with an IRQ-safe lock.

Co-authored-by: Florian Kargl <f.kargl@posteo.de>
Address the SMP request-queue problem from Florian Kargl's #287 (dc71a1f) with one active request, matching the existing driver mutex. Protect the completion check and sleep transition with the same IRQ-safe lock as the handler.

Co-authored-by: Florian Kargl <f.kargl@posteo.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants