DGX Spark's firmware stands on Open Source: Zephyr, edk2, Arm Trusted Services, Hafnium, FreeRTOS. Owners can't change a byte of it.
So I rebuilt the EC firmware from source. Byte for byte.
- The build starts from an empty image. Nothing is copied from NVIDIA's file.
- Zephyr v3.7.1, Intel's ecfw and the recovered NVIDIA application, compiled with GCC 12.2.0, every function at NVIDIA's address.
- 75,008 of 75,008 bytes. Same SHA-256 as NVIDIA's own LVFS capsule.
- Every function has a name and a home: power, thermal, mailbox, RTC. It uses Zephyr's own GPIO and I2C APIs and has behaviour tests. Every edit is checked against NVIDIA's hash.
- The one thing I borrow is NVIDIA's signature. It's the only part I can't make.
What's underneath:
1. EC: Zephyr v3.7.1, from November 2024. Upstream ships v4.4.2. The app is a fork of Intel's Apache-2.0 ecfw-zephyr.
2. SoC: AMI Aptio V on edk2, Arm Trusted Services, Hafnium. The SCP runs FreeRTOS V10.5.1, from November 2022. Upstream ships V11.3.1.
3. USB-C PD: Infineon EZ-PD. The stack lives mostly in ROM.
What owners keep hitting:
4. Clock latch. GPU stuck at 554 MHz instead of 2,495. On my stuck node the EC was publishing 0 W power limits, so firmware fell back to 20 W. Throttle reasons: none. Reboot: no. Unplug the power brick: yes.
5. Fans. Fan requests pass through a firmware relay on their way to the EC. A race in that relay can leave a "pending" flag set after the reply arrives. Every fan request after that waits on a flag nobody clears. Two of my Sparks hit it.
6. USB-C PD. Linux sees no Type-C port, no PD contract, no power supply. A 240 W brick negotiates with the board and the OS is never told.
Why owners can't help:
7. Linux has hwmon, thermal, powercap and typec for exactly this. On Spark, Linux has no direct line to the EC. It has to use FF-A, Arm's standard way for an OS to ask the TrustZone secure world to do something. A signed firmware component there (a "secure partition") takes the request and talks to the EC over the eSPI bus. Linux can't change that code or see inside it while it runs. My fan driver can only raise a minimum RPM. The EC keeps the policy.
8. The EC image is P-384 signed. BL31 and BL32 are encrypted. Change one byte and the signature fails.
9. When it breaks, nvidia-smi --debug writes down what happened. Encrypted. My GPU, my logs, their key.
Jetson already shows the way:
10. Jetson's UEFI is on GitHub (edk2-nvidia). Owners can burn their own key hash into the fuses and sign their own boot chain. NVIDIA documents every step.
11.
@nvidia: Spark is sold as a developer machine. Give its owners the Jetson deal: an owner key or a developer mode, the EC source (it's a Zephyr app), a rebase onto current upstream, and logs we can read. I'll send the fixes back.
@NVIDIAAI, you said it in May. Firmware too, please.
@JensenHuang, your first post here was about keeping AI open. GTC Berlin is on 21 October. Owner keys for DGX Spark would make a great announcement.
Byte-identical. Unflashable. Ready to contribute.
#DGXSpark
x.com/NVIDIAAI/status/205536âŠ