Projects & OSS

Per-token-head quant stores an inline fp32 scale in each cell’s padded tail, and the attention backend carves scale views assuming a per-layer contiguous buffer — but the offload path allocates one cross-layer interleaved buffer shared by all layers, so the scales aliased neighbouring K/V data from the first decode. After review the guard moved out of the connector into use_uniform_kv_cache(), covering every connector in four lines. Full write-up on the blog.

Lets the outer data-parallel gradient reduction be deferred when running hybrid-sharded data parallel (HSDP) in experimental Megatron-FSDP.

Consolidated the per-family tie_word_embeddings guards behind a single declaration each model class makes about what it supports, plus a from_pretrained check that catches the config being flipped after load.

Follow-up covering the architectures that the first pass of tying guards had not yet reached, so every family validates tied/untied embeddings consistently.

The restored consumed_samples offset was applied to every subsequent epoch instead of only the interrupted one, so the start of the dataset was repeatedly skipped. Making the sampler epoch-aware restored correct, deterministic data coverage, with per-epoch reshuffling added on top. Full write-up on the blog.

Co-authored with @yzong-rh. Builds on my earlier fix (#47062) so that generations ending mid-state in the Harmony parser also recover their raw tail through the Responses API, rather than returning nothing.

Silently ignoring the flag left users believing embeddings were tied when they were not; raising at config time makes the mismatch obvious.

When the Harmony parser reached the end of a generation while still mid-state, the partial output was dropped entirely. Returning the raw text instead means callers see what the model actually produced.

Op compatibility checks run at import time were initializing a CUDA context, which breaks any process that forks afterwards. Deferring that work keeps import deepspeed safe in multiprocessing and dataloader workers.

Merged PR fixing checkpoint weight-tying so tie_word_embeddings is resolved top-level-first, matching Hugging Face’s tying semantics.

Cherry-pick of #2601 into the r0.5.0 release branch.

Merged PR fixing weight tying on the Gemma4 MoE path: the language-model replacement orphaned the lm_head ↔ embed_tokens link, so the tied weights drifted during training. The fix re-ties lm_head to the active embed_tokens.

Merged PR fixing stale wheel installs in nightly Docker images. Bind-mount installs were skipped on warm BuildKit agents; copying only wheel.sha256 busts the cache without bloating the image. Added regression test for Anthropic protocol exports.

TargetEncoder.fit_transform() deliberately cross-fits to avoid target leakage, so it does not return the same values as calling fit() then transform(). The docs now state which to use when.