Skip to content

Arm NEON bf16 intrinsics - #2256

Open
Jamesbarford wants to merge 5 commits into
rust-lang:mainfrom
Jamesbarford:feature/aarch64-neon-bf16-intrinsics
Open

Jamesbarford wants to merge 5 commits into
rust-lang:mainfrom
Jamesbarford:feature/aarch64-neon-bf16-intrinsics

Conversation

@Jamesbarford

Copy link
Copy Markdown
Contributor

Adds support for bfloat intrinsics for NEON

@rustbot

rustbot commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator

r? @folkertdev

rustbot has assigned @folkertdev.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

  • Owners of files modified in this PR: @Amanieu, @adamgemmell, @davidtwco, @folkertdev
  • @Amanieu, @adamgemmell, @davidtwco, @folkertdev expanded to Amanieu, adamgemmell, davidtwco, folkertdev
  • Random selection from Amanieu, adamgemmell, davidtwco, folkertdev

Comment on lines +42 to +44
if self.kind() == TypeKind::BFloat && self.simd_len.is_none() && self.vec_len.is_none() {
return "f16b".to_string();
}

@folkertdev folkertdev Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this need special-casing? A comment would help.

View changes since the review

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added, it's so we now create NanEqF16b

Comment thread crates/intrinsic-test/src/arm/types.rs Outdated
}

fn comparison_function(&self) -> String {
if self.kind() == TypeKind::BFloat {

@folkertdev folkertdev Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similarly a comment on why this type is special would help.

View changes since the review

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed and renamed NanEqBF16 to NanEqF16b which makes the code simpler, removing this case.

Comment thread crates/intrinsic-test/src/arm/mod.rs Outdated
Comment on lines +147 to +154
if arg.ty.kind() == TypeKind::BFloat {
let ty = arg.ty.rust_type();
return format!(
"let {name}: {ty} = std::ptr::read_unaligned(\
{vals_name}.as_ptr().add((i+{idx}) % {PASSES}) as *const {ty});",
vals_name = test_values_array_name(&arg.ty),
);
}

@folkertdev folkertdev Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this special-cased? could use a comment.

View changes since the review

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need this. I was experimenting and forgot to remove it.

Comment thread crates/intrinsic-test/src/arm/mod.rs Outdated
CcArgStyle::Clang if !a32 && !big_endian => vec![
"-march=armv8.6a+crypto+crc+dotprod+fp16+sve2-aes+sve2-sm4+sve2-sha3+sve2-bitperm+\
f32mm+f64mm+sve2p1",
"-march=armv8.6a+crypto+crc+dotprod+fp16+bf16+lut+sve2-aes+sve2-sm4+sve2-sha3+\

@folkertdev folkertdev Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lut because the relevant bf16 intrinsics are part of that target feature?

View changes since the review

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I was fiddling about with the tests I added it and forgot that we skip testing the intrinsics which require lut due to the version of QEMU not supporting it. So I've removed it.

@Jamesbarford
Jamesbarford force-pushed the feature/aarch64-neon-bf16-intrinsics branch 2 times, most recently from 1a54bf3 to f3ac668 Compare October 20, 2026 11:55
@rustbot

This comment has been minimized.

@rustbot

This comment has been minimized.

@Jamesbarford
Jamesbarford force-pushed the feature/aarch64-neon-bf16-intrinsics branch from 4542524 to 892a26d Compare October 27, 2026 08:35
@rustbot

rustbot commented Oct 27, 2026

Copy link
Copy Markdown
Collaborator

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

Comment on lines +710 to +718
"vbfdot_f32",
"vbfdot_lane_f32",
"vbfdot_laneq_f32",
"vbfdotq_f32",
"vbfdotq_lane_f32",
"vbfdotq_laneq_f32",
"vbfmlalbq_f32",
"vbfmlalbq_lane_f32",
"vbfmlalbq_laneq_f32",

@folkertdev folkertdev Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so why are all of these skipped?

View changes since the review

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and consequently, how were the intrinsics added by this PR tested?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so why are all of these skipped?

We use the intrinsic-test tool, from the code comment;

// FIXME(arm-maintainers): With the advent of the intrinsic-test tool, new tests of this kind
// are no longer being added and just adding to this list indefinitely isn't the best solution for
// dealing with that.

So I've added to the list 🫠, obviously not ideal.


and consequently, how were the intrinsics added by this PR tested?

For runtime tests I've tested with the intrinsic test tool, running;

./ci/intrinsic-test-docker.sh aarch64-unknown-linux-gnu clang

Then for instruction assertions and ensuring the signatures are correct;

./ci/run-docker.sh aarch64-unknown-linux-gnu

So they get tested in the CI too, if you grep the raw ci logs for vbfdot_f32 you will see it is there.

However The lut intrinsics are skipped with no tests. The CI uses Ubuntu 25:10 of which qemu-user is version 10.1.0 which does not have FEAT_LUT. FEAT_LUT is made available in QEMU version 11.1.0

Locally I am using 11.1.2 and thus can execute lut at one point in this PR I did add lut as I forgot why we were skipping them.

@folkertdev
folkertdev enabled auto-merge October 28, 2026 16:54

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants