Repository navigation
Conversation
Allocating, copying, launching kernels and displaying arrays were not covered by the precompilation workload, which only compiles a kernel without a device, so they compiled seconds of host code on first use: on an Arc A770, the first array took 2.9 s, the first broadcast another 2.4 s. Exercise those operations as well when the GPU can be used while precompiling, and reset the state they create afterwards, as it refers to driver objects of the precompilation process. Precompilation doesn't initialize the driver when loading oneAPI, so add `can_precompile`, which initializes it (quietly, once) and checks that the runtime can be used: the SPIR-V tool for the codegen path, a device, and in-order immediate command lists. It can be disabled with the `precompile` preference, and is meant for the workloads of other packages too. What it finds is not kept in the package image. On Julia 1.13, this takes the first array to 13 ms, the first broadcast to 0.17 s and the first display of an array to 7 ms, for about 40 ms more load time and a package image that is 9 MB larger. On Julia 1.12, loading takes 65 to 110 ms longer, about 50 ms of which is loading the package image: that is the code for compiling and launching kernels the workload caches, and leaving out any one of its operations doesn't reduce it measurably.
Contributor
|
Documentation preview: https://juliagpu.github.io/oneAPI.jl/previews/PR753/ |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #753 +/- ##
==========================================
- Coverage 86.79% 86.62% -0.18%
==========================================
Files 55 55
Lines 5068 5075 +7
==========================================
- Hits 4399 4396 -3
- Misses 669 679 +10 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
oneAPI's precompilation workload only compiles a kernel without a device, so allocating, copying, launching and displaying all compiled seconds of host code on first use: on an Arc A770 with Julia 1.13, the first array took 2.9 s and the first broadcast another 2.4 s.
When the GPU can be used while precompiling, the workload now also runs those operations, and afterwards resets the state they created, since it refers to driver objects of the precompilation process (in an order that lets the streams' finalizers run while their bookkeeping is still valid). Precompilation doesn't initialize the driver, so this adds
can_precompile, which initializes it quietly and once, under a lock, and checks that the runtime can actually be used: the SPIR-V tool for the selected codegen path, a device, and the in-order immediate command lists the runtime needs. Nothing it finds is kept in the package image. It can be disabled with a newprecompilepreference (documented in the installation docs), and is meant for other packages' workloads as well; the companion AcceleratedKernels PR uses it to precompile reductions, scans and predicates for oneAPI's tunings. Failures in the workload itself are not caught, so they surface during precompilation.sumshowrandThis costs about 40 ms of load time and 9 MB of package image (32.7 → 41.3 MB) on 1.13, and precompilation takes 24 instead of 19 s. On Julia 1.12 loading takes 65 to 110 ms longer; that's the cached code for compiling and launching kernels, and leaving out any one operation of the workload doesn't reduce it measurably. On 1.10/1.11 the workload leaves out compiling kernels, as the existing one does.
Kernels that the workloads cover still compile their SPIR-V and go through the driver's compiler in every session, because oneAPI doesn't support persisting compiled results (
can_persist_results, which needs relocatable code as in Metal.jl). That is most of what remains of the numbers above, and a separate change.