AMD MIOpen in SD.Next
What is MIOpen?
In brief, MIOpen is AMD's equivalent to Nvidia cuDNN, The NVIDIA CUDA Deep Neural Network library. Instead of requiring developers to write low-level C++ code to execute complex neural network operations directly on the GPU hardware, cuDNN & MIOpen provide optimized implementations of common machine learning neural network routines, which are solutions for common calculations, such as Matrix Multiplication.
In SD.Next the use of MIOpen is limited. Used mostly on VAE Decode and operation pass-through,
Pass-through for operations that carry & preserve data between image and video inference stages, including resizers, detailers, LoRa's etc.
So MIOpen in this case can be seen as a carrier method for these data.
Note
SD.Next has it's own method for configuring MIOpen already bundled into the program, see the ROCm Advanced Configuration Script details below.
By default on Linux and Windows MIOpen is enabled. SD.Next does not rely on MIOpen and it can be disabled.
Whether MIOpen accelerates these processes or not depends on how MIOpen is set and the type of processes in question.
It is up to the user to decide whether their operation benefits from MIOpen's solutions, research shows that most of the time, when correctly used, AMD users benefit from MIOpen's features, extracting a non-negligable performance improvement.
How to enable or disable AMD MIOpen
In order to enable or disable MIOpen, Edit the radio buttons for cuDNN Enabled found under Compute Settings in the SD.Next UI.
By default MIOpen is enabled, setting the radio button to true will have no effect and is not recommended, setting it to false will disable MIOpen.
ROCm Advanced Configuration Script
How to configure MIOpen for best performance.
The ROCm Advanced Configuration Script is found under Extras on the left pane of the SD.Next modern ui.
SD.Next Integrated MIOpen logging
SD.Next has an integrated MIOpen logging method to take the large amount of console spam MIOpen generates and make it usable in the SD.Next console.
In order to enable the MIOpen logging set MIOpen logging to enabled hit Apply and restart SD.Next.
When logging is enabled it is possible to see the difference in performance / latency between the operations done by Fast Mode and Normal Mode.
MIOpen Fast and Normal modes
By default MIOpen is set to Fast Mode where no benchmarking of your GPU hardware is done. This method is easiest for novice users, as such MIOpen will remain silent performing its tasks, this is recommended for most operations.
"Fast Mode" is represented by 2 drop-down boxes MIOpen Find Mode set to 2 - FAST and MIOpen Find Enforce set to 1 - NONE as seen in the screenshot above.
Tip
It may be beneficial to the user to enable Normal Mode where MIOpen will benchmark operations and store best performance solutions to a database in the users .miopen folder.
In order to set Normal Mode set MIOpen Find Mode = 1 - NORMAL and set MIOpen Find Enforce to 2 - DB_UPDATE hit the Apply button above and restart SD.Next for the settings to take effect.
However Normal Mode will have an initial cache warm-up phase per operation which will take longer at first than just leaving things on Fast Mode.
After the initial cache warm-up and DB update MIOpen will read saved solutions from the database and utilize the performance improvement but only on subsequent runs of the exact same operation.
Since VAE Decode operations are different per image settings, the cached runs may only offer performance boosts if the exact same operation is done many times, such as when changing prompt details for an image while the rest of the image settings remain identical, changing LoRa's etc will cause a new cache warm-up phase because of the difference in the operation.
On occasion, detailers and LoRa's will perform operations never seen by MIOpen in Fast Mode and generate errors, so when using these it is advisable to enable Normal Mode to allow MIOpen to generate a solution based on the shape of the operation.
MIOpen System & Tensile Library path
These two fields are intentionally left blank in the current implementation and are meant for advanced users. Normal users should not need these, they have been included for users who have datacenter hardware and can leverage the more advanced MIOpen solvers.
MIOpen Hybrid Search & GEMM-backend settings
MIOpen Find Mode and MIOpen Find Enforce have other available modes but these are suited to datacenter grade AMD hardware and should not be used on consumer grade RDNA graphics cards.
There are also options to enable hipBLASlt for RDNA 3 and 4 in preference to rocBLAS though it must be noted that hipBLASlt implementation for RDNA cards is still in development and may have bugs and unforeseen issues if used on RDNA 3 or 4 cards, hipBLASlt is not enabled for RDNA 2 cards and should not be used.
Users are advised to use hipBLASlt only when certain, when it works it is much faster than rocBLAS, these both are used by MIOpen in the background as solution parsers.
MIOpen Workspace size
Workspace size configuration is set to 1Gb which is plenty for most of the regular MIOpen solutions, use this to control how much VRAM to assign to MIOpen, CDNA Datacenter solutions may require more space, as is 1GB is a good maximum most of the time.