2.10. Managing Devices

Multiple vendors vie to provide accelerator-type processors. Viskores endeavors to support as many such architectures as possible. Each device and device technology requires some level of code specialization, and that specialization is encapsulated in a unit called a device adapter.

So far in Part 2 (Using Viskores) we have been writing code that runs on a local serial CPU. In those examples where we run a filter, Viskores is launching parallel execution in the execution environment. Internally Viskores uses a device adapter to manage this execution.

A build of Viskores generally supports multiple device adapters. In this chapter we describe how to represent and manage devices.

2.10.1. Device Adapter Tag

A device adapter is identified by a device adapter tag. This tag, which is simply an empty struct type, is used as the template parameter for several classes in the Viskores control environment and causes these classes to direct their work to a particular device. The following device adapter tags are available in Viskores.

struct DeviceAdapterTagSerial : public viskores::cont::DeviceAdapterId 

Tag for a device adapter that performs all computation on the same single thread as the control environment.

This device is useful for debugging. This device is always available. This tag is defined in viskores/cont/DeviceAdapterSerial.h.

struct DeviceAdapterTagCuda : public viskores::cont::DeviceAdapterId 

Tag for a device adapter that uses a CUDA capable GPU device.

For this device to work, Viskores must be configured to use CUDA and the code must be compiled by the CUDA nvcc compiler. This tag is defined in viskores/cont/cuda/DeviceAdapterCuda.h.

struct DeviceAdapterTagOpenMP : public viskores::cont::DeviceAdapterId 

Tag for a device adapter that uses OpenMP compiler extensions to run algorithms on multiple threads.

For this device to work, Viskores must be configured to use OpenMP and the code must be compiled with a compiler that supports OpenMP pragmas. This tag is defined in viskores/cont/openmp/DeviceAdapterOpenMP.h.

struct DeviceAdapterTagTBB : public viskores::cont::DeviceAdapterId 

Tag for a device adapter that uses the Intel Threading Building Blocks library to run algorithms on multiple threads.

For this device to work, Viskores must be configured to use TBB and the executable must be linked to the TBB library. This tag is defined in viskores/cont/tbb/DeviceAdapterTBB.h.

struct DeviceAdapterTagKokkos : public viskores::cont::DeviceAdapterId 

Tag for a device adapter that uses the Kokkos library to run algorithms in parallel.

For this device to work, Viskores must be configured to use Kokkos and the executable must be linked to the Kokkos libraries. Viskores will use the default execution space of the provided kokkos library build. This tag is defined in viskores/cont/kokkos/DeviceAdapterKokkos.h.

The following example uses the tag for the Kokkos device adapter to specify a specific device for Viskores to use. (Details on specifying devices in Viskores is provided in Section 2.10.4 (Specifying Devices).)

Example 2.65 Specifying a device using a device adapter tag.

  viskores::cont::ScopedRuntimeDeviceTracker(viskores::cont::DeviceAdapterTagKokkos{});

For classes and methods that have a template argument that is expected to be a device adapter tag, the tag type can be checked with the VISKORES_IS_DEVICE_ADAPTER_TAG macro to verify the type is a valid device adapter tag. It is good practice to check unknown types with this macro to prevent further unexpected errors.

2.10.2. Device Adapter Id

Using a device adapter tag directly means that the type of device needs to be known at compile time. To store a device adapter type at run time, one can instead use viskores::cont::DeviceAdapterId. viskores::cont::DeviceAdapterId is a superclass to all the device adapter tags, and any device adapter tag can be “stored” in a viskores::cont::DeviceAdapterId. Thus, it is more common for functions and classes to use viskores::cont::DeviceAdapterId then to try to track a specific device with templated code.

struct DeviceAdapterId

An object used to specify a device.

viskores::cont::DeviceAdapterId can be used to specify a device to use when executing some code. Each DeviceAdapterTag object inherits from viskores::cont::DeviceAdapterId. Functions can accept a viskores::cont::DeviceAdapterId object rather than a templated tag to select a device adapter at runtime.

Subclassed by viskores::cont::DeviceAdapterTagAny, viskores::cont::DeviceAdapterTagCuda, viskores::cont::DeviceAdapterTagKokkos, viskores::cont::DeviceAdapterTagOpenMP, viskores::cont::DeviceAdapterTagSerial, viskores::cont::DeviceAdapterTagTBB, viskores::cont::DeviceAdapterTagUndefined

Public Functions

inline constexpr bool IsValueValid() const

Return whether this object represents a valid type of device.

This method will return true if the id represents a specific, valid device. It will return true even if the device is disabled in by the runtime tracker or if the device is not supported by the Viskores build configuration.

It should be noted that this method return false for tags that are not specific devices. This includes viskores::cont::DeviceAdapterTagAny and viskores::cont::DeviceAdapterTagUndefined.

inline constexpr viskores::Int8 GetValue() const: Returns the numeric value of the index.

DeviceAdapterNameType GetName() const

Return a name representing the device.

The string returned from this method is stored in a type named viskores::cont::DeviceAdapterNameType, which is currently aliased to std::string. The device adapter name is useful for printing information about a device being used.

Did You Know?

As a cheat, all device adapter tags actually inherit from the viskores::cont::DeviceAdapterId class. Thus, all of these methods can be called directly on a device adapter tag.

Common Errors

Just because the viskores::cont::DeviceAdapterId::IsValueValid() returns true that does not necessarily mean that this device is available to be run on. It simply means that the device is implemented in Viskores. However, that device might not be compiled, or that device might not be available on the current running system, or that device might not be enabled. Use the device runtime tracker described in Section 2.10.3 (Runtime Device Tracker) to determine if a particular device can actually be used.

In addition to the provided device adapter tags listed previously, a viskores::cont::DeviceAdapterId can store some special device adapter tags that do not directly specify a specific device.

struct DeviceAdapterTagAny : public viskores::cont::DeviceAdapterId 

Tag for a device adapter used to specify that any device may be used for an operation.

In practice this is limited to devices that are currently available.

struct DeviceAdapterTagUndefined : public viskores::cont::DeviceAdapterId 

Tag for a device adapter used to avoid specifying a device.

Useful as a placeholder when a device can be specified but none is given.

Did You Know?

Any device adapter tag can be used where a device adapter id is expected. Thus, you can use a device adapter tag whenever you want to specify a particular device and pass that to any method expecting a device id. Likewise, it is usually more convenient for classes and methods to manage device adapter ids rather than device adapter tag.

2.10.3. Runtime Device Tracker

It is often the case that you are agnostic about what device Viskores algorithms run so long as they complete correctly and as fast as possible. Thus, rather than directly specify a device adapter, you would like Viskores to try using the best available device, and if that does not work try a different device. Because of this, there are many features in Viskores that behave this way. For example, you may have noticed that running filters, as in the examples of Chapter 2.6 (Running Filters), you do not need to specify a device; they choose a device for you.

However, even though we often would like Viskores to choose a device for us, we still need a way to manage device preferences. Viskores also needs a mechanism to record runtime information about what devices are available so that it does not have to continually try (and fail) to use devices that are not available at runtime. These needs are met with the viskores::cont::RuntimeDeviceTracker class. viskores::cont::RuntimeDeviceTracker maintains information about which devices can and should be run on. Viskores maintains a viskores::cont::RuntimeDeviceTracker for each thread your code is operating on. To get the runtime device for the current thread, use the viskores::cont::GetRuntimeDeviceTracker() method.

viskores::cont::RuntimeDeviceTracker &viskores::cont::GetRuntimeDeviceTracker()

Get the RuntimeDeviceTracker for the current thread.

Many features in Viskores will attempt to run algorithms on the “best

available device.” This often is determined at runtime as failures in one device are recorded and that device is disabled. To prevent having to check over and over again, Viskores uses per thread runtime device tracker so that these choices are marked and shared.

class RuntimeDeviceTracker

RuntimeDeviceTracker is the central location for determining which device adapter will be active for algorithm execution.

Many features in Viskores will attempt to run algorithms on the “best

available device.” This generally is determined at runtime as some backends require specific hardware, or failures in one device are recorded and that device is disabled.

While viskores::cont::RunimeDeviceInformation reports on the existence of a device being supported, this tracks on a per-thread basis when worklets fail, why the fail, and will update the list of valid runtime devices based on that information.

Subclassed by viskores::cont::ScopedRuntimeDeviceTracker

Public Functions

bool CanRunOn(DeviceAdapterId deviceId) const: Returns true if the given device adapter is supported on the current machine.

inline void ReportAllocationFailure(viskores::cont::DeviceAdapterId deviceId, const viskores::cont::ErrorBadAllocation&): Report a failure to allocate memory on a device, this will flag the device as being unusable for all future invocations.

inline void ReportBadDeviceFailure(viskores::cont::DeviceAdapterId deviceId, const viskores::cont::ErrorBadDevice&): Report a ErrorBadDevice failure and flag the device as unusable.

void ResetDevice(viskores::cont::DeviceAdapterId deviceId)

Reset the tracker for the given device.

This will discard any updates caused by reported failures. Passing DeviceAdapterTagAny to this will reset all devices (same as Reset()).

void Reset()

Reset the tracker to its default state for default devices.

Will discard any updates caused by reported failures.

void DisableDevice(DeviceAdapterId deviceId)

Disable the given device.

The main intention of RuntimeDeviceTracker is to keep track of what devices are working for Viskores. However, it can also be used to turn devices on and off. Use this method to disable (turn off) a given device. Use ResetDevice() to turn the device back on (if it is supported).

Passing DeviceAdapterTagAny to this will disable all devices.

void ForceDevice(DeviceAdapterId deviceId)

Disable all devices except the specified one.

The main intention of RuntimeDeviceTracker is to keep track of what devices are working for Viskores. However, it can also be used to turn devices on and off. Use this method to disable all devices except one to effectively force Viskores to use that device. Either pass the DeviceAdapterTagAny to this function or call Reset() to restore all devices to their default state.

This method will throw a viskores::cont::ErrorBadValue if the given device does not exist on the system.

bool GetThreadFriendlyMemAlloc() const: Get/Set use of thread-friendly memory allocation for a device.

void SetThreadFriendlyMemAlloc(bool state): Get/Set use of thread-friendly memory allocation for a device.

void CopyStateFrom(const viskores::cont::RuntimeDeviceTracker &tracker)

Copies the state from the given device.

This is a convenient way to allow the RuntimeDeviceTracker on one thread copy the behavior from another thread.

void SetAbortChecker(const std::function<bool()> &func)

Set/Clear the abort checker functor.

If set the abort checker functor is called by viskores::cont::TryExecute() before scheduling a task on a device from the associated the thread. If the functor returns true, an exception is thrown.

void ClearAbortChecker()

Set/Clear the abort checker functor.

If set the abort checker functor is called by viskores::cont::TryExecute() before scheduling a task on a device from the associated the thread. If the functor returns true, an exception is thrown.

void PrintSummary(std::ostream &out) const: Produce a human-readable report on the state of the runtime device tracker.

2.10.4. Specifying Devices

A viskores::cont::RuntimeDeviceTracker can be used to specify which devices to consider for a particular operation. However, a better way to specify devices is to use the viskores::cont::ScopedRuntimeDeviceTracker class. When a viskores::cont::ScopedRuntimeDeviceTracker is constructed, it specifies a new set of devices for Viskores to use. When the viskores::cont::ScopedRuntimeDeviceTracker is destroyed as it leaves scope, it restores Viskores’s devices to those that existed when it was created.

class ScopedRuntimeDeviceTracker : public viskores::cont::RuntimeDeviceTracker 

A class to create a scoped runtime device tracker object.

This object captures the state of the per-thread device tracker and will revert any changes applied during its lifetime on destruction.

Unnamed Group

ScopedRuntimeDeviceTracker(const viskores::cont::RuntimeDeviceTracker &tracker = GetRuntimeDeviceTracker())

Construct a ScopedRuntimeDeviceTracker associated with the thread, associated with the provided tracker (defaults to current thread’s tracker).

Any modifications to the ScopedRuntimeDeviceTracker will effect what ever thread the tracker is associated with, which might not be the thread on which the ScopedRuntimeDeviceTracker was constructed.

Constructors are not thread safe

ScopedRuntimeDeviceTracker(viskores::cont::DeviceAdapterId device, RuntimeDeviceTrackerMode mode = RuntimeDeviceTrackerMode::Force, const viskores::cont::RuntimeDeviceTracker &tracker = GetRuntimeDeviceTracker())

Use this constructor to modify the state of the device adapters associated with the provided tracker.

Use mode with device as follows:

‘Force’ (default)

Force-Enable the provided single device adapter
Force-Enable all device adapters when using viskores::cont::DeviceAdaterTagAny ‘Enable’
Enable the provided single device adapter if it was previously disabled
Enable all device adapters that are currently disabled when using viskores::cont::DeviceAdaterTagAny ‘Disable’
Disable the provided single device adapter
Disable all device adapters when using viskores::cont::DeviceAdaterTagAny

ScopedRuntimeDeviceTracker(const std::function<bool()> &abortChecker, const viskores::cont::RuntimeDeviceTracker &tracker = GetRuntimeDeviceTracker()): Use this constructor to set the abort checker functor for the provided tracker.

~ScopedRuntimeDeviceTracker(): Destructor is not thread safe.

The following example demonstrates how the viskores::cont::ScopedRuntimeDeviceTracker is used to force the Viskores operations that happen within a function to operate exclusively with the Kokkos device.

Example 2.66 Restricting which devices Viskores uses per thread.

void ChangeDefaultRuntime()
{
  std::cout << "Checking changing default runtime." << std::endl;

  viskores::cont::ScopedRuntimeDeviceTracker(viskores::cont::DeviceAdapterTagKokkos{});

  // Viskores operations limited to Kokkos devices here...

  // Devices restored as we leave scope.
}

In the previous example we forced Viskores to use the Kokkos device. This is the default behavior of viskores::cont::ScopedRuntimeDeviceTracker, but the constructor takes an optional second argument that is a value in the viskores::cont::RuntimeDeviceTrackerMode to specify how modify the current device adapter list.

enum class viskores::cont::RuntimeDeviceTrackerMode

Identifier used to specify whether to enable or disable a particular device.

Values:

enumerator Force

Replaces the current list of devices to try with the device specified.

This has the effect of forcing Viskores to use the provided device. This is the default behavior for viskores::cont::ScopedRuntimeDeviceTracker.

enumerator Enable: Adds the provided device adapter to the list of devices to try.

enumerator Disable: Removes the provided device adapter from the list of devices to try.

As a motivating example, let us say that we want to perform a deep copy of an array (described in Section 3.2.3 (Deep Array Copies)). However, we do not want to do the copy on a Kokkos device because we happen to know the data is not on that device and we do not want to spend the time to transfer the data to that device. We can use a viskores::cont::ScopedRuntimeDeviceTracker to temporarily disable the Kokkos device for this operation.

Example 2.67 Disabling a device with viskores::cont::RuntimeDeviceTracker.

  viskores::cont::ScopedRuntimeDeviceTracker tracker(
    viskores::cont::DeviceAdapterTagKokkos(),
    viskores::cont::RuntimeDeviceTrackerMode::Disable);

  viskores::cont::ArrayCopy(srcArray, destArray);