RAII class for a CUDA memory buffer.
#include <DeviceMemory.h>
Name | |
---|---|
DeviceMemory() =default Default constructor, creates an empty object, without allocating any memory on the device. | |
DeviceMemory(size_t numberOfBytes) Constructor allocating the required number of bytes. | |
DeviceMemory(size_t numberOfBytes, cudaStream_t cudaStream) Constructor allocating the required number of bytes by making an async allocation. The allocation will be put in the given CUDA stream. | |
DeviceMemory(void * deviceMemory) Constructor taking ownership of an already allocated CUDA memory pointer. This class will free the pointer once it goes out of scope. | |
bool | allocateMemory(size_t numberOfBytes) Allocates device memory. The memory allocated will automatically be freed by the destructor. |
bool | allocateMemoryAsync(size_t numberOfBytes, cudaStream_t cudaStream) Allocates device memory. The memory allocated will automatically be freed by the destructor. |
bool | reallocateMemory(size_t numberOfBytes) Reallocates device memory. Already existing memory allocation will be freed before the new allocation is made. In case this DeviceMemory has no earlier memory allocation, this method will just allocate new CUDA memory and return a success status. |
bool | reallocateMemoryAsync(size_t numberOfBytes, cudaStream_t cudaStream) Asynchronously reallocate device memory. Already existing memory allocation will be freed before the new allocation is made. In case this DeviceMemory has no earlier memory allocation, this method will just allocate new CUDA memory and return a success status. |
bool | allocateAndResetMemory(size_t numberOfBytes) Allocates device memory and resets all bytes to zeroes. The memory allocated will automatically be freed by the destructor. |
bool | allocateAndResetMemoryAsync(size_t numberOfBytes, cudaStream_t cudaStream) Allocates device memory and resets all bytes to zeroes. The memory allocated will automatically be freed by the destructor. |
bool | freeMemory() Free the device memory held by this class. Calling this when no memory is allocated is a no-op. |
bool | freeMemoryAsync(cudaStream_t cudaStream) Deallocate memory asynchronously, in a given CUDA stream. Calling this when no memory is allocated is a no-op. |
void | setFreeingCudaStream(cudaStream_t cudaStream) Set which CUDA stream to use for freeing this DeviceMemory. In case the DeviceMemory already holds a CUDA stream to use for freeing the memory, this will be overwritten. |
~DeviceMemory() Destructor, frees the internal CUDA memory. | |
template T * | getDevicePointer() const |
size_t | getSize() const |
DeviceMemory(DeviceMemory && other) | |
DeviceMemory & | operator=(DeviceMemory && other) |
void | swap(DeviceMemory & other) |
DeviceMemory(DeviceMemory const & ) =delete DeviceMemory is not copyable. | |
DeviceMemory | operator=(DeviceMemory const & ) =delete |
DeviceMemory() =default
Default constructor, creates an empty object, without allocating any memory on the device.
explicit DeviceMemory(
size_t numberOfBytes
)
Constructor allocating the required number of bytes.
Parameters:
Exceptions:
explicit DeviceMemory(
size_t numberOfBytes,
cudaStream_t cudaStream
)
Constructor allocating the required number of bytes by making an async allocation. The allocation will be put in the given CUDA stream.
Parameters:
Exceptions:
See: freeMemoryAsync method is not explicitly called, the memory will be freed synchronously when this DeviceMemory instance goes out of scope, meaning that the entire GPU is synchronized, which will impact performance negatively.
Note:
cudaStream
.explicit DeviceMemory(
void * deviceMemory
)
Constructor taking ownership of an already allocated CUDA memory pointer. This class will free the pointer once it goes out of scope.
Parameters:
bool allocateMemory(
size_t numberOfBytes
)
Allocates device memory. The memory allocated will automatically be freed by the destructor.
Parameters:
Return: True on success, false if there is already memory allocated by this instance, or if the CUDA malloc failed.
bool allocateMemoryAsync(
size_t numberOfBytes,
cudaStream_t cudaStream
)
Allocates device memory. The memory allocated will automatically be freed by the destructor.
Parameters:
Return: True on success, false if there is already memory allocated by this instance, or if the CUDA malloc failed.
bool reallocateMemory(
size_t numberOfBytes
)
Reallocates device memory. Already existing memory allocation will be freed before the new allocation is made. In case this DeviceMemory has no earlier memory allocation, this method will just allocate new CUDA memory and return a success status.
Parameters:
Return: True on success, false if CUDA free or CUDA malloc failed.
bool reallocateMemoryAsync(
size_t numberOfBytes,
cudaStream_t cudaStream
)
Asynchronously reallocate device memory. Already existing memory allocation will be freed before the new allocation is made. In case this DeviceMemory has no earlier memory allocation, this method will just allocate new CUDA memory and return a success status.
Parameters:
Return: True on success, false if CUDA free or CUDA malloc failed.
bool allocateAndResetMemory(
size_t numberOfBytes
)
Allocates device memory and resets all bytes to zeroes. The memory allocated will automatically be freed by the destructor.
Parameters:
Return: True on success, false if there is already memory allocated by this instance, or if any of the CUDA operations failed.
bool allocateAndResetMemoryAsync(
size_t numberOfBytes,
cudaStream_t cudaStream
)
Allocates device memory and resets all bytes to zeroes. The memory allocated will automatically be freed by the destructor.
Parameters:
Return: True on success, false if there is already memory allocated by this instance, or if any of the CUDA operations failed.
bool freeMemory()
Free the device memory held by this class. Calling this when no memory is allocated is a no-op.
See: freeMemoryAsync instead.
Return: True in case the memory was successfully freed (or not allocated to begin with), false otherwise.
Note: This method will free the memory in an synchronous fashion, synchronizing the entire CUDA context and ignoring the internally saved CUDA stream reference in case one exist. For async freeing of the memory, use
bool freeMemoryAsync(
cudaStream_t cudaStream
)
Deallocate memory asynchronously, in a given CUDA stream. Calling this when no memory is allocated is a no-op.
Parameters:
Return: True in case the memory deallocation request was successfully put in queue in the CUDA stream.
Note:
void setFreeingCudaStream(
cudaStream_t cudaStream
)
Set which CUDA stream to use for freeing this DeviceMemory. In case the DeviceMemory already holds a CUDA stream to use for freeing the memory, this will be overwritten.
Parameters:
Note: It is the programmer’s responsibility to ensure this DeviceMemory is not used in another CUDA stream before this instance is destructed. In case it is used in another CUDA stream, sufficient synchronization must be made before setting this as the new CUDA stream to use when freeing the memory.
~DeviceMemory()
Destructor, frees the internal CUDA memory.
template <typename T =uint8_t>
inline T * getDevicePointer() const
Template Parameters:
Return: the CUDA memory pointer handled by this class. Nullptr in case no memory is allocated.
size_t getSize() const
Return: The size of the CUDA memory allocation held by this class.
DeviceMemory(
DeviceMemory && other
)
DeviceMemory & operator=(
DeviceMemory && other
)
void swap(
DeviceMemory & other
)
DeviceMemory(
DeviceMemory const &
) =delete
DeviceMemory is not copyable.
DeviceMemory operator=(
DeviceMemory const &
) =delete
Updated on 2024-01-25 at 12:02:05 +0100