|
|
CNN: Hans Reiser Convicted
|
|
MPSState(3) MetalPerformanceShaders.framework MPSState(3) NAME
MPSState SYNOPSIS
#import <MPSState.h> Inherits NSObject. Inherited by MPSCNNBatchNormalizationState, MPSCNNConvolutionWeightsAndBiasesState, MPSCNNLossLabels, MPSCNNNormalizationGammaAndBetaState, MPSNNBinaryGradientState, MPSNNGradientState, MPSRNNRecurrentImageState, and MPSRNNRecurrentMatrixState. Instance Methods (nonnull instancetype) - initWithDevice:bufferSize: (nonnull instancetype) - initWithDevice:textureDescriptor: (nonnull instancetype) - initWithResource: (nullable instancetype) - init (nonnull instancetype) - initWithDevice:resourceList: (nonnull instancetype) - initWithResources: (__nullable id< MTLResource >) - resourceAtIndex:allocateMemory: (NSUInteger) - bufferSizeAtIndex: (MPSStateTextureInfo) - textureInfoAtIndex: (MPSStateResourceType) - resourceTypeAtIndex: (void) - synchronizeOnCommandBuffer: (NSUInteger) - resourceSize (MPSImageDescriptor *__nonnull) - destinationImageDescriptorForSourceImages:sourceStates:forKernel:suggestedDescriptor: Class Methods (nonnull instancetype) + temporaryStateWithCommandBuffer:bufferSize: (nonnull instancetype) + temporaryStateWithCommandBuffer:textureDescriptor: (nonnull instancetype) + temporaryStateWithCommandBuffer: (nonnull instancetype) + temporaryStateWithCommandBuffer:resourceList: Properties NSUInteger resourceCount NSUInteger readCount BOOL isTemporary NSString * label id< MTLResource > resource Detailed Description This depends on Metal Framework A semi-opaque data container for large storage in MPS CNN filters Some MPS CNN kernels produce additional information beyond a MPSImage. These may be pooling indices where the result came from, convolution weights, or other information not contained in the usual MPSImage result from a MPSCNNKernel. A MPSState object typically contains one or more expensive MTLResources such as textures or buffers to store this information. It provides a base class with interfaces for managing this storage. Child classes may add additional functionality specific to their contents. Some MPSState objects are temporary. Temporary state objects, like MPSTemporaryImages and Matrices, are for very short lived storage, perhaps just a few lines of code within the scope of a single MTLCommandBuffer. They are very efficient for storage, as several temporary objects can share the same memory over the course of a MTLCommandBuffer. This can improve both memory usage and time spent in the kernel wiring down memory and such. You may find that some large CNN tasks can not be computed without them, as non-temporary storage would simply take up too much memory. In exchange, the lifetime of the underlying storage in temporary MPSState objects needs to be carefully managed. ARC often waits until the end of scope to release objects. Temporary storage often needs to be released sooner than that. Consequently the lifetime of the data in the underlying MTLResources is managed by a readCount property. Each time a MPSCNNKernel reads a temporary MPSState object the readCount is automatically decremented. When it reaches zero, the underlying storage is recycled for use by other MPS temporary objects, and the data is becomes undefined. If you need to consume the data multiple times, you should set the readCount to a larger number to prevent the data from becomming undefined. You may set the readCount to 0 yourself to return the storage to MPS, if for any reason, you realize that the MPSState object will no longer be used. The contents of a temporary MPSState object are only valid from creation to the time the readCount reaches 0. The data is only valid for the MTLCommandBuffer on which it was created. Non-temporary MPSState objects are valid on any MTLCommandBuffer on the same device until they are released. Method Documentation - (NSUInteger) bufferSizeAtIndex: (NSUInteger) index Return the buffer size of the MTLBuffer at index or 0 if it is not a MTLBuffer Does not force allocation of the MTLResource - (MPSImageDescriptor * __nonnull) destinationImageDescriptorForSourceImages: (NSArray< MPSImage * > *__nonnull) sourceImages(NSArray< MPSState * > *__nullable) sourceStates(MPSKernel *__nonnull) kernel(MPSImageDescriptor *__nonnull) inDescriptor Determine padding and sizing of result images A MPSState has the opportunity to reconfigure the MPSImageDescriptor used to create the filter result state and set the MPSKernel.offset to the correct value. By default, the MPSState does not modify the descriptor. There is a order of operations defined for who may update the descriptor: 1) Default padding code runs based on the MPSNNPaddingMethod in the MPSCNNKernel.padding. This creates the descriptor and picks a starting value for the MPSCNNKernel.offset. 2) MPSStates are called in order to apply this function and update the offset. 3) The MPSNNPadding custom padding method of the same name is called. 4) Some code that may prove helpful: const int centeringPolicy = 0; // When kernelSize is even: 0 pad bottom right. 1 pad top left. Centers the kernel for even sized kernels. typedef enum Style{ StyleValidOnly = -1, StyleSame = 0, StyleFull = 1 }Style; // Typical destination size in one dimension for forward filters (most filters) static int DestSize( int sourceSize, int stride, int filterWindowSize, Style style ){ sourceSize += style * (filterWindowSize - 1); // adjust how many pixels we are allowed to read return (sourceSize + stride - 1) / stride; // sourceSize / stride, round up } // Typical destination size in one dimension for reverse filters (e.g. convolution transpose) static int DestSizeReverse( int sourceSize, int stride, int filterWindowSize, Style style ){ return (sourceSize-1) * stride + // center tap for the last N-1 results. Take stride into account 1 + // center tap for the first result style * (filterWindowSize-1); // add or subtract (or ignore) the filter extent } // Find the MPSOffset in one dimension static int Offset( int sourceSize, int stride, int filterWindowSize, Style style ){ // The correction needed to adjust from position of left edge to center per MPSOffset definition int correction = filterWindowSize / 2; // exit if all we want is to start consuming pixels at the left edge of the image. if( 0 ) return correction; // Center the area consumed in the source image: // Calculate the size of the destination image int destSize = DestSize( sourceSize, stride, filterWindowSize, style ); // use DestSizeReverse here instead as appropriate // calculate extent of pixels we need to read in source to populate the destination int readSize = (destSize-1) * stride + filterWindowSize; // calculate number of missing pixels in source int extraSize = readSize - sourceSize; // number of missing pixels on left side int leftExtraPixels = (extraSize + centeringPolicy) / 2; // account for the fact that the offset is based on the center pixel, not the left edge return correction - leftExtraPixels; } Parameters: sourceImages The list of source images to be used sourceStates The list of source states to be used kernel The MPSKernel the padding method will be applied to. Set the kernel.offset inDescriptor MPS will prepare a starting guess based on the padding policy (exclusive of MPSNNPaddingMethodCustom) set for the object. You should adjust the offset and image size accordingly. It is on an autoreleasepool. Returns: The MPSImageDescriptor to use to make a MPSImage to capture the results from the filter. The MPSImageDescriptor is assumed to be on an autoreleasepool. Your method must also set the kernel.offset property. - (nullable instancetype) init Reimplemented in MPSCNNDropoutGradientState, MPSCNNLossLabels, and MPSCNNArithmeticGradientState. - (nonnull instancetype) initWithDevice: (__nonnull id< MTLDevice >) device(size_t) bufferSize Reimplemented in MPSCNNInstanceNormalizationGradientState. - (nonnull instancetype) initWithDevice: (__nonnull id< MTLDevice >) device(MPSStateResourceList *__nonnull) resourceList Initialize a non-temporary state to hold a number of textures and buffers The allocation of each resource will be deferred until it is needed. This occurs when -resource or -resourceAtIndex: is called. Parameters: resourceList The list of resources to create. - (nonnull instancetype) initWithDevice: (__nonnull id< MTLDevice >) device(MTLTextureDescriptor *__nonnull) descriptor Reimplemented in MPSCNNInstanceNormalizationGradientState. - (nonnull instancetype) initWithResource: (__nullable id< MTLResource >) resource Create a MPSState with a non-temporary MTLResource Parameters: resource A MTLBuffer or MTLTexture. May be nil. Reimplemented in MPSCNNBatchNormalizationState, and MPSCNNInstanceNormalizationGradientState. - (nonnull instancetype) initWithResources: (NSArray< id< MTLResource >> *__nullable) resources Create a state object with a list of MTLResources Because MPS prefers deferred allocation of resources your application should use -initWithTextures:bufferSizes:bufferCount: whenever possible. This method is useful for cases when the MTLResources must be initialized by the CPU. - (__nullable id <MTLResource>) resourceAtIndex: (NSUInteger) index(BOOL) allocateMemory Get the MTLResource at the indicated index By convention, except where otherwise documented, the MTLResources held by the MPSState are private to the MPSState object, owned by the MPSState subclass author. If the MPSState subclass author is MPS, then the identity (e.g. texture vs. buffer) and information contained in the resource should be considered implementation dependent. It may change by operating system version or device. If you are the author of the subclass then it is for your own use, and MPS will not look at it, except perhaps so as to pass it to a custom kernel. Otherwise, the method is made available to facilitate debugging and to allow you to write your own state objects. Provide accessors to read this information in a defined format. Parameters: index The index of the MTLResource to retrieve allocateMemory It is very important to avoid allocating memory to hold MTLResources until it is absolutely necessary, especially when working with temporary MPSStates. When allocateMemory is set to NO and the resource has not yet been allocated, nil will be returned instead. If you just need information about the resource such as buffer size or MTLTexture properties, but not the resource itself, please use -bufferSizeAtIndex: or -textureInfoAtIndex: instead, as these will not force the creation of the MTLResource. - (NSUInteger) resourceSize Get the number of bytes used to allocate underyling MTLResources This is the size of the backing store of underlying MTLResources. It does not include all storage used by the object, for example the storage used to hold the MPSState instantiation and MTLTexture or MTLBuffer is not included. It only measures the size of the allocation used to hold the texels in the texture or bytes in the buffer. This value is subject to change between different devices and operating systems. Except when -initWithResource: is used, most MPSStates are allocated without a backing store. The backing store is allocated lazily when it is needed, typically when the .texture property is called. Consequently, in most cases, it should be inexpensive to make a MPSImage to see how much memory it will need, and release it if it is too large. This method may fail in certain circumstances, such as when the MPSImage is created with -initWithTexture:featureChannels:, in which case 0 will be returned. - (MPSStateResourceType) resourceTypeAtIndex: (NSUInteger) index Return YES if the resource at index is a buffer Does not force allocation of the MTLResource - (void) synchronizeOnCommandBuffer: (__nonnull id< MTLCommandBuffer >) commandBuffer Flush any copy of MTLResources held by the state from the device's caches, and invalidate any CPU caches if needed. This will call [id <MTLBlitEncoder> synchronizeResource: ] on the state's MTLResources. This is necessary for all MTLStorageModeManaged resources. For other resources, including temporary resources (these are all MTLStorageModePrivate), nothing is done. Parameters: commandBuffer The commandbuffer on which to synchronize + (nonnull instancetype) temporaryStateWithCommandBuffer: (__nonnull id< MTLCommandBuffer >) cmdBuf Create a new autoreleased temporary state object without underlying resource Parameters: cmdBuf The command buffer with which the temporary resource is associated Reimplemented in MPSCNNInstanceNormalizationGradientState. + (nonnull instancetype) temporaryStateWithCommandBuffer: (__nonnull id< MTLCommandBuffer >) cmdBuf(size_t) bufferSize Create a MPSState holding a temporary MTLBuffer Parameters: cmdBuf The command buffer against which the temporary resource is allocated bufferSize The size of the buffer in bytes Reimplemented in MPSCNNBatchNormalizationState, and MPSCNNInstanceNormalizationGradientState. + (nonnull instancetype) temporaryStateWithCommandBuffer: (__nonnull id< MTLCommandBuffer >) commandBuffer(MPSStateResourceList *__nonnull) resourceList Initialize a temporary state to hold a number of textures and buffers The textures occur first in sequence + (nonnull instancetype) temporaryStateWithCommandBuffer: (__nonnull id< MTLCommandBuffer >) cmdBuf(MTLTextureDescriptor *__nonnull) descriptor Create a MPSState holding a temporary MTLTexture Parameters: cmdBuf The command buffer against which the temporary resource is allocated descriptor A descriptor for the new temporary texture Reimplemented in MPSCNNBatchNormalizationState, and MPSCNNInstanceNormalizationGradientState. - (MPSStateTextureInfo) textureInfoAtIndex: (NSUInteger) index Return the texture size {width,height,depth} or {0,0,0} if it is not a MTLTexture Does not force allocation of the MTLResource Property Documentation - (BOOL) isTemporary [read], [nonatomic], [assign] - label [read], [write], [atomic], [copy] A string to help identify this object. - (NSUInteger) readCount [read], [write], [nonatomic], [assign] - (id<MTLResource>) resource [read], [nonatomic], [retain] Get the private MTLResource underlying the MPSState When the state is not directly initialized with a MTLResource, the actuall MTLResource creation is deferred. Especially with temporary resources, it is important to delay this creation as late as possible to avoid increasing the memory footprint. The memory is returned for reuse when the readCount = 0. Calling the -resource method will force the resource to be allocated, so you should not use it lightly, for purposes such as finding the MTLPixelFormat of a texture in the state. By convention, except where otherwise documented, the MTLResources held by the MPSState are private to the MPSState object, owned by the MPSState subclass author. If the MPSState subclass author is MPS, then the identity (e.g. texture vs. buffer) and information contained in the resource should be considered implementation dependent. It may change by operating system version or device. If you are the author of the subclass then it is for your own use, and MPS will not look at it, except perhaps so as to pass it to a custom kernel. Otherwise, the method is made available to facilitate debugging and to allow you to write your own state objects. - (NSUInteger) resourceCount [read], [nonatomic], [assign] Return the number of MTLResource objects held by the state Author Generated automatically by Doxygen for MetalPerformanceShaders.framework from the source code. Version MetalPerformanceShaders-100 Thu Feb 8 2018 MPSState(3)
