Field3D
SparseFileManager Class Reference

#include <SparseFile.h>

List of all members.

Public Types

typedef std::list
< SparseFile::CacheBlock
CacheList

Public Member Functions

template<class Data_T >
void activateBlock (int fileId, int blockIdx)
 Called by SparseField when it's about to read from a block. This should not be called by the user, and may be removed from the public interface later.
float cacheEfficiency ()
 Computes the efficiency, the ratio of the number of blocks ever loaded to the number of loads. If this is <1, then there were reloads.
float cacheFractionLoaded ()
 Computes the ratio of blocks in the cache to the total number of blocks that have been loaded (including unloaded blocks)
float cacheLoadsPerBlock ()
 Computes the overall loaded-blocks-to-load ratio for cached files.
template<class Data_T >
void decBlockRef (int fileId, int blockIdx)
 Decrements the usage reference count on the specified block, after its value is no longer being used This should not be called by the user, and may be removed from the public interface later.
bool doLimitMemUse () const
 Returns whether to limit memory usage and do dynamic loading for sparse fields.
void flushCache ()
 Flushes the entire block cache for all files, should probably only be used for debugging.
template<class Data_T >
void incBlockRef (int fileId, int blockIdx)
 Increments the usage reference count on the specified block, to prevent it from getting unloaded while it's still in use. This should not be called by the user, and may be removed from the public interface later.
long long numLoadedBlocks ()
 Returns the total number of blocks currently loaded into cache.
void resetCacheStatistics ()
 Resets block load.
void setLimitMemUse (bool enabled)
 Sets whether to limit memory usage and do dynamic loading for sparse fields.
void setMaxMemUse (float maxMemUse)
 Sets the maximum memory usage, in MB, by dynamically loaded sparse fields.
long long totalLoadedBlocks ()
 Returns the total number of blocks loaded (max 1 per block) into cache.
long long totalLoads ()
 Returns the total number of block loads in the cache.

Static Public Member Functions

static SparseFileManagersingleton ()
 Returns a reference to the singleton instance.

Protected Member Functions

template<class Data_T >
int getNextId (const std::string filename, const std::string layerPath)
 Returns the id of the next cache item. This is stored in the SparseField in order to reference its fields at a later time.
template<class Data_T >
SparseFile::Reference< Data_T > & reference (int index)
 Returns a reference to the Reference object with the given index.
template<class Data_T >
void removeFieldFromCache (int refIdx)

Private Member Functions

void addBlockToCache (DataTypeEnum blockType, int fileId, int blockIdx)
 Adds the newly loaded block to the cache, managed by the paging algorithm.
template<class Data_T >
void deallocateBlock (CacheList::iterator &it)
 Utility function to deallocate a single block.
template<class Data_T >
int deallocateBlock (const SparseFile::CacheBlock &cb)
 Utility function to attempt to deallocate a single block and advance the "hand".
void deallocateBlocks (int bytesNeeded)
 Utility function to reclaim the specified number of bytes by deallocating unneeded blocks.
 SparseFileManager ()
 Private to prevent instantiation.

Private Attributes

CacheList m_blockCacheList
 List of dynamically loaded blocks to be considered for unloading when the cache is full. Currently using Second-chance/Clock paging algorithm. For a description of the algorithm, look at: http://en.wikipedia.org/wiki/Page_replacement_algorithm#Second-chance.
SparseFile::FileReferences m_fileData
 Vector containing information for each of the managed fields. The order matches the index stored in each SparseField::m_fileId.
bool m_limitMemUse
 Whether to limit memory use of sparse fields from disk. Enables the cache and dynamic loading when true.
float m_maxMemUse
 Max amount om memory to use in megabytes.
int m_maxMemUseInBytes
 Max amount om memory to use in bytes.
int m_memUse
 Current amount of memory in use in bytes.
boost::mutex m_mutex
 Mutex to prevent multiple threads from deallocating blocks at the same time.
CacheList::iterator m_nextBlock
 Pointer to the next block to test for unloading in the cache, the "hand" of the clock.

Static Private Attributes

static SparseFileManagerms_singleton = 0
 Pointer to singleton.

Friends

class SparseField

Detailed Description

Handles specifics about reading sparse fields from disk. Its primary use is to control sparse fields read using memory limiting (dynamic loading).

To enable the dynamic cache for a file, call setLimitMemUse(true) before opening the file. If you want other files to be fully loaded, call setLimitMemUse(false).

Example of how to use the cache manager to automatically unload sparse blocks from a f3d file:

  SparseFileManager &sparseManager = SparseFileManager::singleton();
  sparseManager.setLimitMemUse(true);  // enables cache for files to be opened 
  sparseManager.setMaxMemUse(1000.0);  // sets cache to 1 GB
  Field3DInputFile cacheManagedFile;
  if (!cacheManagedFile.open(filename)) {
    Msg::print( "Couldn't open file: " + filename);
    return 1;
  }
  sparseManager.setLimitMemUse(false);  // disables cache for other files
  ... // You can use the file normally, loading layers and then accessing
  ... // with const_iterator, value(), or empty block functions like
  ... // getBlockEmptyValue().
  ... // Layers loaded from cacheManagedFile will be managed by the cache,
  ... // but other files you open will be fully loaded when opened because of
  ... // the setLimitMemUse(false) call.
  Msg::print("blocks in cache: " +
    boost::lexical_cast<std::string>(sparseManager.numLoadedBlocks()));
  Msg::print("cache blocks ever loaded: " +
    boost::lexical_cast<std::string>(sparseManager.totalLoadedBlocks()));
  Msg::print("cache loads: " +
    boost::lexical_cast<std::string>(sparseManager.totalLoads()));
  Msg::print("cache fraction loaded: " +
    boost::lexical_cast<std::string>(sparseManager.cacheFractionLoaded()));
  Msg::print("cache loads per block: " +
    boost::lexical_cast<std::string>(sparseManager.cacheLoadsPerBlock()));
  Msg::print("cache efficiency: " +
    boost::lexical_cast<std::string>(sparseManager.cacheEfficiency()));
  

If you want to flush the cache manually instead of waiting for the process to end and clean up its memory:

  sparseManager.flushCache();
  sparseManager.resetCacheStatistics();
  

Definition at line 316 of file SparseFile.h.


Member Typedef Documentation

Definition at line 326 of file SparseFile.h.


Constructor & Destructor Documentation

SparseFileManager::SparseFileManager ( ) [private]

Private to prevent instantiation.

Definition at line 274 of file SparseFile.cpp.

References m_blockCacheList, m_nextBlock, and setMaxMemUse().

Referenced by singleton().

  : m_memUse(0),
    m_limitMemUse(false)
{
  setMaxMemUse(1000.0);
  m_nextBlock = m_blockCacheList.begin();
}

Member Function Documentation

SparseFileManager & SparseFileManager::singleton ( ) [static]

Returns a reference to the singleton instance.

Definition at line 63 of file SparseFile.cpp.

References ms_singleton, and SparseFileManager().

Referenced by SparseField< Data_T >::addReference(), and SparseFieldIO::readData().

void SparseFileManager::setLimitMemUse ( bool  enabled)

Sets whether to limit memory usage and do dynamic loading for sparse fields.

Definition at line 73 of file SparseFile.cpp.

References m_limitMemUse.

{
  m_limitMemUse = enabled;
}
bool SparseFileManager::doLimitMemUse ( ) const

Returns whether to limit memory usage and do dynamic loading for sparse fields.

Definition at line 80 of file SparseFile.cpp.

References m_limitMemUse.

Referenced by SparseFieldIO::readData().

{ 
  return m_limitMemUse; 
}
void SparseFileManager::setMaxMemUse ( float  maxMemUse)

Sets the maximum memory usage, in MB, by dynamically loaded sparse fields.

Definition at line 87 of file SparseFile.cpp.

References m_maxMemUse, and m_maxMemUseInBytes.

Referenced by SparseFileManager().

{
  m_maxMemUse = maxMemUse;
  m_maxMemUseInBytes = static_cast<int>(m_maxMemUse * 1024*1024);
}
void SparseFileManager::flushCache ( )

Flushes the entire block cache for all files, should probably only be used for debugging.

Definition at line 215 of file SparseFile.cpp.

References SparseFile::CacheBlock::blockType, DataTypeDouble, DataTypeFloat, DataTypeHalf, DataTypeUnknown, DataTypeVecDouble, DataTypeVecFloat, DataTypeVecHalf, m_blockCacheList, m_mutex, and m_nextBlock.

{
  boost::mutex::scoped_lock lock(m_mutex);

  CacheList::iterator it = m_blockCacheList.begin();
  while (it != m_blockCacheList.end()) {
    SparseFile::CacheBlock &cb = *it;

    switch(cb.blockType) {
    case DataTypeHalf:
      deallocateBlock<half>(it);
      break;
    case DataTypeFloat:
      deallocateBlock<float>(it);
      break;
    case DataTypeDouble:
      deallocateBlock<double>(it);
      break;
    case DataTypeVecHalf:
      deallocateBlock<V3h>(it);
      break;
    case DataTypeVecFloat:
      deallocateBlock<V3f>(it);
      break;
    case DataTypeVecDouble:
      deallocateBlock<V3d>(it);
      break;
    case DataTypeUnknown:
    default:
      break;
    }
  }
  m_nextBlock = m_blockCacheList.begin();
}
long long SparseFileManager::totalLoads ( )

Returns the total number of block loads in the cache.

Definition at line 284 of file SparseFile.cpp.

References half, m_fileData, SparseFile::FileReferences::numRefs(), and SparseFile::FileReferences::ref().

Referenced by cacheEfficiency(), and cacheLoadsPerBlock().

{

  long long int numLoads = 0;

  for (int i=0; i<m_fileData.numRefs<half>(); i++) {
    numLoads += m_fileData.ref<half>(i).totalLoads();
  }

  for (int i=0; i<m_fileData.numRefs<V3h>(); i++) {
    numLoads += m_fileData.ref<V3h>(i).totalLoads();
  }

  for (int i=0; i<m_fileData.numRefs<float>(); i++) {
    numLoads += m_fileData.ref<float>(i).totalLoads();
  }

  for (int i=0; i<m_fileData.numRefs<V3f>(); i++) {
    numLoads += m_fileData.ref<V3f>(i).totalLoads();
  }

  for (int i=0; i<m_fileData.numRefs<double>(); i++) {
    numLoads += m_fileData.ref<double>(i).totalLoads();
  }

  for (int i=0; i<m_fileData.numRefs<V3d>(); i++) {
    numLoads += m_fileData.ref<V3d>(i).totalLoads();
  }
  return numLoads;
}
long long SparseFileManager::numLoadedBlocks ( )

Returns the total number of blocks currently loaded into cache.

Definition at line 317 of file SparseFile.cpp.

References half, m_fileData, SparseFile::FileReferences::numRefs(), and SparseFile::FileReferences::ref().

Referenced by cacheFractionLoaded().

{

  long long int numBlocks = 0;

  for (int i=0; i<m_fileData.numRefs<half>(); i++) {
    numBlocks += m_fileData.ref<half>(i).numLoadedBlocks();
  }

  for (int i=0; i<m_fileData.numRefs<V3h>(); i++) {
    numBlocks += m_fileData.ref<V3h>(i).numLoadedBlocks();
  }

  for (int i=0; i<m_fileData.numRefs<float>(); i++) {
    numBlocks += m_fileData.ref<float>(i).numLoadedBlocks();
  }

  for (int i=0; i<m_fileData.numRefs<V3f>(); i++) {
    numBlocks += m_fileData.ref<V3f>(i).numLoadedBlocks();
  }

  for (int i=0; i<m_fileData.numRefs<double>(); i++) {
    numBlocks += m_fileData.ref<double>(i).numLoadedBlocks();
  }

  for (int i=0; i<m_fileData.numRefs<V3d>(); i++) {
    numBlocks += m_fileData.ref<V3d>(i).numLoadedBlocks();
  }
  return numBlocks;
}
long long SparseFileManager::totalLoadedBlocks ( )

Returns the total number of blocks loaded (max 1 per block) into cache.

Definition at line 350 of file SparseFile.cpp.

References half, m_fileData, SparseFile::FileReferences::numRefs(), and SparseFile::FileReferences::ref().

Referenced by cacheEfficiency(), cacheFractionLoaded(), and cacheLoadsPerBlock().

{

  long long int numBlocks = 0;

  for (int i=0; i<m_fileData.numRefs<half>(); i++) {
    numBlocks += m_fileData.ref<half>(i).totalLoadedBlocks();
  }

  for (int i=0; i<m_fileData.numRefs<V3h>(); i++) {
    numBlocks += m_fileData.ref<V3h>(i).totalLoadedBlocks();
  }

  for (int i=0; i<m_fileData.numRefs<float>(); i++) {
    numBlocks += m_fileData.ref<float>(i).totalLoadedBlocks();
  }

  for (int i=0; i<m_fileData.numRefs<V3f>(); i++) {
    numBlocks += m_fileData.ref<V3f>(i).totalLoadedBlocks();
  }

  for (int i=0; i<m_fileData.numRefs<double>(); i++) {
    numBlocks += m_fileData.ref<double>(i).totalLoadedBlocks();
  }

  for (int i=0; i<m_fileData.numRefs<V3d>(); i++) {
    numBlocks += m_fileData.ref<V3d>(i).totalLoadedBlocks();
  }
  return numBlocks;
}
float SparseFileManager::cacheFractionLoaded ( )

Computes the ratio of blocks in the cache to the total number of blocks that have been loaded (including unloaded blocks)

Definition at line 383 of file SparseFile.cpp.

References numLoadedBlocks(), and totalLoadedBlocks().

{
  return ((double)numLoadedBlocks())/std::max(1.0, ((double)totalLoadedBlocks()));
}
float SparseFileManager::cacheLoadsPerBlock ( )

Computes the overall loaded-blocks-to-load ratio for cached files.

Definition at line 390 of file SparseFile.cpp.

References totalLoadedBlocks(), and totalLoads().

{
  return ((double)totalLoads())/std::max(1.0, ((double)totalLoadedBlocks()));
}
float SparseFileManager::cacheEfficiency ( )

Computes the efficiency, the ratio of the number of blocks ever loaded to the number of loads. If this is <1, then there were reloads.

Definition at line 397 of file SparseFile.cpp.

References totalLoadedBlocks(), and totalLoads().

{
  return ((double)totalLoadedBlocks())/std::max(1.0, ((double)totalLoads()));
}
void SparseFileManager::resetCacheStatistics ( )

Resets block load.

Definition at line 404 of file SparseFile.cpp.

References half, m_fileData, SparseFile::FileReferences::numRefs(), and SparseFile::FileReferences::ref().

{

  for (int i=0; i<m_fileData.numRefs<half>(); i++) {
    m_fileData.ref<half>(i).resetCacheStatistics();
  }

  for (int i=0; i<m_fileData.numRefs<V3h>(); i++) {
    m_fileData.ref<V3h>(i).resetCacheStatistics();
  }

  for (int i=0; i<m_fileData.numRefs<float>(); i++) {
    m_fileData.ref<float>(i).resetCacheStatistics();
  }

  for (int i=0; i<m_fileData.numRefs<V3f>(); i++) {
    m_fileData.ref<V3f>(i).resetCacheStatistics();
  }

  for (int i=0; i<m_fileData.numRefs<double>(); i++) {
    m_fileData.ref<double>(i).resetCacheStatistics();
  }

  for (int i=0; i<m_fileData.numRefs<V3d>(); i++) {
    m_fileData.ref<V3d>(i).resetCacheStatistics();
  }
}
template<class Data_T >
void SparseFileManager::incBlockRef ( int  fileId,
int  blockIdx 
)

Increments the usage reference count on the specified block, to prevent it from getting unloaded while it's still in use. This should not be called by the user, and may be removed from the public interface later.

Definition at line 1056 of file SparseFile.h.

References SparseFile::Reference< Data_T >::fileBlockIndices, and SparseFile::Reference< Data_T >::incBlockRef().

{
  SparseFile::Reference<Data_T> &reference = m_fileData.ref<Data_T>(fileId);

  if (reference.fileBlockIndices[blockIdx] >= 0) {
    reference.incBlockRef(blockIdx);
  }
}
template<class Data_T >
void SparseFileManager::decBlockRef ( int  fileId,
int  blockIdx 
)

Decrements the usage reference count on the specified block, after its value is no longer being used This should not be called by the user, and may be removed from the public interface later.

Definition at line 1069 of file SparseFile.h.

References SparseFile::Reference< Data_T >::decBlockRef(), and SparseFile::Reference< Data_T >::fileBlockIndices.

{
  SparseFile::Reference<Data_T> &reference = m_fileData.ref<Data_T>(fileId);

  if (reference.fileBlockIndices[blockIdx] >= 0) {
    reference.decBlockRef(blockIdx);
  }
}
template<class Data_T >
void SparseFileManager::activateBlock ( int  fileId,
int  blockIdx 
)

Called by SparseField when it's about to read from a block. This should not be called by the user, and may be removed from the public interface later.

Definition at line 1020 of file SparseFile.h.

References SparseFile::Reference< Data_T >::blockLoaded, SparseFile::Reference< Data_T >::blockMutex, SparseFile::Reference< Data_T >::blockSize(), SparseFile::Reference< Data_T >::blockUsed, SparseFile::Reference< Data_T >::fileBlockIndices, SparseFile::Reference< Data_T >::fileIsOpen(), SparseFile::Reference< Data_T >::loadBlock(), SparseFile::Reference< Data_T >::loadCounts, and SparseFile::Reference< Data_T >::openFile().

Referenced by SparseField< Data_T >::const_iterator::operator->().

{
  SparseFile::Reference<Data_T> &reference = m_fileData.ref<Data_T>(fileId);

  if (reference.fileBlockIndices[blockIdx] >= 0) {
    if (!reference.blockLoaded[blockIdx]) {
      int blockSize = reference.blockSize(blockIdx);
      if (m_limitMemUse) {
        // if we already have enough free memory, deallocateBlocks()
        // will just return
        deallocateBlocks(blockSize);
      }

      if (!reference.fileIsOpen()) {
        reference.openFile();
      }

      boost::mutex::scoped_lock lock_A(m_mutex);
      boost::mutex::scoped_lock lock_B(reference.blockMutex[blockIdx]);
      // check to see if it was loaded between when the function
      // started and we got the lock on the block
      if (!reference.blockLoaded[blockIdx]) {
        reference.loadBlock(blockIdx);
        reference.loadCounts[blockIdx]++;
        addBlockToCache(DataTypeTraits<Data_T>::typeEnum(), fileId, blockIdx);
        m_memUse += blockSize;
      }
    }
  }
  reference.blockUsed[blockIdx] = true;
}
template<class Data_T >
SparseFile::Reference< Data_T > & SparseFileManager::reference ( int  index) [protected]

Returns a reference to the Reference object with the given index.

Definition at line 1011 of file SparseFile.h.

Referenced by deallocateBlock().

{ 
  return m_fileData.ref<Data_T>(index); 
}
template<class Data_T >
int SparseFileManager::getNextId ( const std::string  filename,
const std::string  layerPath 
) [protected]

Returns the id of the next cache item. This is stored in the SparseField in order to reference its fields at a later time.

Definition at line 955 of file SparseFile.h.

{
  using namespace SparseFile;

  int id = m_fileData.append(Reference<Data_T>(filename, layerPath));
  return id;
}
template<class Data_T >
void SparseFileManager::removeFieldFromCache ( int  refIdx) [protected]

Definition at line 968 of file SparseFile.h.

References SparseFile::Reference< Data_T >::blockLoaded, SparseFile::Reference< Data_T >::blocks, SparseFile::Reference< Data_T >::blockSize(), SparseFile::Reference< Data_T >::blockUsed, and SparseFile::Reference< Data_T >::fileBlockIndices.

{
  boost::mutex::scoped_lock lock(m_mutex);

  DataTypeEnum blockType = DataTypeTraits<Data_T>::typeEnum();
  SparseFile::Reference<Data_T> &reference = m_fileData.ref<Data_T>(refIdx);

  CacheList::iterator it = m_blockCacheList.begin();
  CacheList::iterator end = m_blockCacheList.end();
  CacheList::iterator next;

  int bytesFreed = 0;

  while (it != end) {
    if (it->blockType == blockType && it->refIdx == refIdx) {
      if (it == m_nextBlock) {
        ++m_nextBlock;
      }
      next = it;
      ++next;
      bytesFreed += reference.blockSize(it->blockIdx);
      m_blockCacheList.erase(it);
      it = next;
    } else {
      ++it;
    }
  }
  m_memUse -= bytesFreed;

  // reset the block indices to -1, to ensure that the cache manager
  // won't try to activate a block
  reference.fileBlockIndices.clear();
  reference.fileBlockIndices.resize(reference.blocks.size(), -1);
  // clear the reference's pointers into the field, and relevant 
  reference.blocks.clear();
  reference.blockLoaded.clear();
  reference.blockUsed.clear();
}
void SparseFileManager::addBlockToCache ( DataTypeEnum  blockType,
int  fileId,
int  blockIdx 
) [private]

Adds the newly loaded block to the cache, managed by the paging algorithm.

Definition at line 252 of file SparseFile.cpp.

References m_blockCacheList, and m_nextBlock.

{
  // Note: this lock is obtained while we also have a lock on the
  // specific block (in activateBlock()), so we should make sure we
  // never lock the SparseFileManager and *then* a block, to ensure we
  // don't have a deadlock.
  //
  //  Note: this was changed so the order was consistent w/ dealloc
  //  again, see activateBlock()
  //  boost::mutex::scoped_lock lock(m_mutex);

  SparseFile::CacheBlock block(blockType, fileId, blockIdx);
  if (m_nextBlock == m_blockCacheList.end()) {
    m_blockCacheList.push_back(block);
  } else {
    m_blockCacheList.insert(m_nextBlock, block);
  }
}
void SparseFileManager::deallocateBlocks ( int  bytesNeeded) [private]

Utility function to reclaim the specified number of bytes by deallocating unneeded blocks.

Definition at line 152 of file SparseFile.cpp.

References SparseFile::CacheBlock::blockType, DataTypeDouble, DataTypeFloat, DataTypeHalf, DataTypeUnknown, DataTypeVecDouble, DataTypeVecFloat, DataTypeVecHalf, m_blockCacheList, m_maxMemUseInBytes, m_memUse, m_mutex, and m_nextBlock.

{
  boost::mutex::scoped_lock lock_A(m_mutex);

  while (m_blockCacheList.begin() != m_blockCacheList.end() &&
         m_maxMemUseInBytes-m_memUse < bytesNeeded) {

    if (m_nextBlock == m_blockCacheList.end())
      m_nextBlock = m_blockCacheList.begin();

    SparseFile::CacheBlock &cb = *m_nextBlock;

    // if bytesFreed is set to >0, then we've already freed a block
    // and advanced the "clock hand" iterator
    int bytesFreed = 0;

    switch(cb.blockType) {
    case DataTypeHalf:
      bytesFreed = deallocateBlock<half>(cb);
      if (bytesFreed > 0) {
        continue;
      }
      break;
    case DataTypeFloat:
      bytesFreed = deallocateBlock<float>(cb);
      if (bytesFreed > 0) {
        continue;
      }
      break;
    case DataTypeDouble:
      bytesFreed = deallocateBlock<double>(cb);
      if (bytesFreed > 0) {
        continue;
      }
      break;
    case DataTypeVecHalf:
      bytesFreed = deallocateBlock<V3h>(cb);
      if (bytesFreed > 0) {
        continue;
      }
      break;
    case DataTypeVecFloat:
      bytesFreed = deallocateBlock<V3f>(cb);
      if (bytesFreed > 0) {
        continue;
      }
      break;
    case DataTypeVecDouble:
      bytesFreed = deallocateBlock<V3d>(cb);
      if (bytesFreed > 0) {
        continue;
      }
      break;
    case DataTypeUnknown:
    default:
      break;
    }
    ++m_nextBlock;
  }
}
template<class Data_T >
int SparseFileManager::deallocateBlock ( const SparseFile::CacheBlock cb) [private]

Utility function to attempt to deallocate a single block and advance the "hand".

Definition at line 96 of file SparseFile.cpp.

References SparseFile::CacheBlock::blockIdx, SparseFile::Reference< Data_T >::blockMutex, SparseFile::Reference< Data_T >::blockSize(), SparseFile::Reference< Data_T >::blockUsed, m_blockCacheList, m_fileData, m_memUse, m_nextBlock, SparseFile::FileReferences::ref(), SparseFile::Reference< Data_T >::refCounts, reference(), SparseFile::CacheBlock::refIdx, and SparseFile::Reference< Data_T >::unloadBlock().

{
  int bytesFreed = 0;
  SparseFile::Reference<Data_T> &reference = m_fileData.ref<Data_T>(cb.refIdx);

  // Note: we don't need to lock the block's mutex because
  // deallocateBlock() is only called while the SparseFileManager's
  // mutex is also locked (in flushCache() or deallocateBlocks()).
  // Don't lock the block, to make sure we don't have a deadlock by
  // holding two locks at the same time.  (Because addBlockToCache()
  // locks the manager but is also in a block-specific lock.)

  // lock the current block to make sure its blockUsed flag and ref
  // counts don't change
  // Note: this lock order is made consistent w/ allocate to prevent
  // deadlocks and crashes.

  boost::mutex::scoped_lock lock_B(reference.blockMutex[cb.blockIdx]);
  
  // check whether the block is still in use
  if (reference.refCounts[cb.blockIdx] > 0)
    return bytesFreed;

  if (reference.blockUsed[cb.blockIdx]) {
    // the block was recently used according to Second-chance paging
    // algorithm, so skip it
    reference.blockUsed[cb.blockIdx] = false;
  }
  else {

    // the block wasn't in use, so free it
    reference.unloadBlock(cb.blockIdx);
    bytesFreed = reference.blockSize(cb.blockIdx);
    m_memUse -= bytesFreed;
    CacheList::iterator toRemove = m_nextBlock;
    ++m_nextBlock;
    m_blockCacheList.erase(toRemove);
  }
  return bytesFreed;
}
template<class Data_T >
void SparseFileManager::deallocateBlock ( CacheList::iterator &  it) [private]

Friends And Related Function Documentation

friend class SparseField [friend]

Definition at line 322 of file SparseFile.h.


Member Data Documentation

FIELD3D_NAMESPACE_OPEN SparseFileManager * SparseFileManager::ms_singleton = 0 [static, private]

Pointer to singleton.

Definition at line 414 of file SparseFile.h.

Referenced by singleton().

Max amount om memory to use in megabytes.

Definition at line 433 of file SparseFile.h.

Referenced by setMaxMemUse().

Max amount om memory to use in bytes.

Definition at line 436 of file SparseFile.h.

Referenced by deallocateBlocks(), and setMaxMemUse().

Current amount of memory in use in bytes.

Definition at line 439 of file SparseFile.h.

Referenced by deallocateBlock(), and deallocateBlocks().

Whether to limit memory use of sparse fields from disk. Enables the cache and dynamic loading when true.

Definition at line 443 of file SparseFile.h.

Referenced by doLimitMemUse(), and setLimitMemUse().

Vector containing information for each of the managed fields. The order matches the index stored in each SparseField::m_fileId.

Definition at line 447 of file SparseFile.h.

Referenced by deallocateBlock(), numLoadedBlocks(), resetCacheStatistics(), totalLoadedBlocks(), and totalLoads().

List of dynamically loaded blocks to be considered for unloading when the cache is full. Currently using Second-chance/Clock paging algorithm. For a description of the algorithm, look at: http://en.wikipedia.org/wiki/Page_replacement_algorithm#Second-chance.

Definition at line 454 of file SparseFile.h.

Referenced by addBlockToCache(), deallocateBlock(), deallocateBlocks(), flushCache(), and SparseFileManager().

CacheList::iterator SparseFileManager::m_nextBlock [private]

Pointer to the next block to test for unloading in the cache, the "hand" of the clock.

Definition at line 458 of file SparseFile.h.

Referenced by addBlockToCache(), deallocateBlock(), deallocateBlocks(), flushCache(), and SparseFileManager().

boost::mutex SparseFileManager::m_mutex [private]

Mutex to prevent multiple threads from deallocating blocks at the same time.

Definition at line 462 of file SparseFile.h.

Referenced by deallocateBlocks(), and flushCache().


The documentation for this class was generated from the following files: