bes  Updated for version 3.17.0
BESStoredDapResultCache Class Reference

#include <BESStoredDapResultCache.h>

Inheritance diagram for BESStoredDapResultCache:
Inheritance graph

Public Member Functions

virtual bool cache_too_big (unsigned long long current_size) const
 look at the cache size; is it too large? Look at the cache size and see if it is too big. More...
 
virtual bool create_and_lock (const string &target, int &fd)
 Create a file in the cache and lock it for write access. If the file does not exist, make it, open it for read-write access and get an exclusive lock on it. The locking operation blocks, although that should never happen. More...
 
virtual void dump (ostream &strm) const
 dumps information about this object More...
 
virtual void exclusive_to_shared_lock (int fd)
 Transfer from an exclusive lock to a shared lock. If the file has an exclusive write lock on it, change that to a shared read lock. This is an atomic operation. If the call to fcntl(2) is protected by locking the cache, a dead lock will result given typical use of this class. This method exists to help with the situation where one process has the cache locked and is blocking on a shared read lock for a file that a second process has locked exclusively (for writing). By changing the exclusive lock to a shared lock, the first process can get its shared lock and then release the cache. More...
 
virtual string get_cache_file_name (const string &src, bool mangle=true)
 
virtual unsigned long long get_cache_size ()
 Get the cache size. Read the size information from the cache info file and return it. This methods locks the cache. More...
 
libdap::DDS * get_cached_dap2_data_ddx (const std::string &cache_file_name, libdap::BaseTypeFactory *factory, const std::string &dataset)
 
libdap::DMR * get_cached_dap4_data (const string &cache_file_name, libdap::D4BaseTypeFactory *factory, const string &filename)
 
virtual bool get_read_lock (const string &target, int &fd)
 Get a read-only lock on the file if it exists. More...
 
const string getCacheDirectory ()
 
const string getCacheFilePrefix ()
 
virtual bool getExclusiveLock (string file_name, int &ref_fd)
 
virtual void lock_cache_read ()
 
virtual void lock_cache_write ()
 
virtual void purge_file (const string &file)
 Purge a single file from the cache. More...
 
virtual string store_dap2_result (libdap::DDS &dds, const std::string &constraint, BESDapResponseBuilder *rb, libdap::ConstraintEvaluator *eval)
 
virtual string store_dap4_result (libdap::DMR &dmr, const string &constraint, BESDapResponseBuilder *rb)
 
virtual void unlock_and_close (const string &target)
 
virtual void unlock_cache ()
 
virtual void update_and_purge (const string &new_file)
 Purge files from the cache. More...
 
virtual unsigned long long update_cache_info (const string &target)
 Update the cache info file to include 'target'. More...
 

Static Public Member Functions

static bool dir_exists (const string &dir)
 
static BESStoredDapResultCacheget_instance (const string &bes_catalog_root_dir, const string &stored_results_subdir, const string &prefix, unsigned long long size)
 
static BESStoredDapResultCacheget_instance ()
 

Static Public Attributes

static const string PREFIX_KEY = "DAP.StoredResultsCache.prefix"
 
static const string SIZE_KEY = "DAP.StoredResultsCache.size"
 
static const string SUBDIR_KEY = "DAP.StoredResultsCache.subdir"
 

Protected Member Functions

 BESStoredDapResultCache (const string &data_root_dir, const string &stored_results_subdir, const string &prefix, unsigned long long size)
 
void initialize (const string &cache_dir, const string &prefix, unsigned long long size)
 

Protected Attributes

string d_cache_dir
 
unsigned long long d_max_cache_size_in_bytes
 How many bytes can the cache hold before we have to purge.
 
string d_prefix
 
unsigned long long d_target_size
 

Friends

class ResponseBuilderTest
 
class StoredDap2ResultTest
 
class StoredDap4ResultTest
 

Detailed Description

This class is used to cache DAP2 response objects.

Author
jhrg 5/3/13

Definition at line 52 of file BESStoredDapResultCache.h.

Member Function Documentation

bool BESFileLockingCache::cache_too_big ( unsigned long long  current_size) const
virtualinherited
Returns
True if the size is too big, false otherwise.

Definition at line 694 of file BESFileLockingCache.cc.

References BESFileLockingCache::d_max_cache_size_in_bytes.

bool BESFileLockingCache::create_and_lock ( const string &  target,
int &  fd 
)
virtualinherited
Parameters
targetThe name of the file to make/open/lock
fdValue-result param that holds the file descriptor of the opened file
Returns
True if the operation was successful, false otherwise. This method will return false if the file already existed (the file won't be locked and the descriptor reference is undefined - but likely -1).
Exceptions
BESBESInternalErrororif any error except EEXIST is returned by open(2) or if fcntl(2) returns an error.

Definition at line 527 of file BESFileLockingCache.cc.

References BESFileLockingCache::lock_cache_write(), and BESFileLockingCache::unlock_cache().

bool BESFileLockingCache::dir_exists ( const string &  dir)
staticinherited

Does the directory exist?

Parameters
dirThe pathname to test.
Returns
True if the directory exists, false otherwise

Definition at line 929 of file BESFileLockingCache.cc.

void BESFileLockingCache::dump ( ostream &  strm) const
virtualinherited

Displays the pointer value of this instance along with information about this cache.

Parameters
strmC++ i/o stream to dump the information to

Implements BESObj.

Definition at line 943 of file BESFileLockingCache.cc.

References BESFileLockingCache::d_max_cache_size_in_bytes.

void BESFileLockingCache::exclusive_to_shared_lock ( int  fd)
virtualinherited
Parameters
fdThe file descriptor that is exclusively locked and which, on exit, will have a shared lock.

Definition at line 557 of file BESFileLockingCache.cc.

unsigned long long BESFileLockingCache::get_cache_size ( )
virtualinherited
Returns
The size of the cache.

Definition at line 706 of file BESFileLockingCache.cc.

References BESFileLockingCache::lock_cache_read(), and BESFileLockingCache::unlock_cache().

DDS * BESStoredDapResultCache::get_cached_dap2_data_ddx ( const std::string &  cache_file_name,
libdap::BaseTypeFactory *  factory,
const std::string &  dataset 
)

Read data from cache. Allocates a new DDS using the given factory.

Definition at line 501 of file BESStoredDapResultCache.cc.

DMR * BESStoredDapResultCache::get_cached_dap4_data ( const string &  cache_file_name,
libdap::D4BaseTypeFactory *  factory,
const string &  filename 
)

Read data from cache. Allocates a new DDS using the given factory. If the file does not exists this will return null (0).

Definition at line 543 of file BESStoredDapResultCache.cc.

BESStoredDapResultCache * BESStoredDapResultCache::get_instance ( )
static

Get the default instance of the BESStoredDapResultCache object. This will read "TheBESKeys" looking for the values of SUBDIR_KEY, PREFIX_KEY, an SIZE_KEY to initialize the cache.

Definition at line 259 of file BESStoredDapResultCache.cc.

References BESError::get_message(), and BESDebug::IsSet().

bool BESFileLockingCache::get_read_lock ( const string &  target,
int &  fd 
)
virtualinherited

Try to get a read-only lock on the file, blocking until we can get it. If the file does not exist, return false.

Note
If this code returns false, that means the file did not exist in the cache at the time of the test. by the time the caller gets the result, the file may have been added to the cache by another process.
Parameters
srcsrc file that will be cached eventually
targeta value-result parameter set to the resulting cached file
Returns
true if the file is in the cache and has been locked, false if the file is/was not in the cache.
Exceptions
Errorif the attempt to get the (shared) lock failed for any reason other than that the file does/did not exist.

Definition at line 499 of file BESFileLockingCache.cc.

References BESFileLockingCache::lock_cache_read(), and BESFileLockingCache::unlock_cache().

bool BESFileLockingCache::getExclusiveLock ( string  file_name,
int &  ref_fd 
)
virtualinherited

Get an exclusive read/write lock on an existing file.

Parameters
file_nameThe name of the file.
ref_fpif successful, the file descriptor of the file on which we have an exclusive read/write lock.
Returns
If the file does not exist, return immediately indicating failure (false), otherwise block until an exclusive read/write lock can be obtained and then return true.
Exceptions
Erroris thrown to indicate a number of untoward events.

Definition at line 200 of file BESFileLockingCache.cc.

References BESUtil::assemblePath(), BESFileLockingCache::d_max_cache_size_in_bytes, and BESFileLockingCache::unlock_cache().

void BESFileLockingCache::lock_cache_read ( )
virtualinherited

Get a shared lock on the 'cache info' file.

Definition at line 591 of file BESFileLockingCache.cc.

void BESFileLockingCache::lock_cache_write ( )
virtualinherited

Get an exclusive lock on the 'cache info' file. The 'cache info' file is used to control certain cache actions, ensuring that they are atomic. These include making sure that the create_and_lock() and read_and_lock() operations are atomic as well as the purge and related operations.

Note
This is intended to be used internally only but might be useful in some settings.

Definition at line 579 of file BESFileLockingCache.cc.

void BESFileLockingCache::purge_file ( const string &  file)
virtualinherited

Purge a single file from the cache. The file might be old, etc., and need to be removed. Don't use this to shrink the cache when it gets too big, use update_and_purge() instead since that file optimizes accesses to the cache control file for several changes in a row.

Todo:
This is a new feature; add to BESCache3
Parameters
fileThe name of the file to purge.

Definition at line 873 of file BESFileLockingCache.cc.

References BESFileLockingCache::get_cache_size(), BESFileLockingCache::getExclusiveLock(), BESFileLockingCache::lock_cache_write(), and BESFileLockingCache::unlock_cache().

string BESStoredDapResultCache::store_dap2_result ( libdap::DDS &  dds,
const std::string &  constraint,
BESDapResponseBuilder rb,
libdap::ConstraintEvaluator *  eval 
)
virtual
Returns
The local ID (relative to the BES data root directory) of the stored dataset.

FIXME I think we need to delete this here, before we close and unlock the cache.

Definition at line 572 of file BESStoredDapResultCache.cc.

References BESUtil::assemblePath(), BESDapResponseBuilder::get_dataset_name(), and BESDapResponseBuilder::serialize_dap2_data_ddx().

string BESStoredDapResultCache::store_dap4_result ( libdap::DMR &  dmr,
const string &  constraint,
BESDapResponseBuilder rb 
)
virtual
Returns
The local ID (relative to the BES data root directory) of the stored dataset.

Definition at line 766 of file BESStoredDapResultCache.cc.

References BESDapResponseBuilder::serialize_dap4_data().

void BESFileLockingCache::unlock_and_close ( const string &  file_name)
virtualinherited

Unlock the named file.

This does not do any name mangling; it just closes and unlocks whatever is named (or throws BESBESInternalErroror if the file cannot be closed). If the file was opened more than once, all descriptors are closed. If you need to close a specific descriptor, use the other version of unlock_and_close().

Note
This method assumes that the file was opened/locked using one of read_and_lock() or create_and_lock(). Those methods record the name/file- descriptor pairs so that the files can be properly closed and locks released.
Parameters
file_nameThe name of the file to unlock.
Exceptions
BESBESInternalErroror

Definition at line 629 of file BESFileLockingCache.cc.

void BESFileLockingCache::unlock_cache ( )
virtualinherited

Unlock the cache info file.

Note
This is intended to be used internally only bt might be useful in some settings.

Definition at line 605 of file BESFileLockingCache.cc.

void BESFileLockingCache::update_and_purge ( const string &  new_file)
virtualinherited

Purge files, oldest to newest, if the current size of the cache exceeds the size of the cache specified in the constructor. This method uses an exclusive lock on the cache for the duration of the purge process.

Parameters
new_fileThe name of a file this process just added to the cache. Using fcntl(2) locking there is no way this process can detect its own lock, so the shared read lock on the new file won't keep this process from deleting it (but will keep other processes from deleting it).

Definition at line 789 of file BESFileLockingCache.cc.

References BESFileLockingCache::cache_too_big(), BESFileLockingCache::lock_cache_write(), and BESFileLockingCache::unlock_cache().

unsigned long long BESFileLockingCache::update_cache_info ( const string &  target)
virtualinherited

Add the size of the named file to the total cache size recorded in the cache info file. The cache info file is exclusively locked by this method for its duration. This updates the cache info file and returns the new size.

Parameters
targetThe name of the file
Returns
The new size of the cache

Definition at line 652 of file BESFileLockingCache.cc.

References BESFileLockingCache::lock_cache_write(), and BESFileLockingCache::unlock_cache().


The documentation for this class was generated from the following files: