Class

eZDFSFileHandler

class eZDFSFileHandler implements eZClusterFileHandlerInterface, ezpDatabaseBasedClusterFileHandler

Handles file operations for Distributed File Systems (f.e.

NFS)

Uses a dual DB / FS approach: - files metadata are DB based - files data are read/written to a local mount point (outside var/) - actual files are locally written, exactly like the DB handler does

Glossary of terms used in the internal doc: - DFS: Distributed File System. Local NFS mount point - DB: MetaData database - LFS: Local file system (var)

Constants

LOCAL_CACHE

Controls whether file data from database is cached on the local filesystem.

INFOCACHE_MAX

Controls the maximum number of metdata entries to keep in memory for this request.

If the limit is reached the least used entries are removed.

Properties

string $filePath Path to the current file

Methods

__construct(string $filePath = false)

Constructor

disconnect()

Disconnects the cluster handler from the database

void loadMetaData(bool $force = false)

Loads file meta information.

void fileStore(string $filePath, string $scope = false, bool $delete = false, string $datatype = false)

Stores a file by path to the backend

void fileStoreContents(string $filePath, string $contents, string $scope = false, string $datatype = false)

Store file contents.

storeContents(string $contents, string $scope = false, string $datatype = false, bool $storeLocally = false)

Store file contents using binary data

string|false fileFetch(string $filePath)

Fetches file from DFS and saves it in LFS under the same name.

string fetchUnique()

Fetches file from db and saves it in FS under a unique name

fetch(bool $noLocalCache = false)

Fetches file from DFS and saves it in FS under the same name.

bool|string fileFetchContents(string $filePath)

Returns file contents.

string|bool fetchContents()

Returns file contents.

processCache($retrieveCallback, $generateCallback = null, $ttl = null, $expiry = null, $extraData = null)

Handles cache requests / write operations

static bool isFileExpired(string $fname, int $mtime, int $expiry, int $curtime, int $ttl)

Calculates if the file data is expired or not.

bool isExpired(int $expiry, int $curtime, int $ttl)

Calculates if the current file data is expired or not.

bool isLocalFileExpired(int $expiry, int $curtime, int $ttl)

Calculates if the local file is expired or not.

bool isDBFileExpired(int $expiry, int $curtime, int $ttl)

Calculates if the DB file is expired or not.

string|null storeCache(string|array $fileData)

Stores the data in $fileData to the remote and local file and commits the transaction.

processFile($callback, $expiry = false, $extraData = null)

Provides access to the file contents by downloading the file locally and calling $callback with the local filename.

stat()

Returns file metadata.

int|null size()

Returns file size.

string|null dataType()

Returns file mime-type / content-type.

int|null mtime()

Returns file modification time.

string|null name()

Returns file name.

void fileDeleteByWildcard(string $wildcard)

Deletes a list of files by wildcard

void fileDeleteByDirList(array $dirList, string $commonPath, string $commonSuffix)

Deletes a list of files based on directory / filename components

void fileDelete(string $path, bool|string $fnamePart = false)

Deletes specified file/directory.

delete()

Deletes specified file/directory.

fileDeleteLocal($path)

Deletes a file that has been fetched before.

deleteLocal()

Deletes a file that has been fetched before.

void purge(callback $printCallback = false, int $microsleep = false, int $max = false, int $expiry = false)

Purges local and remote file data for current file path.

bool fileExists(string $path, bool $checkDFSFile = false)

Check if given file/dir exists.

bool exists(bool $checkDFSFile = false)

Check if given file/dir exists.

void passthrough(int $startOffset, int $length = false)

Outputs file contents directly

fileCopy($srcPath, $dstPath)

Copy file.

fileLinkCopy($srcPath, $dstPath, $symLink)

Create symbolic or hard link to file.

fileMove($srcPath, $dstPath)

Move file.

move(string $dstPath)

Move/rename file to $dstPath

getFileList(array $scopes = false, boolean $excludeScopes = false, array $limit = false, string $path = false)

Get list of files stored in database.

bool startCacheGeneration()

Starts cache generation for the current file.

endCacheGeneration($rename = true)

Ends the cache generation started by startCacheGeneration().

abortCacheGeneration()

Aborts the current cache generation process.

checkCacheGenerationTimeout()

Checks if the .generating file was changed, which would mean that generation timed out.

__get($propertyName)

Magic getter

bool requiresClusterizing()

Since eZDFS uses the database, running clusterize.php is required

bool requiresPurge()

eZDFS does require binary purge.

array(filepath) fetchExpiredItems(array $scopes, array $limit = array(0, 100), int $expiry = false)

Fetches the first $limit expired files from the DB

bool hasStaleCacheSupport()

Indicates if the handler supports the stalecache feature

string applyServerUri(string $filePath)

Transforms $filePath so that it contains a valid href to the file, wherever it is stored.

Details

at line 52
public __construct(string $filePath = false)

Constructor

If provided with $filePath, will use this file for further operations. If not given, the file* methods must be used instead

Parameters

string $filePath Path of the file to handle

Exceptions

eZDBNoConnectionException DB connection failed

at line 112
public disconnect()

Disconnects the cluster handler from the database

at line 127
public void loadMetaData(bool $force = false)

Loads file meta information.

Parameters

bool $force File stats will be refreshed if true

Return Value

void

at line 176
public void fileStore(string $filePath, string $scope = false, bool $delete = false, string $datatype = false)

Stores a file by path to the backend

Parameters

string $filePath Path to the file being stored.
string $scope Means something like "file category". May be used to clean caches of a certain type.
bool $delete true if the file should be deleted after storing.
string $datatype

Return Value

void

at line 204
public void fileStoreContents(string $filePath, string $contents, string $scope = false, string $datatype = false)

Store file contents.

Parameters

string $filePath Path to the file being stored.
string $contents Binary file content
string $scope "file category". May be used by cache management
string $datatype Datatype for the file. Also used to clean cache up

Return Value

void

at line 228
public storeContents(string $contents, string $scope = false, string $datatype = false, bool $storeLocally = false)

Store file contents using binary data

Parameters

string $contents Binary file content
string $scope "file category". May be used by cache management
string $datatype Datatype for the file. Also used to clean cache up
bool $storeLocally If true the file will also be stored on the local file system.

at line 256
public string|false fileFetch(string $filePath)

Fetches file from DFS and saves it in LFS under the same name.

Parameters

string $filePath

Return Value

string|false the file path, or false if fetching failed

at line 294
public string fetchUnique()

Fetches file from db and saves it in FS under a unique name

Return Value

string filename with path of a saved file. You can use this filename to get contents of file from filesystem.

at line 308
public fetch(bool $noLocalCache = false)

Fetches file from DFS and saves it in FS under the same name.

Parameters

bool $noLocalCache

at line 317
public bool|string fileFetchContents(string $filePath)

Returns file contents.

Parameters

string $filePath

Return Value

bool|string contents string, or false in case of an error.

at line 330
public string|bool fetchContents()

Returns file contents.

Return Value

string|bool contents string, or false in case of an error.

at line 404
public processCache($retrieveCallback, $generateCallback = null, $ttl = null, $expiry = null, $extraData = null)

Handles cache requests / write operations

Creates a single transaction out of the typical file operations for accessing caches. Caches are normally ready from the database or local file, if the entry does not exist or is expired then it generates the new cache data and stores it. This method takes care of these operations and handles the custom code by performing callbacks when needed.

The $retrieveCallback is used when the file contents can be used (ie. not re-generation) and is called when the file is ready locally. The function will be called with the file path as the first parameter, the mtime as the second and optionally $extraData as the third. The function must return the file contents or an instance of eZClusterFileFailure which can be used to tell the system that the retrieve data cannot be used after all.

$retrieveCallback can be set to null which makes the system go directly to the generation.

The $generateCallback is used when the file content is expired or does not exist, in this case the content must be re-generated and stored. The function will be called with the file path as the first parameter and optionally $extraData as the second. The function must return an array with information on the contents, the array consists of: - scope - The current scope of the file, is optional. - datatype - The current datatype of the file, is optional. - content - The file content, this can be any type except null. - binarydata - The binary data which is written to the file. - store - Whether content or binarydata should be stored to the file, if false it will simply return the data. Optional, by default it is true. Note: Set $generateCallback to false to disable generation callback. Note: Set $generateCallback to null to tell the function to perform a write lock but not do any generation, the generation must done be done by the caller by calling see storeCache().

Either content or binarydata must be supplied, if not an error is issued and it returns null.

If content is set it will be used as the return value of this function, if not it will return the binary data. If binarydata is set it will be used as the binary data for the file, if not it will perform a var_export on *content* and use that as the binary data.

For convenience the $generateCallback function can return a string which will be considered as the binary data for the file and returned as the content.

For controlling how long a cache entry can be used the parameters

Parameters

$retrieveCallback
$generateCallback
$ttl
$expiry
$extraData

See also

$expiry and @see $ttl is used.
$expiry can be set to a timestamp which controls the absolute max time for the cache, after this time/date the cache will never be used. If the value is set to a negative value or null there the expiration check is disabled. $ttl (time to live) tells how many seconds the cache can live from the time it was stored. If the value is set to negative or null there is no limit for the lifetime of the cache. A value of 0 means that the cache will always expire and practically disables caching. For the cache to be used both the $expiry and $ttl check must hold.

at line 716
static public bool isFileExpired(string $fname, int $mtime, int $expiry, int $curtime, int $ttl)

Calculates if the file data is expired or not.

Parameters

string $fname Name of file, available for easy debugging.
int $mtime Modification time of file, can be set to false if file does not exist.
int $expiry Time when file is to be expired, a value of -1 will disable this check.
int $curtime The current time to check against.
int $ttl Number of seconds the data can live, set to null to disable TTL.

Return Value

bool

at line 740
public bool isExpired(int $expiry, int $curtime, int $ttl)

Calculates if the current file data is expired or not.

Parameters

int $expiry Time when file is to be expired, a value of -1 will disable this check.
int $curtime The current time to check against.
int $ttl Number of seconds the data can live, set to null to disable TTL.

Return Value

bool

at line 752
public bool isLocalFileExpired(int $expiry, int $curtime, int $ttl)

Calculates if the local file is expired or not.

Parameters

int $expiry Time when file is to be expired, a value of -1 will disable this check.
int $curtime The current time to check against.
int $ttl Number of seconds the data can live, set to null to disable TTL.

Return Value

bool

at line 764
public bool isDBFileExpired(int $expiry, int $curtime, int $ttl)

Calculates if the DB file is expired or not.

Parameters

int $expiry Time when file is to be expired, a value of -1 will disable this check.
int $curtime The current time to check against.
int $ttl Number of seconds the data can live, set to null to disable TTL.

Return Value

bool

at line 780
public string|null storeCache(string|array $fileData)

Stores the data in $fileData to the remote and local file and commits the transaction.

The parameter $fileData must contain the same as information as the $generateCallback returns as explained in processCache().

Parameters

string|array $fileData

Return Value

string|null

at line 896
public processFile($callback, $expiry = false, $extraData = null)

Provides access to the file contents by downloading the file locally and calling $callback with the local filename.

The callback can then process the contents and return the data in the same way as in processCache().

Downloading is only done once so the local copy is kept, while updates to the remote DB entry is synced with the local one.

The parameters $expiry and $extraData is the same as for processCache().

Parameters

$callback
$expiry
$extraData

See also

self::processCache()

at line 909
public stat()

Returns file metadata.

at line 919
public int|null size()

Returns file size.

Return Value

int|null

at line 929
public string|null dataType()

Returns file mime-type / content-type.

Return Value

string|null

at line 939
public int|null mtime()

Returns file modification time.

Return Value

int|null

at line 949
public string|null name()

Returns file name.

Return Value

string|null

at line 963
public void fileDeleteByWildcard(string $wildcard)

Deletes a list of files by wildcard

Parameters

string $wildcard The wildcard used to look for files. Can contain * and ?

Return Value

void

at line 982
public void fileDeleteByDirList(array $dirList, string $commonPath, string $commonSuffix)

Deletes a list of files based on directory / filename components

Parameters

array $dirList Array of directory that will be prefixed with $commonPath when looking for files
string $commonPath Starting path common to every delete request
string $commonSuffix Suffix appended to every delete request

Return Value

void

at line 1006
public void fileDelete(string $path, bool|string $fnamePart = false)

Deletes specified file/directory.

Parameters

string $path the file path to delete
bool|string $fnamePart If set to true, $path is a directory and its content is deleted. If it is a string, this string is appended a wildcard and used for deletion

Return Value

void

at line 1031
public delete()

Deletes specified file/directory.

If a directory specified it is deleted recursively.

at line 1044
public fileDeleteLocal($path)

Deletes a file that has been fetched before.

Parameters

$path

at line 1055
public deleteLocal()

Deletes a file that has been fetched before.

at line 1088
public void purge(callback $printCallback = false, int $microsleep = false, int $max = false, int $expiry = false)

Purges local and remote file data for current file path.

Can be given a file or a folder. In order to clear a folder, do NOT add a trailing / at the end of the file's path: path/to/file instead of path/to/file/.

By default, only expired files will be removed (ezdfsfile.expired = 1). If you specify an $expiry time, it will replace the expired test and only purge files older than the given expiry timestamp.

Parameters

callback $printCallback Callback called after each delete iteration (see $max) to print out a report of the deleted files. This callback expects two parameters, $file (delete pattern used to perform deletion) and $count (number of deleted items)
int $microsleep Wait interval before each purge batch of $max items
int $max Maximum number of items to delete in one batch (default: 100)
int $expiry If specificed, only files older than this date will be purged

Return Value

void

at line 1142
public bool fileExists(string $path, bool $checkDFSFile = false)

Check if given file/dir exists.

Parameters

string $path File path to test existence for
bool $checkDFSFile if true, also check on the DFS

Return Value

bool

See also

eZDFSFileHandler::exists()

at line 1155
public bool exists(bool $checkDFSFile = false)

Check if given file/dir exists.

Parameters

bool $checkDFSFile if true, also check on the DFS

Return Value

bool

See also

eZDFSFileHandler::fileExists()

at line 1169
public void passthrough(int $startOffset, int $length = false)

Outputs file contents directly

Parameters

int $startOffset Byte offset to start transfer from
int $length Byte length to transfer. NOT end offset, end offset = $startOffset + $length

Return Value

void

at line 1178
public fileCopy($srcPath, $dstPath)

Copy file.

Parameters

$srcPath
$dstPath

at line 1191
public fileLinkCopy($srcPath, $dstPath, $symLink)

Create symbolic or hard link to file.

Parameters

$srcPath
$dstPath
$symLink

at line 1203
public fileMove($srcPath, $dstPath)

Move file.

Parameters

$srcPath
$dstPath

at line 1219
public move(string $dstPath)

Move/rename file to $dstPath

Parameters

string $dstPath Destination path

at line 1242
public getFileList(array $scopes = false, boolean $excludeScopes = false, array $limit = false, string $path = false)

Get list of files stored in database.

Used in bin/php/clusterize.php.

Parameters

array $scopes return only files that belong to any of these scopes
boolean $excludeScopes if true, then reverse the meaning of $scopes, which is
array $limit limits the search to offset limit[0], limit limit[1]
string $path filter to include entries only including $path return only files that do not belong to any of the scopes listed in $scopes

at line 1281
public bool startCacheGeneration()

Starts cache generation for the current file.

This is done by creating a file named by the original file name, prefixed with '.generating'.

Return Value

bool false if the file is being generated, true if it is not

at line 1318
public endCacheGeneration($rename = true)

Ends the cache generation started by startCacheGeneration().

Parameters

$rename

at line 1350
public abortCacheGeneration()

Aborts the current cache generation process.

Does so by rolling back the current transaction, which should be the .generating file lock

at line 1364
public checkCacheGenerationTimeout()

Checks if the .generating file was changed, which would mean that generation timed out.

If not timed out, refreshes the timestamp so that storage won't be stolen

at line 1387
public __get($propertyName)

Magic getter

Parameters

$propertyName

at line 1417
public bool requiresClusterizing()

Since eZDFS uses the database, running clusterize.php is required

Return Value

bool

at line 1430
public bool requiresPurge()

eZDFS does require binary purge.

It does store files in DB + on NFS, and therefore doesn't remove files in real time

Return Value

bool

at line 1445
public array(filepath) fetchExpiredItems(array $scopes, array $limit = array(0, 100), int $expiry = false)

Fetches the first $limit expired files from the DB

Parameters

array $scopes Array of scopes to fetch from
array $limit A 2 items array( offset, limit )
int $expiry Number of seconds, only items older than this will be returned

Return Value

array(filepath)

at line 1450
public bool hasStaleCacheSupport()

Indicates if the handler supports the stalecache feature

Return Value

bool true if it does, false otherwise

at line 1455
public string applyServerUri(string $filePath)

Transforms $filePath so that it contains a valid href to the file, wherever it is stored.

Parameters

string $filePath Example: /var/site/storage/images/example.png

Return Value

string http://static.example.com/var/site/storage/images/example.png