Package ghidra.formats.gfilesystem

Class FileCache

java.lang.Object

ghidra.formats.gfilesystem.FileCache

public class FileCache
extends java.lang.Object

File caching implementation.

Caches files based on a hash of the contents of the file.
Files are retrieved using the hash string.

Cached files are stored in a file with a name that is the hex encoded value of the hash. Cached files are organized into a nested directory structure to prevent overwhelming a single directory with thousands of files.

Nested directory structure is based on the file's name: File: AABBCCDDEEFF... Directory (2 level nesting): AA/BB/AABBCCDDEEFF...

Cache size is not bounded.

Cache maint is done during startup if interval since last maint has been exceeded

No file data is maintained in memory.

No file is moved or removed from the cache after being added (except during startup) as there is no use count or reference tracking of the files.

Field Summary

Fields

Modifier and Type	Field	Description
static int	MD5_HEXSTR_LEN

Constructor Summary

Constructors

Constructor	Description
FileCache(java.io.File cacheDir)	Creates a new FileCache instance where files are stored under the specified cacheDir

Method Summary

Modifier and Type	Method	Description
FileCacheEntry	addFile(java.io.File f, TaskMonitor monitor)	Adds a File to the cache, returning a FileCacheEntry.
FileCacheEntry	addStream(java.io.InputStream is, TaskMonitor monitor)	Adds a contents of a stream to the cache, returning the md5 identifier of the stream.
FileCacheEntry	getFile(java.lang.String md5)	Returns a FileCacheEntry for the matching file, based on its MD5, or NULL if there is no matching file.
int	getFileAddCount()	Number of files added to this cache.
int	getFileReUseCount()	Number of times a file-add was a no-op and the contents were already present in the cache.
long	getMaxFileAgeMS()	How old (in milliseconds) files must be before being aged-off during cache maintenance.
long	getStorageEstimateBytes()	Estimate of the number of bytes in the cache.
void	purge()	Deletes all stored files from this file cache that are under a "NN" two hex digit nesting dir.
FileCacheEntry	pushStream(DerivedFilePushProducer pusher, TaskMonitor monitor)	Adds a file to the cache, using a 'pusher' strategy where the producer is given a OutputStream to write to.
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

MD5_HEXSTR_LEN

public static final int MD5_HEXSTR_LEN

See Also:

Constant Field Values

Constructor Detail

FileCache

public FileCache(java.io.File cacheDir)
          throws java.io.IOException

Creates a new FileCache instance where files are stored under the specified cacheDir

Parameters:

cacheDir - where to store the files

Throws:

java.io.IOException - if there was a problem creating subdirectories under cacheDir or when pruning expired files.

Method Detail

purge

public void purge()

Deletes all stored files from this file cache that are under a "NN" two hex digit nesting dir.

Will cause other processes which are accessing or updating the cache to error.

addFile

public FileCacheEntry addFile(java.io.File f,
                              TaskMonitor monitor)
                       throws java.io.IOException,
                              CancelledException

Adds a File to the cache, returning a FileCacheEntry.

Parameters:

f - File to add to cache.

monitor - TaskMonitor to monitor for cancel and to update progress.

Returns:

FileCacheEntry with new File and md5.

Throws:

java.io.IOException - if error

CancelledException - if canceled

getFile

public FileCacheEntry getFile(java.lang.String md5)

Returns a FileCacheEntry for the matching file, based on its MD5, or NULL if there is no matching file.

Tweaks the file's last modified time to implement a LRU.

Parameters:

md5 - md5 string.

Returns:

FileCacheEntry with a File and it's md5 string or null if no matching file exists in cache.

addStream

public FileCacheEntry addStream(java.io.InputStream is,
                                TaskMonitor monitor)
                         throws java.io.IOException,
                                CancelledException

Adds a contents of a stream to the cache, returning the md5 identifier of the stream.

The stream is copied into a temp file in the cacheDir/new directory while its md5 is calculated. The temp file is then moved into its final location based on the md5 of the stream: AA/BB/CC/AABBCCDDEEFF....

The monitor progress is updated with the number of bytes that are being copied. No message or maximum is set.

Parameters:

is - InputStream to add to the cache. Not closed when done.

monitor - TaskMonitor that will be checked for canceling and updating progress.

Returns:

FileCacheEntry with file info and md5, never null.

Throws:

java.io.IOException - if error

CancelledException - if canceled

pushStream

public FileCacheEntry pushStream(DerivedFilePushProducer pusher,
                                 TaskMonitor monitor)
                          throws java.io.IOException,
                                 CancelledException

Adds a file to the cache, using a 'pusher' strategy where the producer is given a OutputStream to write to.

Unbeknownst to the producer, but knownst to us, the outputstream is really a HashingOutputStream that will allow us to get the MD5 hash when the producer is finished pushing.

Parameters:

pusher - functional callback that will accept an OutputStream and write to it.

 (os) -> { os.write(.....); }

monitor - TaskMonitor that will be checked for cancel and updated with file io progress.

Returns:

a new FileCacheEntry with the newly added cache file's File and MD5, never null.

Throws:

java.io.IOException - if an IO error

CancelledException - if the user cancels

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

getFileAddCount

public int getFileAddCount()

Number of files added to this cache.

Returns:

Number of files added to this cache

getFileReUseCount

public int getFileReUseCount()

Number of times a file-add was a no-op and the contents were already present in the cache.

Returns:

Number of times a file-add was a no-op and the contents were already present in the cache.

getStorageEstimateBytes

public long getStorageEstimateBytes()

Estimate of the number of bytes in the cache.

Returns:

estimate of the number of bytes in the cache - could be very wrong

getMaxFileAgeMS

public long getMaxFileAgeMS()

How old (in milliseconds) files must be before being aged-off during cache maintenance.

Returns:

Max cache file age in milliseconds.