FS Cache

FS-cache是一种内核功能，网络文件系统或其他文件系统可以通过它来缓存数据到本地磁盘空间，减少网络传输的数据，从而提升性能。这在网络速度比较慢时会得到比较好的效果。

FS-Cache 可以被任何希望添加本地缓存的文件系统使用，例如：AFS、NFS、CIFS和Isofs。

FS-Cache 对于客户端文件系统是透明存在的，当这项功能开启的时候，透过缓存请求文件对于客户端是无感知的。可以参考下图，FS-Cache可以认为是网络文件系统和缓存后端的中间介质:

cache的基本原理（cachefilesd缓存项目介绍）(1)

看一个更详细的图，FS-Cache为网络文件系统提供了一个缓存工具，从而让缓存对用户无感知

cache的基本原理（cachefilesd缓存项目介绍）(2)

FS-Cache并不遵循在允许访问之前将所有完全打开的每个netfs文件完全加载到高速缓存中，主要有以下几个原因：

  1. 没有Cache 也应该能够正常操作
  2. 被访问的文件的大小不应该受限于Cache的空间大小
  3. 所有已经打开的文件大小不应该受限于Cache的空间大小
  4. 不应该强制用户为了一个文件操作（访问文件的一小部分）将整个文件全部进行下载缓存

FS-Cache提供的能力如下：

1. More than one cache can be used at once. Caches can be selected explicitly by use of tags. 一次可以使用多个Cache，不同的Cache使用不同的tag区分 2. Caches can be added / removed at any time. Cache可以在任何时间被移除或者添加 3. The netfs is provided with an interface that allows either party to withdraw caching facilities from a file (required for (2)). 网络文件系统提供接口，能够允许其他方删除文件cache相关的能力 4. The interface to the netfs returns as few errors as possible, preferring rather to let the netfs remain oblivious. 网络文件系统尽量不要返回错误 5. Cookies are used to represent indices, files and other objects to the netfs. The simplest cookie is just a NULL pointer - indicating nothing cached there. 使用cookie表示文件系统的目录、文件、其他对象 6. The netfs is allowed to propose - dynamically - any index hierarchy it desires, though it must be aware that the index search function is recursive, stack space is limited, and indices can only be children of indices. 7. Data I/O is done direct to and from the netfs’s pages. The netfs indicates that page A is at index B of the data-file represented by cookie C, and that it should be read or written. The cache backend may or may not start I/O on that page, but if it does, a netfs callback will be invoked to indicate completion. The I/O may be either synchronous or asynchronous. 8. Cookies can be “retired” upon release. At this point FS-Cache will mark them as obsolete and the index hierarchy rooted at that point will get recycled. 9. The netfs provides a “match” function for index searches. In addition to saying whether a match was made or not, this can also specify that an entry should be updated or deleted. 10. As much as possible is done asynchronously.

FS-Cache维护了一个网络文件系统数据的全部索引，该信息可以位于一个或者多个cache中，如下图所示：

cache的基本原理（cachefilesd缓存项目介绍）(3)

In the example above, you can see two netfs’s being backed: NFS and AFS. These have different index hierarchies: * The NFS primary index contains per-server indices. Each server index is indexed by NFS file handles to get data file objects. Each data file objects can have an array of pages, but may also have further child objects, such as extended attributes and directory entries. Extended attribute objects themselves have page-array contents. * The AFS primary index contains per-cell indices. Each cell index contains per-logical-volume indices. Each of volume index contains up to three indices for the read-write, read-only and backup mirrors of those volumes. Each of these contains vnode data file objects, each of which contains an array of pages.

CacheFiles介绍

CacheFiles,是属于Linux Kernel的一个模块，主要用于缓存已经挂载的文件系统，CacheFiles 是一个缓存后端，当一个文件系统挂载到本地时，可以基于CacheFiles做一个缓存目录，CacheFi 使用一个用户空间的守护进程进行cache管理，例如收割陈旧的节点和剔除，这个守护进程被称为cachefilesd

缓存的文件系统和数据完整性与后端服务的文件系统一样好，由于不同文件系统的日志记录接口都是特殊定义的，因此CacheFiles不会尝试记录任文件系统日志

CacheFiles 会创建一个混杂的字符设备”/dev/cachefiles”,用于与守护进程进行通信，这个设备一次打开只能做一次事情，当它打开时，至少存在部分缓存，守护进程打开 并发送指令用于控制缓存，CacheFiles目前只能用于一个单独的缓存

CacheFiles会尝试维护文件系统一定比例的空闲空间，可能会通过剔除部分cache用户缩小cache的大小，用于释放空间，这就意味着可以在同一介质上存放灵活的实时数据，可能会扩展来使用空闲的空间，也可能收缩

Requiremenets

使用CacheFiles 需要以下依赖

配置

配置文件 /etc/cachefilesd.conf,配置文件的主要内容为：

brun <N>%, bcull <N>%, bstop <N>%, frun <N>%, fcull <N>%, fstop <N>% Configure the culling limits. Optional. See the section on culling The defaults are 7% (run), 5% (cull) and 1% (stop) respectively. The commands beginning with a ‘b’ are file space (block) limits, those beginning with an ‘f’ are file count limits(也可以限制文件数量). dir <path> 存放缓存的根目录 Specify the directory containing the root of the cache. Mandatory. tag <name> Specify a tag to FS-Cache to use in distinguishing multiple caches. Optional. The default is “CacheFiles”. debug <mask> 用于开启日志 Specify a numeric bitmask to control debugging in the kernel module. Optional. The default is zero (all off). The following values can be OR’d into the mask to collect various information: 1 Turn on trace of function entry (_enter() macros) 2 Turn on trace of function exit (_leave() macros) 4 Turn on trace of internal debug points (_debug()) This mask can also be set through sysfs, eg: echo 5 >/sys/modules/cachefiles/parameters/debug

启动服务

启动守护进程，该守护进程打开 cache 设备（/dev/cachefiles）,配置cache，并开始进行cache，此时cache绑定fscache，cache开始运行。具体的启动命令和参数如下所示：

The daemon is run as follows: /sbin/cachefilesd [-d]* [-s] [-n] [-f <configfile>] The flags are: -d Increase the debugging level. This can be specified multiple times and is cumulative with itself. -s Send messages to stderr instead of syslog. -n Don’t daemonise and go into background. -f <configfile> Use an alternative configuration file rather than the default one.

缓存剔除

缓存偶尔需要进行清理，用于释放空间，这里主要将近期未被使用的cache进行清理，基于文件的访问时间进行判断，如果空目录没有在使用也会被清理。Cache的清理是基于配置的当前文件系统的block比例和文件比例，主要有6个限制，如下所示

brun, frun If the amount of free space and the number of available files in the cache rises above both these limits, then culling is turned off.当缓存中空闲的block和file 都高于该值时，不进行缓存剔除 bcull, fcull If the amount of available space or the number of available files in the cache falls below either of these limits, then culling is started.当缓存中可以使用的block或者files 有一个低于该值时，进行缓存剔除 bstop, fstop If the amount of available space or the number of available files in the cache falls below either of these limits, then no further allocation of disk space or files is permitted until culling has raised things above these limits again.当缓存中可以使用的block或者files有一个低于该值时，除非缓存剔除机制进行了缓存剔除，否则不会再分配磁盘空间，

通常配置是这样

0 <= bstop < bcull < brun < 100 0 <= fstop < fcull < frun < 100

需要注意，这些值是表示的可以使用的空间和文件，并不是100 减去使用df 查看的信息，用户空间的守护进程扫描cache，来建立一个需要提出的对象表，基于最少使用原则进行剔除。“ A new scan of the cache is started as soon as space is made in the table”，如果对象的atimes(最后访问时间)发生了变化，不会进行剔除，或者内核模块通知说该文件仍然在使用，也不会删除该cache

缓存结构

会存在两个目录

活动的cache 对象会存放在 cache/ 目录。CacheFile的内核模块会将不再使用或者剔除的对象移动到graveyard 目录，守护进程会在graveyard进行删除，守护进程使用dnotify来监控graveyard目录，然后会将graveyard存在的对象删除。

如果一个对象有子对象，那么他会以目录的形式表示，在该目录下会有一系列子目录，子目录的名称以@ 子对象的哈希值命名，如下所示

/INDEX /INDEX /INDEX /DATA FILES /=========/==========/=================================/================ cache/@4a/I03nfs/@30/Ji000000000000000--fHg8hi8400 cache/@4a/I03nfs/@30/Ji000000000000000--fHg8hi8400/@75/Es0g000w...DB1ry cache/@4a/I03nfs/@30/Ji000000000000000--fHg8hi8400/@75/Es0g000w...N22ry cache/@4a/I03nfs/@30/Ji000000000000000--fHg8hi8400/@75/Es0g000w...FP1ry

如果文件名称太长，超过了NAME_MAX的大小，那么会被分成多份，第一份被用于创建嵌套目录，最后一份会位于最后一个目录，每个中间目录的名称会以” ”作为前缀，例如：

J1223/@23/ xy...z/ kl...m/Epqr

Note that keys are raw data, and not only may they exceed NAME_MAX in size, they may also contain things like ‘/’ and NUL characters, and so they may not be suitable for turning directly into a filename. To handle this, CacheFiles will use a suitably printable filename directly and “base-64” encode ones that aren’t directly suitable. The two versions of object filenames indicate the encoding: OBJECT TYPE PRINTABLE ENCODED Index “I…” “J…” Data “D…” “E…” Special “S…” “T…” Intermediate directories are always “@” or “ ” as appropriate. Each object in the cache has an extended attribute label that holds the object type ID (required to distinguish special objects) and the auxiliary data from the netfs. The latter is used to detect stale objects in the cache and update or retire them. Note that CacheFiles will erase from the cache any file it doesn’t recognise or any file of an incorrect type (such as a FIFO file or a device file).

,