Reproducible workflow

As part of a reproducible workflow, caching of function calls, code chunks, and other elements of a project is a critical component. The objective of a reproducible workflow is is likely that an entire work flow from raw data to publication, decision support, report writing, presentation building etc., could be built and be reproducible anywhere, on any computer, operating system, with any starting conditions, on demand. The reproducible::Cache function is built to work with any R function.

Differences with other approaches

Cache uses 2 key the archivist functions saveToLocalRepo and loadFromLocalRepo, but does not use archivist::cache. Similar to archivist::cache, there is some reliance on digest::digest to determine whether the arguments are identical in subsequent iterations. It also but does many things that make standard caching with digest::digest don’t work reliably between systems. For these, the function .robustDigest is introduced to make caching transferable between systems. This is relevant for file paths, environments, parallel clusters, functions (which are contained within an environment), and many others (e.g., see ?.robustDigest for methods). Cache also adds important elements like automated tagging and the option to retrieve disk-cached values via stashed objects in memory using memoise::memoise. This means that running Cache 1, 2, and 3 times on the same function will get progressively faster. This can be extremely useful for web apps built with, say shiny.

Function-level caching

Any function can be cached using: Cache(FUN = functionName, ...).

This will be a slight change to a function call, such as: projectRaster(raster, crs = crs(newRaster)) to Cache(projectRaster, raster, crs = crs(newRaster)).

This is particularly useful for expensive operations.

library(raster)
## Loading required package: sp
library(reproducible)

tmpDir <- file.path(tempdir(), "reproducible_examples", "Cache")
checkPath(tmpDir, create = TRUE)
## [1] "/tmp/Rtmpit54vE/reproducible_examples/Cache"
ras <- raster(extent(0, 1000, 0, 1000), vals = 1:1e6, res = 1)
crs(ras) <- "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84"

newCRS <- "+init=epsg:4326 +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0"
# No Cache
system.time(map1 <- projectRaster(ras, crs = newCRS))
##    user  system elapsed 
##   2.162   0.198   2.363
# With Cache -- a little slower the first time because saving to disk
system.time(map1 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir,
                         notOlderThan = Sys.time()))
##    user  system elapsed 
##   2.400   0.215   2.771
# vastly faster the second time
system.time(map2 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
##   loading cached result from previous projectRaster call, adding to memoised copy
##    user  system elapsed 
##   0.088   0.005   0.093
# even faster the third time
system.time(map3 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
##   loading memoised result from previous projectRaster call.
##    user  system elapsed 
##   0.023   0.003   0.026
all.equal(map1, map2) # TRUE
## [1] TRUE
all.equal(map1, map3) # TRUE
## [1] TRUE

Caching examples

Basic use

library(raster)
library(magrittr)
## 
## Attaching package: 'magrittr'
## The following object is masked from 'package:raster':
## 
##     extract
try(clearCache(tmpDir, ask = FALSE), silent = TRUE) # just to make sure it is clear

ranNumsA <- Cache(rnorm, 10, 16, cacheRepo = tmpDir)

# All same
ranNumsB <- Cache(rnorm, 10, 16, cacheRepo = tmpDir) # recovers cached copy
##   loading cached result from previous rnorm call, adding to memoised copy
ranNumsC <- Cache(cacheRepo = tmpDir) %C% rnorm(10, 16)  # recovers cached copy
##   loading memoised result from previous 'rnorm' pipe sequence call.
ranNumsD <- Cache(quote(rnorm(n = 10, 16)), cacheRepo = tmpDir) # recovers cached copy
##   loading memoised result from previous rnorm call.
# Any minor change makes it different
ranNumsE <- Cache(cacheRepo = tmpDir) %C% rnorm(10, 6) # different

Example 1: Basic cache use with tags

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

showCache(tmpDir, userTags = c("objectName"))
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 824ba35c4c76273931bec6af02ac8ee3         format
##  2: 824ba35c4c76273931bec6af02ac8ee3           name
##  3: 824ba35c4c76273931bec6af02ac8ee3          class
##  4: 824ba35c4c76273931bec6af02ac8ee3           date
##  5: 824ba35c4c76273931bec6af02ac8ee3        cacheId
##  6: 824ba35c4c76273931bec6af02ac8ee3     objectName
##  7: 824ba35c4c76273931bec6af02ac8ee3       function
##  8: 824ba35c4c76273931bec6af02ac8ee3    object.size
##  9: 824ba35c4c76273931bec6af02ac8ee3       accessed
## 10: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 11: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 12: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 13: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 14: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 15: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 16: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 17: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 18: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 19: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 20: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 21: 824ba35c4c76273931bec6af02ac8ee3      preDigest
## 22: 824ba35c4c76273931bec6af02ac8ee3      preDigest
## 23: db111de35916948a4bba53ca13568d0b         format
## 24: db111de35916948a4bba53ca13568d0b           name
## 25: db111de35916948a4bba53ca13568d0b          class
## 26: db111de35916948a4bba53ca13568d0b           date
## 27: db111de35916948a4bba53ca13568d0b        cacheId
## 28: db111de35916948a4bba53ca13568d0b     objectName
## 29: db111de35916948a4bba53ca13568d0b       function
## 30: db111de35916948a4bba53ca13568d0b    object.size
## 31: db111de35916948a4bba53ca13568d0b       accessed
## 32: db111de35916948a4bba53ca13568d0b otherFunctions
## 33: db111de35916948a4bba53ca13568d0b otherFunctions
## 34: db111de35916948a4bba53ca13568d0b otherFunctions
## 35: db111de35916948a4bba53ca13568d0b otherFunctions
## 36: db111de35916948a4bba53ca13568d0b otherFunctions
## 37: db111de35916948a4bba53ca13568d0b otherFunctions
## 38: db111de35916948a4bba53ca13568d0b otherFunctions
## 39: db111de35916948a4bba53ca13568d0b otherFunctions
## 40: db111de35916948a4bba53ca13568d0b otherFunctions
## 41: db111de35916948a4bba53ca13568d0b otherFunctions
## 42: db111de35916948a4bba53ca13568d0b otherFunctions
## 43: db111de35916948a4bba53ca13568d0b      preDigest
## 44: db111de35916948a4bba53ca13568d0b      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:06:30
##  2: 824ba35c4c76273931bec6af02ac8ee3 2019-03-18 10:06:30
##  3:                          numeric 2019-03-18 10:06:30
##  4:              2019-03-18 10:06:30 2019-03-18 10:06:30
##  5:                 f7bee22047b8d0c1 2019-03-18 10:06:30
##  6:                                a 2019-03-18 10:06:30
##  7:                            rnorm 2019-03-18 10:06:30
##  8:                             1008 2019-03-18 10:06:30
##  9:              2019-03-18 10:06:30 2019-03-18 10:06:30
## 10:                          saveRDS 2019-03-18 10:06:30
## 11:                          do.call 2019-03-18 10:06:30
## 12:                     process_file 2019-03-18 10:06:30
## 13:                    process_group 2019-03-18 10:06:30
## 14:              process_group.block 2019-03-18 10:06:30
## 15:                       call_block 2019-03-18 10:06:30
## 16:                       block_exec 2019-03-18 10:06:30
## 17:                           in_dir 2019-03-18 10:06:30
## 18:                        timing_fn 2019-03-18 10:06:30
## 19:                           handle 2019-03-18 10:06:30
## 20:                      withVisible 2019-03-18 10:06:30
## 21:               n:7eef4eae85fd9229 2019-03-18 10:06:30
## 22:            .FUN:4f604aa46882b368 2019-03-18 10:06:30
## 23:                              rda 2019-03-18 10:06:30
## 24: db111de35916948a4bba53ca13568d0b 2019-03-18 10:06:30
## 25:                          numeric 2019-03-18 10:06:30
## 26:              2019-03-18 10:06:30 2019-03-18 10:06:30
## 27:                 3aef38d1fc02aee5 2019-03-18 10:06:30
## 28:                                b 2019-03-18 10:06:30
## 29:                            runif 2019-03-18 10:06:30
## 30:                             1008 2019-03-18 10:06:30
## 31:              2019-03-18 10:06:30 2019-03-18 10:06:30
## 32:                          saveRDS 2019-03-18 10:06:30
## 33:                          do.call 2019-03-18 10:06:30
## 34:                     process_file 2019-03-18 10:06:30
## 35:                    process_group 2019-03-18 10:06:30
## 36:              process_group.block 2019-03-18 10:06:30
## 37:                       call_block 2019-03-18 10:06:30
## 38:                       block_exec 2019-03-18 10:06:30
## 39:                           in_dir 2019-03-18 10:06:30
## 40:                        timing_fn 2019-03-18 10:06:30
## 41:                           handle 2019-03-18 10:06:30
## 42:                      withVisible 2019-03-18 10:06:30
## 43:               n:7eef4eae85fd9229 2019-03-18 10:06:30
## 44:            .FUN:881ec847b7161f3c 2019-03-18 10:06:30
##                             tagValue         createdDate
showCache(tmpDir, userTags = c("^a$")) # regular expression ... "a" exactly
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: 824ba35c4c76273931bec6af02ac8ee3         format
##  2: 824ba35c4c76273931bec6af02ac8ee3           name
##  3: 824ba35c4c76273931bec6af02ac8ee3          class
##  4: 824ba35c4c76273931bec6af02ac8ee3           date
##  5: 824ba35c4c76273931bec6af02ac8ee3        cacheId
##  6: 824ba35c4c76273931bec6af02ac8ee3     objectName
##  7: 824ba35c4c76273931bec6af02ac8ee3       function
##  8: 824ba35c4c76273931bec6af02ac8ee3    object.size
##  9: 824ba35c4c76273931bec6af02ac8ee3       accessed
## 10: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 11: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 12: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 13: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 14: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 15: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 16: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 17: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 18: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 19: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 20: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 21: 824ba35c4c76273931bec6af02ac8ee3      preDigest
## 22: 824ba35c4c76273931bec6af02ac8ee3      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:06:30
##  2: 824ba35c4c76273931bec6af02ac8ee3 2019-03-18 10:06:30
##  3:                          numeric 2019-03-18 10:06:30
##  4:              2019-03-18 10:06:30 2019-03-18 10:06:30
##  5:                 f7bee22047b8d0c1 2019-03-18 10:06:30
##  6:                                a 2019-03-18 10:06:30
##  7:                            rnorm 2019-03-18 10:06:30
##  8:                             1008 2019-03-18 10:06:30
##  9:              2019-03-18 10:06:30 2019-03-18 10:06:30
## 10:                          saveRDS 2019-03-18 10:06:30
## 11:                          do.call 2019-03-18 10:06:30
## 12:                     process_file 2019-03-18 10:06:30
## 13:                    process_group 2019-03-18 10:06:30
## 14:              process_group.block 2019-03-18 10:06:30
## 15:                       call_block 2019-03-18 10:06:30
## 16:                       block_exec 2019-03-18 10:06:30
## 17:                           in_dir 2019-03-18 10:06:30
## 18:                        timing_fn 2019-03-18 10:06:30
## 19:                           handle 2019-03-18 10:06:30
## 20:                      withVisible 2019-03-18 10:06:30
## 21:               n:7eef4eae85fd9229 2019-03-18 10:06:30
## 22:            .FUN:4f604aa46882b368 2019-03-18 10:06:30
##                             tagValue         createdDate
showCache(tmpDir, userTags = c("runif")) # show only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: db111de35916948a4bba53ca13568d0b         format
##  2: db111de35916948a4bba53ca13568d0b           name
##  3: db111de35916948a4bba53ca13568d0b          class
##  4: db111de35916948a4bba53ca13568d0b           date
##  5: db111de35916948a4bba53ca13568d0b        cacheId
##  6: db111de35916948a4bba53ca13568d0b     objectName
##  7: db111de35916948a4bba53ca13568d0b       function
##  8: db111de35916948a4bba53ca13568d0b    object.size
##  9: db111de35916948a4bba53ca13568d0b       accessed
## 10: db111de35916948a4bba53ca13568d0b otherFunctions
## 11: db111de35916948a4bba53ca13568d0b otherFunctions
## 12: db111de35916948a4bba53ca13568d0b otherFunctions
## 13: db111de35916948a4bba53ca13568d0b otherFunctions
## 14: db111de35916948a4bba53ca13568d0b otherFunctions
## 15: db111de35916948a4bba53ca13568d0b otherFunctions
## 16: db111de35916948a4bba53ca13568d0b otherFunctions
## 17: db111de35916948a4bba53ca13568d0b otherFunctions
## 18: db111de35916948a4bba53ca13568d0b otherFunctions
## 19: db111de35916948a4bba53ca13568d0b otherFunctions
## 20: db111de35916948a4bba53ca13568d0b otherFunctions
## 21: db111de35916948a4bba53ca13568d0b      preDigest
## 22: db111de35916948a4bba53ca13568d0b      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:06:30
##  2: db111de35916948a4bba53ca13568d0b 2019-03-18 10:06:30
##  3:                          numeric 2019-03-18 10:06:30
##  4:              2019-03-18 10:06:30 2019-03-18 10:06:30
##  5:                 3aef38d1fc02aee5 2019-03-18 10:06:30
##  6:                                b 2019-03-18 10:06:30
##  7:                            runif 2019-03-18 10:06:30
##  8:                             1008 2019-03-18 10:06:30
##  9:              2019-03-18 10:06:30 2019-03-18 10:06:30
## 10:                          saveRDS 2019-03-18 10:06:30
## 11:                          do.call 2019-03-18 10:06:30
## 12:                     process_file 2019-03-18 10:06:30
## 13:                    process_group 2019-03-18 10:06:30
## 14:              process_group.block 2019-03-18 10:06:30
## 15:                       call_block 2019-03-18 10:06:30
## 16:                       block_exec 2019-03-18 10:06:30
## 17:                           in_dir 2019-03-18 10:06:30
## 18:                        timing_fn 2019-03-18 10:06:30
## 19:                           handle 2019-03-18 10:06:30
## 20:                      withVisible 2019-03-18 10:06:30
## 21:               n:7eef4eae85fd9229 2019-03-18 10:06:30
## 22:            .FUN:881ec847b7161f3c 2019-03-18 10:06:30
##                             tagValue         createdDate
clearCache(tmpDir, userTags = c("runif"), ask = FALSE) # remove only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) # only those made during rnorm call
## Cache size:
##   Total (including Rasters): 804 bytes
##   Selected objects (not including Rasters): 804 bytes
##                             artifact         tagKey
##  1: 37e0d4cab84fd441e92c95013f15812e         format
##  2: 37e0d4cab84fd441e92c95013f15812e           name
##  3: 37e0d4cab84fd441e92c95013f15812e          class
##  4: 37e0d4cab84fd441e92c95013f15812e           date
##  5: 37e0d4cab84fd441e92c95013f15812e        cacheId
##  6: 37e0d4cab84fd441e92c95013f15812e       function
##  7: 37e0d4cab84fd441e92c95013f15812e    object.size
##  8: 37e0d4cab84fd441e92c95013f15812e       accessed
##  9: 37e0d4cab84fd441e92c95013f15812e otherFunctions
## 10: 37e0d4cab84fd441e92c95013f15812e otherFunctions
## 11: 37e0d4cab84fd441e92c95013f15812e otherFunctions
## 12: 37e0d4cab84fd441e92c95013f15812e otherFunctions
## 13: 37e0d4cab84fd441e92c95013f15812e otherFunctions
## 14: 37e0d4cab84fd441e92c95013f15812e otherFunctions
## 15: 37e0d4cab84fd441e92c95013f15812e otherFunctions
## 16: 37e0d4cab84fd441e92c95013f15812e otherFunctions
## 17: 37e0d4cab84fd441e92c95013f15812e otherFunctions
## 18: 37e0d4cab84fd441e92c95013f15812e otherFunctions
## 19: 37e0d4cab84fd441e92c95013f15812e otherFunctions
## 20: 37e0d4cab84fd441e92c95013f15812e otherFunctions
## 21: 37e0d4cab84fd441e92c95013f15812e      preDigest
## 22: 37e0d4cab84fd441e92c95013f15812e      preDigest
## 23: 37e0d4cab84fd441e92c95013f15812e      preDigest
## 24: 824ba35c4c76273931bec6af02ac8ee3         format
## 25: 824ba35c4c76273931bec6af02ac8ee3           name
## 26: 824ba35c4c76273931bec6af02ac8ee3          class
## 27: 824ba35c4c76273931bec6af02ac8ee3           date
## 28: 824ba35c4c76273931bec6af02ac8ee3        cacheId
## 29: 824ba35c4c76273931bec6af02ac8ee3     objectName
## 30: 824ba35c4c76273931bec6af02ac8ee3       function
## 31: 824ba35c4c76273931bec6af02ac8ee3    object.size
## 32: 824ba35c4c76273931bec6af02ac8ee3       accessed
## 33: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 34: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 35: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 36: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 37: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 38: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 39: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 40: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 41: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 42: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 43: 824ba35c4c76273931bec6af02ac8ee3 otherFunctions
## 44: 824ba35c4c76273931bec6af02ac8ee3      preDigest
## 45: 824ba35c4c76273931bec6af02ac8ee3      preDigest
## 46: dcec754b18a24c45723a35766f3eab6d         format
## 47: dcec754b18a24c45723a35766f3eab6d           name
## 48: dcec754b18a24c45723a35766f3eab6d          class
## 49: dcec754b18a24c45723a35766f3eab6d           date
## 50: dcec754b18a24c45723a35766f3eab6d        cacheId
## 51: dcec754b18a24c45723a35766f3eab6d       function
## 52: dcec754b18a24c45723a35766f3eab6d    object.size
## 53: dcec754b18a24c45723a35766f3eab6d       accessed
## 54: dcec754b18a24c45723a35766f3eab6d otherFunctions
## 55: dcec754b18a24c45723a35766f3eab6d otherFunctions
## 56: dcec754b18a24c45723a35766f3eab6d otherFunctions
## 57: dcec754b18a24c45723a35766f3eab6d otherFunctions
## 58: dcec754b18a24c45723a35766f3eab6d otherFunctions
## 59: dcec754b18a24c45723a35766f3eab6d otherFunctions
## 60: dcec754b18a24c45723a35766f3eab6d otherFunctions
## 61: dcec754b18a24c45723a35766f3eab6d otherFunctions
## 62: dcec754b18a24c45723a35766f3eab6d otherFunctions
## 63: dcec754b18a24c45723a35766f3eab6d otherFunctions
## 64: dcec754b18a24c45723a35766f3eab6d otherFunctions
## 65: dcec754b18a24c45723a35766f3eab6d      preDigest
## 66: dcec754b18a24c45723a35766f3eab6d      preDigest
## 67: dcec754b18a24c45723a35766f3eab6d      preDigest
## 68: dcec754b18a24c45723a35766f3eab6d       accessed
## 69: dcec754b18a24c45723a35766f3eab6d       accessed
## 70: dcec754b18a24c45723a35766f3eab6d       accessed
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:06:30
##  2: 37e0d4cab84fd441e92c95013f15812e 2019-03-18 10:06:30
##  3:                          numeric 2019-03-18 10:06:30
##  4:              2019-03-18 10:06:30 2019-03-18 10:06:30
##  5:                 c16f0e2c57596069 2019-03-18 10:06:30
##  6:              rnorm pipe sequence 2019-03-18 10:06:30
##  7:                             1104 2019-03-18 10:06:30
##  8:              2019-03-18 10:06:30 2019-03-18 10:06:30
##  9:                          saveRDS 2019-03-18 10:06:30
## 10:                          do.call 2019-03-18 10:06:30
## 11:                     process_file 2019-03-18 10:06:30
## 12:                    process_group 2019-03-18 10:06:30
## 13:              process_group.block 2019-03-18 10:06:30
## 14:                       call_block 2019-03-18 10:06:30
## 15:                       block_exec 2019-03-18 10:06:30
## 16:                           in_dir 2019-03-18 10:06:30
## 17:                        timing_fn 2019-03-18 10:06:30
## 18:                           handle 2019-03-18 10:06:30
## 19:                      withVisible 2019-03-18 10:06:30
## 20:                              %C% 2019-03-18 10:06:30
## 21:               n:c5775c3b366fb719 2019-03-18 10:06:30
## 22:            mean:152602b8ff81e5bb 2019-03-18 10:06:30
## 23:            .FUN:4f604aa46882b368 2019-03-18 10:06:30
## 24:                              rda 2019-03-18 10:06:30
## 25: 824ba35c4c76273931bec6af02ac8ee3 2019-03-18 10:06:30
## 26:                          numeric 2019-03-18 10:06:30
## 27:              2019-03-18 10:06:30 2019-03-18 10:06:30
## 28:                 f7bee22047b8d0c1 2019-03-18 10:06:30
## 29:                                a 2019-03-18 10:06:30
## 30:                            rnorm 2019-03-18 10:06:30
## 31:                             1008 2019-03-18 10:06:30
## 32:              2019-03-18 10:06:30 2019-03-18 10:06:30
## 33:                          saveRDS 2019-03-18 10:06:30
## 34:                          do.call 2019-03-18 10:06:30
## 35:                     process_file 2019-03-18 10:06:30
## 36:                    process_group 2019-03-18 10:06:30
## 37:              process_group.block 2019-03-18 10:06:30
## 38:                       call_block 2019-03-18 10:06:30
## 39:                       block_exec 2019-03-18 10:06:30
## 40:                           in_dir 2019-03-18 10:06:30
## 41:                        timing_fn 2019-03-18 10:06:30
## 42:                           handle 2019-03-18 10:06:30
## 43:                      withVisible 2019-03-18 10:06:30
## 44:               n:7eef4eae85fd9229 2019-03-18 10:06:30
## 45:            .FUN:4f604aa46882b368 2019-03-18 10:06:30
## 46:                              rda 2019-03-18 10:06:30
## 47: dcec754b18a24c45723a35766f3eab6d 2019-03-18 10:06:30
## 48:                          numeric 2019-03-18 10:06:30
## 49:              2019-03-18 10:06:30 2019-03-18 10:06:30
## 50:                 f11fb1a2880f8060 2019-03-18 10:06:30
## 51:                            rnorm 2019-03-18 10:06:30
## 52:                             1104 2019-03-18 10:06:30
## 53:              2019-03-18 10:06:30 2019-03-18 10:06:30
## 54:                          saveRDS 2019-03-18 10:06:30
## 55:                          do.call 2019-03-18 10:06:30
## 56:                     process_file 2019-03-18 10:06:30
## 57:                    process_group 2019-03-18 10:06:30
## 58:              process_group.block 2019-03-18 10:06:30
## 59:                       call_block 2019-03-18 10:06:30
## 60:                       block_exec 2019-03-18 10:06:30
## 61:                           in_dir 2019-03-18 10:06:30
## 62:                        timing_fn 2019-03-18 10:06:30
## 63:                           handle 2019-03-18 10:06:30
## 64:                      withVisible 2019-03-18 10:06:30
## 65:               n:c5775c3b366fb719 2019-03-18 10:06:30
## 66:            mean:15620f138033a66c 2019-03-18 10:06:30
## 67:            .FUN:4f604aa46882b368 2019-03-18 10:06:30
## 68:              2019-03-18 10:06:30 2019-03-18 10:06:30
## 69:              2019-03-18 10:06:30 2019-03-18 10:06:30
## 70:              2019-03-18 10:06:30 2019-03-18 10:06:30
##                             tagValue         createdDate
clearCache(tmpDir, ask = FALSE)

Example 2: using the “accessed” tag

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# access it again, from Cache
ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
##   loading cached result from previous rnorm call, adding to memoised copy
wholeCache <- showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
# keep only items accessed "recently" (i.e., only objectName:a)
onlyRecentlyAccessed <- showCache(tmpDir, userTags = max(wholeCache[tagKey == "accessed"]$tagValue))
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
# inverse join with 2 data.tables ... using: a[!b]
# i.e., return all of wholeCache that was not recently accessed
toRemove <- unique(wholeCache[!onlyRecentlyAccessed], by = "artifact")$artifact
clearCache(tmpDir, toRemove, ask = FALSE) # remove ones not recently accessed
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
showCache(tmpDir) # still has more recently accessed
## Cache size:
##   Total (including Rasters): 0 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 3 cols: md5hash,name,createdDate
clearCache(tmpDir, ask = FALSE)

Example 3: using keepCache

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# keep only those cached items from the last 24 hours
oneDay <- 60 * 60 * 24
keepCache(tmpDir, after = Sys.time() - oneDay, ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 03481e07fb970c288843b2178fea07f5         format
##  2: 03481e07fb970c288843b2178fea07f5           name
##  3: 03481e07fb970c288843b2178fea07f5          class
##  4: 03481e07fb970c288843b2178fea07f5           date
##  5: 03481e07fb970c288843b2178fea07f5        cacheId
##  6: 03481e07fb970c288843b2178fea07f5     objectName
##  7: 03481e07fb970c288843b2178fea07f5       function
##  8: 03481e07fb970c288843b2178fea07f5    object.size
##  9: 03481e07fb970c288843b2178fea07f5       accessed
## 10: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 11: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 12: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 13: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 14: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 15: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 16: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 17: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 18: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 19: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 20: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 21: 03481e07fb970c288843b2178fea07f5      preDigest
## 22: 03481e07fb970c288843b2178fea07f5      preDigest
## 23: b2c1071b62d2b52e8ce56cfd2e58606d         format
## 24: b2c1071b62d2b52e8ce56cfd2e58606d           name
## 25: b2c1071b62d2b52e8ce56cfd2e58606d          class
## 26: b2c1071b62d2b52e8ce56cfd2e58606d           date
## 27: b2c1071b62d2b52e8ce56cfd2e58606d        cacheId
## 28: b2c1071b62d2b52e8ce56cfd2e58606d     objectName
## 29: b2c1071b62d2b52e8ce56cfd2e58606d       function
## 30: b2c1071b62d2b52e8ce56cfd2e58606d    object.size
## 31: b2c1071b62d2b52e8ce56cfd2e58606d       accessed
## 32: b2c1071b62d2b52e8ce56cfd2e58606d otherFunctions
## 33: b2c1071b62d2b52e8ce56cfd2e58606d otherFunctions
## 34: b2c1071b62d2b52e8ce56cfd2e58606d otherFunctions
## 35: b2c1071b62d2b52e8ce56cfd2e58606d otherFunctions
## 36: b2c1071b62d2b52e8ce56cfd2e58606d otherFunctions
## 37: b2c1071b62d2b52e8ce56cfd2e58606d otherFunctions
## 38: b2c1071b62d2b52e8ce56cfd2e58606d otherFunctions
## 39: b2c1071b62d2b52e8ce56cfd2e58606d otherFunctions
## 40: b2c1071b62d2b52e8ce56cfd2e58606d otherFunctions
## 41: b2c1071b62d2b52e8ce56cfd2e58606d otherFunctions
## 42: b2c1071b62d2b52e8ce56cfd2e58606d otherFunctions
## 43: b2c1071b62d2b52e8ce56cfd2e58606d      preDigest
## 44: b2c1071b62d2b52e8ce56cfd2e58606d      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:06:31
##  2: 03481e07fb970c288843b2178fea07f5 2019-03-18 10:06:31
##  3:                          numeric 2019-03-18 10:06:31
##  4:              2019-03-18 10:06:31 2019-03-18 10:06:31
##  5:                 f7bee22047b8d0c1 2019-03-18 10:06:31
##  6:                                a 2019-03-18 10:06:31
##  7:                            rnorm 2019-03-18 10:06:31
##  8:                             1008 2019-03-18 10:06:31
##  9:              2019-03-18 10:06:31 2019-03-18 10:06:31
## 10:                          saveRDS 2019-03-18 10:06:31
## 11:                          do.call 2019-03-18 10:06:31
## 12:                     process_file 2019-03-18 10:06:31
## 13:                    process_group 2019-03-18 10:06:31
## 14:              process_group.block 2019-03-18 10:06:31
## 15:                       call_block 2019-03-18 10:06:31
## 16:                       block_exec 2019-03-18 10:06:31
## 17:                           in_dir 2019-03-18 10:06:31
## 18:                        timing_fn 2019-03-18 10:06:31
## 19:                           handle 2019-03-18 10:06:31
## 20:                      withVisible 2019-03-18 10:06:31
## 21:               n:7eef4eae85fd9229 2019-03-18 10:06:31
## 22:            .FUN:4f604aa46882b368 2019-03-18 10:06:31
## 23:                              rda 2019-03-18 10:06:31
## 24: b2c1071b62d2b52e8ce56cfd2e58606d 2019-03-18 10:06:31
## 25:                          numeric 2019-03-18 10:06:31
## 26:              2019-03-18 10:06:31 2019-03-18 10:06:31
## 27:                 3aef38d1fc02aee5 2019-03-18 10:06:31
## 28:                                b 2019-03-18 10:06:31
## 29:                            runif 2019-03-18 10:06:31
## 30:                             1008 2019-03-18 10:06:31
## 31:              2019-03-18 10:06:31 2019-03-18 10:06:31
## 32:                          saveRDS 2019-03-18 10:06:31
## 33:                          do.call 2019-03-18 10:06:31
## 34:                     process_file 2019-03-18 10:06:31
## 35:                    process_group 2019-03-18 10:06:31
## 36:              process_group.block 2019-03-18 10:06:31
## 37:                       call_block 2019-03-18 10:06:31
## 38:                       block_exec 2019-03-18 10:06:31
## 39:                           in_dir 2019-03-18 10:06:31
## 40:                        timing_fn 2019-03-18 10:06:31
## 41:                           handle 2019-03-18 10:06:31
## 42:                      withVisible 2019-03-18 10:06:31
## 43:               n:7eef4eae85fd9229 2019-03-18 10:06:31
## 44:            .FUN:881ec847b7161f3c 2019-03-18 10:06:31
##                             tagValue         createdDate
# Keep all Cache items created with an rnorm() call
keepCache(tmpDir, userTags = "rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: 03481e07fb970c288843b2178fea07f5         format
##  2: 03481e07fb970c288843b2178fea07f5           name
##  3: 03481e07fb970c288843b2178fea07f5          class
##  4: 03481e07fb970c288843b2178fea07f5           date
##  5: 03481e07fb970c288843b2178fea07f5        cacheId
##  6: 03481e07fb970c288843b2178fea07f5     objectName
##  7: 03481e07fb970c288843b2178fea07f5       function
##  8: 03481e07fb970c288843b2178fea07f5    object.size
##  9: 03481e07fb970c288843b2178fea07f5       accessed
## 10: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 11: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 12: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 13: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 14: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 15: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 16: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 17: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 18: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 19: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 20: 03481e07fb970c288843b2178fea07f5 otherFunctions
## 21: 03481e07fb970c288843b2178fea07f5      preDigest
## 22: 03481e07fb970c288843b2178fea07f5      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:06:31
##  2: 03481e07fb970c288843b2178fea07f5 2019-03-18 10:06:31
##  3:                          numeric 2019-03-18 10:06:31
##  4:              2019-03-18 10:06:31 2019-03-18 10:06:31
##  5:                 f7bee22047b8d0c1 2019-03-18 10:06:31
##  6:                                a 2019-03-18 10:06:31
##  7:                            rnorm 2019-03-18 10:06:31
##  8:                             1008 2019-03-18 10:06:31
##  9:              2019-03-18 10:06:31 2019-03-18 10:06:31
## 10:                          saveRDS 2019-03-18 10:06:31
## 11:                          do.call 2019-03-18 10:06:31
## 12:                     process_file 2019-03-18 10:06:31
## 13:                    process_group 2019-03-18 10:06:31
## 14:              process_group.block 2019-03-18 10:06:31
## 15:                       call_block 2019-03-18 10:06:31
## 16:                       block_exec 2019-03-18 10:06:31
## 17:                           in_dir 2019-03-18 10:06:31
## 18:                        timing_fn 2019-03-18 10:06:31
## 19:                           handle 2019-03-18 10:06:31
## 20:                      withVisible 2019-03-18 10:06:31
## 21:               n:7eef4eae85fd9229 2019-03-18 10:06:31
## 22:            .FUN:4f604aa46882b368 2019-03-18 10:06:31
##                             tagValue         createdDate
# Remove all Cache items that happened within a rnorm() call
clearCache(tmpDir, userTags = "rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) ## empty
## Cache size:
##   Total (including Rasters): 0 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 3 cols: md5hash,name,createdDate

Example 4: searching for multiple objects in the cache

# default userTags is "and" matching; for "or" matching use |
ranNumsA <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# show all objects (runif and rnorm in this case)
showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 06106c694ad125b4dfd6ee9f9a2c5506         format
##  2: 06106c694ad125b4dfd6ee9f9a2c5506           name
##  3: 06106c694ad125b4dfd6ee9f9a2c5506          class
##  4: 06106c694ad125b4dfd6ee9f9a2c5506           date
##  5: 06106c694ad125b4dfd6ee9f9a2c5506        cacheId
##  6: 06106c694ad125b4dfd6ee9f9a2c5506     objectName
##  7: 06106c694ad125b4dfd6ee9f9a2c5506       function
##  8: 06106c694ad125b4dfd6ee9f9a2c5506    object.size
##  9: 06106c694ad125b4dfd6ee9f9a2c5506       accessed
## 10: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 11: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 12: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 13: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 14: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 15: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 16: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 17: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 18: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 19: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 20: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 21: 06106c694ad125b4dfd6ee9f9a2c5506      preDigest
## 22: 06106c694ad125b4dfd6ee9f9a2c5506      preDigest
## 23: 4b4941e215f23ce56389a17c8951e44a         format
## 24: 4b4941e215f23ce56389a17c8951e44a           name
## 25: 4b4941e215f23ce56389a17c8951e44a          class
## 26: 4b4941e215f23ce56389a17c8951e44a           date
## 27: 4b4941e215f23ce56389a17c8951e44a        cacheId
## 28: 4b4941e215f23ce56389a17c8951e44a     objectName
## 29: 4b4941e215f23ce56389a17c8951e44a       function
## 30: 4b4941e215f23ce56389a17c8951e44a    object.size
## 31: 4b4941e215f23ce56389a17c8951e44a       accessed
## 32: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 33: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 34: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 35: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 36: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 37: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 38: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 39: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 40: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 41: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 42: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 43: 4b4941e215f23ce56389a17c8951e44a      preDigest
## 44: 4b4941e215f23ce56389a17c8951e44a      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:06:32
##  2: 06106c694ad125b4dfd6ee9f9a2c5506 2019-03-18 10:06:32
##  3:                          numeric 2019-03-18 10:06:32
##  4:              2019-03-18 10:06:32 2019-03-18 10:06:32
##  5:                 f7bee22047b8d0c1 2019-03-18 10:06:32
##  6:                                b 2019-03-18 10:06:32
##  7:                            rnorm 2019-03-18 10:06:32
##  8:                             1008 2019-03-18 10:06:32
##  9:              2019-03-18 10:06:32 2019-03-18 10:06:32
## 10:                          saveRDS 2019-03-18 10:06:32
## 11:                          do.call 2019-03-18 10:06:32
## 12:                     process_file 2019-03-18 10:06:32
## 13:                    process_group 2019-03-18 10:06:32
## 14:              process_group.block 2019-03-18 10:06:32
## 15:                       call_block 2019-03-18 10:06:32
## 16:                       block_exec 2019-03-18 10:06:32
## 17:                           in_dir 2019-03-18 10:06:32
## 18:                        timing_fn 2019-03-18 10:06:32
## 19:                           handle 2019-03-18 10:06:32
## 20:                      withVisible 2019-03-18 10:06:32
## 21:               n:7eef4eae85fd9229 2019-03-18 10:06:32
## 22:            .FUN:4f604aa46882b368 2019-03-18 10:06:32
## 23:                              rda 2019-03-18 10:06:32
## 24: 4b4941e215f23ce56389a17c8951e44a 2019-03-18 10:06:32
## 25:                          numeric 2019-03-18 10:06:32
## 26:              2019-03-18 10:06:32 2019-03-18 10:06:32
## 27:                 3aef38d1fc02aee5 2019-03-18 10:06:32
## 28:                                a 2019-03-18 10:06:32
## 29:                            runif 2019-03-18 10:06:32
## 30:                             1008 2019-03-18 10:06:32
## 31:              2019-03-18 10:06:32 2019-03-18 10:06:32
## 32:                          saveRDS 2019-03-18 10:06:32
## 33:                          do.call 2019-03-18 10:06:32
## 34:                     process_file 2019-03-18 10:06:32
## 35:                    process_group 2019-03-18 10:06:32
## 36:              process_group.block 2019-03-18 10:06:32
## 37:                       call_block 2019-03-18 10:06:32
## 38:                       block_exec 2019-03-18 10:06:32
## 39:                           in_dir 2019-03-18 10:06:32
## 40:                        timing_fn 2019-03-18 10:06:32
## 41:                           handle 2019-03-18 10:06:32
## 42:                      withVisible 2019-03-18 10:06:32
## 43:               n:7eef4eae85fd9229 2019-03-18 10:06:32
## 44:            .FUN:881ec847b7161f3c 2019-03-18 10:06:32
##                             tagValue         createdDate
# show objects that are both runif and rnorm
# (i.e., none in this case, because objecs are either or, not both)
showCache(tmpDir, userTags = c("runif", "rnorm")) ## empty
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 4 cols: artifact,tagKey,tagValue,createdDate
# show objects that are either runif or rnorm ("or" search)
showCache(tmpDir, userTags = "runif|rnorm")
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 06106c694ad125b4dfd6ee9f9a2c5506         format
##  2: 06106c694ad125b4dfd6ee9f9a2c5506           name
##  3: 06106c694ad125b4dfd6ee9f9a2c5506          class
##  4: 06106c694ad125b4dfd6ee9f9a2c5506           date
##  5: 06106c694ad125b4dfd6ee9f9a2c5506        cacheId
##  6: 06106c694ad125b4dfd6ee9f9a2c5506     objectName
##  7: 06106c694ad125b4dfd6ee9f9a2c5506       function
##  8: 06106c694ad125b4dfd6ee9f9a2c5506    object.size
##  9: 06106c694ad125b4dfd6ee9f9a2c5506       accessed
## 10: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 11: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 12: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 13: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 14: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 15: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 16: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 17: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 18: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 19: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 20: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 21: 06106c694ad125b4dfd6ee9f9a2c5506      preDigest
## 22: 06106c694ad125b4dfd6ee9f9a2c5506      preDigest
## 23: 4b4941e215f23ce56389a17c8951e44a         format
## 24: 4b4941e215f23ce56389a17c8951e44a           name
## 25: 4b4941e215f23ce56389a17c8951e44a          class
## 26: 4b4941e215f23ce56389a17c8951e44a           date
## 27: 4b4941e215f23ce56389a17c8951e44a        cacheId
## 28: 4b4941e215f23ce56389a17c8951e44a     objectName
## 29: 4b4941e215f23ce56389a17c8951e44a       function
## 30: 4b4941e215f23ce56389a17c8951e44a    object.size
## 31: 4b4941e215f23ce56389a17c8951e44a       accessed
## 32: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 33: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 34: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 35: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 36: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 37: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 38: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 39: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 40: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 41: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 42: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 43: 4b4941e215f23ce56389a17c8951e44a      preDigest
## 44: 4b4941e215f23ce56389a17c8951e44a      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:06:32
##  2: 06106c694ad125b4dfd6ee9f9a2c5506 2019-03-18 10:06:32
##  3:                          numeric 2019-03-18 10:06:32
##  4:              2019-03-18 10:06:32 2019-03-18 10:06:32
##  5:                 f7bee22047b8d0c1 2019-03-18 10:06:32
##  6:                                b 2019-03-18 10:06:32
##  7:                            rnorm 2019-03-18 10:06:32
##  8:                             1008 2019-03-18 10:06:32
##  9:              2019-03-18 10:06:32 2019-03-18 10:06:32
## 10:                          saveRDS 2019-03-18 10:06:32
## 11:                          do.call 2019-03-18 10:06:32
## 12:                     process_file 2019-03-18 10:06:32
## 13:                    process_group 2019-03-18 10:06:32
## 14:              process_group.block 2019-03-18 10:06:32
## 15:                       call_block 2019-03-18 10:06:32
## 16:                       block_exec 2019-03-18 10:06:32
## 17:                           in_dir 2019-03-18 10:06:32
## 18:                        timing_fn 2019-03-18 10:06:32
## 19:                           handle 2019-03-18 10:06:32
## 20:                      withVisible 2019-03-18 10:06:32
## 21:               n:7eef4eae85fd9229 2019-03-18 10:06:32
## 22:            .FUN:4f604aa46882b368 2019-03-18 10:06:32
## 23:                              rda 2019-03-18 10:06:32
## 24: 4b4941e215f23ce56389a17c8951e44a 2019-03-18 10:06:32
## 25:                          numeric 2019-03-18 10:06:32
## 26:              2019-03-18 10:06:32 2019-03-18 10:06:32
## 27:                 3aef38d1fc02aee5 2019-03-18 10:06:32
## 28:                                a 2019-03-18 10:06:32
## 29:                            runif 2019-03-18 10:06:32
## 30:                             1008 2019-03-18 10:06:32
## 31:              2019-03-18 10:06:32 2019-03-18 10:06:32
## 32:                          saveRDS 2019-03-18 10:06:32
## 33:                          do.call 2019-03-18 10:06:32
## 34:                     process_file 2019-03-18 10:06:32
## 35:                    process_group 2019-03-18 10:06:32
## 36:              process_group.block 2019-03-18 10:06:32
## 37:                       call_block 2019-03-18 10:06:32
## 38:                       block_exec 2019-03-18 10:06:32
## 39:                           in_dir 2019-03-18 10:06:32
## 40:                        timing_fn 2019-03-18 10:06:32
## 41:                           handle 2019-03-18 10:06:32
## 42:                      withVisible 2019-03-18 10:06:32
## 43:               n:7eef4eae85fd9229 2019-03-18 10:06:32
## 44:            .FUN:881ec847b7161f3c 2019-03-18 10:06:32
##                             tagValue         createdDate
# keep only objects that are either runif or rnorm ("or" search)
keepCache(tmpDir, userTags = "runif|rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 06106c694ad125b4dfd6ee9f9a2c5506         format
##  2: 06106c694ad125b4dfd6ee9f9a2c5506           name
##  3: 06106c694ad125b4dfd6ee9f9a2c5506          class
##  4: 06106c694ad125b4dfd6ee9f9a2c5506           date
##  5: 06106c694ad125b4dfd6ee9f9a2c5506        cacheId
##  6: 06106c694ad125b4dfd6ee9f9a2c5506     objectName
##  7: 06106c694ad125b4dfd6ee9f9a2c5506       function
##  8: 06106c694ad125b4dfd6ee9f9a2c5506    object.size
##  9: 06106c694ad125b4dfd6ee9f9a2c5506       accessed
## 10: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 11: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 12: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 13: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 14: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 15: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 16: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 17: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 18: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 19: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 20: 06106c694ad125b4dfd6ee9f9a2c5506 otherFunctions
## 21: 06106c694ad125b4dfd6ee9f9a2c5506      preDigest
## 22: 06106c694ad125b4dfd6ee9f9a2c5506      preDigest
## 23: 4b4941e215f23ce56389a17c8951e44a         format
## 24: 4b4941e215f23ce56389a17c8951e44a           name
## 25: 4b4941e215f23ce56389a17c8951e44a          class
## 26: 4b4941e215f23ce56389a17c8951e44a           date
## 27: 4b4941e215f23ce56389a17c8951e44a        cacheId
## 28: 4b4941e215f23ce56389a17c8951e44a     objectName
## 29: 4b4941e215f23ce56389a17c8951e44a       function
## 30: 4b4941e215f23ce56389a17c8951e44a    object.size
## 31: 4b4941e215f23ce56389a17c8951e44a       accessed
## 32: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 33: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 34: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 35: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 36: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 37: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 38: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 39: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 40: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 41: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 42: 4b4941e215f23ce56389a17c8951e44a otherFunctions
## 43: 4b4941e215f23ce56389a17c8951e44a      preDigest
## 44: 4b4941e215f23ce56389a17c8951e44a      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:06:32
##  2: 06106c694ad125b4dfd6ee9f9a2c5506 2019-03-18 10:06:32
##  3:                          numeric 2019-03-18 10:06:32
##  4:              2019-03-18 10:06:32 2019-03-18 10:06:32
##  5:                 f7bee22047b8d0c1 2019-03-18 10:06:32
##  6:                                b 2019-03-18 10:06:32
##  7:                            rnorm 2019-03-18 10:06:32
##  8:                             1008 2019-03-18 10:06:32
##  9:              2019-03-18 10:06:32 2019-03-18 10:06:32
## 10:                          saveRDS 2019-03-18 10:06:32
## 11:                          do.call 2019-03-18 10:06:32
## 12:                     process_file 2019-03-18 10:06:32
## 13:                    process_group 2019-03-18 10:06:32
## 14:              process_group.block 2019-03-18 10:06:32
## 15:                       call_block 2019-03-18 10:06:32
## 16:                       block_exec 2019-03-18 10:06:32
## 17:                           in_dir 2019-03-18 10:06:32
## 18:                        timing_fn 2019-03-18 10:06:32
## 19:                           handle 2019-03-18 10:06:32
## 20:                      withVisible 2019-03-18 10:06:32
## 21:               n:7eef4eae85fd9229 2019-03-18 10:06:32
## 22:            .FUN:4f604aa46882b368 2019-03-18 10:06:32
## 23:                              rda 2019-03-18 10:06:32
## 24: 4b4941e215f23ce56389a17c8951e44a 2019-03-18 10:06:32
## 25:                          numeric 2019-03-18 10:06:32
## 26:              2019-03-18 10:06:32 2019-03-18 10:06:32
## 27:                 3aef38d1fc02aee5 2019-03-18 10:06:32
## 28:                                a 2019-03-18 10:06:32
## 29:                            runif 2019-03-18 10:06:32
## 30:                             1008 2019-03-18 10:06:32
## 31:              2019-03-18 10:06:32 2019-03-18 10:06:32
## 32:                          saveRDS 2019-03-18 10:06:32
## 33:                          do.call 2019-03-18 10:06:32
## 34:                     process_file 2019-03-18 10:06:32
## 35:                    process_group 2019-03-18 10:06:32
## 36:              process_group.block 2019-03-18 10:06:32
## 37:                       call_block 2019-03-18 10:06:32
## 38:                       block_exec 2019-03-18 10:06:32
## 39:                           in_dir 2019-03-18 10:06:32
## 40:                        timing_fn 2019-03-18 10:06:32
## 41:                           handle 2019-03-18 10:06:32
## 42:                      withVisible 2019-03-18 10:06:32
## 43:               n:7eef4eae85fd9229 2019-03-18 10:06:32
## 44:            .FUN:881ec847b7161f3c 2019-03-18 10:06:32
##                             tagValue         createdDate
clearCache(tmpDir, ask = FALSE)

Example 5: using caching to speed up rerunning expensive computations

ras <- raster(extent(0, 5, 0, 5), res = 1,
              vals = sample(1:5, replace = TRUE, size = 25),
              crs = "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84")

# A slow operation, like GIS operation
notCached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  projectRaster(ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)

cached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  # using quote works also
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)

# second time is much faster
reRun <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)
##   loading cached result from previous projectRaster call, adding to memoised copy
# recovered cached version is same as non-cached version
all.equal(notCached, reRun) ## TRUE
## [1] TRUE

Nested Caching

Nested caching, which is when Caching of a function occurs inside an outer function, which is itself cached. This is a critical element to working within a reproducible work flow. It is not enough during development to cache flat code chunks, as there will be many levels of “slow” functions. Ideally, at all points in a development cycle, it should be possible to get to any line of code starting from the very initial steps, running through everything up to that point, in less that 1 second. If the workflow can be kept very fast like this, then there is a guarantee that it will work at any point.

##########################
## Nested Caching
# Make 2 functions
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean)
}
outer <- function(n) {
  Cache(inner, 0.1, cacheRepo = tmpdir2)
}

# make 2 different cache paths
tmpdir1 <- file.path(tempdir(), "first")
tmpdir2 <- file.path(tempdir(), "second")

# Run the Cache ... notOlderThan propagates to all 3 Cache calls,
#   but cacheRepo is tmpdir1 in top level Cache and all nested
#   Cache calls, unless individually overridden ... here inner
#   uses tmpdir2 repository
Cache(outer, n = 2, cacheRepo = tmpdir1, notOlderThan = Sys.time())
## [1]  2.2505539  0.3104797 -0.2358056
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:a7af5367c13aba8f"
## attr(,"call")
## [1] ""
showCache(tmpdir1) # 2 function calls
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 35bee6474644a96c96382adadc09de93         format
##  2: 35bee6474644a96c96382adadc09de93           name
##  3: 35bee6474644a96c96382adadc09de93          class
##  4: 35bee6474644a96c96382adadc09de93           date
##  5: 35bee6474644a96c96382adadc09de93        cacheId
##  6: 35bee6474644a96c96382adadc09de93       function
##  7: 35bee6474644a96c96382adadc09de93    object.size
##  8: 35bee6474644a96c96382adadc09de93       accessed
##  9: 35bee6474644a96c96382adadc09de93 otherFunctions
## 10: 35bee6474644a96c96382adadc09de93 otherFunctions
## 11: 35bee6474644a96c96382adadc09de93 otherFunctions
## 12: 35bee6474644a96c96382adadc09de93 otherFunctions
## 13: 35bee6474644a96c96382adadc09de93 otherFunctions
## 14: 35bee6474644a96c96382adadc09de93 otherFunctions
## 15: 35bee6474644a96c96382adadc09de93 otherFunctions
## 16: 35bee6474644a96c96382adadc09de93 otherFunctions
## 17: 35bee6474644a96c96382adadc09de93 otherFunctions
## 18: 35bee6474644a96c96382adadc09de93 otherFunctions
## 19: 35bee6474644a96c96382adadc09de93 otherFunctions
## 20: 35bee6474644a96c96382adadc09de93      preDigest
## 21: 35bee6474644a96c96382adadc09de93      preDigest
## 22: 35bee6474644a96c96382adadc09de93      preDigest
## 23: 4364fb54bb5da90f3124eb83b3e8eebe         format
## 24: 4364fb54bb5da90f3124eb83b3e8eebe           name
## 25: 4364fb54bb5da90f3124eb83b3e8eebe          class
## 26: 4364fb54bb5da90f3124eb83b3e8eebe           date
## 27: 4364fb54bb5da90f3124eb83b3e8eebe        cacheId
## 28: 4364fb54bb5da90f3124eb83b3e8eebe       function
## 29: 4364fb54bb5da90f3124eb83b3e8eebe    object.size
## 30: 4364fb54bb5da90f3124eb83b3e8eebe       accessed
## 31: 4364fb54bb5da90f3124eb83b3e8eebe otherFunctions
## 32: 4364fb54bb5da90f3124eb83b3e8eebe otherFunctions
## 33: 4364fb54bb5da90f3124eb83b3e8eebe otherFunctions
## 34: 4364fb54bb5da90f3124eb83b3e8eebe otherFunctions
## 35: 4364fb54bb5da90f3124eb83b3e8eebe otherFunctions
## 36: 4364fb54bb5da90f3124eb83b3e8eebe otherFunctions
## 37: 4364fb54bb5da90f3124eb83b3e8eebe otherFunctions
## 38: 4364fb54bb5da90f3124eb83b3e8eebe otherFunctions
## 39: 4364fb54bb5da90f3124eb83b3e8eebe otherFunctions
## 40: 4364fb54bb5da90f3124eb83b3e8eebe otherFunctions
## 41: 4364fb54bb5da90f3124eb83b3e8eebe otherFunctions
## 42: 4364fb54bb5da90f3124eb83b3e8eebe      preDigest
## 43: 4364fb54bb5da90f3124eb83b3e8eebe      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:06:32
##  2: 35bee6474644a96c96382adadc09de93 2019-03-18 10:06:32
##  3:                          numeric 2019-03-18 10:06:32
##  4:              2019-03-18 10:06:32 2019-03-18 10:06:32
##  5:                 4ac2b7c0f42d1e46 2019-03-18 10:06:32
##  6:                            rnorm 2019-03-18 10:06:32
##  7:                             1008 2019-03-18 10:06:32
##  8:              2019-03-18 10:06:32 2019-03-18 10:06:32
##  9:                          saveRDS 2019-03-18 10:06:32
## 10:                          do.call 2019-03-18 10:06:32
## 11:                     process_file 2019-03-18 10:06:32
## 12:                    process_group 2019-03-18 10:06:32
## 13:              process_group.block 2019-03-18 10:06:32
## 14:                       call_block 2019-03-18 10:06:32
## 15:                       block_exec 2019-03-18 10:06:32
## 16:                           in_dir 2019-03-18 10:06:32
## 17:                        timing_fn 2019-03-18 10:06:32
## 18:                           handle 2019-03-18 10:06:32
## 19:                      withVisible 2019-03-18 10:06:32
## 20:               n:7f12988bd88a0fb8 2019-03-18 10:06:32
## 21:            mean:22413394efd9f6a3 2019-03-18 10:06:32
## 22:            .FUN:4f604aa46882b368 2019-03-18 10:06:32
## 23:                              rda 2019-03-18 10:06:32
## 24: 4364fb54bb5da90f3124eb83b3e8eebe 2019-03-18 10:06:32
## 25:                          numeric 2019-03-18 10:06:32
## 26:              2019-03-18 10:06:32 2019-03-18 10:06:32
## 27:                 a7af5367c13aba8f 2019-03-18 10:06:32
## 28:                            outer 2019-03-18 10:06:32
## 29:                             1008 2019-03-18 10:06:32
## 30:              2019-03-18 10:06:32 2019-03-18 10:06:32
## 31:                          saveRDS 2019-03-18 10:06:32
## 32:                          do.call 2019-03-18 10:06:32
## 33:                     process_file 2019-03-18 10:06:32
## 34:                    process_group 2019-03-18 10:06:32
## 35:              process_group.block 2019-03-18 10:06:32
## 36:                       call_block 2019-03-18 10:06:32
## 37:                       block_exec 2019-03-18 10:06:32
## 38:                           in_dir 2019-03-18 10:06:32
## 39:                        timing_fn 2019-03-18 10:06:32
## 40:                           handle 2019-03-18 10:06:32
## 41:                      withVisible 2019-03-18 10:06:32
## 42:               n:82dc709f2b91918a 2019-03-18 10:06:32
## 43:            .FUN:892a6afc47a63a90 2019-03-18 10:06:32
##                             tagValue         createdDate
showCache(tmpdir2) # 1 function call
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: 770f5d2d6600d61b3a95a4cd97916d35         format
##  2: 770f5d2d6600d61b3a95a4cd97916d35           name
##  3: 770f5d2d6600d61b3a95a4cd97916d35          class
##  4: 770f5d2d6600d61b3a95a4cd97916d35           date
##  5: 770f5d2d6600d61b3a95a4cd97916d35        cacheId
##  6: 770f5d2d6600d61b3a95a4cd97916d35       function
##  7: 770f5d2d6600d61b3a95a4cd97916d35    object.size
##  8: 770f5d2d6600d61b3a95a4cd97916d35       accessed
##  9: 770f5d2d6600d61b3a95a4cd97916d35 otherFunctions
## 10: 770f5d2d6600d61b3a95a4cd97916d35 otherFunctions
## 11: 770f5d2d6600d61b3a95a4cd97916d35 otherFunctions
## 12: 770f5d2d6600d61b3a95a4cd97916d35 otherFunctions
## 13: 770f5d2d6600d61b3a95a4cd97916d35 otherFunctions
## 14: 770f5d2d6600d61b3a95a4cd97916d35 otherFunctions
## 15: 770f5d2d6600d61b3a95a4cd97916d35 otherFunctions
## 16: 770f5d2d6600d61b3a95a4cd97916d35 otherFunctions
## 17: 770f5d2d6600d61b3a95a4cd97916d35 otherFunctions
## 18: 770f5d2d6600d61b3a95a4cd97916d35 otherFunctions
## 19: 770f5d2d6600d61b3a95a4cd97916d35 otherFunctions
## 20: 770f5d2d6600d61b3a95a4cd97916d35      preDigest
## 21: 770f5d2d6600d61b3a95a4cd97916d35      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:06:32
##  2: 770f5d2d6600d61b3a95a4cd97916d35 2019-03-18 10:06:32
##  3:                          numeric 2019-03-18 10:06:32
##  4:              2019-03-18 10:06:32 2019-03-18 10:06:32
##  5:                 33ceb4fb525fd08f 2019-03-18 10:06:32
##  6:                            inner 2019-03-18 10:06:32
##  7:                             1008 2019-03-18 10:06:32
##  8:              2019-03-18 10:06:32 2019-03-18 10:06:32
##  9:                          saveRDS 2019-03-18 10:06:32
## 10:                          do.call 2019-03-18 10:06:32
## 11:                     process_file 2019-03-18 10:06:32
## 12:                    process_group 2019-03-18 10:06:32
## 13:              process_group.block 2019-03-18 10:06:32
## 14:                       call_block 2019-03-18 10:06:32
## 15:                       block_exec 2019-03-18 10:06:32
## 16:                           in_dir 2019-03-18 10:06:32
## 17:                        timing_fn 2019-03-18 10:06:32
## 18:                           handle 2019-03-18 10:06:32
## 19:                      withVisible 2019-03-18 10:06:32
## 20:            mean:22413394efd9f6a3 2019-03-18 10:06:32
## 21:            .FUN:87e2c30917a34d25 2019-03-18 10:06:32
##                             tagValue         createdDate
# userTags get appended
# all items have the outer tag propagate, plus inner ones only have inner ones
clearCache(tmpdir1, ask = FALSE)
outerTag <- "outerTag"
innerTag <- "innerTag"
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean, notOlderThan = Sys.time() - 1e5, userTags = innerTag)
}
outer <- function(n) {
  Cache(inner, 0.1)
}
aa <- Cache(outer, n = 2, cacheRepo = tmpdir1, userTags = outerTag)
showCache(tmpdir1) # rnorm function has outerTag and innerTag, inner and outer only have outerTag
## Cache size:
##   Total (including Rasters): 756 bytes
##   Selected objects (not including Rasters): 756 bytes
##                             artifact         tagKey
##  1: 1cb9db673568bcb6c2209c7c551df3e9         format
##  2: 1cb9db673568bcb6c2209c7c551df3e9           name
##  3: 1cb9db673568bcb6c2209c7c551df3e9          class
##  4: 1cb9db673568bcb6c2209c7c551df3e9           date
##  5: 1cb9db673568bcb6c2209c7c551df3e9        cacheId
##  6: 1cb9db673568bcb6c2209c7c551df3e9       outerTag
##  7: 1cb9db673568bcb6c2209c7c551df3e9       function
##  8: 1cb9db673568bcb6c2209c7c551df3e9    object.size
##  9: 1cb9db673568bcb6c2209c7c551df3e9       accessed
## 10: 1cb9db673568bcb6c2209c7c551df3e9 otherFunctions
## 11: 1cb9db673568bcb6c2209c7c551df3e9 otherFunctions
## 12: 1cb9db673568bcb6c2209c7c551df3e9 otherFunctions
## 13: 1cb9db673568bcb6c2209c7c551df3e9 otherFunctions
## 14: 1cb9db673568bcb6c2209c7c551df3e9 otherFunctions
## 15: 1cb9db673568bcb6c2209c7c551df3e9 otherFunctions
## 16: 1cb9db673568bcb6c2209c7c551df3e9 otherFunctions
## 17: 1cb9db673568bcb6c2209c7c551df3e9 otherFunctions
## 18: 1cb9db673568bcb6c2209c7c551df3e9 otherFunctions
## 19: 1cb9db673568bcb6c2209c7c551df3e9 otherFunctions
## 20: 1cb9db673568bcb6c2209c7c551df3e9 otherFunctions
## 21: 1cb9db673568bcb6c2209c7c551df3e9      preDigest
## 22: 1cb9db673568bcb6c2209c7c551df3e9      preDigest
## 23: 3c12577b1f885582ba5ca1e54266d6b2         format
## 24: 3c12577b1f885582ba5ca1e54266d6b2           name
## 25: 3c12577b1f885582ba5ca1e54266d6b2          class
## 26: 3c12577b1f885582ba5ca1e54266d6b2           date
## 27: 3c12577b1f885582ba5ca1e54266d6b2        cacheId
## 28: 3c12577b1f885582ba5ca1e54266d6b2       innerTag
## 29: 3c12577b1f885582ba5ca1e54266d6b2       outerTag
## 30: 3c12577b1f885582ba5ca1e54266d6b2       function
## 31: 3c12577b1f885582ba5ca1e54266d6b2    object.size
## 32: 3c12577b1f885582ba5ca1e54266d6b2       accessed
## 33: 3c12577b1f885582ba5ca1e54266d6b2 otherFunctions
## 34: 3c12577b1f885582ba5ca1e54266d6b2 otherFunctions
## 35: 3c12577b1f885582ba5ca1e54266d6b2 otherFunctions
## 36: 3c12577b1f885582ba5ca1e54266d6b2 otherFunctions
## 37: 3c12577b1f885582ba5ca1e54266d6b2 otherFunctions
## 38: 3c12577b1f885582ba5ca1e54266d6b2 otherFunctions
## 39: 3c12577b1f885582ba5ca1e54266d6b2 otherFunctions
## 40: 3c12577b1f885582ba5ca1e54266d6b2 otherFunctions
## 41: 3c12577b1f885582ba5ca1e54266d6b2 otherFunctions
## 42: 3c12577b1f885582ba5ca1e54266d6b2 otherFunctions
## 43: 3c12577b1f885582ba5ca1e54266d6b2 otherFunctions
## 44: 3c12577b1f885582ba5ca1e54266d6b2      preDigest
## 45: 3c12577b1f885582ba5ca1e54266d6b2      preDigest
## 46: 3c12577b1f885582ba5ca1e54266d6b2      preDigest
## 47: 55695707ac280879c57294317b904482         format
## 48: 55695707ac280879c57294317b904482           name
## 49: 55695707ac280879c57294317b904482          class
## 50: 55695707ac280879c57294317b904482           date
## 51: 55695707ac280879c57294317b904482        cacheId
## 52: 55695707ac280879c57294317b904482       outerTag
## 53: 55695707ac280879c57294317b904482       function
## 54: 55695707ac280879c57294317b904482    object.size
## 55: 55695707ac280879c57294317b904482       accessed
## 56: 55695707ac280879c57294317b904482 otherFunctions
## 57: 55695707ac280879c57294317b904482 otherFunctions
## 58: 55695707ac280879c57294317b904482 otherFunctions
## 59: 55695707ac280879c57294317b904482 otherFunctions
## 60: 55695707ac280879c57294317b904482 otherFunctions
## 61: 55695707ac280879c57294317b904482 otherFunctions
## 62: 55695707ac280879c57294317b904482 otherFunctions
## 63: 55695707ac280879c57294317b904482 otherFunctions
## 64: 55695707ac280879c57294317b904482 otherFunctions
## 65: 55695707ac280879c57294317b904482 otherFunctions
## 66: 55695707ac280879c57294317b904482 otherFunctions
## 67: 55695707ac280879c57294317b904482      preDigest
## 68: 55695707ac280879c57294317b904482      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-03-18 10:06:32
##  2: 1cb9db673568bcb6c2209c7c551df3e9 2019-03-18 10:06:33
##  3:                          numeric 2019-03-18 10:06:33
##  4:              2019-03-18 10:06:33 2019-03-18 10:06:33
##  5:                 b06af03d5a73dc7d 2019-03-18 10:06:33
##  6:                         outerTag 2019-03-18 10:06:33
##  7:                            inner 2019-03-18 10:06:33
##  8:                             1008 2019-03-18 10:06:33
##  9:              2019-03-18 10:06:32 2019-03-18 10:06:33
## 10:                          saveRDS 2019-03-18 10:06:33
## 11:                          do.call 2019-03-18 10:06:33
## 12:                     process_file 2019-03-18 10:06:33
## 13:                    process_group 2019-03-18 10:06:33
## 14:              process_group.block 2019-03-18 10:06:33
## 15:                       call_block 2019-03-18 10:06:33
## 16:                       block_exec 2019-03-18 10:06:33
## 17:                           in_dir 2019-03-18 10:06:33
## 18:                        timing_fn 2019-03-18 10:06:33
## 19:                           handle 2019-03-18 10:06:33
## 20:                      withVisible 2019-03-18 10:06:33
## 21:            mean:22413394efd9f6a3 2019-03-18 10:06:33
## 22:            .FUN:7ad10bc1ae528d8c 2019-03-18 10:06:33
## 23:                              rda 2019-03-18 10:06:32
## 24: 3c12577b1f885582ba5ca1e54266d6b2 2019-03-18 10:06:32
## 25:                          numeric 2019-03-18 10:06:32
## 26:              2019-03-18 10:06:32 2019-03-18 10:06:32
## 27:                 4ac2b7c0f42d1e46 2019-03-18 10:06:32
## 28:                         innerTag 2019-03-18 10:06:32
## 29:                         outerTag 2019-03-18 10:06:32
## 30:                            rnorm 2019-03-18 10:06:32
## 31:                             1008 2019-03-18 10:06:32
## 32:              2019-03-18 10:06:32 2019-03-18 10:06:32
## 33:                          saveRDS 2019-03-18 10:06:32
## 34:                          do.call 2019-03-18 10:06:32
## 35:                     process_file 2019-03-18 10:06:32
## 36:                    process_group 2019-03-18 10:06:32
## 37:              process_group.block 2019-03-18 10:06:32
## 38:                       call_block 2019-03-18 10:06:32
## 39:                       block_exec 2019-03-18 10:06:32
## 40:                           in_dir 2019-03-18 10:06:32
## 41:                        timing_fn 2019-03-18 10:06:32
## 42:                           handle 2019-03-18 10:06:32
## 43:                      withVisible 2019-03-18 10:06:32
## 44:               n:7f12988bd88a0fb8 2019-03-18 10:06:32
## 45:            mean:22413394efd9f6a3 2019-03-18 10:06:32
## 46:            .FUN:4f604aa46882b368 2019-03-18 10:06:32
## 47:                              rda 2019-03-18 10:06:33
## 48: 55695707ac280879c57294317b904482 2019-03-18 10:06:33
## 49:                          numeric 2019-03-18 10:06:33
## 50:              2019-03-18 10:06:33 2019-03-18 10:06:33
## 51:                 88a34e1d033329e5 2019-03-18 10:06:33
## 52:                         outerTag 2019-03-18 10:06:33
## 53:                            outer 2019-03-18 10:06:33
## 54:                             1008 2019-03-18 10:06:33
## 55:              2019-03-18 10:06:33 2019-03-18 10:06:33
## 56:                          saveRDS 2019-03-18 10:06:33
## 57:                          do.call 2019-03-18 10:06:33
## 58:                     process_file 2019-03-18 10:06:33
## 59:                    process_group 2019-03-18 10:06:33
## 60:              process_group.block 2019-03-18 10:06:33
## 61:                       call_block 2019-03-18 10:06:33
## 62:                       block_exec 2019-03-18 10:06:33
## 63:                           in_dir 2019-03-18 10:06:33
## 64:                        timing_fn 2019-03-18 10:06:33
## 65:                           handle 2019-03-18 10:06:33
## 66:                      withVisible 2019-03-18 10:06:33
## 67:               n:82dc709f2b91918a 2019-03-18 10:06:33
## 68:            .FUN:5f06fb5fbffe9e3b 2019-03-18 10:06:33
##                             tagValue         createdDate

cacheId

Sometimes, it is not absolutely desirable to maintain the work flow intact because changes that are irrelevant to the analysis, such as changing messages sent to a user, may be changed, without a desire to rerun functions. The cacheId argument is for this. Once a piece of code is run, then the cacheId can be manually extracted (it is reported at the end of a Cache call) and manually placed in the code, passed in as, say, cacheId = "ad184ce64541972b50afd8e7b75f821b".

### cacheId
set.seed(1)
Cache(rnorm, 1, cacheRepo = tmpdir1)
## [1] -0.6264538
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:7072c305d8c69df0"
## attr(,"call")
## [1] ""
# manually look at output attribute which shows cacheId: ad184ce64541972b50afd8e7b75f821b
Cache(rnorm, 1, cacheRepo = tmpdir1, cacheId = "ad184ce64541972b50afd8e7b75f821b") # same value
## cacheId is not same as calculated hash. Manually searching for cacheId:ad184ce64541972b50afd8e7b75f821b
## [1] 0.1836433
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:ad184ce64541972b50afd8e7b75f821b"
## attr(,"call")
## [1] ""
# override even with different inputs:
Cache(rnorm, 2, cacheRepo = tmpdir1, cacheId = "ad184ce64541972b50afd8e7b75f821b")
## cacheId is not same as calculated hash. Manually searching for cacheId:ad184ce64541972b50afd8e7b75f821b
##   loading cached result from previous rnorm call, adding to memoised copy
## [1] 0.1836433
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] FALSE
## 
## attr(,"tags")
## [1] "cacheId:ad184ce64541972b50afd8e7b75f821b"
## attr(,"call")
## [1] ""
## cleanup
unlink(c("filename.rda", "filename1.rda"))

Working with the Cache manually

Since the cache is simply an archivist repository, all archivist functions will work as is. In addition, there are several helpers in the reproducible package, including showCache, keepCache and clearCache that may be useful. Also, one can access cached items manually (rather than simply rerunning the same Cache function again).

if (requireNamespace("archivist")) {
  # get the RasterLayer that was produced with the gaussMap function:
  mapHash <- unique(showCache(tmpDir, userTags = "projectRaster")$artifact)
  map <- archivist::loadFromLocalRepo(md5hash = mapHash[1], repoDir = tmpDir, value = TRUE)
  
  plot(map)
}
## Cache size:
##   Total (including Rasters): 3.3 Kb
##   Selected objects (not including Rasters): 3.3 Kb

## cleanup
unlink(dirname(tmpDir), recursive = TRUE)

Reproducible Workflow

In general, we feel that a liberal use of Cache will make a re-usable and reproducible work flow. shiny apps can be made, taking advantage of Cache. Indeed, much of the difficulty in managing data sets and saving them for future use, can be accommodated by caching.

Nested Caching

Cache individual functions

Cache(<functionName>, <other arguments>)

This will allow fine scale control of individual function calls.