Reproducible workflow

As part of a reproducible workflow, caching of function calls, code chunks, and other elements of a project is a critical component. The objective of a reproducible workflow is is likely that an entire work flow from raw data to publication, decision support, report writing, presentation building etc., could be built and be reproducible anywhere, on any computer, operating system, with any starting conditions, on demand. The reproducible::Cache function is built to work with any R function.

Differences with other approaches

Cache uses 2 key the archivist functions saveToLocalRepo and loadFromLocalRepo, but does not use archivist::cache. Similar to archivist::cache, there is some reliance on digest::digest to determine whether the arguments are identical in subsequent iterations; however, it also uses fastdigest::fastdigest to make it substantially faster in many cases. It also but does many things that make standard caching with digest::digest don’t work reliably between systems. For these, the function .robustDigest is introduced to make caching transferable between systems. This is relevant for file paths, environments, parallel clusters, functions (which are contained within an environment), and many others (e.g., see ?.robustDigest for methods). Cache also adds important elements like automated tagging and the option to retrieve disk-cached values via stashed objects in memory using memoise::memoise. This means that running Cache 1, 2, and 3 times on the same function will get progressively faster. This can be extremely useful for web apps built with, say shiny.

Function-level caching

Any function can be cached using: Cache(FUN = functionName, ...).

This will be a slight change to a function call, such as: projectRaster(raster, crs = crs(newRaster)) to Cache(projectRaster, raster, crs = crs(newRaster)).

This is particularly useful for expensive operations.

library(raster)
## Loading required package: sp
library(reproducible)

tmpDir <- file.path(tempdir(), "reproducible_examples", "Cache")
checkPath(tmpDir, create = TRUE)
## [1] "/tmp/RtmpVZEWOk/reproducible_examples/Cache"
ras <- raster(extent(0,1000,0,1000), vals = 1:1e6, res = 1)
crs(ras) <- "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84"

newCRS <- "+init=epsg:4326 +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0"
# No Cache
system.time(map1 <- projectRaster(ras, crs = newCRS))
##    user  system elapsed 
##   2.633   0.168   2.805
# With Cache -- a little slower the first time because saving to disk
system.time(map1 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir,
                         notOlderThan = Sys.time()))
##    user  system elapsed 
##   2.462   0.149   2.615
# vastly faster the second time
system.time(map2 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
##   loading cached result from previous projectRaster call, adding to memoised copy
##    user  system elapsed 
##   0.113   0.008   0.121
# even faster the third time
system.time(map3 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
##   loading memoised result from previous projectRaster call.
##    user  system elapsed 
##   0.046   0.000   0.046
all.equal(map1, map2) # TRUE
## [1] TRUE
all.equal(map1, map3) # TRUE
## [1] TRUE

Caching examples

Basic use

library(raster)
# magrittr, if loaded, gives an error below
  try(detach("package:magrittr", unload = TRUE), silent = TRUE)

try(clearCache(tmpDir), silent = TRUE) # just to make sure it is clear
## Cache size:
##   Total (including Rasters): 1.9 Mb
##   Selected objects (not including Rasters): 1.9 Mb
ranNumsA <- Cache(rnorm, 10, 16, cacheRepo = tmpDir)

# All same
ranNumsB <- Cache(rnorm, 10, 16, cacheRepo = tmpDir) # recovers cached copy
##   loading cached result from previous rnorm call, adding to memoised copy
ranNumsC <- rnorm(10, 16) %>% Cache(cacheRepo = tmpDir) # recovers cached copy
##   loading memoised result from previous 'rnorm' pipe sequence call.
ranNumsD <- Cache(quote(rnorm(n = 10, 16)), cacheRepo = tmpDir) # recovers cached copy
##   loading memoised result from previous rnorm call.
# Any minor change makes it different
ranNumsE <- rnorm(10, 6) %>% Cache(cacheRepo = tmpDir) # different

Example 1: Basic cache use with tags

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

showCache(tmpDir, userTags = c("objectName"))
## Cache size:
##   Total (including Rasters): 1000 bytes
##   Selected objects (not including Rasters): 476 bytes
##                             artifact         tagKey
##  1: a1a364b0ff2fd4a1153f981a8b2edda2         format
##  2: a1a364b0ff2fd4a1153f981a8b2edda2           name
##  3: a1a364b0ff2fd4a1153f981a8b2edda2          class
##  4: a1a364b0ff2fd4a1153f981a8b2edda2           date
##  5: a1a364b0ff2fd4a1153f981a8b2edda2        cacheId
##  6: a1a364b0ff2fd4a1153f981a8b2edda2     objectName
##  7: a1a364b0ff2fd4a1153f981a8b2edda2       function
##  8: a1a364b0ff2fd4a1153f981a8b2edda2    object.size
##  9: a1a364b0ff2fd4a1153f981a8b2edda2       accessed
## 10: a1a364b0ff2fd4a1153f981a8b2edda2 otherFunctions
## 11: a1a364b0ff2fd4a1153f981a8b2edda2 otherFunctions
## 12: a1a364b0ff2fd4a1153f981a8b2edda2 otherFunctions
## 13: a1a364b0ff2fd4a1153f981a8b2edda2 otherFunctions
## 14: a1a364b0ff2fd4a1153f981a8b2edda2 otherFunctions
## 15: a1a364b0ff2fd4a1153f981a8b2edda2 otherFunctions
## 16: a1a364b0ff2fd4a1153f981a8b2edda2 otherFunctions
## 17: a1a364b0ff2fd4a1153f981a8b2edda2 otherFunctions
## 18: a1a364b0ff2fd4a1153f981a8b2edda2      preDigest
## 19: a1a364b0ff2fd4a1153f981a8b2edda2      preDigest
## 20: f0d80b70a2e9c21742904505bfc62e50         format
## 21: f0d80b70a2e9c21742904505bfc62e50           name
## 22: f0d80b70a2e9c21742904505bfc62e50          class
## 23: f0d80b70a2e9c21742904505bfc62e50           date
## 24: f0d80b70a2e9c21742904505bfc62e50        cacheId
## 25: f0d80b70a2e9c21742904505bfc62e50     objectName
## 26: f0d80b70a2e9c21742904505bfc62e50       function
## 27: f0d80b70a2e9c21742904505bfc62e50    object.size
## 28: f0d80b70a2e9c21742904505bfc62e50       accessed
## 29: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 30: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 31: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 32: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 33: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 34: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 35: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 36: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 37: f0d80b70a2e9c21742904505bfc62e50      preDigest
## 38: f0d80b70a2e9c21742904505bfc62e50      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-08-05 21:53:32
##  2:      a1a364b0ff2fd4a1153f981a8b2edda2 2018-08-05 21:53:32
##  3:                               numeric 2018-08-05 21:53:32
##  4:                   2018-08-05 21:53:32 2018-08-05 21:53:32
##  5:      e37bb635c97bc2eeecab63816b881bbc 2018-08-05 21:53:32
##  6:                                     b 2018-08-05 21:53:32
##  7:                                 runif 2018-08-05 21:53:32
##  8:                                   952 2018-08-05 21:53:32
##  9:                   2018-08-05 21:53:32 2018-08-05 21:53:32
## 10:                   withCallingHandlers 2018-08-05 21:53:32
## 11:                               saveRDS 2018-08-05 21:53:32
## 12:                               do.call 2018-08-05 21:53:32
## 13:                          process_file 2018-08-05 21:53:32
## 14:                         process_group 2018-08-05 21:53:32
## 15:                   process_group.block 2018-08-05 21:53:32
## 16:                            call_block 2018-08-05 21:53:32
## 17:                            block_exec 2018-08-05 21:53:32
## 18:    n:969a49ec15bcd4323ff31538af321264 2018-08-05 21:53:32
## 19: .FUN:d2631d24c3b38b89c7bdd4ab7faaaac3 2018-08-05 21:53:32
## 20:                                   rda 2018-08-05 21:53:32
## 21:      f0d80b70a2e9c21742904505bfc62e50 2018-08-05 21:53:32
## 22:                               numeric 2018-08-05 21:53:32
## 23:                   2018-08-05 21:53:32 2018-08-05 21:53:32
## 24:      85874f26b2e0c1ef689a7d379d275ebf 2018-08-05 21:53:32
## 25:                                     a 2018-08-05 21:53:32
## 26:                                 rnorm 2018-08-05 21:53:32
## 27:                                   952 2018-08-05 21:53:32
## 28:                   2018-08-05 21:53:32 2018-08-05 21:53:32
## 29:                   withCallingHandlers 2018-08-05 21:53:32
## 30:                               saveRDS 2018-08-05 21:53:32
## 31:                               do.call 2018-08-05 21:53:32
## 32:                          process_file 2018-08-05 21:53:32
## 33:                         process_group 2018-08-05 21:53:32
## 34:                   process_group.block 2018-08-05 21:53:32
## 35:                            call_block 2018-08-05 21:53:32
## 36:                            block_exec 2018-08-05 21:53:32
## 37:    n:969a49ec15bcd4323ff31538af321264 2018-08-05 21:53:32
## 38: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-08-05 21:53:32
##                                  tagValue         createdDate
showCache(tmpDir, userTags = c("^a$")) # regular expression ... "a" exactly
## Cache size:
##   Total (including Rasters): 1000 bytes
##   Selected objects (not including Rasters): 238 bytes
##                             artifact         tagKey
##  1: f0d80b70a2e9c21742904505bfc62e50         format
##  2: f0d80b70a2e9c21742904505bfc62e50           name
##  3: f0d80b70a2e9c21742904505bfc62e50          class
##  4: f0d80b70a2e9c21742904505bfc62e50           date
##  5: f0d80b70a2e9c21742904505bfc62e50        cacheId
##  6: f0d80b70a2e9c21742904505bfc62e50     objectName
##  7: f0d80b70a2e9c21742904505bfc62e50       function
##  8: f0d80b70a2e9c21742904505bfc62e50    object.size
##  9: f0d80b70a2e9c21742904505bfc62e50       accessed
## 10: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 11: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 12: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 13: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 14: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 15: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 16: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 17: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 18: f0d80b70a2e9c21742904505bfc62e50      preDigest
## 19: f0d80b70a2e9c21742904505bfc62e50      preDigest
##                                  tagValue         createdDate
##  1:                                   rda 2018-08-05 21:53:32
##  2:      f0d80b70a2e9c21742904505bfc62e50 2018-08-05 21:53:32
##  3:                               numeric 2018-08-05 21:53:32
##  4:                   2018-08-05 21:53:32 2018-08-05 21:53:32
##  5:      85874f26b2e0c1ef689a7d379d275ebf 2018-08-05 21:53:32
##  6:                                     a 2018-08-05 21:53:32
##  7:                                 rnorm 2018-08-05 21:53:32
##  8:                                   952 2018-08-05 21:53:32
##  9:                   2018-08-05 21:53:32 2018-08-05 21:53:32
## 10:                   withCallingHandlers 2018-08-05 21:53:32
## 11:                               saveRDS 2018-08-05 21:53:32
## 12:                               do.call 2018-08-05 21:53:32
## 13:                          process_file 2018-08-05 21:53:32
## 14:                         process_group 2018-08-05 21:53:32
## 15:                   process_group.block 2018-08-05 21:53:32
## 16:                            call_block 2018-08-05 21:53:32
## 17:                            block_exec 2018-08-05 21:53:32
## 18:    n:969a49ec15bcd4323ff31538af321264 2018-08-05 21:53:32
## 19: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-08-05 21:53:32
showCache(tmpDir, userTags = c("runif")) # show only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 1000 bytes
##   Selected objects (not including Rasters): 238 bytes
##                             artifact         tagKey
##  1: a1a364b0ff2fd4a1153f981a8b2edda2         format
##  2: a1a364b0ff2fd4a1153f981a8b2edda2           name
##  3: a1a364b0ff2fd4a1153f981a8b2edda2          class
##  4: a1a364b0ff2fd4a1153f981a8b2edda2           date
##  5: a1a364b0ff2fd4a1153f981a8b2edda2        cacheId
##  6: a1a364b0ff2fd4a1153f981a8b2edda2     objectName
##  7: a1a364b0ff2fd4a1153f981a8b2edda2       function
##  8: a1a364b0ff2fd4a1153f981a8b2edda2    object.size
##  9: a1a364b0ff2fd4a1153f981a8b2edda2       accessed
## 10: a1a364b0ff2fd4a1153f981a8b2edda2 otherFunctions
## 11: a1a364b0ff2fd4a1153f981a8b2edda2 otherFunctions
## 12: a1a364b0ff2fd4a1153f981a8b2edda2 otherFunctions
## 13: a1a364b0ff2fd4a1153f981a8b2edda2 otherFunctions
## 14: a1a364b0ff2fd4a1153f981a8b2edda2 otherFunctions
## 15: a1a364b0ff2fd4a1153f981a8b2edda2 otherFunctions
## 16: a1a364b0ff2fd4a1153f981a8b2edda2 otherFunctions
## 17: a1a364b0ff2fd4a1153f981a8b2edda2 otherFunctions
## 18: a1a364b0ff2fd4a1153f981a8b2edda2      preDigest
## 19: a1a364b0ff2fd4a1153f981a8b2edda2      preDigest
##                                  tagValue         createdDate
##  1:                                   rda 2018-08-05 21:53:32
##  2:      a1a364b0ff2fd4a1153f981a8b2edda2 2018-08-05 21:53:32
##  3:                               numeric 2018-08-05 21:53:32
##  4:                   2018-08-05 21:53:32 2018-08-05 21:53:32
##  5:      e37bb635c97bc2eeecab63816b881bbc 2018-08-05 21:53:32
##  6:                                     b 2018-08-05 21:53:32
##  7:                                 runif 2018-08-05 21:53:32
##  8:                                   952 2018-08-05 21:53:32
##  9:                   2018-08-05 21:53:32 2018-08-05 21:53:32
## 10:                   withCallingHandlers 2018-08-05 21:53:32
## 11:                               saveRDS 2018-08-05 21:53:32
## 12:                               do.call 2018-08-05 21:53:32
## 13:                          process_file 2018-08-05 21:53:32
## 14:                         process_group 2018-08-05 21:53:32
## 15:                   process_group.block 2018-08-05 21:53:32
## 16:                            call_block 2018-08-05 21:53:32
## 17:                            block_exec 2018-08-05 21:53:32
## 18:    n:969a49ec15bcd4323ff31538af321264 2018-08-05 21:53:32
## 19: .FUN:d2631d24c3b38b89c7bdd4ab7faaaac3 2018-08-05 21:53:32
clearCache(tmpDir, userTags = c("runif")) # remove only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 1000 bytes
##   Selected objects (not including Rasters): 238 bytes
showCache(tmpDir) # only those made during rnorm call
## Cache size:
##   Total (including Rasters): 762 bytes
##   Selected objects (not including Rasters): 762 bytes
##                             artifact         tagKey
##  1: 2b5512931b191cfc5020960435f2db4c         format
##  2: 2b5512931b191cfc5020960435f2db4c           name
##  3: 2b5512931b191cfc5020960435f2db4c          class
##  4: 2b5512931b191cfc5020960435f2db4c           date
##  5: 2b5512931b191cfc5020960435f2db4c        cacheId
##  6: 2b5512931b191cfc5020960435f2db4c       function
##  7: 2b5512931b191cfc5020960435f2db4c    object.size
##  8: 2b5512931b191cfc5020960435f2db4c       accessed
##  9: 2b5512931b191cfc5020960435f2db4c otherFunctions
## 10: 2b5512931b191cfc5020960435f2db4c otherFunctions
## 11: 2b5512931b191cfc5020960435f2db4c otherFunctions
## 12: 2b5512931b191cfc5020960435f2db4c otherFunctions
## 13: 2b5512931b191cfc5020960435f2db4c otherFunctions
## 14: 2b5512931b191cfc5020960435f2db4c otherFunctions
## 15: 2b5512931b191cfc5020960435f2db4c otherFunctions
## 16: 2b5512931b191cfc5020960435f2db4c otherFunctions
## 17: 2b5512931b191cfc5020960435f2db4c      preDigest
## 18: 2b5512931b191cfc5020960435f2db4c      preDigest
## 19: 2b5512931b191cfc5020960435f2db4c      preDigest
## 20: 2b5512931b191cfc5020960435f2db4c       accessed
## 21: 2b5512931b191cfc5020960435f2db4c       accessed
## 22: 2b5512931b191cfc5020960435f2db4c       accessed
## 23: 2f6ae122288cd8b5bd1f8236364506ff         format
## 24: 2f6ae122288cd8b5bd1f8236364506ff           name
## 25: 2f6ae122288cd8b5bd1f8236364506ff          class
## 26: 2f6ae122288cd8b5bd1f8236364506ff           date
## 27: 2f6ae122288cd8b5bd1f8236364506ff        cacheId
## 28: 2f6ae122288cd8b5bd1f8236364506ff       function
## 29: 2f6ae122288cd8b5bd1f8236364506ff    object.size
## 30: 2f6ae122288cd8b5bd1f8236364506ff       accessed
## 31: 2f6ae122288cd8b5bd1f8236364506ff otherFunctions
## 32: 2f6ae122288cd8b5bd1f8236364506ff otherFunctions
## 33: 2f6ae122288cd8b5bd1f8236364506ff otherFunctions
## 34: 2f6ae122288cd8b5bd1f8236364506ff otherFunctions
## 35: 2f6ae122288cd8b5bd1f8236364506ff otherFunctions
## 36: 2f6ae122288cd8b5bd1f8236364506ff otherFunctions
## 37: 2f6ae122288cd8b5bd1f8236364506ff otherFunctions
## 38: 2f6ae122288cd8b5bd1f8236364506ff otherFunctions
## 39: 2f6ae122288cd8b5bd1f8236364506ff      preDigest
## 40: 2f6ae122288cd8b5bd1f8236364506ff      preDigest
## 41: 2f6ae122288cd8b5bd1f8236364506ff      preDigest
## 42: f0d80b70a2e9c21742904505bfc62e50         format
## 43: f0d80b70a2e9c21742904505bfc62e50           name
## 44: f0d80b70a2e9c21742904505bfc62e50          class
## 45: f0d80b70a2e9c21742904505bfc62e50           date
## 46: f0d80b70a2e9c21742904505bfc62e50        cacheId
## 47: f0d80b70a2e9c21742904505bfc62e50     objectName
## 48: f0d80b70a2e9c21742904505bfc62e50       function
## 49: f0d80b70a2e9c21742904505bfc62e50    object.size
## 50: f0d80b70a2e9c21742904505bfc62e50       accessed
## 51: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 52: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 53: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 54: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 55: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 56: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 57: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 58: f0d80b70a2e9c21742904505bfc62e50 otherFunctions
## 59: f0d80b70a2e9c21742904505bfc62e50      preDigest
## 60: f0d80b70a2e9c21742904505bfc62e50      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-08-05 21:53:31
##  2:      2b5512931b191cfc5020960435f2db4c 2018-08-05 21:53:31
##  3:                               numeric 2018-08-05 21:53:31
##  4:                   2018-08-05 21:53:31 2018-08-05 21:53:31
##  5:      f185e4f8cea576a595f7b81be17e63b5 2018-08-05 21:53:31
##  6:                                 rnorm 2018-08-05 21:53:31
##  7:                                  1048 2018-08-05 21:53:31
##  8:                   2018-08-05 21:53:31 2018-08-05 21:53:31
##  9:                   withCallingHandlers 2018-08-05 21:53:31
## 10:                               saveRDS 2018-08-05 21:53:31
## 11:                               do.call 2018-08-05 21:53:31
## 12:                          process_file 2018-08-05 21:53:31
## 13:                         process_group 2018-08-05 21:53:31
## 14:                   process_group.block 2018-08-05 21:53:31
## 15:                            call_block 2018-08-05 21:53:31
## 16:                            block_exec 2018-08-05 21:53:31
## 17:    n:52ad08b1270ae9be9c0a12805d408433 2018-08-05 21:53:31
## 18: mean:2c67b8e97ab3ea8d032fea4318223887 2018-08-05 21:53:31
## 19: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-08-05 21:53:31
## 20:                   2018-08-05 21:53:31 2018-08-05 21:53:31
## 21:                   2018-08-05 21:53:31 2018-08-05 21:53:31
## 22:                   2018-08-05 21:53:31 2018-08-05 21:53:31
## 23:                                   rda 2018-08-05 21:53:31
## 24:      2f6ae122288cd8b5bd1f8236364506ff 2018-08-05 21:53:31
## 25:                               numeric 2018-08-05 21:53:31
## 26:                   2018-08-05 21:53:31 2018-08-05 21:53:31
## 27:      778e2e6b9a967175d1f08811a995269d 2018-08-05 21:53:31
## 28:                   rnorm pipe sequence 2018-08-05 21:53:31
## 29:                                  1048 2018-08-05 21:53:31
## 30:                   2018-08-05 21:53:31 2018-08-05 21:53:31
## 31:                   withCallingHandlers 2018-08-05 21:53:31
## 32:                               saveRDS 2018-08-05 21:53:31
## 33:                               do.call 2018-08-05 21:53:31
## 34:                          process_file 2018-08-05 21:53:31
## 35:                         process_group 2018-08-05 21:53:31
## 36:                   process_group.block 2018-08-05 21:53:31
## 37:                            call_block 2018-08-05 21:53:31
## 38:                            block_exec 2018-08-05 21:53:31
## 39:    n:52ad08b1270ae9be9c0a12805d408433 2018-08-05 21:53:31
## 40: mean:6f529546a7c8cf6edda6081007b3d58d 2018-08-05 21:53:31
## 41: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-08-05 21:53:31
## 42:                                   rda 2018-08-05 21:53:32
## 43:      f0d80b70a2e9c21742904505bfc62e50 2018-08-05 21:53:32
## 44:                               numeric 2018-08-05 21:53:32
## 45:                   2018-08-05 21:53:32 2018-08-05 21:53:32
## 46:      85874f26b2e0c1ef689a7d379d275ebf 2018-08-05 21:53:32
## 47:                                     a 2018-08-05 21:53:32
## 48:                                 rnorm 2018-08-05 21:53:32
## 49:                                   952 2018-08-05 21:53:32
## 50:                   2018-08-05 21:53:32 2018-08-05 21:53:32
## 51:                   withCallingHandlers 2018-08-05 21:53:32
## 52:                               saveRDS 2018-08-05 21:53:32
## 53:                               do.call 2018-08-05 21:53:32
## 54:                          process_file 2018-08-05 21:53:32
## 55:                         process_group 2018-08-05 21:53:32
## 56:                   process_group.block 2018-08-05 21:53:32
## 57:                            call_block 2018-08-05 21:53:32
## 58:                            block_exec 2018-08-05 21:53:32
## 59:    n:969a49ec15bcd4323ff31538af321264 2018-08-05 21:53:32
## 60: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-08-05 21:53:32
##                                  tagValue         createdDate
clearCache(tmpDir)
## Cache size:
##   Total (including Rasters): 762 bytes
##   Selected objects (not including Rasters): 762 bytes

Example 2: using the “accessed” tag

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# access it again, from Cache
ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
##   loading cached result from previous rnorm call, adding to memoised copy
wholeCache <- showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 476 bytes
##   Selected objects (not including Rasters): 476 bytes
# keep only items accessed "recently" (i.e., only objectName:a)
onlyRecentlyAccessed <- showCache(tmpDir, userTags = max(wholeCache[tagKey == "accessed"]$tagValue))
## Cache size:
##   Total (including Rasters): 476 bytes
##   Selected objects (not including Rasters): 476 bytes
# inverse join with 2 data.tables ... using: a[!b]
# i.e., return all of wholeCache that was not recently accessed
toRemove <- unique(wholeCache[!onlyRecentlyAccessed], by = "artifact")$artifact
clearCache(tmpDir, toRemove) # remove ones not recently accessed
## Cache size:
##   Total (including Rasters): 476 bytes
##   Selected objects (not including Rasters): 476 bytes
showCache(tmpDir) # still has more recently accessed
## Cache size:
##   Total (including Rasters): 0 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 3 cols: md5hash,name,createdDate
clearCache(tmpDir)
## Cache size:
##   Total (including Rasters): 0 bytes
##   Selected objects (not including Rasters): 0 bytes

Example 3: using keepCache

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# keep only those cached items from the last 24 hours
oneDay <- 60 * 60 * 24
keepCache(tmpDir, after = Sys.time() - oneDay)
## Cache size:
##   Total (including Rasters): 476 bytes
##   Selected objects (not including Rasters): 476 bytes
##                             artifact         tagKey
##  1: f48322282fff6e5ad26658698f1b685f         format
##  2: f48322282fff6e5ad26658698f1b685f           name
##  3: f48322282fff6e5ad26658698f1b685f          class
##  4: f48322282fff6e5ad26658698f1b685f           date
##  5: f48322282fff6e5ad26658698f1b685f        cacheId
##  6: f48322282fff6e5ad26658698f1b685f     objectName
##  7: f48322282fff6e5ad26658698f1b685f       function
##  8: f48322282fff6e5ad26658698f1b685f    object.size
##  9: f48322282fff6e5ad26658698f1b685f       accessed
## 10: f48322282fff6e5ad26658698f1b685f otherFunctions
## 11: f48322282fff6e5ad26658698f1b685f otherFunctions
## 12: f48322282fff6e5ad26658698f1b685f otherFunctions
## 13: f48322282fff6e5ad26658698f1b685f otherFunctions
## 14: f48322282fff6e5ad26658698f1b685f otherFunctions
## 15: f48322282fff6e5ad26658698f1b685f otherFunctions
## 16: f48322282fff6e5ad26658698f1b685f otherFunctions
## 17: f48322282fff6e5ad26658698f1b685f otherFunctions
## 18: f48322282fff6e5ad26658698f1b685f      preDigest
## 19: f48322282fff6e5ad26658698f1b685f      preDigest
## 20: fd8b8a98cb8ac56cbf0ce07021a85be6         format
## 21: fd8b8a98cb8ac56cbf0ce07021a85be6           name
## 22: fd8b8a98cb8ac56cbf0ce07021a85be6          class
## 23: fd8b8a98cb8ac56cbf0ce07021a85be6           date
## 24: fd8b8a98cb8ac56cbf0ce07021a85be6        cacheId
## 25: fd8b8a98cb8ac56cbf0ce07021a85be6     objectName
## 26: fd8b8a98cb8ac56cbf0ce07021a85be6       function
## 27: fd8b8a98cb8ac56cbf0ce07021a85be6    object.size
## 28: fd8b8a98cb8ac56cbf0ce07021a85be6       accessed
## 29: fd8b8a98cb8ac56cbf0ce07021a85be6 otherFunctions
## 30: fd8b8a98cb8ac56cbf0ce07021a85be6 otherFunctions
## 31: fd8b8a98cb8ac56cbf0ce07021a85be6 otherFunctions
## 32: fd8b8a98cb8ac56cbf0ce07021a85be6 otherFunctions
## 33: fd8b8a98cb8ac56cbf0ce07021a85be6 otherFunctions
## 34: fd8b8a98cb8ac56cbf0ce07021a85be6 otherFunctions
## 35: fd8b8a98cb8ac56cbf0ce07021a85be6 otherFunctions
## 36: fd8b8a98cb8ac56cbf0ce07021a85be6 otherFunctions
## 37: fd8b8a98cb8ac56cbf0ce07021a85be6      preDigest
## 38: fd8b8a98cb8ac56cbf0ce07021a85be6      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-08-05 21:53:32
##  2:      f48322282fff6e5ad26658698f1b685f 2018-08-05 21:53:32
##  3:                               numeric 2018-08-05 21:53:32
##  4:                   2018-08-05 21:53:32 2018-08-05 21:53:32
##  5:      e37bb635c97bc2eeecab63816b881bbc 2018-08-05 21:53:32
##  6:                                     b 2018-08-05 21:53:32
##  7:                                 runif 2018-08-05 21:53:32
##  8:                                   952 2018-08-05 21:53:32
##  9:                   2018-08-05 21:53:32 2018-08-05 21:53:32
## 10:                   withCallingHandlers 2018-08-05 21:53:32
## 11:                               saveRDS 2018-08-05 21:53:32
## 12:                               do.call 2018-08-05 21:53:32
## 13:                          process_file 2018-08-05 21:53:32
## 14:                         process_group 2018-08-05 21:53:32
## 15:                   process_group.block 2018-08-05 21:53:32
## 16:                            call_block 2018-08-05 21:53:32
## 17:                            block_exec 2018-08-05 21:53:32
## 18:    n:969a49ec15bcd4323ff31538af321264 2018-08-05 21:53:32
## 19: .FUN:d2631d24c3b38b89c7bdd4ab7faaaac3 2018-08-05 21:53:32
## 20:                                   rda 2018-08-05 21:53:32
## 21:      fd8b8a98cb8ac56cbf0ce07021a85be6 2018-08-05 21:53:32
## 22:                               numeric 2018-08-05 21:53:32
## 23:                   2018-08-05 21:53:32 2018-08-05 21:53:32
## 24:      85874f26b2e0c1ef689a7d379d275ebf 2018-08-05 21:53:32
## 25:                                     a 2018-08-05 21:53:32
## 26:                                 rnorm 2018-08-05 21:53:32
## 27:                                   952 2018-08-05 21:53:32
## 28:                   2018-08-05 21:53:32 2018-08-05 21:53:32
## 29:                   withCallingHandlers 2018-08-05 21:53:32
## 30:                               saveRDS 2018-08-05 21:53:32
## 31:                               do.call 2018-08-05 21:53:32
## 32:                          process_file 2018-08-05 21:53:32
## 33:                         process_group 2018-08-05 21:53:32
## 34:                   process_group.block 2018-08-05 21:53:32
## 35:                            call_block 2018-08-05 21:53:32
## 36:                            block_exec 2018-08-05 21:53:32
## 37:    n:969a49ec15bcd4323ff31538af321264 2018-08-05 21:53:32
## 38: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-08-05 21:53:32
##                                  tagValue         createdDate
# Keep all Cache items created with an rnorm() call
keepCache(tmpDir, userTags = "rnorm")
## Cache size:
##   Total (including Rasters): 476 bytes
##   Selected objects (not including Rasters): 238 bytes
##                             artifact         tagKey
##  1: fd8b8a98cb8ac56cbf0ce07021a85be6         format
##  2: fd8b8a98cb8ac56cbf0ce07021a85be6           name
##  3: fd8b8a98cb8ac56cbf0ce07021a85be6          class
##  4: fd8b8a98cb8ac56cbf0ce07021a85be6           date
##  5: fd8b8a98cb8ac56cbf0ce07021a85be6        cacheId
##  6: fd8b8a98cb8ac56cbf0ce07021a85be6     objectName
##  7: fd8b8a98cb8ac56cbf0ce07021a85be6       function
##  8: fd8b8a98cb8ac56cbf0ce07021a85be6    object.size
##  9: fd8b8a98cb8ac56cbf0ce07021a85be6       accessed
## 10: fd8b8a98cb8ac56cbf0ce07021a85be6 otherFunctions
## 11: fd8b8a98cb8ac56cbf0ce07021a85be6 otherFunctions
## 12: fd8b8a98cb8ac56cbf0ce07021a85be6 otherFunctions
## 13: fd8b8a98cb8ac56cbf0ce07021a85be6 otherFunctions
## 14: fd8b8a98cb8ac56cbf0ce07021a85be6 otherFunctions
## 15: fd8b8a98cb8ac56cbf0ce07021a85be6 otherFunctions
## 16: fd8b8a98cb8ac56cbf0ce07021a85be6 otherFunctions
## 17: fd8b8a98cb8ac56cbf0ce07021a85be6 otherFunctions
## 18: fd8b8a98cb8ac56cbf0ce07021a85be6      preDigest
## 19: fd8b8a98cb8ac56cbf0ce07021a85be6      preDigest
##                                  tagValue         createdDate
##  1:                                   rda 2018-08-05 21:53:32
##  2:      fd8b8a98cb8ac56cbf0ce07021a85be6 2018-08-05 21:53:32
##  3:                               numeric 2018-08-05 21:53:32
##  4:                   2018-08-05 21:53:32 2018-08-05 21:53:32
##  5:      85874f26b2e0c1ef689a7d379d275ebf 2018-08-05 21:53:32
##  6:                                     a 2018-08-05 21:53:32
##  7:                                 rnorm 2018-08-05 21:53:32
##  8:                                   952 2018-08-05 21:53:32
##  9:                   2018-08-05 21:53:32 2018-08-05 21:53:32
## 10:                   withCallingHandlers 2018-08-05 21:53:32
## 11:                               saveRDS 2018-08-05 21:53:32
## 12:                               do.call 2018-08-05 21:53:32
## 13:                          process_file 2018-08-05 21:53:32
## 14:                         process_group 2018-08-05 21:53:32
## 15:                   process_group.block 2018-08-05 21:53:32
## 16:                            call_block 2018-08-05 21:53:32
## 17:                            block_exec 2018-08-05 21:53:32
## 18:    n:969a49ec15bcd4323ff31538af321264 2018-08-05 21:53:32
## 19: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-08-05 21:53:32
# Remove all Cache items that happened within a rnorm() call
clearCache(tmpDir, userTags = "rnorm")
## Cache size:
##   Total (including Rasters): 238 bytes
##   Selected objects (not including Rasters): 238 bytes
showCache(tmpDir) ## empty
## Cache size:
##   Total (including Rasters): 0 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 3 cols: md5hash,name,createdDate

Example 4: searching for multiple objects in the cache

# default userTags is "and" matching; for "or" matching use |
ranNumsA <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# show all objects (runif and rnorm in this case)
showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 476 bytes
##   Selected objects (not including Rasters): 476 bytes
##                             artifact         tagKey
##  1: 948f846fcc5fbec96ee003aef4e083e6         format
##  2: 948f846fcc5fbec96ee003aef4e083e6           name
##  3: 948f846fcc5fbec96ee003aef4e083e6          class
##  4: 948f846fcc5fbec96ee003aef4e083e6           date
##  5: 948f846fcc5fbec96ee003aef4e083e6        cacheId
##  6: 948f846fcc5fbec96ee003aef4e083e6     objectName
##  7: 948f846fcc5fbec96ee003aef4e083e6       function
##  8: 948f846fcc5fbec96ee003aef4e083e6    object.size
##  9: 948f846fcc5fbec96ee003aef4e083e6       accessed
## 10: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 11: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 12: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 13: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 14: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 15: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 16: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 17: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 18: 948f846fcc5fbec96ee003aef4e083e6      preDigest
## 19: 948f846fcc5fbec96ee003aef4e083e6      preDigest
## 20: f64d114e61042e7541ccc1e68f250de3         format
## 21: f64d114e61042e7541ccc1e68f250de3           name
## 22: f64d114e61042e7541ccc1e68f250de3          class
## 23: f64d114e61042e7541ccc1e68f250de3           date
## 24: f64d114e61042e7541ccc1e68f250de3        cacheId
## 25: f64d114e61042e7541ccc1e68f250de3     objectName
## 26: f64d114e61042e7541ccc1e68f250de3       function
## 27: f64d114e61042e7541ccc1e68f250de3    object.size
## 28: f64d114e61042e7541ccc1e68f250de3       accessed
## 29: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 30: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 31: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 32: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 33: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 34: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 35: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 36: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 37: f64d114e61042e7541ccc1e68f250de3      preDigest
## 38: f64d114e61042e7541ccc1e68f250de3      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-08-05 21:53:33
##  2:      948f846fcc5fbec96ee003aef4e083e6 2018-08-05 21:53:33
##  3:                               numeric 2018-08-05 21:53:33
##  4:                   2018-08-05 21:53:33 2018-08-05 21:53:33
##  5:      85874f26b2e0c1ef689a7d379d275ebf 2018-08-05 21:53:33
##  6:                                     b 2018-08-05 21:53:33
##  7:                                 rnorm 2018-08-05 21:53:33
##  8:                                   952 2018-08-05 21:53:33
##  9:                   2018-08-05 21:53:33 2018-08-05 21:53:33
## 10:                   withCallingHandlers 2018-08-05 21:53:33
## 11:                               saveRDS 2018-08-05 21:53:33
## 12:                               do.call 2018-08-05 21:53:33
## 13:                          process_file 2018-08-05 21:53:33
## 14:                         process_group 2018-08-05 21:53:33
## 15:                   process_group.block 2018-08-05 21:53:33
## 16:                            call_block 2018-08-05 21:53:33
## 17:                            block_exec 2018-08-05 21:53:33
## 18:    n:969a49ec15bcd4323ff31538af321264 2018-08-05 21:53:33
## 19: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-08-05 21:53:33
## 20:                                   rda 2018-08-05 21:53:33
## 21:      f64d114e61042e7541ccc1e68f250de3 2018-08-05 21:53:33
## 22:                               numeric 2018-08-05 21:53:33
## 23:                   2018-08-05 21:53:33 2018-08-05 21:53:33
## 24:      e37bb635c97bc2eeecab63816b881bbc 2018-08-05 21:53:33
## 25:                                     a 2018-08-05 21:53:33
## 26:                                 runif 2018-08-05 21:53:33
## 27:                                   952 2018-08-05 21:53:33
## 28:                   2018-08-05 21:53:33 2018-08-05 21:53:33
## 29:                   withCallingHandlers 2018-08-05 21:53:33
## 30:                               saveRDS 2018-08-05 21:53:33
## 31:                               do.call 2018-08-05 21:53:33
## 32:                          process_file 2018-08-05 21:53:33
## 33:                         process_group 2018-08-05 21:53:33
## 34:                   process_group.block 2018-08-05 21:53:33
## 35:                            call_block 2018-08-05 21:53:33
## 36:                            block_exec 2018-08-05 21:53:33
## 37:    n:969a49ec15bcd4323ff31538af321264 2018-08-05 21:53:33
## 38: .FUN:d2631d24c3b38b89c7bdd4ab7faaaac3 2018-08-05 21:53:33
##                                  tagValue         createdDate
# show objects that are both runif and rnorm
# (i.e., none in this case, because objecs are either or, not both)
showCache(tmpDir, userTags = c("runif", "rnorm")) ## empty
## Cache size:
##   Total (including Rasters): 476 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 4 cols: artifact,tagKey,tagValue,createdDate
# show objects that are either runif or rnorm ("or" search)
showCache(tmpDir, userTags = "runif|rnorm")
## Cache size:
##   Total (including Rasters): 476 bytes
##   Selected objects (not including Rasters): 476 bytes
##                             artifact         tagKey
##  1: 948f846fcc5fbec96ee003aef4e083e6         format
##  2: 948f846fcc5fbec96ee003aef4e083e6           name
##  3: 948f846fcc5fbec96ee003aef4e083e6          class
##  4: 948f846fcc5fbec96ee003aef4e083e6           date
##  5: 948f846fcc5fbec96ee003aef4e083e6        cacheId
##  6: 948f846fcc5fbec96ee003aef4e083e6     objectName
##  7: 948f846fcc5fbec96ee003aef4e083e6       function
##  8: 948f846fcc5fbec96ee003aef4e083e6    object.size
##  9: 948f846fcc5fbec96ee003aef4e083e6       accessed
## 10: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 11: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 12: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 13: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 14: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 15: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 16: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 17: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 18: 948f846fcc5fbec96ee003aef4e083e6      preDigest
## 19: 948f846fcc5fbec96ee003aef4e083e6      preDigest
## 20: f64d114e61042e7541ccc1e68f250de3         format
## 21: f64d114e61042e7541ccc1e68f250de3           name
## 22: f64d114e61042e7541ccc1e68f250de3          class
## 23: f64d114e61042e7541ccc1e68f250de3           date
## 24: f64d114e61042e7541ccc1e68f250de3        cacheId
## 25: f64d114e61042e7541ccc1e68f250de3     objectName
## 26: f64d114e61042e7541ccc1e68f250de3       function
## 27: f64d114e61042e7541ccc1e68f250de3    object.size
## 28: f64d114e61042e7541ccc1e68f250de3       accessed
## 29: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 30: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 31: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 32: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 33: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 34: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 35: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 36: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 37: f64d114e61042e7541ccc1e68f250de3      preDigest
## 38: f64d114e61042e7541ccc1e68f250de3      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-08-05 21:53:33
##  2:      948f846fcc5fbec96ee003aef4e083e6 2018-08-05 21:53:33
##  3:                               numeric 2018-08-05 21:53:33
##  4:                   2018-08-05 21:53:33 2018-08-05 21:53:33
##  5:      85874f26b2e0c1ef689a7d379d275ebf 2018-08-05 21:53:33
##  6:                                     b 2018-08-05 21:53:33
##  7:                                 rnorm 2018-08-05 21:53:33
##  8:                                   952 2018-08-05 21:53:33
##  9:                   2018-08-05 21:53:33 2018-08-05 21:53:33
## 10:                   withCallingHandlers 2018-08-05 21:53:33
## 11:                               saveRDS 2018-08-05 21:53:33
## 12:                               do.call 2018-08-05 21:53:33
## 13:                          process_file 2018-08-05 21:53:33
## 14:                         process_group 2018-08-05 21:53:33
## 15:                   process_group.block 2018-08-05 21:53:33
## 16:                            call_block 2018-08-05 21:53:33
## 17:                            block_exec 2018-08-05 21:53:33
## 18:    n:969a49ec15bcd4323ff31538af321264 2018-08-05 21:53:33
## 19: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-08-05 21:53:33
## 20:                                   rda 2018-08-05 21:53:33
## 21:      f64d114e61042e7541ccc1e68f250de3 2018-08-05 21:53:33
## 22:                               numeric 2018-08-05 21:53:33
## 23:                   2018-08-05 21:53:33 2018-08-05 21:53:33
## 24:      e37bb635c97bc2eeecab63816b881bbc 2018-08-05 21:53:33
## 25:                                     a 2018-08-05 21:53:33
## 26:                                 runif 2018-08-05 21:53:33
## 27:                                   952 2018-08-05 21:53:33
## 28:                   2018-08-05 21:53:33 2018-08-05 21:53:33
## 29:                   withCallingHandlers 2018-08-05 21:53:33
## 30:                               saveRDS 2018-08-05 21:53:33
## 31:                               do.call 2018-08-05 21:53:33
## 32:                          process_file 2018-08-05 21:53:33
## 33:                         process_group 2018-08-05 21:53:33
## 34:                   process_group.block 2018-08-05 21:53:33
## 35:                            call_block 2018-08-05 21:53:33
## 36:                            block_exec 2018-08-05 21:53:33
## 37:    n:969a49ec15bcd4323ff31538af321264 2018-08-05 21:53:33
## 38: .FUN:d2631d24c3b38b89c7bdd4ab7faaaac3 2018-08-05 21:53:33
##                                  tagValue         createdDate
# keep only objects that are either runif or rnorm ("or" search)
keepCache(tmpDir, userTags = "runif|rnorm")
## Cache size:
##   Total (including Rasters): 476 bytes
##   Selected objects (not including Rasters): 476 bytes
##                             artifact         tagKey
##  1: 948f846fcc5fbec96ee003aef4e083e6         format
##  2: 948f846fcc5fbec96ee003aef4e083e6           name
##  3: 948f846fcc5fbec96ee003aef4e083e6          class
##  4: 948f846fcc5fbec96ee003aef4e083e6           date
##  5: 948f846fcc5fbec96ee003aef4e083e6        cacheId
##  6: 948f846fcc5fbec96ee003aef4e083e6     objectName
##  7: 948f846fcc5fbec96ee003aef4e083e6       function
##  8: 948f846fcc5fbec96ee003aef4e083e6    object.size
##  9: 948f846fcc5fbec96ee003aef4e083e6       accessed
## 10: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 11: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 12: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 13: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 14: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 15: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 16: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 17: 948f846fcc5fbec96ee003aef4e083e6 otherFunctions
## 18: 948f846fcc5fbec96ee003aef4e083e6      preDigest
## 19: 948f846fcc5fbec96ee003aef4e083e6      preDigest
## 20: f64d114e61042e7541ccc1e68f250de3         format
## 21: f64d114e61042e7541ccc1e68f250de3           name
## 22: f64d114e61042e7541ccc1e68f250de3          class
## 23: f64d114e61042e7541ccc1e68f250de3           date
## 24: f64d114e61042e7541ccc1e68f250de3        cacheId
## 25: f64d114e61042e7541ccc1e68f250de3     objectName
## 26: f64d114e61042e7541ccc1e68f250de3       function
## 27: f64d114e61042e7541ccc1e68f250de3    object.size
## 28: f64d114e61042e7541ccc1e68f250de3       accessed
## 29: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 30: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 31: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 32: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 33: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 34: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 35: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 36: f64d114e61042e7541ccc1e68f250de3 otherFunctions
## 37: f64d114e61042e7541ccc1e68f250de3      preDigest
## 38: f64d114e61042e7541ccc1e68f250de3      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-08-05 21:53:33
##  2:      948f846fcc5fbec96ee003aef4e083e6 2018-08-05 21:53:33
##  3:                               numeric 2018-08-05 21:53:33
##  4:                   2018-08-05 21:53:33 2018-08-05 21:53:33
##  5:      85874f26b2e0c1ef689a7d379d275ebf 2018-08-05 21:53:33
##  6:                                     b 2018-08-05 21:53:33
##  7:                                 rnorm 2018-08-05 21:53:33
##  8:                                   952 2018-08-05 21:53:33
##  9:                   2018-08-05 21:53:33 2018-08-05 21:53:33
## 10:                   withCallingHandlers 2018-08-05 21:53:33
## 11:                               saveRDS 2018-08-05 21:53:33
## 12:                               do.call 2018-08-05 21:53:33
## 13:                          process_file 2018-08-05 21:53:33
## 14:                         process_group 2018-08-05 21:53:33
## 15:                   process_group.block 2018-08-05 21:53:33
## 16:                            call_block 2018-08-05 21:53:33
## 17:                            block_exec 2018-08-05 21:53:33
## 18:    n:969a49ec15bcd4323ff31538af321264 2018-08-05 21:53:33
## 19: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-08-05 21:53:33
## 20:                                   rda 2018-08-05 21:53:33
## 21:      f64d114e61042e7541ccc1e68f250de3 2018-08-05 21:53:33
## 22:                               numeric 2018-08-05 21:53:33
## 23:                   2018-08-05 21:53:33 2018-08-05 21:53:33
## 24:      e37bb635c97bc2eeecab63816b881bbc 2018-08-05 21:53:33
## 25:                                     a 2018-08-05 21:53:33
## 26:                                 runif 2018-08-05 21:53:33
## 27:                                   952 2018-08-05 21:53:33
## 28:                   2018-08-05 21:53:33 2018-08-05 21:53:33
## 29:                   withCallingHandlers 2018-08-05 21:53:33
## 30:                               saveRDS 2018-08-05 21:53:33
## 31:                               do.call 2018-08-05 21:53:33
## 32:                          process_file 2018-08-05 21:53:33
## 33:                         process_group 2018-08-05 21:53:33
## 34:                   process_group.block 2018-08-05 21:53:33
## 35:                            call_block 2018-08-05 21:53:33
## 36:                            block_exec 2018-08-05 21:53:33
## 37:    n:969a49ec15bcd4323ff31538af321264 2018-08-05 21:53:33
## 38: .FUN:d2631d24c3b38b89c7bdd4ab7faaaac3 2018-08-05 21:53:33
##                                  tagValue         createdDate
clearCache(tmpDir)
## Cache size:
##   Total (including Rasters): 476 bytes
##   Selected objects (not including Rasters): 476 bytes

Example 5: using caching to speed up rerunning expensive computations

ras <- raster(extent(0, 5, 0, 5), res = 1,
              vals = sample(1:5, replace = TRUE, size = 25),
              crs = "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84")

# A slow operation, like GIS operation
notCached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  projectRaster(ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)

cached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  # using quote works also
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)

# second time is much faster
reRun <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)
##   loading cached result from previous projectRaster call, adding to memoised copy
# recovered cached version is same as non-cached version
all.equal(notCached, reRun) ## TRUE
## [1] TRUE

Nested Caching

Nested caching, which is when Caching of a function occurs inside an outer function, which is itself cached. This is a critical element to working within a reproducible work flow. It is not enough during development to cache flat code chunks, as there will be many levels of “slow” functions. Ideally, at all points in a development cycle, it should be possible to get to any line of code starting from the very initial steps, running through everything up to that point, in less that 1 second. If the workflow can be kept very fast like this, then there is a guarantee that it will work at any point.

##########################
## Nested Caching
# Make 2 functions
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean)
}
outer <- function(n) {
  Cache(inner, 0.1, cacheRepo = tmpdir2)
}

# make 2 different cache paths
tmpdir1 <- file.path(tempdir(), "first")
tmpdir2 <- file.path(tempdir(), "second")

# Run the Cache ... notOlderThan propagates to all 3 Cache calls,
#   but cacheRepo is tmpdir1 in top level Cache and all nested
#   Cache calls, unless individually overridden ... here inner
#   uses tmpdir2 repository
Cache(outer, n = 2, cacheRepo = tmpdir1, notOlderThan = Sys.time())
## [1]  1.2754785 -0.1995874 -1.7834047
## attr(,"tags")
## [1] "cacheId:e09bf93970f9e94d5c639cfa8ca722f0"
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"call")
## [1] ""
showCache(tmpdir1) # 2 function calls
## Cache size:
##   Total (including Rasters): 476 bytes
##   Selected objects (not including Rasters): 476 bytes
##                             artifact         tagKey
##  1: 13ced4d578c3c1b9606bc874bd056e2f         format
##  2: 13ced4d578c3c1b9606bc874bd056e2f           name
##  3: 13ced4d578c3c1b9606bc874bd056e2f          class
##  4: 13ced4d578c3c1b9606bc874bd056e2f           date
##  5: 13ced4d578c3c1b9606bc874bd056e2f        cacheId
##  6: 13ced4d578c3c1b9606bc874bd056e2f       function
##  7: 13ced4d578c3c1b9606bc874bd056e2f    object.size
##  8: 13ced4d578c3c1b9606bc874bd056e2f       accessed
##  9: 13ced4d578c3c1b9606bc874bd056e2f otherFunctions
## 10: 13ced4d578c3c1b9606bc874bd056e2f otherFunctions
## 11: 13ced4d578c3c1b9606bc874bd056e2f otherFunctions
## 12: 13ced4d578c3c1b9606bc874bd056e2f otherFunctions
## 13: 13ced4d578c3c1b9606bc874bd056e2f otherFunctions
## 14: 13ced4d578c3c1b9606bc874bd056e2f otherFunctions
## 15: 13ced4d578c3c1b9606bc874bd056e2f otherFunctions
## 16: 13ced4d578c3c1b9606bc874bd056e2f otherFunctions
## 17: 13ced4d578c3c1b9606bc874bd056e2f      preDigest
## 18: 13ced4d578c3c1b9606bc874bd056e2f      preDigest
## 19: 13ced4d578c3c1b9606bc874bd056e2f      preDigest
## 20: e0d1ae0848e904640ae7be7b51360377         format
## 21: e0d1ae0848e904640ae7be7b51360377           name
## 22: e0d1ae0848e904640ae7be7b51360377          class
## 23: e0d1ae0848e904640ae7be7b51360377           date
## 24: e0d1ae0848e904640ae7be7b51360377        cacheId
## 25: e0d1ae0848e904640ae7be7b51360377       function
## 26: e0d1ae0848e904640ae7be7b51360377    object.size
## 27: e0d1ae0848e904640ae7be7b51360377       accessed
## 28: e0d1ae0848e904640ae7be7b51360377 otherFunctions
## 29: e0d1ae0848e904640ae7be7b51360377 otherFunctions
## 30: e0d1ae0848e904640ae7be7b51360377 otherFunctions
## 31: e0d1ae0848e904640ae7be7b51360377 otherFunctions
## 32: e0d1ae0848e904640ae7be7b51360377 otherFunctions
## 33: e0d1ae0848e904640ae7be7b51360377 otherFunctions
## 34: e0d1ae0848e904640ae7be7b51360377 otherFunctions
## 35: e0d1ae0848e904640ae7be7b51360377 otherFunctions
## 36: e0d1ae0848e904640ae7be7b51360377      preDigest
## 37: e0d1ae0848e904640ae7be7b51360377      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-08-05 21:53:33
##  2:      13ced4d578c3c1b9606bc874bd056e2f 2018-08-05 21:53:33
##  3:                               numeric 2018-08-05 21:53:33
##  4:                   2018-08-05 21:53:33 2018-08-05 21:53:33
##  5:      cec73d63ad3864af8bcd7efc5fae864d 2018-08-05 21:53:33
##  6:                                 rnorm 2018-08-05 21:53:33
##  7:                                   952 2018-08-05 21:53:33
##  8:                   2018-08-05 21:53:33 2018-08-05 21:53:33
##  9:                   withCallingHandlers 2018-08-05 21:53:33
## 10:                               saveRDS 2018-08-05 21:53:33
## 11:                               do.call 2018-08-05 21:53:33
## 12:                          process_file 2018-08-05 21:53:33
## 13:                         process_group 2018-08-05 21:53:33
## 14:                   process_group.block 2018-08-05 21:53:33
## 15:                            call_block 2018-08-05 21:53:33
## 16:                            block_exec 2018-08-05 21:53:33
## 17:    n:4ae3e6b6364de42fdc243469d73448cc 2018-08-05 21:53:33
## 18: mean:c28b87a0be6a99966bdaa5e556974b43 2018-08-05 21:53:33
## 19: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-08-05 21:53:33
## 20:                                   rda 2018-08-05 21:53:33
## 21:      e0d1ae0848e904640ae7be7b51360377 2018-08-05 21:53:33
## 22:                               numeric 2018-08-05 21:53:33
## 23:                   2018-08-05 21:53:33 2018-08-05 21:53:33
## 24:      e09bf93970f9e94d5c639cfa8ca722f0 2018-08-05 21:53:33
## 25:                                 outer 2018-08-05 21:53:33
## 26:                                   952 2018-08-05 21:53:33
## 27:                   2018-08-05 21:53:33 2018-08-05 21:53:33
## 28:                   withCallingHandlers 2018-08-05 21:53:33
## 29:                               saveRDS 2018-08-05 21:53:33
## 30:                               do.call 2018-08-05 21:53:33
## 31:                          process_file 2018-08-05 21:53:33
## 32:                         process_group 2018-08-05 21:53:33
## 33:                   process_group.block 2018-08-05 21:53:33
## 34:                            call_block 2018-08-05 21:53:33
## 35:                            block_exec 2018-08-05 21:53:33
## 36:    n:8128a6180a705341ab7c05cfa945edfb 2018-08-05 21:53:33
## 37: .FUN:b5f6bcbdd9f23e39c2c5d4020e73a6ff 2018-08-05 21:53:33
##                                  tagValue         createdDate
showCache(tmpdir2) # 1 function call
## Cache size:
##   Total (including Rasters): 238 bytes
##   Selected objects (not including Rasters): 238 bytes
##                             artifact         tagKey
##  1: 6c69a0fb828f88d0266d02ae2d106442         format
##  2: 6c69a0fb828f88d0266d02ae2d106442           name
##  3: 6c69a0fb828f88d0266d02ae2d106442          class
##  4: 6c69a0fb828f88d0266d02ae2d106442           date
##  5: 6c69a0fb828f88d0266d02ae2d106442        cacheId
##  6: 6c69a0fb828f88d0266d02ae2d106442       function
##  7: 6c69a0fb828f88d0266d02ae2d106442    object.size
##  8: 6c69a0fb828f88d0266d02ae2d106442       accessed
##  9: 6c69a0fb828f88d0266d02ae2d106442 otherFunctions
## 10: 6c69a0fb828f88d0266d02ae2d106442 otherFunctions
## 11: 6c69a0fb828f88d0266d02ae2d106442 otherFunctions
## 12: 6c69a0fb828f88d0266d02ae2d106442 otherFunctions
## 13: 6c69a0fb828f88d0266d02ae2d106442 otherFunctions
## 14: 6c69a0fb828f88d0266d02ae2d106442 otherFunctions
## 15: 6c69a0fb828f88d0266d02ae2d106442 otherFunctions
## 16: 6c69a0fb828f88d0266d02ae2d106442 otherFunctions
## 17: 6c69a0fb828f88d0266d02ae2d106442      preDigest
## 18: 6c69a0fb828f88d0266d02ae2d106442      preDigest
##                                  tagValue         createdDate
##  1:                                   rda 2018-08-05 21:53:33
##  2:      6c69a0fb828f88d0266d02ae2d106442 2018-08-05 21:53:33
##  3:                               numeric 2018-08-05 21:53:33
##  4:                   2018-08-05 21:53:33 2018-08-05 21:53:33
##  5:      19b808ac6871e0184e63c421a116cb61 2018-08-05 21:53:33
##  6:                                 inner 2018-08-05 21:53:33
##  7:                                   952 2018-08-05 21:53:33
##  8:                   2018-08-05 21:53:33 2018-08-05 21:53:33
##  9:                   withCallingHandlers 2018-08-05 21:53:33
## 10:                               saveRDS 2018-08-05 21:53:33
## 11:                               do.call 2018-08-05 21:53:33
## 12:                          process_file 2018-08-05 21:53:33
## 13:                         process_group 2018-08-05 21:53:33
## 14:                   process_group.block 2018-08-05 21:53:33
## 15:                            call_block 2018-08-05 21:53:33
## 16:                            block_exec 2018-08-05 21:53:33
## 17: mean:c28b87a0be6a99966bdaa5e556974b43 2018-08-05 21:53:33
## 18: .FUN:56a1302d7ef43383766d7af6ca072c4e 2018-08-05 21:53:33
# userTags get appended
# all items have the outer tag propagate, plus inner ones only have inner ones
clearCache(tmpdir1)
## Cache size:
##   Total (including Rasters): 476 bytes
##   Selected objects (not including Rasters): 476 bytes
outerTag <- "outerTag"
innerTag <- "innerTag"
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean, notOlderThan = Sys.time() - 1e5, userTags = innerTag)
}
outer <- function(n) {
  Cache(inner, 0.1)
}
aa <- Cache(outer, n = 2, cacheRepo = tmpdir1, userTags = outerTag)
showCache(tmpdir1) # rnorm function has outerTag and innerTag, inner and outer only have outerTag
## Cache size:
##   Total (including Rasters): 714 bytes
##   Selected objects (not including Rasters): 714 bytes
##                             artifact         tagKey
##  1: 64dca680207ce1ee1ed3b4b4bde7fc2b         format
##  2: 64dca680207ce1ee1ed3b4b4bde7fc2b           name
##  3: 64dca680207ce1ee1ed3b4b4bde7fc2b          class
##  4: 64dca680207ce1ee1ed3b4b4bde7fc2b           date
##  5: 64dca680207ce1ee1ed3b4b4bde7fc2b        cacheId
##  6: 64dca680207ce1ee1ed3b4b4bde7fc2b       outerTag
##  7: 64dca680207ce1ee1ed3b4b4bde7fc2b       function
##  8: 64dca680207ce1ee1ed3b4b4bde7fc2b    object.size
##  9: 64dca680207ce1ee1ed3b4b4bde7fc2b       accessed
## 10: 64dca680207ce1ee1ed3b4b4bde7fc2b otherFunctions
## 11: 64dca680207ce1ee1ed3b4b4bde7fc2b otherFunctions
## 12: 64dca680207ce1ee1ed3b4b4bde7fc2b otherFunctions
## 13: 64dca680207ce1ee1ed3b4b4bde7fc2b otherFunctions
## 14: 64dca680207ce1ee1ed3b4b4bde7fc2b otherFunctions
## 15: 64dca680207ce1ee1ed3b4b4bde7fc2b otherFunctions
## 16: 64dca680207ce1ee1ed3b4b4bde7fc2b otherFunctions
## 17: 64dca680207ce1ee1ed3b4b4bde7fc2b otherFunctions
## 18: 64dca680207ce1ee1ed3b4b4bde7fc2b      preDigest
## 19: 64dca680207ce1ee1ed3b4b4bde7fc2b      preDigest
## 20: 99318d247270c672c29ef6f89ab270c3         format
## 21: 99318d247270c672c29ef6f89ab270c3           name
## 22: 99318d247270c672c29ef6f89ab270c3          class
## 23: 99318d247270c672c29ef6f89ab270c3           date
## 24: 99318d247270c672c29ef6f89ab270c3        cacheId
## 25: 99318d247270c672c29ef6f89ab270c3       innerTag
## 26: 99318d247270c672c29ef6f89ab270c3       outerTag
## 27: 99318d247270c672c29ef6f89ab270c3       function
## 28: 99318d247270c672c29ef6f89ab270c3    object.size
## 29: 99318d247270c672c29ef6f89ab270c3       accessed
## 30: 99318d247270c672c29ef6f89ab270c3 otherFunctions
## 31: 99318d247270c672c29ef6f89ab270c3 otherFunctions
## 32: 99318d247270c672c29ef6f89ab270c3 otherFunctions
## 33: 99318d247270c672c29ef6f89ab270c3 otherFunctions
## 34: 99318d247270c672c29ef6f89ab270c3 otherFunctions
## 35: 99318d247270c672c29ef6f89ab270c3 otherFunctions
## 36: 99318d247270c672c29ef6f89ab270c3 otherFunctions
## 37: 99318d247270c672c29ef6f89ab270c3 otherFunctions
## 38: 99318d247270c672c29ef6f89ab270c3      preDigest
## 39: 99318d247270c672c29ef6f89ab270c3      preDigest
## 40: 99318d247270c672c29ef6f89ab270c3      preDigest
## 41: c12c1649cf0d2b5addb7a08369d518d6         format
## 42: c12c1649cf0d2b5addb7a08369d518d6           name
## 43: c12c1649cf0d2b5addb7a08369d518d6          class
## 44: c12c1649cf0d2b5addb7a08369d518d6           date
## 45: c12c1649cf0d2b5addb7a08369d518d6        cacheId
## 46: c12c1649cf0d2b5addb7a08369d518d6       outerTag
## 47: c12c1649cf0d2b5addb7a08369d518d6       function
## 48: c12c1649cf0d2b5addb7a08369d518d6    object.size
## 49: c12c1649cf0d2b5addb7a08369d518d6       accessed
## 50: c12c1649cf0d2b5addb7a08369d518d6 otherFunctions
## 51: c12c1649cf0d2b5addb7a08369d518d6 otherFunctions
## 52: c12c1649cf0d2b5addb7a08369d518d6 otherFunctions
## 53: c12c1649cf0d2b5addb7a08369d518d6 otherFunctions
## 54: c12c1649cf0d2b5addb7a08369d518d6 otherFunctions
## 55: c12c1649cf0d2b5addb7a08369d518d6 otherFunctions
## 56: c12c1649cf0d2b5addb7a08369d518d6 otherFunctions
## 57: c12c1649cf0d2b5addb7a08369d518d6 otherFunctions
## 58: c12c1649cf0d2b5addb7a08369d518d6      preDigest
## 59: c12c1649cf0d2b5addb7a08369d518d6      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-08-05 21:53:33
##  2:      64dca680207ce1ee1ed3b4b4bde7fc2b 2018-08-05 21:53:33
##  3:                               numeric 2018-08-05 21:53:33
##  4:                   2018-08-05 21:53:33 2018-08-05 21:53:33
##  5:      994d1330fbd961f795ab0dc508271963 2018-08-05 21:53:33
##  6:                              outerTag 2018-08-05 21:53:33
##  7:                                 inner 2018-08-05 21:53:33
##  8:                                   952 2018-08-05 21:53:33
##  9:                   2018-08-05 21:53:33 2018-08-05 21:53:33
## 10:                   withCallingHandlers 2018-08-05 21:53:34
## 11:                               saveRDS 2018-08-05 21:53:34
## 12:                               do.call 2018-08-05 21:53:34
## 13:                          process_file 2018-08-05 21:53:34
## 14:                         process_group 2018-08-05 21:53:34
## 15:                   process_group.block 2018-08-05 21:53:34
## 16:                            call_block 2018-08-05 21:53:34
## 17:                            block_exec 2018-08-05 21:53:34
## 18: mean:c28b87a0be6a99966bdaa5e556974b43 2018-08-05 21:53:34
## 19: .FUN:b910401646b09073940de757678db03d 2018-08-05 21:53:34
## 20:                                   rda 2018-08-05 21:53:33
## 21:      99318d247270c672c29ef6f89ab270c3 2018-08-05 21:53:33
## 22:                               numeric 2018-08-05 21:53:33
## 23:                   2018-08-05 21:53:33 2018-08-05 21:53:33
## 24:      cec73d63ad3864af8bcd7efc5fae864d 2018-08-05 21:53:33
## 25:                              innerTag 2018-08-05 21:53:33
## 26:                              outerTag 2018-08-05 21:53:33
## 27:                                 rnorm 2018-08-05 21:53:33
## 28:                                   952 2018-08-05 21:53:33
## 29:                   2018-08-05 21:53:33 2018-08-05 21:53:33
## 30:                   withCallingHandlers 2018-08-05 21:53:33
## 31:                               saveRDS 2018-08-05 21:53:33
## 32:                               do.call 2018-08-05 21:53:33
## 33:                          process_file 2018-08-05 21:53:33
## 34:                         process_group 2018-08-05 21:53:33
## 35:                   process_group.block 2018-08-05 21:53:33
## 36:                            call_block 2018-08-05 21:53:33
## 37:                            block_exec 2018-08-05 21:53:33
## 38:    n:4ae3e6b6364de42fdc243469d73448cc 2018-08-05 21:53:33
## 39: mean:c28b87a0be6a99966bdaa5e556974b43 2018-08-05 21:53:33
## 40: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-08-05 21:53:33
## 41:                                   rda 2018-08-05 21:53:34
## 42:      c12c1649cf0d2b5addb7a08369d518d6 2018-08-05 21:53:34
## 43:                               numeric 2018-08-05 21:53:34
## 44:                   2018-08-05 21:53:34 2018-08-05 21:53:34
## 45:      44f57deb36c53cd9c395e04c51fea77a 2018-08-05 21:53:34
## 46:                              outerTag 2018-08-05 21:53:34
## 47:                                 outer 2018-08-05 21:53:34
## 48:                                   952 2018-08-05 21:53:34
## 49:                   2018-08-05 21:53:34 2018-08-05 21:53:34
## 50:                   withCallingHandlers 2018-08-05 21:53:34
## 51:                               saveRDS 2018-08-05 21:53:34
## 52:                               do.call 2018-08-05 21:53:34
## 53:                          process_file 2018-08-05 21:53:34
## 54:                         process_group 2018-08-05 21:53:34
## 55:                   process_group.block 2018-08-05 21:53:34
## 56:                            call_block 2018-08-05 21:53:34
## 57:                            block_exec 2018-08-05 21:53:34
## 58:    n:8128a6180a705341ab7c05cfa945edfb 2018-08-05 21:53:34
## 59: .FUN:62302feda89e19149a56ca40fde725e1 2018-08-05 21:53:34
##                                  tagValue         createdDate

cacheId

Sometimes, it is not absolutely desirable to maintain the work flow intact because changes that are irrelevant to the analysis, such as changing messages sent to a user, may be changed, without a desire to rerun functions. The cacheId argument is for this. Once a piece of code is run, then the cacheId can be manually extracted (it is reported at the end of a Cache call) and manually placed in the code, passed in as, say, cacheId = "ad184ce64541972b50afd8e7b75f821b".

### cacheId
set.seed(1)
Cache(rnorm, 1, cacheRepo = tmpdir1)
## [1] -0.6264538
## attr(,"tags")
## [1] "cacheId:23dc247384c1b270f0d36de4bca1b138"
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"call")
## [1] ""
# manually look at output attribute which shows cacheId: ad184ce64541972b50afd8e7b75f821b
Cache(rnorm, 1, cacheRepo = tmpdir1, cacheId = "ad184ce64541972b50afd8e7b75f821b") # same value
## cacheId is not same as calculated hash. Manually searching for cacheId:ad184ce64541972b50afd8e7b75f821b
## [1] 0.1836433
## attr(,"tags")
## [1] "cacheId:ad184ce64541972b50afd8e7b75f821b"
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"call")
## [1] ""
# override even with different inputs:
Cache(rnorm, 2, cacheRepo = tmpdir1, cacheId = "ad184ce64541972b50afd8e7b75f821b")
## cacheId is not same as calculated hash. Manually searching for cacheId:ad184ce64541972b50afd8e7b75f821b
##   loading cached result from previous rnorm call, adding to memoised copy
## [1] 0.1836433
## attr(,"tags")
## [1] "cacheId:ad184ce64541972b50afd8e7b75f821b"
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] FALSE
## 
## attr(,"call")
## [1] ""
## cleanup
unlink(c("filename.rda", "filename1.rda"))

Working with the Cache manually

Since the cache is simply an archivist repository, all archivist functions will work as is. In addition, there are several helpers in the reproducible package, including showCache, keepCache and clearCache that may be useful. Also, one can access cached items manually (rather than simply rerunning the same Cache function again).

if (requireNamespace("archivist")) {
  # get the RasterLayer that was produced with the gaussMap function:
  mapHash <- unique(showCache(tmpDir, userTags = "projectRaster")$artifact)
  map <- archivist::loadFromLocalRepo(md5hash = mapHash[1], repoDir = tmpDir, value = TRUE)
  
  plot(map)
}
## Cache size:
##   Total (including Rasters): 3.1 Kb
##   Selected objects (not including Rasters): 3.1 Kb

## cleanup
unlink(dirname(tmpDir), recursive = TRUE)

Reproducible Workflow

In general, we feel that a liberal use of Cache will make a re-usable and reproducible work flow. shiny apps can be made, taking advantage of Cache. Indeed, much of the difficulty in managing data sets and saving them for future use, can be accommodated by caching.

Nested Caching

Cache individual functions

Cache(<functionName>, <other arguments>)

This will allow fine scale control of individual function calls.