Reproducible workflow

As part of a reproducible workflow, caching of function calls, code chunks, and other elements of a project is a critical component. The objective of a reproducible workflow is is likely that an entire work flow from raw data to publication, decision support, report writing, presentation building etc., could be built and be reproducible anywhere, on any computer, operating system, with any starting conditions, on demand. The reproducible::Cache function is built to work with any R function.

Differences with other approaches

Cache uses 2 key the archivist functions saveToLocalRepo and loadFromLocalRepo, but does not use archivist::cache. Similar to archivist::cache, there is some reliance on digest::digest to determine whether the arguments are identical in subsequent iterations. It also but does many things that make standard caching with digest::digest don’t work reliably between systems. For these, the function .robustDigest is introduced to make caching transferable between systems. This is relevant for file paths, environments, parallel clusters, functions (which are contained within an environment), and many others (e.g., see ?.robustDigest for methods). Cache also adds important elements like automated tagging and the option to retrieve disk-cached values via stashed objects in memory using memoise::memoise. This means that running Cache 1, 2, and 3 times on the same function will get progressively faster. This can be extremely useful for web apps built with, say shiny.

Function-level caching

Any function can be cached using: Cache(FUN = functionName, ...).

This will be a slight change to a function call, such as: projectRaster(raster, crs = crs(newRaster)) to Cache(projectRaster, raster, crs = crs(newRaster)).

This is particularly useful for expensive operations.

library(raster)
## Loading required package: sp
library(reproducible)

tmpDir <- file.path(tempdir(), "reproducible_examples", "Cache")
checkPath(tmpDir, create = TRUE)
## [1] "/tmp/RtmpAA82jt/reproducible_examples/Cache"
ras <- raster(extent(0, 1000, 0, 1000), vals = 1:1e6, res = 1)
crs(ras) <- "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84"

newCRS <- "+init=epsg:4326 +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0"
# No Cache
system.time(map1 <- projectRaster(ras, crs = newCRS))
##    user  system elapsed 
##   2.394   0.153   2.550
# With Cache -- a little slower the first time because saving to disk
system.time(map1 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir,
                         notOlderThan = Sys.time()))
##    user  system elapsed 
##   2.378   0.200   2.724
# vastly faster the second time
system.time(map2 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
##   loading cached result from previous projectRaster call, adding to memoised copy
##    user  system elapsed 
##   0.091   0.004   0.094
# even faster the third time
system.time(map3 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
##   loading memoised result from previous projectRaster call.
##    user  system elapsed 
##   0.026   0.001   0.027
all.equal(map1, map2) # TRUE
## [1] TRUE
all.equal(map1, map3) # TRUE
## [1] TRUE

Caching examples

Basic use

library(raster)
library(magrittr)
## 
## Attaching package: 'magrittr'
## The following object is masked from 'package:raster':
## 
##     extract
try(clearCache(tmpDir, ask = FALSE), silent = TRUE) # just to make sure it is clear

ranNumsA <- Cache(rnorm, 10, 16, cacheRepo = tmpDir)

# All same
ranNumsB <- Cache(rnorm, 10, 16, cacheRepo = tmpDir) # recovers cached copy
##   loading cached result from previous rnorm call, adding to memoised copy
ranNumsC <- Cache(cacheRepo = tmpDir) %C% rnorm(10, 16)  # recovers cached copy
##   loading memoised result from previous 'rnorm' pipe sequence call.
ranNumsD <- Cache(quote(rnorm(n = 10, 16)), cacheRepo = tmpDir) # recovers cached copy
##   loading memoised result from previous rnorm call.
# Any minor change makes it different
ranNumsE <- Cache(cacheRepo = tmpDir) %C% rnorm(10, 6) # different

Example 1: Basic cache use with tags

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

showCache(tmpDir, userTags = c("objectName"))
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: ca7c12c96d8755103c9a07446373e7c4         format
##  2: ca7c12c96d8755103c9a07446373e7c4           name
##  3: ca7c12c96d8755103c9a07446373e7c4          class
##  4: ca7c12c96d8755103c9a07446373e7c4           date
##  5: ca7c12c96d8755103c9a07446373e7c4        cacheId
##  6: ca7c12c96d8755103c9a07446373e7c4     objectName
##  7: ca7c12c96d8755103c9a07446373e7c4       function
##  8: ca7c12c96d8755103c9a07446373e7c4    object.size
##  9: ca7c12c96d8755103c9a07446373e7c4       accessed
## 10: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 11: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 12: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 13: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 14: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 15: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 16: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 17: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 18: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 19: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 20: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 21: ca7c12c96d8755103c9a07446373e7c4      preDigest
## 22: ca7c12c96d8755103c9a07446373e7c4      preDigest
## 23: d648cb644316eed27f9c7da7abdfd040         format
## 24: d648cb644316eed27f9c7da7abdfd040           name
## 25: d648cb644316eed27f9c7da7abdfd040          class
## 26: d648cb644316eed27f9c7da7abdfd040           date
## 27: d648cb644316eed27f9c7da7abdfd040        cacheId
## 28: d648cb644316eed27f9c7da7abdfd040     objectName
## 29: d648cb644316eed27f9c7da7abdfd040       function
## 30: d648cb644316eed27f9c7da7abdfd040    object.size
## 31: d648cb644316eed27f9c7da7abdfd040       accessed
## 32: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 33: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 34: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 35: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 36: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 37: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 38: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 39: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 40: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 41: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 42: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 43: d648cb644316eed27f9c7da7abdfd040      preDigest
## 44: d648cb644316eed27f9c7da7abdfd040      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 10:58:04
##  2: ca7c12c96d8755103c9a07446373e7c4 2019-01-28 10:58:04
##  3:                          numeric 2019-01-28 10:58:04
##  4:              2019-01-28 10:58:04 2019-01-28 10:58:04
##  5:                 f7bee22047b8d0c1 2019-01-28 10:58:04
##  6:                                a 2019-01-28 10:58:04
##  7:                            rnorm 2019-01-28 10:58:04
##  8:                             1008 2019-01-28 10:58:04
##  9:              2019-01-28 10:58:04 2019-01-28 10:58:04
## 10:                          saveRDS 2019-01-28 10:58:04
## 11:                          do.call 2019-01-28 10:58:04
## 12:                     process_file 2019-01-28 10:58:04
## 13:                    process_group 2019-01-28 10:58:04
## 14:              process_group.block 2019-01-28 10:58:04
## 15:                       call_block 2019-01-28 10:58:04
## 16:                       block_exec 2019-01-28 10:58:04
## 17:                           in_dir 2019-01-28 10:58:04
## 18:                        timing_fn 2019-01-28 10:58:04
## 19:                           handle 2019-01-28 10:58:04
## 20:                      withVisible 2019-01-28 10:58:04
## 21:               n:7eef4eae85fd9229 2019-01-28 10:58:04
## 22:            .FUN:4f604aa46882b368 2019-01-28 10:58:04
## 23:                              rda 2019-01-28 10:58:04
## 24: d648cb644316eed27f9c7da7abdfd040 2019-01-28 10:58:04
## 25:                          numeric 2019-01-28 10:58:04
## 26:              2019-01-28 10:58:04 2019-01-28 10:58:04
## 27:                 3aef38d1fc02aee5 2019-01-28 10:58:04
## 28:                                b 2019-01-28 10:58:04
## 29:                            runif 2019-01-28 10:58:04
## 30:                             1008 2019-01-28 10:58:04
## 31:              2019-01-28 10:58:04 2019-01-28 10:58:04
## 32:                          saveRDS 2019-01-28 10:58:04
## 33:                          do.call 2019-01-28 10:58:04
## 34:                     process_file 2019-01-28 10:58:04
## 35:                    process_group 2019-01-28 10:58:04
## 36:              process_group.block 2019-01-28 10:58:04
## 37:                       call_block 2019-01-28 10:58:04
## 38:                       block_exec 2019-01-28 10:58:04
## 39:                           in_dir 2019-01-28 10:58:04
## 40:                        timing_fn 2019-01-28 10:58:04
## 41:                           handle 2019-01-28 10:58:04
## 42:                      withVisible 2019-01-28 10:58:04
## 43:               n:7eef4eae85fd9229 2019-01-28 10:58:04
## 44:            .FUN:881ec847b7161f3c 2019-01-28 10:58:04
##                             tagValue         createdDate
showCache(tmpDir, userTags = c("^a$")) # regular expression ... "a" exactly
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: ca7c12c96d8755103c9a07446373e7c4         format
##  2: ca7c12c96d8755103c9a07446373e7c4           name
##  3: ca7c12c96d8755103c9a07446373e7c4          class
##  4: ca7c12c96d8755103c9a07446373e7c4           date
##  5: ca7c12c96d8755103c9a07446373e7c4        cacheId
##  6: ca7c12c96d8755103c9a07446373e7c4     objectName
##  7: ca7c12c96d8755103c9a07446373e7c4       function
##  8: ca7c12c96d8755103c9a07446373e7c4    object.size
##  9: ca7c12c96d8755103c9a07446373e7c4       accessed
## 10: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 11: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 12: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 13: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 14: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 15: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 16: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 17: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 18: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 19: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 20: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 21: ca7c12c96d8755103c9a07446373e7c4      preDigest
## 22: ca7c12c96d8755103c9a07446373e7c4      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 10:58:04
##  2: ca7c12c96d8755103c9a07446373e7c4 2019-01-28 10:58:04
##  3:                          numeric 2019-01-28 10:58:04
##  4:              2019-01-28 10:58:04 2019-01-28 10:58:04
##  5:                 f7bee22047b8d0c1 2019-01-28 10:58:04
##  6:                                a 2019-01-28 10:58:04
##  7:                            rnorm 2019-01-28 10:58:04
##  8:                             1008 2019-01-28 10:58:04
##  9:              2019-01-28 10:58:04 2019-01-28 10:58:04
## 10:                          saveRDS 2019-01-28 10:58:04
## 11:                          do.call 2019-01-28 10:58:04
## 12:                     process_file 2019-01-28 10:58:04
## 13:                    process_group 2019-01-28 10:58:04
## 14:              process_group.block 2019-01-28 10:58:04
## 15:                       call_block 2019-01-28 10:58:04
## 16:                       block_exec 2019-01-28 10:58:04
## 17:                           in_dir 2019-01-28 10:58:04
## 18:                        timing_fn 2019-01-28 10:58:04
## 19:                           handle 2019-01-28 10:58:04
## 20:                      withVisible 2019-01-28 10:58:04
## 21:               n:7eef4eae85fd9229 2019-01-28 10:58:04
## 22:            .FUN:4f604aa46882b368 2019-01-28 10:58:04
##                             tagValue         createdDate
showCache(tmpDir, userTags = c("runif")) # show only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: d648cb644316eed27f9c7da7abdfd040         format
##  2: d648cb644316eed27f9c7da7abdfd040           name
##  3: d648cb644316eed27f9c7da7abdfd040          class
##  4: d648cb644316eed27f9c7da7abdfd040           date
##  5: d648cb644316eed27f9c7da7abdfd040        cacheId
##  6: d648cb644316eed27f9c7da7abdfd040     objectName
##  7: d648cb644316eed27f9c7da7abdfd040       function
##  8: d648cb644316eed27f9c7da7abdfd040    object.size
##  9: d648cb644316eed27f9c7da7abdfd040       accessed
## 10: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 11: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 12: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 13: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 14: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 15: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 16: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 17: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 18: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 19: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 20: d648cb644316eed27f9c7da7abdfd040 otherFunctions
## 21: d648cb644316eed27f9c7da7abdfd040      preDigest
## 22: d648cb644316eed27f9c7da7abdfd040      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 10:58:04
##  2: d648cb644316eed27f9c7da7abdfd040 2019-01-28 10:58:04
##  3:                          numeric 2019-01-28 10:58:04
##  4:              2019-01-28 10:58:04 2019-01-28 10:58:04
##  5:                 3aef38d1fc02aee5 2019-01-28 10:58:04
##  6:                                b 2019-01-28 10:58:04
##  7:                            runif 2019-01-28 10:58:04
##  8:                             1008 2019-01-28 10:58:04
##  9:              2019-01-28 10:58:04 2019-01-28 10:58:04
## 10:                          saveRDS 2019-01-28 10:58:04
## 11:                          do.call 2019-01-28 10:58:04
## 12:                     process_file 2019-01-28 10:58:04
## 13:                    process_group 2019-01-28 10:58:04
## 14:              process_group.block 2019-01-28 10:58:04
## 15:                       call_block 2019-01-28 10:58:04
## 16:                       block_exec 2019-01-28 10:58:04
## 17:                           in_dir 2019-01-28 10:58:04
## 18:                        timing_fn 2019-01-28 10:58:04
## 19:                           handle 2019-01-28 10:58:04
## 20:                      withVisible 2019-01-28 10:58:04
## 21:               n:7eef4eae85fd9229 2019-01-28 10:58:04
## 22:            .FUN:881ec847b7161f3c 2019-01-28 10:58:04
##                             tagValue         createdDate
clearCache(tmpDir, userTags = c("runif"), ask = FALSE) # remove only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) # only those made during rnorm call
## Cache size:
##   Total (including Rasters): 804 bytes
##   Selected objects (not including Rasters): 804 bytes
##                             artifact         tagKey
##  1: 0175286f0be4ed7c8b2256a294a384a2         format
##  2: 0175286f0be4ed7c8b2256a294a384a2           name
##  3: 0175286f0be4ed7c8b2256a294a384a2          class
##  4: 0175286f0be4ed7c8b2256a294a384a2           date
##  5: 0175286f0be4ed7c8b2256a294a384a2        cacheId
##  6: 0175286f0be4ed7c8b2256a294a384a2       function
##  7: 0175286f0be4ed7c8b2256a294a384a2    object.size
##  8: 0175286f0be4ed7c8b2256a294a384a2       accessed
##  9: 0175286f0be4ed7c8b2256a294a384a2 otherFunctions
## 10: 0175286f0be4ed7c8b2256a294a384a2 otherFunctions
## 11: 0175286f0be4ed7c8b2256a294a384a2 otherFunctions
## 12: 0175286f0be4ed7c8b2256a294a384a2 otherFunctions
## 13: 0175286f0be4ed7c8b2256a294a384a2 otherFunctions
## 14: 0175286f0be4ed7c8b2256a294a384a2 otherFunctions
## 15: 0175286f0be4ed7c8b2256a294a384a2 otherFunctions
## 16: 0175286f0be4ed7c8b2256a294a384a2 otherFunctions
## 17: 0175286f0be4ed7c8b2256a294a384a2 otherFunctions
## 18: 0175286f0be4ed7c8b2256a294a384a2 otherFunctions
## 19: 0175286f0be4ed7c8b2256a294a384a2 otherFunctions
## 20: 0175286f0be4ed7c8b2256a294a384a2      preDigest
## 21: 0175286f0be4ed7c8b2256a294a384a2      preDigest
## 22: 0175286f0be4ed7c8b2256a294a384a2      preDigest
## 23: 0175286f0be4ed7c8b2256a294a384a2       accessed
## 24: 0175286f0be4ed7c8b2256a294a384a2       accessed
## 25: 0175286f0be4ed7c8b2256a294a384a2       accessed
## 26: 17559f8f462df427c5961fb732519701         format
## 27: 17559f8f462df427c5961fb732519701           name
## 28: 17559f8f462df427c5961fb732519701          class
## 29: 17559f8f462df427c5961fb732519701           date
## 30: 17559f8f462df427c5961fb732519701        cacheId
## 31: 17559f8f462df427c5961fb732519701       function
## 32: 17559f8f462df427c5961fb732519701    object.size
## 33: 17559f8f462df427c5961fb732519701       accessed
## 34: 17559f8f462df427c5961fb732519701 otherFunctions
## 35: 17559f8f462df427c5961fb732519701 otherFunctions
## 36: 17559f8f462df427c5961fb732519701 otherFunctions
## 37: 17559f8f462df427c5961fb732519701 otherFunctions
## 38: 17559f8f462df427c5961fb732519701 otherFunctions
## 39: 17559f8f462df427c5961fb732519701 otherFunctions
## 40: 17559f8f462df427c5961fb732519701 otherFunctions
## 41: 17559f8f462df427c5961fb732519701 otherFunctions
## 42: 17559f8f462df427c5961fb732519701 otherFunctions
## 43: 17559f8f462df427c5961fb732519701 otherFunctions
## 44: 17559f8f462df427c5961fb732519701 otherFunctions
## 45: 17559f8f462df427c5961fb732519701 otherFunctions
## 46: 17559f8f462df427c5961fb732519701      preDigest
## 47: 17559f8f462df427c5961fb732519701      preDigest
## 48: 17559f8f462df427c5961fb732519701      preDigest
## 49: ca7c12c96d8755103c9a07446373e7c4         format
## 50: ca7c12c96d8755103c9a07446373e7c4           name
## 51: ca7c12c96d8755103c9a07446373e7c4          class
## 52: ca7c12c96d8755103c9a07446373e7c4           date
## 53: ca7c12c96d8755103c9a07446373e7c4        cacheId
## 54: ca7c12c96d8755103c9a07446373e7c4     objectName
## 55: ca7c12c96d8755103c9a07446373e7c4       function
## 56: ca7c12c96d8755103c9a07446373e7c4    object.size
## 57: ca7c12c96d8755103c9a07446373e7c4       accessed
## 58: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 59: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 60: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 61: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 62: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 63: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 64: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 65: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 66: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 67: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 68: ca7c12c96d8755103c9a07446373e7c4 otherFunctions
## 69: ca7c12c96d8755103c9a07446373e7c4      preDigest
## 70: ca7c12c96d8755103c9a07446373e7c4      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 10:58:04
##  2: 0175286f0be4ed7c8b2256a294a384a2 2019-01-28 10:58:04
##  3:                          numeric 2019-01-28 10:58:04
##  4:              2019-01-28 10:58:04 2019-01-28 10:58:04
##  5:                 f11fb1a2880f8060 2019-01-28 10:58:04
##  6:                            rnorm 2019-01-28 10:58:04
##  7:                             1104 2019-01-28 10:58:04
##  8:              2019-01-28 10:58:04 2019-01-28 10:58:04
##  9:                          saveRDS 2019-01-28 10:58:04
## 10:                          do.call 2019-01-28 10:58:04
## 11:                     process_file 2019-01-28 10:58:04
## 12:                    process_group 2019-01-28 10:58:04
## 13:              process_group.block 2019-01-28 10:58:04
## 14:                       call_block 2019-01-28 10:58:04
## 15:                       block_exec 2019-01-28 10:58:04
## 16:                           in_dir 2019-01-28 10:58:04
## 17:                        timing_fn 2019-01-28 10:58:04
## 18:                           handle 2019-01-28 10:58:04
## 19:                      withVisible 2019-01-28 10:58:04
## 20:               n:c5775c3b366fb719 2019-01-28 10:58:04
## 21:            mean:15620f138033a66c 2019-01-28 10:58:04
## 22:            .FUN:4f604aa46882b368 2019-01-28 10:58:04
## 23:              2019-01-28 10:58:04 2019-01-28 10:58:04
## 24:              2019-01-28 10:58:04 2019-01-28 10:58:04
## 25:              2019-01-28 10:58:04 2019-01-28 10:58:04
## 26:                              rda 2019-01-28 10:58:04
## 27: 17559f8f462df427c5961fb732519701 2019-01-28 10:58:04
## 28:                          numeric 2019-01-28 10:58:04
## 29:              2019-01-28 10:58:04 2019-01-28 10:58:04
## 30:                 c16f0e2c57596069 2019-01-28 10:58:04
## 31:              rnorm pipe sequence 2019-01-28 10:58:04
## 32:                             1104 2019-01-28 10:58:04
## 33:              2019-01-28 10:58:04 2019-01-28 10:58:04
## 34:                          saveRDS 2019-01-28 10:58:04
## 35:                          do.call 2019-01-28 10:58:04
## 36:                     process_file 2019-01-28 10:58:04
## 37:                    process_group 2019-01-28 10:58:04
## 38:              process_group.block 2019-01-28 10:58:04
## 39:                       call_block 2019-01-28 10:58:04
## 40:                       block_exec 2019-01-28 10:58:04
## 41:                           in_dir 2019-01-28 10:58:04
## 42:                        timing_fn 2019-01-28 10:58:04
## 43:                           handle 2019-01-28 10:58:04
## 44:                      withVisible 2019-01-28 10:58:04
## 45:                              %C% 2019-01-28 10:58:04
## 46:               n:c5775c3b366fb719 2019-01-28 10:58:04
## 47:            mean:152602b8ff81e5bb 2019-01-28 10:58:04
## 48:            .FUN:4f604aa46882b368 2019-01-28 10:58:04
## 49:                              rda 2019-01-28 10:58:04
## 50: ca7c12c96d8755103c9a07446373e7c4 2019-01-28 10:58:04
## 51:                          numeric 2019-01-28 10:58:04
## 52:              2019-01-28 10:58:04 2019-01-28 10:58:04
## 53:                 f7bee22047b8d0c1 2019-01-28 10:58:04
## 54:                                a 2019-01-28 10:58:04
## 55:                            rnorm 2019-01-28 10:58:04
## 56:                             1008 2019-01-28 10:58:04
## 57:              2019-01-28 10:58:04 2019-01-28 10:58:04
## 58:                          saveRDS 2019-01-28 10:58:04
## 59:                          do.call 2019-01-28 10:58:04
## 60:                     process_file 2019-01-28 10:58:04
## 61:                    process_group 2019-01-28 10:58:04
## 62:              process_group.block 2019-01-28 10:58:04
## 63:                       call_block 2019-01-28 10:58:04
## 64:                       block_exec 2019-01-28 10:58:04
## 65:                           in_dir 2019-01-28 10:58:04
## 66:                        timing_fn 2019-01-28 10:58:04
## 67:                           handle 2019-01-28 10:58:04
## 68:                      withVisible 2019-01-28 10:58:04
## 69:               n:7eef4eae85fd9229 2019-01-28 10:58:04
## 70:            .FUN:4f604aa46882b368 2019-01-28 10:58:04
##                             tagValue         createdDate
clearCache(tmpDir, ask = FALSE)

Example 2: using the “accessed” tag

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# access it again, from Cache
ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
##   loading cached result from previous rnorm call, adding to memoised copy
wholeCache <- showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
# keep only items accessed "recently" (i.e., only objectName:a)
onlyRecentlyAccessed <- showCache(tmpDir, userTags = max(wholeCache[tagKey == "accessed"]$tagValue))
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
# inverse join with 2 data.tables ... using: a[!b]
# i.e., return all of wholeCache that was not recently accessed
toRemove <- unique(wholeCache[!onlyRecentlyAccessed], by = "artifact")$artifact
clearCache(tmpDir, toRemove, ask = FALSE) # remove ones not recently accessed
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
showCache(tmpDir) # still has more recently accessed
## Cache size:
##   Total (including Rasters): 0 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 3 cols: md5hash,name,createdDate
clearCache(tmpDir, ask = FALSE)

Example 3: using keepCache

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# keep only those cached items from the last 24 hours
oneDay <- 60 * 60 * 24
keepCache(tmpDir, after = Sys.time() - oneDay, ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 5044ea9f7020e07d650ccf30d0ebde69         format
##  2: 5044ea9f7020e07d650ccf30d0ebde69           name
##  3: 5044ea9f7020e07d650ccf30d0ebde69          class
##  4: 5044ea9f7020e07d650ccf30d0ebde69           date
##  5: 5044ea9f7020e07d650ccf30d0ebde69        cacheId
##  6: 5044ea9f7020e07d650ccf30d0ebde69     objectName
##  7: 5044ea9f7020e07d650ccf30d0ebde69       function
##  8: 5044ea9f7020e07d650ccf30d0ebde69    object.size
##  9: 5044ea9f7020e07d650ccf30d0ebde69       accessed
## 10: 5044ea9f7020e07d650ccf30d0ebde69 otherFunctions
## 11: 5044ea9f7020e07d650ccf30d0ebde69 otherFunctions
## 12: 5044ea9f7020e07d650ccf30d0ebde69 otherFunctions
## 13: 5044ea9f7020e07d650ccf30d0ebde69 otherFunctions
## 14: 5044ea9f7020e07d650ccf30d0ebde69 otherFunctions
## 15: 5044ea9f7020e07d650ccf30d0ebde69 otherFunctions
## 16: 5044ea9f7020e07d650ccf30d0ebde69 otherFunctions
## 17: 5044ea9f7020e07d650ccf30d0ebde69 otherFunctions
## 18: 5044ea9f7020e07d650ccf30d0ebde69 otherFunctions
## 19: 5044ea9f7020e07d650ccf30d0ebde69 otherFunctions
## 20: 5044ea9f7020e07d650ccf30d0ebde69 otherFunctions
## 21: 5044ea9f7020e07d650ccf30d0ebde69      preDigest
## 22: 5044ea9f7020e07d650ccf30d0ebde69      preDigest
## 23: e97a50ac85344b208775db3ced7915f7         format
## 24: e97a50ac85344b208775db3ced7915f7           name
## 25: e97a50ac85344b208775db3ced7915f7          class
## 26: e97a50ac85344b208775db3ced7915f7           date
## 27: e97a50ac85344b208775db3ced7915f7        cacheId
## 28: e97a50ac85344b208775db3ced7915f7     objectName
## 29: e97a50ac85344b208775db3ced7915f7       function
## 30: e97a50ac85344b208775db3ced7915f7    object.size
## 31: e97a50ac85344b208775db3ced7915f7       accessed
## 32: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 33: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 34: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 35: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 36: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 37: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 38: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 39: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 40: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 41: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 42: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 43: e97a50ac85344b208775db3ced7915f7      preDigest
## 44: e97a50ac85344b208775db3ced7915f7      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 10:58:05
##  2: 5044ea9f7020e07d650ccf30d0ebde69 2019-01-28 10:58:05
##  3:                          numeric 2019-01-28 10:58:05
##  4:              2019-01-28 10:58:05 2019-01-28 10:58:05
##  5:                 3aef38d1fc02aee5 2019-01-28 10:58:05
##  6:                                b 2019-01-28 10:58:05
##  7:                            runif 2019-01-28 10:58:05
##  8:                             1008 2019-01-28 10:58:05
##  9:              2019-01-28 10:58:05 2019-01-28 10:58:05
## 10:                          saveRDS 2019-01-28 10:58:05
## 11:                          do.call 2019-01-28 10:58:05
## 12:                     process_file 2019-01-28 10:58:05
## 13:                    process_group 2019-01-28 10:58:05
## 14:              process_group.block 2019-01-28 10:58:05
## 15:                       call_block 2019-01-28 10:58:05
## 16:                       block_exec 2019-01-28 10:58:05
## 17:                           in_dir 2019-01-28 10:58:05
## 18:                        timing_fn 2019-01-28 10:58:05
## 19:                           handle 2019-01-28 10:58:05
## 20:                      withVisible 2019-01-28 10:58:05
## 21:               n:7eef4eae85fd9229 2019-01-28 10:58:05
## 22:            .FUN:881ec847b7161f3c 2019-01-28 10:58:05
## 23:                              rda 2019-01-28 10:58:05
## 24: e97a50ac85344b208775db3ced7915f7 2019-01-28 10:58:05
## 25:                          numeric 2019-01-28 10:58:05
## 26:              2019-01-28 10:58:05 2019-01-28 10:58:05
## 27:                 f7bee22047b8d0c1 2019-01-28 10:58:05
## 28:                                a 2019-01-28 10:58:05
## 29:                            rnorm 2019-01-28 10:58:05
## 30:                             1008 2019-01-28 10:58:05
## 31:              2019-01-28 10:58:05 2019-01-28 10:58:05
## 32:                          saveRDS 2019-01-28 10:58:05
## 33:                          do.call 2019-01-28 10:58:05
## 34:                     process_file 2019-01-28 10:58:05
## 35:                    process_group 2019-01-28 10:58:05
## 36:              process_group.block 2019-01-28 10:58:05
## 37:                       call_block 2019-01-28 10:58:05
## 38:                       block_exec 2019-01-28 10:58:05
## 39:                           in_dir 2019-01-28 10:58:05
## 40:                        timing_fn 2019-01-28 10:58:05
## 41:                           handle 2019-01-28 10:58:05
## 42:                      withVisible 2019-01-28 10:58:05
## 43:               n:7eef4eae85fd9229 2019-01-28 10:58:05
## 44:            .FUN:4f604aa46882b368 2019-01-28 10:58:05
##                             tagValue         createdDate
# Keep all Cache items created with an rnorm() call
keepCache(tmpDir, userTags = "rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: e97a50ac85344b208775db3ced7915f7         format
##  2: e97a50ac85344b208775db3ced7915f7           name
##  3: e97a50ac85344b208775db3ced7915f7          class
##  4: e97a50ac85344b208775db3ced7915f7           date
##  5: e97a50ac85344b208775db3ced7915f7        cacheId
##  6: e97a50ac85344b208775db3ced7915f7     objectName
##  7: e97a50ac85344b208775db3ced7915f7       function
##  8: e97a50ac85344b208775db3ced7915f7    object.size
##  9: e97a50ac85344b208775db3ced7915f7       accessed
## 10: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 11: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 12: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 13: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 14: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 15: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 16: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 17: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 18: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 19: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 20: e97a50ac85344b208775db3ced7915f7 otherFunctions
## 21: e97a50ac85344b208775db3ced7915f7      preDigest
## 22: e97a50ac85344b208775db3ced7915f7      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 10:58:05
##  2: e97a50ac85344b208775db3ced7915f7 2019-01-28 10:58:05
##  3:                          numeric 2019-01-28 10:58:05
##  4:              2019-01-28 10:58:05 2019-01-28 10:58:05
##  5:                 f7bee22047b8d0c1 2019-01-28 10:58:05
##  6:                                a 2019-01-28 10:58:05
##  7:                            rnorm 2019-01-28 10:58:05
##  8:                             1008 2019-01-28 10:58:05
##  9:              2019-01-28 10:58:05 2019-01-28 10:58:05
## 10:                          saveRDS 2019-01-28 10:58:05
## 11:                          do.call 2019-01-28 10:58:05
## 12:                     process_file 2019-01-28 10:58:05
## 13:                    process_group 2019-01-28 10:58:05
## 14:              process_group.block 2019-01-28 10:58:05
## 15:                       call_block 2019-01-28 10:58:05
## 16:                       block_exec 2019-01-28 10:58:05
## 17:                           in_dir 2019-01-28 10:58:05
## 18:                        timing_fn 2019-01-28 10:58:05
## 19:                           handle 2019-01-28 10:58:05
## 20:                      withVisible 2019-01-28 10:58:05
## 21:               n:7eef4eae85fd9229 2019-01-28 10:58:05
## 22:            .FUN:4f604aa46882b368 2019-01-28 10:58:05
##                             tagValue         createdDate
# Remove all Cache items that happened within a rnorm() call
clearCache(tmpDir, userTags = "rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) ## empty
## Cache size:
##   Total (including Rasters): 0 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 3 cols: md5hash,name,createdDate

Example 4: searching for multiple objects in the cache

# default userTags is "and" matching; for "or" matching use |
ranNumsA <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# show all objects (runif and rnorm in this case)
showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 474488d53c73b1cd85cf9700cedad5ac         format
##  2: 474488d53c73b1cd85cf9700cedad5ac           name
##  3: 474488d53c73b1cd85cf9700cedad5ac          class
##  4: 474488d53c73b1cd85cf9700cedad5ac           date
##  5: 474488d53c73b1cd85cf9700cedad5ac        cacheId
##  6: 474488d53c73b1cd85cf9700cedad5ac     objectName
##  7: 474488d53c73b1cd85cf9700cedad5ac       function
##  8: 474488d53c73b1cd85cf9700cedad5ac    object.size
##  9: 474488d53c73b1cd85cf9700cedad5ac       accessed
## 10: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 11: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 12: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 13: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 14: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 15: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 16: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 17: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 18: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 19: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 20: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 21: 474488d53c73b1cd85cf9700cedad5ac      preDigest
## 22: 474488d53c73b1cd85cf9700cedad5ac      preDigest
## 23: cd35d944ba4b5e84cebe69e34013f1e1         format
## 24: cd35d944ba4b5e84cebe69e34013f1e1           name
## 25: cd35d944ba4b5e84cebe69e34013f1e1          class
## 26: cd35d944ba4b5e84cebe69e34013f1e1           date
## 27: cd35d944ba4b5e84cebe69e34013f1e1        cacheId
## 28: cd35d944ba4b5e84cebe69e34013f1e1     objectName
## 29: cd35d944ba4b5e84cebe69e34013f1e1       function
## 30: cd35d944ba4b5e84cebe69e34013f1e1    object.size
## 31: cd35d944ba4b5e84cebe69e34013f1e1       accessed
## 32: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 33: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 34: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 35: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 36: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 37: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 38: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 39: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 40: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 41: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 42: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 43: cd35d944ba4b5e84cebe69e34013f1e1      preDigest
## 44: cd35d944ba4b5e84cebe69e34013f1e1      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 10:58:06
##  2: 474488d53c73b1cd85cf9700cedad5ac 2019-01-28 10:58:06
##  3:                          numeric 2019-01-28 10:58:06
##  4:              2019-01-28 10:58:06 2019-01-28 10:58:06
##  5:                 f7bee22047b8d0c1 2019-01-28 10:58:06
##  6:                                b 2019-01-28 10:58:06
##  7:                            rnorm 2019-01-28 10:58:06
##  8:                             1008 2019-01-28 10:58:06
##  9:              2019-01-28 10:58:06 2019-01-28 10:58:06
## 10:                          saveRDS 2019-01-28 10:58:06
## 11:                          do.call 2019-01-28 10:58:06
## 12:                     process_file 2019-01-28 10:58:06
## 13:                    process_group 2019-01-28 10:58:06
## 14:              process_group.block 2019-01-28 10:58:06
## 15:                       call_block 2019-01-28 10:58:06
## 16:                       block_exec 2019-01-28 10:58:06
## 17:                           in_dir 2019-01-28 10:58:06
## 18:                        timing_fn 2019-01-28 10:58:06
## 19:                           handle 2019-01-28 10:58:06
## 20:                      withVisible 2019-01-28 10:58:06
## 21:               n:7eef4eae85fd9229 2019-01-28 10:58:06
## 22:            .FUN:4f604aa46882b368 2019-01-28 10:58:06
## 23:                              rda 2019-01-28 10:58:06
## 24: cd35d944ba4b5e84cebe69e34013f1e1 2019-01-28 10:58:06
## 25:                          numeric 2019-01-28 10:58:06
## 26:              2019-01-28 10:58:06 2019-01-28 10:58:06
## 27:                 3aef38d1fc02aee5 2019-01-28 10:58:06
## 28:                                a 2019-01-28 10:58:06
## 29:                            runif 2019-01-28 10:58:06
## 30:                             1008 2019-01-28 10:58:06
## 31:              2019-01-28 10:58:06 2019-01-28 10:58:06
## 32:                          saveRDS 2019-01-28 10:58:06
## 33:                          do.call 2019-01-28 10:58:06
## 34:                     process_file 2019-01-28 10:58:06
## 35:                    process_group 2019-01-28 10:58:06
## 36:              process_group.block 2019-01-28 10:58:06
## 37:                       call_block 2019-01-28 10:58:06
## 38:                       block_exec 2019-01-28 10:58:06
## 39:                           in_dir 2019-01-28 10:58:06
## 40:                        timing_fn 2019-01-28 10:58:06
## 41:                           handle 2019-01-28 10:58:06
## 42:                      withVisible 2019-01-28 10:58:06
## 43:               n:7eef4eae85fd9229 2019-01-28 10:58:06
## 44:            .FUN:881ec847b7161f3c 2019-01-28 10:58:06
##                             tagValue         createdDate
# show objects that are both runif and rnorm
# (i.e., none in this case, because objecs are either or, not both)
showCache(tmpDir, userTags = c("runif", "rnorm")) ## empty
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 4 cols: artifact,tagKey,tagValue,createdDate
# show objects that are either runif or rnorm ("or" search)
showCache(tmpDir, userTags = "runif|rnorm")
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 474488d53c73b1cd85cf9700cedad5ac         format
##  2: 474488d53c73b1cd85cf9700cedad5ac           name
##  3: 474488d53c73b1cd85cf9700cedad5ac          class
##  4: 474488d53c73b1cd85cf9700cedad5ac           date
##  5: 474488d53c73b1cd85cf9700cedad5ac        cacheId
##  6: 474488d53c73b1cd85cf9700cedad5ac     objectName
##  7: 474488d53c73b1cd85cf9700cedad5ac       function
##  8: 474488d53c73b1cd85cf9700cedad5ac    object.size
##  9: 474488d53c73b1cd85cf9700cedad5ac       accessed
## 10: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 11: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 12: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 13: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 14: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 15: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 16: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 17: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 18: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 19: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 20: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 21: 474488d53c73b1cd85cf9700cedad5ac      preDigest
## 22: 474488d53c73b1cd85cf9700cedad5ac      preDigest
## 23: cd35d944ba4b5e84cebe69e34013f1e1         format
## 24: cd35d944ba4b5e84cebe69e34013f1e1           name
## 25: cd35d944ba4b5e84cebe69e34013f1e1          class
## 26: cd35d944ba4b5e84cebe69e34013f1e1           date
## 27: cd35d944ba4b5e84cebe69e34013f1e1        cacheId
## 28: cd35d944ba4b5e84cebe69e34013f1e1     objectName
## 29: cd35d944ba4b5e84cebe69e34013f1e1       function
## 30: cd35d944ba4b5e84cebe69e34013f1e1    object.size
## 31: cd35d944ba4b5e84cebe69e34013f1e1       accessed
## 32: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 33: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 34: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 35: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 36: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 37: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 38: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 39: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 40: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 41: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 42: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 43: cd35d944ba4b5e84cebe69e34013f1e1      preDigest
## 44: cd35d944ba4b5e84cebe69e34013f1e1      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 10:58:06
##  2: 474488d53c73b1cd85cf9700cedad5ac 2019-01-28 10:58:06
##  3:                          numeric 2019-01-28 10:58:06
##  4:              2019-01-28 10:58:06 2019-01-28 10:58:06
##  5:                 f7bee22047b8d0c1 2019-01-28 10:58:06
##  6:                                b 2019-01-28 10:58:06
##  7:                            rnorm 2019-01-28 10:58:06
##  8:                             1008 2019-01-28 10:58:06
##  9:              2019-01-28 10:58:06 2019-01-28 10:58:06
## 10:                          saveRDS 2019-01-28 10:58:06
## 11:                          do.call 2019-01-28 10:58:06
## 12:                     process_file 2019-01-28 10:58:06
## 13:                    process_group 2019-01-28 10:58:06
## 14:              process_group.block 2019-01-28 10:58:06
## 15:                       call_block 2019-01-28 10:58:06
## 16:                       block_exec 2019-01-28 10:58:06
## 17:                           in_dir 2019-01-28 10:58:06
## 18:                        timing_fn 2019-01-28 10:58:06
## 19:                           handle 2019-01-28 10:58:06
## 20:                      withVisible 2019-01-28 10:58:06
## 21:               n:7eef4eae85fd9229 2019-01-28 10:58:06
## 22:            .FUN:4f604aa46882b368 2019-01-28 10:58:06
## 23:                              rda 2019-01-28 10:58:06
## 24: cd35d944ba4b5e84cebe69e34013f1e1 2019-01-28 10:58:06
## 25:                          numeric 2019-01-28 10:58:06
## 26:              2019-01-28 10:58:06 2019-01-28 10:58:06
## 27:                 3aef38d1fc02aee5 2019-01-28 10:58:06
## 28:                                a 2019-01-28 10:58:06
## 29:                            runif 2019-01-28 10:58:06
## 30:                             1008 2019-01-28 10:58:06
## 31:              2019-01-28 10:58:06 2019-01-28 10:58:06
## 32:                          saveRDS 2019-01-28 10:58:06
## 33:                          do.call 2019-01-28 10:58:06
## 34:                     process_file 2019-01-28 10:58:06
## 35:                    process_group 2019-01-28 10:58:06
## 36:              process_group.block 2019-01-28 10:58:06
## 37:                       call_block 2019-01-28 10:58:06
## 38:                       block_exec 2019-01-28 10:58:06
## 39:                           in_dir 2019-01-28 10:58:06
## 40:                        timing_fn 2019-01-28 10:58:06
## 41:                           handle 2019-01-28 10:58:06
## 42:                      withVisible 2019-01-28 10:58:06
## 43:               n:7eef4eae85fd9229 2019-01-28 10:58:06
## 44:            .FUN:881ec847b7161f3c 2019-01-28 10:58:06
##                             tagValue         createdDate
# keep only objects that are either runif or rnorm ("or" search)
keepCache(tmpDir, userTags = "runif|rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 474488d53c73b1cd85cf9700cedad5ac         format
##  2: 474488d53c73b1cd85cf9700cedad5ac           name
##  3: 474488d53c73b1cd85cf9700cedad5ac          class
##  4: 474488d53c73b1cd85cf9700cedad5ac           date
##  5: 474488d53c73b1cd85cf9700cedad5ac        cacheId
##  6: 474488d53c73b1cd85cf9700cedad5ac     objectName
##  7: 474488d53c73b1cd85cf9700cedad5ac       function
##  8: 474488d53c73b1cd85cf9700cedad5ac    object.size
##  9: 474488d53c73b1cd85cf9700cedad5ac       accessed
## 10: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 11: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 12: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 13: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 14: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 15: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 16: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 17: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 18: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 19: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 20: 474488d53c73b1cd85cf9700cedad5ac otherFunctions
## 21: 474488d53c73b1cd85cf9700cedad5ac      preDigest
## 22: 474488d53c73b1cd85cf9700cedad5ac      preDigest
## 23: cd35d944ba4b5e84cebe69e34013f1e1         format
## 24: cd35d944ba4b5e84cebe69e34013f1e1           name
## 25: cd35d944ba4b5e84cebe69e34013f1e1          class
## 26: cd35d944ba4b5e84cebe69e34013f1e1           date
## 27: cd35d944ba4b5e84cebe69e34013f1e1        cacheId
## 28: cd35d944ba4b5e84cebe69e34013f1e1     objectName
## 29: cd35d944ba4b5e84cebe69e34013f1e1       function
## 30: cd35d944ba4b5e84cebe69e34013f1e1    object.size
## 31: cd35d944ba4b5e84cebe69e34013f1e1       accessed
## 32: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 33: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 34: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 35: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 36: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 37: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 38: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 39: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 40: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 41: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 42: cd35d944ba4b5e84cebe69e34013f1e1 otherFunctions
## 43: cd35d944ba4b5e84cebe69e34013f1e1      preDigest
## 44: cd35d944ba4b5e84cebe69e34013f1e1      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 10:58:06
##  2: 474488d53c73b1cd85cf9700cedad5ac 2019-01-28 10:58:06
##  3:                          numeric 2019-01-28 10:58:06
##  4:              2019-01-28 10:58:06 2019-01-28 10:58:06
##  5:                 f7bee22047b8d0c1 2019-01-28 10:58:06
##  6:                                b 2019-01-28 10:58:06
##  7:                            rnorm 2019-01-28 10:58:06
##  8:                             1008 2019-01-28 10:58:06
##  9:              2019-01-28 10:58:06 2019-01-28 10:58:06
## 10:                          saveRDS 2019-01-28 10:58:06
## 11:                          do.call 2019-01-28 10:58:06
## 12:                     process_file 2019-01-28 10:58:06
## 13:                    process_group 2019-01-28 10:58:06
## 14:              process_group.block 2019-01-28 10:58:06
## 15:                       call_block 2019-01-28 10:58:06
## 16:                       block_exec 2019-01-28 10:58:06
## 17:                           in_dir 2019-01-28 10:58:06
## 18:                        timing_fn 2019-01-28 10:58:06
## 19:                           handle 2019-01-28 10:58:06
## 20:                      withVisible 2019-01-28 10:58:06
## 21:               n:7eef4eae85fd9229 2019-01-28 10:58:06
## 22:            .FUN:4f604aa46882b368 2019-01-28 10:58:06
## 23:                              rda 2019-01-28 10:58:06
## 24: cd35d944ba4b5e84cebe69e34013f1e1 2019-01-28 10:58:06
## 25:                          numeric 2019-01-28 10:58:06
## 26:              2019-01-28 10:58:06 2019-01-28 10:58:06
## 27:                 3aef38d1fc02aee5 2019-01-28 10:58:06
## 28:                                a 2019-01-28 10:58:06
## 29:                            runif 2019-01-28 10:58:06
## 30:                             1008 2019-01-28 10:58:06
## 31:              2019-01-28 10:58:06 2019-01-28 10:58:06
## 32:                          saveRDS 2019-01-28 10:58:06
## 33:                          do.call 2019-01-28 10:58:06
## 34:                     process_file 2019-01-28 10:58:06
## 35:                    process_group 2019-01-28 10:58:06
## 36:              process_group.block 2019-01-28 10:58:06
## 37:                       call_block 2019-01-28 10:58:06
## 38:                       block_exec 2019-01-28 10:58:06
## 39:                           in_dir 2019-01-28 10:58:06
## 40:                        timing_fn 2019-01-28 10:58:06
## 41:                           handle 2019-01-28 10:58:06
## 42:                      withVisible 2019-01-28 10:58:06
## 43:               n:7eef4eae85fd9229 2019-01-28 10:58:06
## 44:            .FUN:881ec847b7161f3c 2019-01-28 10:58:06
##                             tagValue         createdDate
clearCache(tmpDir, ask = FALSE)

Example 5: using caching to speed up rerunning expensive computations

ras <- raster(extent(0, 5, 0, 5), res = 1,
              vals = sample(1:5, replace = TRUE, size = 25),
              crs = "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84")

# A slow operation, like GIS operation
notCached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  projectRaster(ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)

cached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  # using quote works also
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)

# second time is much faster
reRun <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)
##   loading cached result from previous projectRaster call, adding to memoised copy
# recovered cached version is same as non-cached version
all.equal(notCached, reRun) ## TRUE
## [1] TRUE

Nested Caching

Nested caching, which is when Caching of a function occurs inside an outer function, which is itself cached. This is a critical element to working within a reproducible work flow. It is not enough during development to cache flat code chunks, as there will be many levels of “slow” functions. Ideally, at all points in a development cycle, it should be possible to get to any line of code starting from the very initial steps, running through everything up to that point, in less that 1 second. If the workflow can be kept very fast like this, then there is a guarantee that it will work at any point.

##########################
## Nested Caching
# Make 2 functions
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean)
}
outer <- function(n) {
  Cache(inner, 0.1, cacheRepo = tmpdir2)
}

# make 2 different cache paths
tmpdir1 <- file.path(tempdir(), "first")
tmpdir2 <- file.path(tempdir(), "second")

# Run the Cache ... notOlderThan propagates to all 3 Cache calls,
#   but cacheRepo is tmpdir1 in top level Cache and all nested
#   Cache calls, unless individually overridden ... here inner
#   uses tmpdir2 repository
Cache(outer, n = 2, cacheRepo = tmpdir1, notOlderThan = Sys.time())
## [1]  1.2363982  0.3391868 -0.2872611
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:a7af5367c13aba8f"
## attr(,"call")
## [1] ""
showCache(tmpdir1) # 2 function calls
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 12621c86ae216006b9f58733961f4ba3         format
##  2: 12621c86ae216006b9f58733961f4ba3           name
##  3: 12621c86ae216006b9f58733961f4ba3          class
##  4: 12621c86ae216006b9f58733961f4ba3           date
##  5: 12621c86ae216006b9f58733961f4ba3        cacheId
##  6: 12621c86ae216006b9f58733961f4ba3       function
##  7: 12621c86ae216006b9f58733961f4ba3    object.size
##  8: 12621c86ae216006b9f58733961f4ba3       accessed
##  9: 12621c86ae216006b9f58733961f4ba3 otherFunctions
## 10: 12621c86ae216006b9f58733961f4ba3 otherFunctions
## 11: 12621c86ae216006b9f58733961f4ba3 otherFunctions
## 12: 12621c86ae216006b9f58733961f4ba3 otherFunctions
## 13: 12621c86ae216006b9f58733961f4ba3 otherFunctions
## 14: 12621c86ae216006b9f58733961f4ba3 otherFunctions
## 15: 12621c86ae216006b9f58733961f4ba3 otherFunctions
## 16: 12621c86ae216006b9f58733961f4ba3 otherFunctions
## 17: 12621c86ae216006b9f58733961f4ba3 otherFunctions
## 18: 12621c86ae216006b9f58733961f4ba3 otherFunctions
## 19: 12621c86ae216006b9f58733961f4ba3 otherFunctions
## 20: 12621c86ae216006b9f58733961f4ba3      preDigest
## 21: 12621c86ae216006b9f58733961f4ba3      preDigest
## 22: 12621c86ae216006b9f58733961f4ba3      preDigest
## 23: 8d2e7b676a4f835156b24fdeecb052e4         format
## 24: 8d2e7b676a4f835156b24fdeecb052e4           name
## 25: 8d2e7b676a4f835156b24fdeecb052e4          class
## 26: 8d2e7b676a4f835156b24fdeecb052e4           date
## 27: 8d2e7b676a4f835156b24fdeecb052e4        cacheId
## 28: 8d2e7b676a4f835156b24fdeecb052e4       function
## 29: 8d2e7b676a4f835156b24fdeecb052e4    object.size
## 30: 8d2e7b676a4f835156b24fdeecb052e4       accessed
## 31: 8d2e7b676a4f835156b24fdeecb052e4 otherFunctions
## 32: 8d2e7b676a4f835156b24fdeecb052e4 otherFunctions
## 33: 8d2e7b676a4f835156b24fdeecb052e4 otherFunctions
## 34: 8d2e7b676a4f835156b24fdeecb052e4 otherFunctions
## 35: 8d2e7b676a4f835156b24fdeecb052e4 otherFunctions
## 36: 8d2e7b676a4f835156b24fdeecb052e4 otherFunctions
## 37: 8d2e7b676a4f835156b24fdeecb052e4 otherFunctions
## 38: 8d2e7b676a4f835156b24fdeecb052e4 otherFunctions
## 39: 8d2e7b676a4f835156b24fdeecb052e4 otherFunctions
## 40: 8d2e7b676a4f835156b24fdeecb052e4 otherFunctions
## 41: 8d2e7b676a4f835156b24fdeecb052e4 otherFunctions
## 42: 8d2e7b676a4f835156b24fdeecb052e4      preDigest
## 43: 8d2e7b676a4f835156b24fdeecb052e4      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 10:58:07
##  2: 12621c86ae216006b9f58733961f4ba3 2019-01-28 10:58:07
##  3:                          numeric 2019-01-28 10:58:07
##  4:              2019-01-28 10:58:07 2019-01-28 10:58:07
##  5:                 4ac2b7c0f42d1e46 2019-01-28 10:58:07
##  6:                            rnorm 2019-01-28 10:58:07
##  7:                             1008 2019-01-28 10:58:07
##  8:              2019-01-28 10:58:07 2019-01-28 10:58:07
##  9:                          saveRDS 2019-01-28 10:58:07
## 10:                          do.call 2019-01-28 10:58:07
## 11:                     process_file 2019-01-28 10:58:07
## 12:                    process_group 2019-01-28 10:58:07
## 13:              process_group.block 2019-01-28 10:58:07
## 14:                       call_block 2019-01-28 10:58:07
## 15:                       block_exec 2019-01-28 10:58:07
## 16:                           in_dir 2019-01-28 10:58:07
## 17:                        timing_fn 2019-01-28 10:58:07
## 18:                           handle 2019-01-28 10:58:07
## 19:                      withVisible 2019-01-28 10:58:07
## 20:               n:7f12988bd88a0fb8 2019-01-28 10:58:07
## 21:            mean:22413394efd9f6a3 2019-01-28 10:58:07
## 22:            .FUN:4f604aa46882b368 2019-01-28 10:58:07
## 23:                              rda 2019-01-28 10:58:07
## 24: 8d2e7b676a4f835156b24fdeecb052e4 2019-01-28 10:58:07
## 25:                          numeric 2019-01-28 10:58:07
## 26:              2019-01-28 10:58:07 2019-01-28 10:58:07
## 27:                 a7af5367c13aba8f 2019-01-28 10:58:07
## 28:                            outer 2019-01-28 10:58:07
## 29:                             1008 2019-01-28 10:58:07
## 30:              2019-01-28 10:58:07 2019-01-28 10:58:07
## 31:                          saveRDS 2019-01-28 10:58:07
## 32:                          do.call 2019-01-28 10:58:07
## 33:                     process_file 2019-01-28 10:58:07
## 34:                    process_group 2019-01-28 10:58:07
## 35:              process_group.block 2019-01-28 10:58:07
## 36:                       call_block 2019-01-28 10:58:07
## 37:                       block_exec 2019-01-28 10:58:07
## 38:                           in_dir 2019-01-28 10:58:07
## 39:                        timing_fn 2019-01-28 10:58:07
## 40:                           handle 2019-01-28 10:58:07
## 41:                      withVisible 2019-01-28 10:58:07
## 42:               n:82dc709f2b91918a 2019-01-28 10:58:07
## 43:            .FUN:892a6afc47a63a90 2019-01-28 10:58:07
##                             tagValue         createdDate
showCache(tmpdir2) # 1 function call
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: c1dc03a39cfe5b295f9d8f50dcd6dea9         format
##  2: c1dc03a39cfe5b295f9d8f50dcd6dea9           name
##  3: c1dc03a39cfe5b295f9d8f50dcd6dea9          class
##  4: c1dc03a39cfe5b295f9d8f50dcd6dea9           date
##  5: c1dc03a39cfe5b295f9d8f50dcd6dea9        cacheId
##  6: c1dc03a39cfe5b295f9d8f50dcd6dea9       function
##  7: c1dc03a39cfe5b295f9d8f50dcd6dea9    object.size
##  8: c1dc03a39cfe5b295f9d8f50dcd6dea9       accessed
##  9: c1dc03a39cfe5b295f9d8f50dcd6dea9 otherFunctions
## 10: c1dc03a39cfe5b295f9d8f50dcd6dea9 otherFunctions
## 11: c1dc03a39cfe5b295f9d8f50dcd6dea9 otherFunctions
## 12: c1dc03a39cfe5b295f9d8f50dcd6dea9 otherFunctions
## 13: c1dc03a39cfe5b295f9d8f50dcd6dea9 otherFunctions
## 14: c1dc03a39cfe5b295f9d8f50dcd6dea9 otherFunctions
## 15: c1dc03a39cfe5b295f9d8f50dcd6dea9 otherFunctions
## 16: c1dc03a39cfe5b295f9d8f50dcd6dea9 otherFunctions
## 17: c1dc03a39cfe5b295f9d8f50dcd6dea9 otherFunctions
## 18: c1dc03a39cfe5b295f9d8f50dcd6dea9 otherFunctions
## 19: c1dc03a39cfe5b295f9d8f50dcd6dea9 otherFunctions
## 20: c1dc03a39cfe5b295f9d8f50dcd6dea9      preDigest
## 21: c1dc03a39cfe5b295f9d8f50dcd6dea9      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 10:58:07
##  2: c1dc03a39cfe5b295f9d8f50dcd6dea9 2019-01-28 10:58:07
##  3:                          numeric 2019-01-28 10:58:07
##  4:              2019-01-28 10:58:07 2019-01-28 10:58:07
##  5:                 33ceb4fb525fd08f 2019-01-28 10:58:07
##  6:                            inner 2019-01-28 10:58:07
##  7:                             1008 2019-01-28 10:58:07
##  8:              2019-01-28 10:58:07 2019-01-28 10:58:07
##  9:                          saveRDS 2019-01-28 10:58:07
## 10:                          do.call 2019-01-28 10:58:07
## 11:                     process_file 2019-01-28 10:58:07
## 12:                    process_group 2019-01-28 10:58:07
## 13:              process_group.block 2019-01-28 10:58:07
## 14:                       call_block 2019-01-28 10:58:07
## 15:                       block_exec 2019-01-28 10:58:07
## 16:                           in_dir 2019-01-28 10:58:07
## 17:                        timing_fn 2019-01-28 10:58:07
## 18:                           handle 2019-01-28 10:58:07
## 19:                      withVisible 2019-01-28 10:58:07
## 20:            mean:22413394efd9f6a3 2019-01-28 10:58:07
## 21:            .FUN:87e2c30917a34d25 2019-01-28 10:58:07
##                             tagValue         createdDate
# userTags get appended
# all items have the outer tag propagate, plus inner ones only have inner ones
clearCache(tmpdir1, ask = FALSE)
outerTag <- "outerTag"
innerTag <- "innerTag"
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean, notOlderThan = Sys.time() - 1e5, userTags = innerTag)
}
outer <- function(n) {
  Cache(inner, 0.1)
}
aa <- Cache(outer, n = 2, cacheRepo = tmpdir1, userTags = outerTag)
showCache(tmpdir1) # rnorm function has outerTag and innerTag, inner and outer only have outerTag
## Cache size:
##   Total (including Rasters): 756 bytes
##   Selected objects (not including Rasters): 756 bytes
##                             artifact         tagKey
##  1: 342c2bb66d4d266eac07df0899b6b912         format
##  2: 342c2bb66d4d266eac07df0899b6b912           name
##  3: 342c2bb66d4d266eac07df0899b6b912          class
##  4: 342c2bb66d4d266eac07df0899b6b912           date
##  5: 342c2bb66d4d266eac07df0899b6b912        cacheId
##  6: 342c2bb66d4d266eac07df0899b6b912       innerTag
##  7: 342c2bb66d4d266eac07df0899b6b912       outerTag
##  8: 342c2bb66d4d266eac07df0899b6b912       function
##  9: 342c2bb66d4d266eac07df0899b6b912    object.size
## 10: 342c2bb66d4d266eac07df0899b6b912       accessed
## 11: 342c2bb66d4d266eac07df0899b6b912 otherFunctions
## 12: 342c2bb66d4d266eac07df0899b6b912 otherFunctions
## 13: 342c2bb66d4d266eac07df0899b6b912 otherFunctions
## 14: 342c2bb66d4d266eac07df0899b6b912 otherFunctions
## 15: 342c2bb66d4d266eac07df0899b6b912 otherFunctions
## 16: 342c2bb66d4d266eac07df0899b6b912 otherFunctions
## 17: 342c2bb66d4d266eac07df0899b6b912 otherFunctions
## 18: 342c2bb66d4d266eac07df0899b6b912 otherFunctions
## 19: 342c2bb66d4d266eac07df0899b6b912 otherFunctions
## 20: 342c2bb66d4d266eac07df0899b6b912 otherFunctions
## 21: 342c2bb66d4d266eac07df0899b6b912 otherFunctions
## 22: 342c2bb66d4d266eac07df0899b6b912      preDigest
## 23: 342c2bb66d4d266eac07df0899b6b912      preDigest
## 24: 342c2bb66d4d266eac07df0899b6b912      preDigest
## 25: 6ed926b6b4c69abb3de3c1e92a43c089         format
## 26: 6ed926b6b4c69abb3de3c1e92a43c089           name
## 27: 6ed926b6b4c69abb3de3c1e92a43c089          class
## 28: 6ed926b6b4c69abb3de3c1e92a43c089           date
## 29: 6ed926b6b4c69abb3de3c1e92a43c089        cacheId
## 30: 6ed926b6b4c69abb3de3c1e92a43c089       outerTag
## 31: 6ed926b6b4c69abb3de3c1e92a43c089       function
## 32: 6ed926b6b4c69abb3de3c1e92a43c089    object.size
## 33: 6ed926b6b4c69abb3de3c1e92a43c089       accessed
## 34: 6ed926b6b4c69abb3de3c1e92a43c089 otherFunctions
## 35: 6ed926b6b4c69abb3de3c1e92a43c089 otherFunctions
## 36: 6ed926b6b4c69abb3de3c1e92a43c089 otherFunctions
## 37: 6ed926b6b4c69abb3de3c1e92a43c089 otherFunctions
## 38: 6ed926b6b4c69abb3de3c1e92a43c089 otherFunctions
## 39: 6ed926b6b4c69abb3de3c1e92a43c089 otherFunctions
## 40: 6ed926b6b4c69abb3de3c1e92a43c089 otherFunctions
## 41: 6ed926b6b4c69abb3de3c1e92a43c089 otherFunctions
## 42: 6ed926b6b4c69abb3de3c1e92a43c089 otherFunctions
## 43: 6ed926b6b4c69abb3de3c1e92a43c089 otherFunctions
## 44: 6ed926b6b4c69abb3de3c1e92a43c089 otherFunctions
## 45: 6ed926b6b4c69abb3de3c1e92a43c089      preDigest
## 46: 6ed926b6b4c69abb3de3c1e92a43c089      preDigest
## 47: 77480e8354cf8b9c88cc5870d1937700         format
## 48: 77480e8354cf8b9c88cc5870d1937700           name
## 49: 77480e8354cf8b9c88cc5870d1937700          class
## 50: 77480e8354cf8b9c88cc5870d1937700           date
## 51: 77480e8354cf8b9c88cc5870d1937700        cacheId
## 52: 77480e8354cf8b9c88cc5870d1937700       outerTag
## 53: 77480e8354cf8b9c88cc5870d1937700       function
## 54: 77480e8354cf8b9c88cc5870d1937700    object.size
## 55: 77480e8354cf8b9c88cc5870d1937700       accessed
## 56: 77480e8354cf8b9c88cc5870d1937700 otherFunctions
## 57: 77480e8354cf8b9c88cc5870d1937700 otherFunctions
## 58: 77480e8354cf8b9c88cc5870d1937700 otherFunctions
## 59: 77480e8354cf8b9c88cc5870d1937700 otherFunctions
## 60: 77480e8354cf8b9c88cc5870d1937700 otherFunctions
## 61: 77480e8354cf8b9c88cc5870d1937700 otherFunctions
## 62: 77480e8354cf8b9c88cc5870d1937700 otherFunctions
## 63: 77480e8354cf8b9c88cc5870d1937700 otherFunctions
## 64: 77480e8354cf8b9c88cc5870d1937700 otherFunctions
## 65: 77480e8354cf8b9c88cc5870d1937700 otherFunctions
## 66: 77480e8354cf8b9c88cc5870d1937700 otherFunctions
## 67: 77480e8354cf8b9c88cc5870d1937700      preDigest
## 68: 77480e8354cf8b9c88cc5870d1937700      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 10:58:07
##  2: 342c2bb66d4d266eac07df0899b6b912 2019-01-28 10:58:07
##  3:                          numeric 2019-01-28 10:58:07
##  4:              2019-01-28 10:58:07 2019-01-28 10:58:07
##  5:                 4ac2b7c0f42d1e46 2019-01-28 10:58:07
##  6:                         innerTag 2019-01-28 10:58:07
##  7:                         outerTag 2019-01-28 10:58:07
##  8:                            rnorm 2019-01-28 10:58:07
##  9:                             1008 2019-01-28 10:58:07
## 10:              2019-01-28 10:58:07 2019-01-28 10:58:07
## 11:                          saveRDS 2019-01-28 10:58:07
## 12:                          do.call 2019-01-28 10:58:07
## 13:                     process_file 2019-01-28 10:58:07
## 14:                    process_group 2019-01-28 10:58:07
## 15:              process_group.block 2019-01-28 10:58:07
## 16:                       call_block 2019-01-28 10:58:07
## 17:                       block_exec 2019-01-28 10:58:07
## 18:                           in_dir 2019-01-28 10:58:07
## 19:                        timing_fn 2019-01-28 10:58:07
## 20:                           handle 2019-01-28 10:58:07
## 21:                      withVisible 2019-01-28 10:58:07
## 22:               n:7f12988bd88a0fb8 2019-01-28 10:58:07
## 23:            mean:22413394efd9f6a3 2019-01-28 10:58:07
## 24:            .FUN:4f604aa46882b368 2019-01-28 10:58:07
## 25:                              rda 2019-01-28 10:58:07
## 26: 6ed926b6b4c69abb3de3c1e92a43c089 2019-01-28 10:58:07
## 27:                          numeric 2019-01-28 10:58:07
## 28:              2019-01-28 10:58:07 2019-01-28 10:58:07
## 29:                 88a34e1d033329e5 2019-01-28 10:58:07
## 30:                         outerTag 2019-01-28 10:58:07
## 31:                            outer 2019-01-28 10:58:07
## 32:                             1008 2019-01-28 10:58:07
## 33:              2019-01-28 10:58:07 2019-01-28 10:58:07
## 34:                          saveRDS 2019-01-28 10:58:07
## 35:                          do.call 2019-01-28 10:58:07
## 36:                     process_file 2019-01-28 10:58:07
## 37:                    process_group 2019-01-28 10:58:07
## 38:              process_group.block 2019-01-28 10:58:07
## 39:                       call_block 2019-01-28 10:58:07
## 40:                       block_exec 2019-01-28 10:58:07
## 41:                           in_dir 2019-01-28 10:58:07
## 42:                        timing_fn 2019-01-28 10:58:07
## 43:                           handle 2019-01-28 10:58:07
## 44:                      withVisible 2019-01-28 10:58:07
## 45:               n:82dc709f2b91918a 2019-01-28 10:58:07
## 46:            .FUN:5f06fb5fbffe9e3b 2019-01-28 10:58:07
## 47:                              rda 2019-01-28 10:58:07
## 48: 77480e8354cf8b9c88cc5870d1937700 2019-01-28 10:58:07
## 49:                          numeric 2019-01-28 10:58:07
## 50:              2019-01-28 10:58:07 2019-01-28 10:58:07
## 51:                 b06af03d5a73dc7d 2019-01-28 10:58:07
## 52:                         outerTag 2019-01-28 10:58:07
## 53:                            inner 2019-01-28 10:58:07
## 54:                             1008 2019-01-28 10:58:07
## 55:              2019-01-28 10:58:07 2019-01-28 10:58:07
## 56:                          saveRDS 2019-01-28 10:58:07
## 57:                          do.call 2019-01-28 10:58:07
## 58:                     process_file 2019-01-28 10:58:07
## 59:                    process_group 2019-01-28 10:58:07
## 60:              process_group.block 2019-01-28 10:58:07
## 61:                       call_block 2019-01-28 10:58:07
## 62:                       block_exec 2019-01-28 10:58:07
## 63:                           in_dir 2019-01-28 10:58:07
## 64:                        timing_fn 2019-01-28 10:58:07
## 65:                           handle 2019-01-28 10:58:07
## 66:                      withVisible 2019-01-28 10:58:07
## 67:            mean:22413394efd9f6a3 2019-01-28 10:58:07
## 68:            .FUN:7ad10bc1ae528d8c 2019-01-28 10:58:07
##                             tagValue         createdDate

cacheId

Sometimes, it is not absolutely desirable to maintain the work flow intact because changes that are irrelevant to the analysis, such as changing messages sent to a user, may be changed, without a desire to rerun functions. The cacheId argument is for this. Once a piece of code is run, then the cacheId can be manually extracted (it is reported at the end of a Cache call) and manually placed in the code, passed in as, say, cacheId = "ad184ce64541972b50afd8e7b75f821b".

### cacheId
set.seed(1)
Cache(rnorm, 1, cacheRepo = tmpdir1)
## [1] -0.6264538
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:7072c305d8c69df0"
## attr(,"call")
## [1] ""
# manually look at output attribute which shows cacheId: ad184ce64541972b50afd8e7b75f821b
Cache(rnorm, 1, cacheRepo = tmpdir1, cacheId = "ad184ce64541972b50afd8e7b75f821b") # same value
## cacheId is not same as calculated hash. Manually searching for cacheId:ad184ce64541972b50afd8e7b75f821b
## [1] 0.1836433
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:ad184ce64541972b50afd8e7b75f821b"
## attr(,"call")
## [1] ""
# override even with different inputs:
Cache(rnorm, 2, cacheRepo = tmpdir1, cacheId = "ad184ce64541972b50afd8e7b75f821b")
## cacheId is not same as calculated hash. Manually searching for cacheId:ad184ce64541972b50afd8e7b75f821b
##   loading cached result from previous rnorm call, adding to memoised copy
## [1] 0.1836433
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] FALSE
## 
## attr(,"tags")
## [1] "cacheId:ad184ce64541972b50afd8e7b75f821b"
## attr(,"call")
## [1] ""
## cleanup
unlink(c("filename.rda", "filename1.rda"))

Working with the Cache manually

Since the cache is simply an archivist repository, all archivist functions will work as is. In addition, there are several helpers in the reproducible package, including showCache, keepCache and clearCache that may be useful. Also, one can access cached items manually (rather than simply rerunning the same Cache function again).

if (requireNamespace("archivist")) {
  # get the RasterLayer that was produced with the gaussMap function:
  mapHash <- unique(showCache(tmpDir, userTags = "projectRaster")$artifact)
  map <- archivist::loadFromLocalRepo(md5hash = mapHash[1], repoDir = tmpDir, value = TRUE)
  
  plot(map)
}
## Cache size:
##   Total (including Rasters): 3.3 Kb
##   Selected objects (not including Rasters): 3.3 Kb

## cleanup
unlink(dirname(tmpDir), recursive = TRUE)

Reproducible Workflow

In general, we feel that a liberal use of Cache will make a re-usable and reproducible work flow. shiny apps can be made, taking advantage of Cache. Indeed, much of the difficulty in managing data sets and saving them for future use, can be accommodated by caching.

Nested Caching

Cache individual functions

Cache(<functionName>, <other arguments>)

This will allow fine scale control of individual function calls.