Introduction to Cache

Eliot J. B. McIntire

May 19 2020

Source: vignettes/Intro-to-Cache.Rmd

Reproducible workflow

As part of a reproducible workflow, caching of function calls, code chunks, and other elements of a project is a critical component. The objective of a reproducible workflow is is likely that an entire work flow from raw data to publication, decision support, report writing, presentation building etc., could be built and be reproducible anywhere, on any computer, operating system, with any starting conditions, on demand. The reproducible::Cache function is built to work with any R function.

Differences with other approaches

Cache users DBI as a backend, with key functions, dbReadTable, dbRemoveTable, dbSendQuery, dbSendStatement, dbCreateTable and dbAppendTable. These can all be accessed via Cache, showCache, clearCache, and keepCache. It is optimized for speed of transactions, using fastdigest::fastdigest on R memory objects and digest::digest on files. The main function is superficially similar to archivist::cache, which uses digest::digest in all cases to determine whether the arguments are identical in subsequent iterations. It also but does many things that make standard caching with digest::digest don’t work reliably between systems. For these, the function .robustDigest is introduced to make caching transferable between systems. This is relevant for file paths, environments, parallel clusters, functions (which are contained within an environment), and many others (e.g., see ?.robustDigest for methods). Cache also adds important elements like automated tagging and the option to retrieve disk-cached values via stashed objects in memory using memoise::memoise. This means that running Cache 1, 2, and 3 times on the same function will get progressively faster. This can be extremely useful for web apps built with, say shiny.

Function-level caching

Any function can be cached using: Cache(FUN = functionName, ...).

This will be a slight change to a function call, such as: projectRaster(raster, crs = crs(newRaster)) to Cache(projectRaster, raster, crs = crs(newRaster)).

This is particularly useful for expensive operations.

library(raster)
## Loading required package: sp
library(reproducible)

tmpDir <- file.path(tempdir(), "reproducible_examples", "Cache")
checkPath(tmpDir, create = TRUE)
## [1] "/tmp/RtmpRhVjii/reproducible_examples/Cache"
ras <- raster(extent(0, 1000, 0, 1000), vals = 1:1e6, res = 1)
crs(ras) <- "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84"

newCRS <- "+init=epsg:4326 +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0"
# No Cache
system.time(map1 <- projectRaster(ras, crs = newCRS))
##    user  system elapsed 
##   1.971   0.199   2.174
# Try with memoise for this example -- for many simple cases, memoising will not be faster
opts <- options("reproducible.useMemoise" = TRUE)
# With Cache -- a little slower the first time because saving to disk
system.time(map1 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir,
                         notOlderThan = Sys.time()))
##    user  system elapsed 
##   2.581   0.141   2.966
# vastly faster the second time
system.time(map2 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
##   ...(Object to retrieve (88249b4e1af6b3a2.rds) is large: 6.5 Mb)
##      loaded cached result from previous projectRaster call, adding to memoised copy...
##    user  system elapsed 
##   0.533   0.006   0.472
# may be faster the third time because of memoise; but this example is too simple to show
system.time(map3 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
##   ...(Object to retrieve (88249b4e1af6b3a2.rds) is large: 6.5 Mb)
##      loaded memoised result from previous projectRaster call,
##    user  system elapsed 
##   0.589   0.000   0.549
options(opts)

all.equal(map1, map2) # TRUE
## [1] TRUE
all.equal(map1, map3) # TRUE
## [1] TRUE

Caching examples

Basic use

library(raster)
library(magrittr)
## 
## Attaching package: 'magrittr'
## The following object is masked from 'package:raster':
## 
##     extract
try(clearCache(tmpDir, ask = FALSE), silent = TRUE) # just to make sure it is clear

ranNumsA <- Cache(rnorm, 10, 16, cacheRepo = tmpDir)

# All same
ranNumsB <- Cache(rnorm, 10, 16, cacheRepo = tmpDir) # recovers cached copy
##   ...(Object to retrieve (f11fb1a2880f8060.rds))
##      loaded cached result from previous rnorm call,
ranNumsC <- Cache(cacheRepo = tmpDir) %C% rnorm(10, 16)  # recovers cached copy
##   ...(Object to retrieve (f11fb1a2880f8060.rds))
##      loaded cached result from previous 'rnorm' pipe sequence call,
ranNumsD <- Cache(quote(rnorm(n = 10, 16)), cacheRepo = tmpDir) # recovers cached copy
##   ...(Object to retrieve (f11fb1a2880f8060.rds))
##      loaded cached result from previous rnorm call,
# Any minor change makes it different
ranNumsE <- Cache(cacheRepo = tmpDir) %C% rnorm(10, 6) # different

Example 1: Basic cache use with tags

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

showCache(tmpDir, userTags = c("objectName"))
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##              cacheId              tagKey              tagValue
##  1: 3aef38d1fc02aee5          objectName                     b
##  2: 3aef38d1fc02aee5            function                 runif
##  3: 3aef38d1fc02aee5               class               numeric
##  4: 3aef38d1fc02aee5         object.size                  1008
##  5: 3aef38d1fc02aee5            accessed   2020-05-19 10:14:42
##  6: 3aef38d1fc02aee5             inCloud                 FALSE
##  7: 3aef38d1fc02aee5          resultHash                      
##  8: 3aef38d1fc02aee5   elapsedTimeDigest     0.0008673668 secs
##  9: 3aef38d1fc02aee5 elapsedTimeFirstRun     1.740456e-05 secs
## 10: 3aef38d1fc02aee5      otherFunctions               saveRDS
## 11: 3aef38d1fc02aee5      otherFunctions               do.call
## 12: 3aef38d1fc02aee5      otherFunctions          process_file
## 13: 3aef38d1fc02aee5      otherFunctions         process_group
## 14: 3aef38d1fc02aee5      otherFunctions   process_group.block
## 15: 3aef38d1fc02aee5      otherFunctions            call_block
## 16: 3aef38d1fc02aee5      otherFunctions            block_exec
## 17: 3aef38d1fc02aee5      otherFunctions                in_dir
## 18: 3aef38d1fc02aee5      otherFunctions             timing_fn
## 19: 3aef38d1fc02aee5      otherFunctions                handle
## 20: 3aef38d1fc02aee5           preDigest    n:7eef4eae85fd9229
## 21: 3aef38d1fc02aee5           preDigest .FUN:881ec847b7161f3c
## 22: 3aef38d1fc02aee5           file.size                   167
## 23: f7bee22047b8d0c1          objectName                     a
## 24: f7bee22047b8d0c1            function                 rnorm
## 25: f7bee22047b8d0c1               class               numeric
## 26: f7bee22047b8d0c1         object.size                  1008
## 27: f7bee22047b8d0c1            accessed   2020-05-19 10:14:42
## 28: f7bee22047b8d0c1             inCloud                 FALSE
## 29: f7bee22047b8d0c1          resultHash                      
## 30: f7bee22047b8d0c1   elapsedTimeDigest     0.0005731583 secs
## 31: f7bee22047b8d0c1 elapsedTimeFirstRun     2.074242e-05 secs
## 32: f7bee22047b8d0c1      otherFunctions               saveRDS
## 33: f7bee22047b8d0c1      otherFunctions               do.call
## 34: f7bee22047b8d0c1      otherFunctions          process_file
## 35: f7bee22047b8d0c1      otherFunctions         process_group
## 36: f7bee22047b8d0c1      otherFunctions   process_group.block
## 37: f7bee22047b8d0c1      otherFunctions            call_block
## 38: f7bee22047b8d0c1      otherFunctions            block_exec
## 39: f7bee22047b8d0c1      otherFunctions                in_dir
## 40: f7bee22047b8d0c1      otherFunctions             timing_fn
## 41: f7bee22047b8d0c1      otherFunctions                handle
## 42: f7bee22047b8d0c1           preDigest    n:7eef4eae85fd9229
## 43: f7bee22047b8d0c1           preDigest .FUN:4f604aa46882b368
## 44: f7bee22047b8d0c1           file.size                   173
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2020-05-19 10:14:42
##  2: 2020-05-19 10:14:42
##  3: 2020-05-19 10:14:42
##  4: 2020-05-19 10:14:42
##  5: 2020-05-19 10:14:42
##  6: 2020-05-19 10:14:42
##  7: 2020-05-19 10:14:42
##  8: 2020-05-19 10:14:42
##  9: 2020-05-19 10:14:42
## 10: 2020-05-19 10:14:42
## 11: 2020-05-19 10:14:42
## 12: 2020-05-19 10:14:42
## 13: 2020-05-19 10:14:42
## 14: 2020-05-19 10:14:42
## 15: 2020-05-19 10:14:42
## 16: 2020-05-19 10:14:42
## 17: 2020-05-19 10:14:42
## 18: 2020-05-19 10:14:42
## 19: 2020-05-19 10:14:42
## 20: 2020-05-19 10:14:42
## 21: 2020-05-19 10:14:42
## 22: 2020-05-19 10:14:42
## 23: 2020-05-19 10:14:42
## 24: 2020-05-19 10:14:42
## 25: 2020-05-19 10:14:42
## 26: 2020-05-19 10:14:42
## 27: 2020-05-19 10:14:42
## 28: 2020-05-19 10:14:42
## 29: 2020-05-19 10:14:42
## 30: 2020-05-19 10:14:42
## 31: 2020-05-19 10:14:42
## 32: 2020-05-19 10:14:42
## 33: 2020-05-19 10:14:42
## 34: 2020-05-19 10:14:42
## 35: 2020-05-19 10:14:42
## 36: 2020-05-19 10:14:42
## 37: 2020-05-19 10:14:42
## 38: 2020-05-19 10:14:42
## 39: 2020-05-19 10:14:42
## 40: 2020-05-19 10:14:42
## 41: 2020-05-19 10:14:42
## 42: 2020-05-19 10:14:42
## 43: 2020-05-19 10:14:42
## 44: 2020-05-19 10:14:42
##             createdDate
showCache(tmpDir, userTags = c("^a$")) # regular expression ... "a" exactly
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
##              cacheId              tagKey              tagValue
##  1: f7bee22047b8d0c1          objectName                     a
##  2: f7bee22047b8d0c1            function                 rnorm
##  3: f7bee22047b8d0c1               class               numeric
##  4: f7bee22047b8d0c1         object.size                  1008
##  5: f7bee22047b8d0c1            accessed   2020-05-19 10:14:42
##  6: f7bee22047b8d0c1             inCloud                 FALSE
##  7: f7bee22047b8d0c1          resultHash                      
##  8: f7bee22047b8d0c1   elapsedTimeDigest     0.0005731583 secs
##  9: f7bee22047b8d0c1 elapsedTimeFirstRun     2.074242e-05 secs
## 10: f7bee22047b8d0c1      otherFunctions               saveRDS
## 11: f7bee22047b8d0c1      otherFunctions               do.call
## 12: f7bee22047b8d0c1      otherFunctions          process_file
## 13: f7bee22047b8d0c1      otherFunctions         process_group
## 14: f7bee22047b8d0c1      otherFunctions   process_group.block
## 15: f7bee22047b8d0c1      otherFunctions            call_block
## 16: f7bee22047b8d0c1      otherFunctions            block_exec
## 17: f7bee22047b8d0c1      otherFunctions                in_dir
## 18: f7bee22047b8d0c1      otherFunctions             timing_fn
## 19: f7bee22047b8d0c1      otherFunctions                handle
## 20: f7bee22047b8d0c1           preDigest    n:7eef4eae85fd9229
## 21: f7bee22047b8d0c1           preDigest .FUN:4f604aa46882b368
## 22: f7bee22047b8d0c1           file.size                   173
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2020-05-19 10:14:42
##  2: 2020-05-19 10:14:42
##  3: 2020-05-19 10:14:42
##  4: 2020-05-19 10:14:42
##  5: 2020-05-19 10:14:42
##  6: 2020-05-19 10:14:42
##  7: 2020-05-19 10:14:42
##  8: 2020-05-19 10:14:42
##  9: 2020-05-19 10:14:42
## 10: 2020-05-19 10:14:42
## 11: 2020-05-19 10:14:42
## 12: 2020-05-19 10:14:42
## 13: 2020-05-19 10:14:42
## 14: 2020-05-19 10:14:42
## 15: 2020-05-19 10:14:42
## 16: 2020-05-19 10:14:42
## 17: 2020-05-19 10:14:42
## 18: 2020-05-19 10:14:42
## 19: 2020-05-19 10:14:42
## 20: 2020-05-19 10:14:42
## 21: 2020-05-19 10:14:42
## 22: 2020-05-19 10:14:42
##             createdDate
showCache(tmpDir, userTags = c("runif")) # show only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
##              cacheId              tagKey              tagValue
##  1: 3aef38d1fc02aee5          objectName                     b
##  2: 3aef38d1fc02aee5            function                 runif
##  3: 3aef38d1fc02aee5               class               numeric
##  4: 3aef38d1fc02aee5         object.size                  1008
##  5: 3aef38d1fc02aee5            accessed   2020-05-19 10:14:42
##  6: 3aef38d1fc02aee5             inCloud                 FALSE
##  7: 3aef38d1fc02aee5          resultHash                      
##  8: 3aef38d1fc02aee5   elapsedTimeDigest     0.0008673668 secs
##  9: 3aef38d1fc02aee5 elapsedTimeFirstRun     1.740456e-05 secs
## 10: 3aef38d1fc02aee5      otherFunctions               saveRDS
## 11: 3aef38d1fc02aee5      otherFunctions               do.call
## 12: 3aef38d1fc02aee5      otherFunctions          process_file
## 13: 3aef38d1fc02aee5      otherFunctions         process_group
## 14: 3aef38d1fc02aee5      otherFunctions   process_group.block
## 15: 3aef38d1fc02aee5      otherFunctions            call_block
## 16: 3aef38d1fc02aee5      otherFunctions            block_exec
## 17: 3aef38d1fc02aee5      otherFunctions                in_dir
## 18: 3aef38d1fc02aee5      otherFunctions             timing_fn
## 19: 3aef38d1fc02aee5      otherFunctions                handle
## 20: 3aef38d1fc02aee5           preDigest    n:7eef4eae85fd9229
## 21: 3aef38d1fc02aee5           preDigest .FUN:881ec847b7161f3c
## 22: 3aef38d1fc02aee5           file.size                   167
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2020-05-19 10:14:42
##  2: 2020-05-19 10:14:42
##  3: 2020-05-19 10:14:42
##  4: 2020-05-19 10:14:42
##  5: 2020-05-19 10:14:42
##  6: 2020-05-19 10:14:42
##  7: 2020-05-19 10:14:42
##  8: 2020-05-19 10:14:42
##  9: 2020-05-19 10:14:42
## 10: 2020-05-19 10:14:42
## 11: 2020-05-19 10:14:42
## 12: 2020-05-19 10:14:42
## 13: 2020-05-19 10:14:42
## 14: 2020-05-19 10:14:42
## 15: 2020-05-19 10:14:42
## 16: 2020-05-19 10:14:42
## 17: 2020-05-19 10:14:42
## 18: 2020-05-19 10:14:42
## 19: 2020-05-19 10:14:42
## 20: 2020-05-19 10:14:42
## 21: 2020-05-19 10:14:42
## 22: 2020-05-19 10:14:42
##             createdDate
clearCache(tmpDir, userTags = c("runif"), ask = FALSE) # remove only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) # only those made during rnorm call
## Cache size:
##   Total (including Rasters): 804 bytes
##   Selected objects (not including Rasters): 804 bytes
##              cacheId              tagKey              tagValue
##  1: f11fb1a2880f8060            function                 rnorm
##  2: f11fb1a2880f8060               class               numeric
##  3: f11fb1a2880f8060         object.size                  1104
##  4: f11fb1a2880f8060            accessed   2020-05-19 10:14:41
##  5: f11fb1a2880f8060             inCloud                 FALSE
##  6: f11fb1a2880f8060          resultHash                      
##  7: f11fb1a2880f8060   elapsedTimeDigest      0.001351357 secs
##  8: f11fb1a2880f8060 elapsedTimeFirstRun     8.296967e-05 secs
##  9: f11fb1a2880f8060      otherFunctions               saveRDS
## 10: f11fb1a2880f8060      otherFunctions               do.call
## 11: f11fb1a2880f8060      otherFunctions          process_file
## 12: f11fb1a2880f8060      otherFunctions         process_group
## 13: f11fb1a2880f8060      otherFunctions   process_group.block
## 14: f11fb1a2880f8060      otherFunctions            call_block
## 15: f11fb1a2880f8060      otherFunctions            block_exec
## 16: f11fb1a2880f8060      otherFunctions                in_dir
## 17: f11fb1a2880f8060      otherFunctions             timing_fn
## 18: f11fb1a2880f8060      otherFunctions                handle
## 19: f11fb1a2880f8060           preDigest    n:c5775c3b366fb719
## 20: f11fb1a2880f8060           preDigest mean:15620f138033a66c
## 21: f11fb1a2880f8060           preDigest .FUN:4f604aa46882b368
## 22: f11fb1a2880f8060           file.size                   222
## 23: f11fb1a2880f8060            accessed   2020-05-19 10:14:41
## 24: f11fb1a2880f8060     elapsedTimeLoad        0.0127027 secs
## 25: f11fb1a2880f8060            accessed   2020-05-19 10:14:41
## 26: f11fb1a2880f8060            accessed   2020-05-19 10:14:41
## 27: c16f0e2c57596069            function 'rnorm' pipe sequence
## 28: c16f0e2c57596069               class               numeric
## 29: c16f0e2c57596069         object.size                  1104
## 30: c16f0e2c57596069            accessed   2020-05-19 10:14:41
## 31: c16f0e2c57596069             inCloud                 FALSE
## 32: c16f0e2c57596069          resultHash                      
## 33: c16f0e2c57596069   elapsedTimeDigest     0.0007073879 secs
## 34: c16f0e2c57596069 elapsedTimeFirstRun     2.813339e-05 secs
## 35: c16f0e2c57596069      otherFunctions               saveRDS
## 36: c16f0e2c57596069      otherFunctions               do.call
## 37: c16f0e2c57596069      otherFunctions          process_file
## 38: c16f0e2c57596069      otherFunctions         process_group
## 39: c16f0e2c57596069      otherFunctions   process_group.block
## 40: c16f0e2c57596069      otherFunctions            call_block
## 41: c16f0e2c57596069      otherFunctions            block_exec
## 42: c16f0e2c57596069      otherFunctions                in_dir
## 43: c16f0e2c57596069      otherFunctions             timing_fn
## 44: c16f0e2c57596069      otherFunctions                handle
## 45: c16f0e2c57596069      otherFunctions                   %C%
## 46: c16f0e2c57596069           preDigest    n:c5775c3b366fb719
## 47: c16f0e2c57596069           preDigest mean:152602b8ff81e5bb
## 48: c16f0e2c57596069           preDigest .FUN:4f604aa46882b368
## 49: c16f0e2c57596069           file.size                   223
## 50: f7bee22047b8d0c1          objectName                     a
## 51: f7bee22047b8d0c1            function                 rnorm
## 52: f7bee22047b8d0c1               class               numeric
## 53: f7bee22047b8d0c1         object.size                  1008
## 54: f7bee22047b8d0c1            accessed   2020-05-19 10:14:42
## 55: f7bee22047b8d0c1             inCloud                 FALSE
## 56: f7bee22047b8d0c1          resultHash                      
## 57: f7bee22047b8d0c1   elapsedTimeDigest     0.0005731583 secs
## 58: f7bee22047b8d0c1 elapsedTimeFirstRun     2.074242e-05 secs
## 59: f7bee22047b8d0c1      otherFunctions               saveRDS
## 60: f7bee22047b8d0c1      otherFunctions               do.call
## 61: f7bee22047b8d0c1      otherFunctions          process_file
## 62: f7bee22047b8d0c1      otherFunctions         process_group
## 63: f7bee22047b8d0c1      otherFunctions   process_group.block
## 64: f7bee22047b8d0c1      otherFunctions            call_block
## 65: f7bee22047b8d0c1      otherFunctions            block_exec
## 66: f7bee22047b8d0c1      otherFunctions                in_dir
## 67: f7bee22047b8d0c1      otherFunctions             timing_fn
## 68: f7bee22047b8d0c1      otherFunctions                handle
## 69: f7bee22047b8d0c1           preDigest    n:7eef4eae85fd9229
## 70: f7bee22047b8d0c1           preDigest .FUN:4f604aa46882b368
## 71: f7bee22047b8d0c1           file.size                   173
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2020-05-19 10:14:41
##  2: 2020-05-19 10:14:41
##  3: 2020-05-19 10:14:41
##  4: 2020-05-19 10:14:41
##  5: 2020-05-19 10:14:41
##  6: 2020-05-19 10:14:41
##  7: 2020-05-19 10:14:41
##  8: 2020-05-19 10:14:41
##  9: 2020-05-19 10:14:41
## 10: 2020-05-19 10:14:41
## 11: 2020-05-19 10:14:41
## 12: 2020-05-19 10:14:41
## 13: 2020-05-19 10:14:41
## 14: 2020-05-19 10:14:41
## 15: 2020-05-19 10:14:41
## 16: 2020-05-19 10:14:41
## 17: 2020-05-19 10:14:41
## 18: 2020-05-19 10:14:41
## 19: 2020-05-19 10:14:41
## 20: 2020-05-19 10:14:41
## 21: 2020-05-19 10:14:41
## 22: 2020-05-19 10:14:41
## 23: 2020-05-19 10:14:41
## 24: 2020-05-19 10:14:41
## 25: 2020-05-19 10:14:41
## 26: 2020-05-19 10:14:41
## 27: 2020-05-19 10:14:41
## 28: 2020-05-19 10:14:41
## 29: 2020-05-19 10:14:41
## 30: 2020-05-19 10:14:41
## 31: 2020-05-19 10:14:41
## 32: 2020-05-19 10:14:41
## 33: 2020-05-19 10:14:41
## 34: 2020-05-19 10:14:41
## 35: 2020-05-19 10:14:41
## 36: 2020-05-19 10:14:41
## 37: 2020-05-19 10:14:41
## 38: 2020-05-19 10:14:41
## 39: 2020-05-19 10:14:41
## 40: 2020-05-19 10:14:41
## 41: 2020-05-19 10:14:41
## 42: 2020-05-19 10:14:41
## 43: 2020-05-19 10:14:41
## 44: 2020-05-19 10:14:41
## 45: 2020-05-19 10:14:41
## 46: 2020-05-19 10:14:41
## 47: 2020-05-19 10:14:41
## 48: 2020-05-19 10:14:41
## 49: 2020-05-19 10:14:41
## 50: 2020-05-19 10:14:42
## 51: 2020-05-19 10:14:42
## 52: 2020-05-19 10:14:42
## 53: 2020-05-19 10:14:42
## 54: 2020-05-19 10:14:42
## 55: 2020-05-19 10:14:42
## 56: 2020-05-19 10:14:42
## 57: 2020-05-19 10:14:42
## 58: 2020-05-19 10:14:42
## 59: 2020-05-19 10:14:42
## 60: 2020-05-19 10:14:42
## 61: 2020-05-19 10:14:42
## 62: 2020-05-19 10:14:42
## 63: 2020-05-19 10:14:42
## 64: 2020-05-19 10:14:42
## 65: 2020-05-19 10:14:42
## 66: 2020-05-19 10:14:42
## 67: 2020-05-19 10:14:42
## 68: 2020-05-19 10:14:42
## 69: 2020-05-19 10:14:42
## 70: 2020-05-19 10:14:42
## 71: 2020-05-19 10:14:42
##             createdDate
clearCache(tmpDir, ask = FALSE)

Example 2: using the “accessed” tag

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# access it again, from Cache
Sys.sleep(1)
ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
##   ...(Object to retrieve (f7bee22047b8d0c1.rds))
##      loaded cached result from previous rnorm call,
wholeCache <- showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
# keep only items accessed "recently" (i.e., only objectName:a)
onlyRecentlyAccessed <- showCache(tmpDir, userTags = max(wholeCache[tagKey == "accessed"]$tagValue))
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
# inverse join with 2 data.tables ... using: a[!b]
# i.e., return all of wholeCache that was not recently accessed
#   Note: the two different ways to access -- old way with "artifact" will be deprecated
toRemove <- unique(wholeCache[!onlyRecentlyAccessed, on = "cacheId"], by = "cacheId")$cacheId
clearCache(tmpDir, toRemove, ask = FALSE) # remove ones not recently accessed
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) # still has more recently accessed
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
##              cacheId              tagKey              tagValue
##  1: f7bee22047b8d0c1          objectName                     a
##  2: f7bee22047b8d0c1            function                 rnorm
##  3: f7bee22047b8d0c1               class               numeric
##  4: f7bee22047b8d0c1         object.size                  1008
##  5: f7bee22047b8d0c1            accessed   2020-05-19 10:14:43
##  6: f7bee22047b8d0c1             inCloud                 FALSE
##  7: f7bee22047b8d0c1          resultHash                      
##  8: f7bee22047b8d0c1   elapsedTimeDigest     0.0005843639 secs
##  9: f7bee22047b8d0c1 elapsedTimeFirstRun     1.835823e-05 secs
## 10: f7bee22047b8d0c1      otherFunctions               saveRDS
## 11: f7bee22047b8d0c1      otherFunctions               do.call
## 12: f7bee22047b8d0c1      otherFunctions          process_file
## 13: f7bee22047b8d0c1      otherFunctions         process_group
## 14: f7bee22047b8d0c1      otherFunctions   process_group.block
## 15: f7bee22047b8d0c1      otherFunctions            call_block
## 16: f7bee22047b8d0c1      otherFunctions            block_exec
## 17: f7bee22047b8d0c1      otherFunctions                in_dir
## 18: f7bee22047b8d0c1      otherFunctions             timing_fn
## 19: f7bee22047b8d0c1      otherFunctions                handle
## 20: f7bee22047b8d0c1           preDigest    n:7eef4eae85fd9229
## 21: f7bee22047b8d0c1           preDigest .FUN:4f604aa46882b368
## 22: f7bee22047b8d0c1           file.size                   173
## 23: f7bee22047b8d0c1            accessed   2020-05-19 10:14:44
## 24: f7bee22047b8d0c1     elapsedTimeLoad       0.01836729 secs
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2020-05-19 10:14:43
##  2: 2020-05-19 10:14:43
##  3: 2020-05-19 10:14:43
##  4: 2020-05-19 10:14:43
##  5: 2020-05-19 10:14:43
##  6: 2020-05-19 10:14:43
##  7: 2020-05-19 10:14:43
##  8: 2020-05-19 10:14:43
##  9: 2020-05-19 10:14:43
## 10: 2020-05-19 10:14:43
## 11: 2020-05-19 10:14:43
## 12: 2020-05-19 10:14:43
## 13: 2020-05-19 10:14:43
## 14: 2020-05-19 10:14:43
## 15: 2020-05-19 10:14:43
## 16: 2020-05-19 10:14:43
## 17: 2020-05-19 10:14:43
## 18: 2020-05-19 10:14:43
## 19: 2020-05-19 10:14:43
## 20: 2020-05-19 10:14:43
## 21: 2020-05-19 10:14:43
## 22: 2020-05-19 10:14:43
## 23: 2020-05-19 10:14:44
## 24: 2020-05-19 10:14:44
##             createdDate

Example 3: using keepCache

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
##   ...(Object to retrieve (f7bee22047b8d0c1.rds))
##      loaded cached result from previous rnorm call,
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# keep only those cached items from the last 24 hours
oneDay <- 60 * 60 * 24
keepCache(tmpDir, after = Sys.time() - oneDay, ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##              cacheId              tagKey              tagValue
##  1: f7bee22047b8d0c1          objectName                     a
##  2: f7bee22047b8d0c1            function                 rnorm
##  3: f7bee22047b8d0c1               class               numeric
##  4: f7bee22047b8d0c1         object.size                  1008
##  5: f7bee22047b8d0c1            accessed   2020-05-19 10:14:43
##  6: f7bee22047b8d0c1             inCloud                 FALSE
##  7: f7bee22047b8d0c1          resultHash                      
##  8: f7bee22047b8d0c1   elapsedTimeDigest     0.0005843639 secs
##  9: f7bee22047b8d0c1 elapsedTimeFirstRun     1.835823e-05 secs
## 10: f7bee22047b8d0c1      otherFunctions               saveRDS
## 11: f7bee22047b8d0c1      otherFunctions               do.call
## 12: f7bee22047b8d0c1      otherFunctions          process_file
## 13: f7bee22047b8d0c1      otherFunctions         process_group
## 14: f7bee22047b8d0c1      otherFunctions   process_group.block
## 15: f7bee22047b8d0c1      otherFunctions            call_block
## 16: f7bee22047b8d0c1      otherFunctions            block_exec
## 17: f7bee22047b8d0c1      otherFunctions                in_dir
## 18: f7bee22047b8d0c1      otherFunctions             timing_fn
## 19: f7bee22047b8d0c1      otherFunctions                handle
## 20: f7bee22047b8d0c1           preDigest    n:7eef4eae85fd9229
## 21: f7bee22047b8d0c1           preDigest .FUN:4f604aa46882b368
## 22: f7bee22047b8d0c1           file.size                   173
## 23: f7bee22047b8d0c1            accessed   2020-05-19 10:14:44
## 24: f7bee22047b8d0c1     elapsedTimeLoad       0.01175022 secs
## 25: f7bee22047b8d0c1            accessed   2020-05-19 10:14:44
## 26: 3aef38d1fc02aee5          objectName                     b
## 27: 3aef38d1fc02aee5            function                 runif
## 28: 3aef38d1fc02aee5               class               numeric
## 29: 3aef38d1fc02aee5         object.size                  1008
## 30: 3aef38d1fc02aee5            accessed   2020-05-19 10:14:45
## 31: 3aef38d1fc02aee5             inCloud                 FALSE
## 32: 3aef38d1fc02aee5          resultHash                      
## 33: 3aef38d1fc02aee5   elapsedTimeDigest     0.0008101463 secs
## 34: 3aef38d1fc02aee5 elapsedTimeFirstRun     1.716614e-05 secs
## 35: 3aef38d1fc02aee5      otherFunctions               saveRDS
## 36: 3aef38d1fc02aee5      otherFunctions               do.call
## 37: 3aef38d1fc02aee5      otherFunctions          process_file
## 38: 3aef38d1fc02aee5      otherFunctions         process_group
## 39: 3aef38d1fc02aee5      otherFunctions   process_group.block
## 40: 3aef38d1fc02aee5      otherFunctions            call_block
## 41: 3aef38d1fc02aee5      otherFunctions            block_exec
## 42: 3aef38d1fc02aee5      otherFunctions                in_dir
## 43: 3aef38d1fc02aee5      otherFunctions             timing_fn
## 44: 3aef38d1fc02aee5      otherFunctions                handle
## 45: 3aef38d1fc02aee5           preDigest    n:7eef4eae85fd9229
## 46: 3aef38d1fc02aee5           preDigest .FUN:881ec847b7161f3c
## 47: 3aef38d1fc02aee5           file.size                   168
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2020-05-19 10:14:43
##  2: 2020-05-19 10:14:43
##  3: 2020-05-19 10:14:43
##  4: 2020-05-19 10:14:43
##  5: 2020-05-19 10:14:43
##  6: 2020-05-19 10:14:43
##  7: 2020-05-19 10:14:43
##  8: 2020-05-19 10:14:43
##  9: 2020-05-19 10:14:43
## 10: 2020-05-19 10:14:43
## 11: 2020-05-19 10:14:43
## 12: 2020-05-19 10:14:43
## 13: 2020-05-19 10:14:43
## 14: 2020-05-19 10:14:43
## 15: 2020-05-19 10:14:43
## 16: 2020-05-19 10:14:43
## 17: 2020-05-19 10:14:43
## 18: 2020-05-19 10:14:43
## 19: 2020-05-19 10:14:43
## 20: 2020-05-19 10:14:43
## 21: 2020-05-19 10:14:43
## 22: 2020-05-19 10:14:43
## 23: 2020-05-19 10:14:44
## 24: 2020-05-19 10:14:44
## 25: 2020-05-19 10:14:44
## 26: 2020-05-19 10:14:45
## 27: 2020-05-19 10:14:45
## 28: 2020-05-19 10:14:45
## 29: 2020-05-19 10:14:45
## 30: 2020-05-19 10:14:45
## 31: 2020-05-19 10:14:45
## 32: 2020-05-19 10:14:45
## 33: 2020-05-19 10:14:45
## 34: 2020-05-19 10:14:45
## 35: 2020-05-19 10:14:45
## 36: 2020-05-19 10:14:45
## 37: 2020-05-19 10:14:45
## 38: 2020-05-19 10:14:45
## 39: 2020-05-19 10:14:45
## 40: 2020-05-19 10:14:45
## 41: 2020-05-19 10:14:45
## 42: 2020-05-19 10:14:45
## 43: 2020-05-19 10:14:45
## 44: 2020-05-19 10:14:45
## 45: 2020-05-19 10:14:45
## 46: 2020-05-19 10:14:45
## 47: 2020-05-19 10:14:45
##             createdDate
# Keep all Cache items created with an rnorm() call
keepCache(tmpDir, userTags = "rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
##              cacheId              tagKey              tagValue
##  1: f7bee22047b8d0c1          objectName                     a
##  2: f7bee22047b8d0c1            function                 rnorm
##  3: f7bee22047b8d0c1               class               numeric
##  4: f7bee22047b8d0c1         object.size                  1008
##  5: f7bee22047b8d0c1            accessed   2020-05-19 10:14:43
##  6: f7bee22047b8d0c1             inCloud                 FALSE
##  7: f7bee22047b8d0c1          resultHash                      
##  8: f7bee22047b8d0c1   elapsedTimeDigest     0.0005843639 secs
##  9: f7bee22047b8d0c1 elapsedTimeFirstRun     1.835823e-05 secs
## 10: f7bee22047b8d0c1      otherFunctions               saveRDS
## 11: f7bee22047b8d0c1      otherFunctions               do.call
## 12: f7bee22047b8d0c1      otherFunctions          process_file
## 13: f7bee22047b8d0c1      otherFunctions         process_group
## 14: f7bee22047b8d0c1      otherFunctions   process_group.block
## 15: f7bee22047b8d0c1      otherFunctions            call_block
## 16: f7bee22047b8d0c1      otherFunctions            block_exec
## 17: f7bee22047b8d0c1      otherFunctions                in_dir
## 18: f7bee22047b8d0c1      otherFunctions             timing_fn
## 19: f7bee22047b8d0c1      otherFunctions                handle
## 20: f7bee22047b8d0c1           preDigest    n:7eef4eae85fd9229
## 21: f7bee22047b8d0c1           preDigest .FUN:4f604aa46882b368
## 22: f7bee22047b8d0c1           file.size                   173
## 23: f7bee22047b8d0c1            accessed   2020-05-19 10:14:44
## 24: f7bee22047b8d0c1     elapsedTimeLoad       0.01175022 secs
## 25: f7bee22047b8d0c1            accessed   2020-05-19 10:14:44
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2020-05-19 10:14:43
##  2: 2020-05-19 10:14:43
##  3: 2020-05-19 10:14:43
##  4: 2020-05-19 10:14:43
##  5: 2020-05-19 10:14:43
##  6: 2020-05-19 10:14:43
##  7: 2020-05-19 10:14:43
##  8: 2020-05-19 10:14:43
##  9: 2020-05-19 10:14:43
## 10: 2020-05-19 10:14:43
## 11: 2020-05-19 10:14:43
## 12: 2020-05-19 10:14:43
## 13: 2020-05-19 10:14:43
## 14: 2020-05-19 10:14:43
## 15: 2020-05-19 10:14:43
## 16: 2020-05-19 10:14:43
## 17: 2020-05-19 10:14:43
## 18: 2020-05-19 10:14:43
## 19: 2020-05-19 10:14:43
## 20: 2020-05-19 10:14:43
## 21: 2020-05-19 10:14:43
## 22: 2020-05-19 10:14:43
## 23: 2020-05-19 10:14:44
## 24: 2020-05-19 10:14:44
## 25: 2020-05-19 10:14:44
##             createdDate
# Remove all Cache items that happened within a rnorm() call
clearCache(tmpDir, userTags = "rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) ## empty
## Cache size:
##   Total (including Rasters): 0 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows and 4 cols): cacheId,tagKey,tagValue,createdDate
# Also, can set a time before caching happens and remove based on this
#  --> a useful, simple way to control Cache
ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
startTime <- Sys.time()
Sys.sleep(1)
ranNumsB <- Cache(rnorm, 5, cacheRepo = tmpDir, userTags = "objectName:b")
keepCache(tmpDir, after = startTime, ask = FALSE) # keep only those newer than startTime
## Cache size:
##   Total (including Rasters): 256 bytes
##   Selected objects (not including Rasters): 256 bytes
##              cacheId              tagKey              tagValue
##  1: 142b4176fe87d51d          objectName                     b
##  2: 142b4176fe87d51d            function                 rnorm
##  3: 142b4176fe87d51d               class               numeric
##  4: 142b4176fe87d51d         object.size                  1024
##  5: 142b4176fe87d51d            accessed   2020-05-19 10:14:46
##  6: 142b4176fe87d51d             inCloud                 FALSE
##  7: 142b4176fe87d51d          resultHash                      
##  8: 142b4176fe87d51d   elapsedTimeDigest     0.0005776882 secs
##  9: 142b4176fe87d51d elapsedTimeFirstRun     2.217293e-05 secs
## 10: 142b4176fe87d51d      otherFunctions               saveRDS
## 11: 142b4176fe87d51d      otherFunctions               do.call
## 12: 142b4176fe87d51d      otherFunctions          process_file
## 13: 142b4176fe87d51d      otherFunctions         process_group
## 14: 142b4176fe87d51d      otherFunctions   process_group.block
## 15: 142b4176fe87d51d      otherFunctions            call_block
## 16: 142b4176fe87d51d      otherFunctions            block_exec
## 17: 142b4176fe87d51d      otherFunctions                in_dir
## 18: 142b4176fe87d51d      otherFunctions             timing_fn
## 19: 142b4176fe87d51d      otherFunctions                handle
## 20: 142b4176fe87d51d           preDigest    n:a4f076b3db622faf
## 21: 142b4176fe87d51d           preDigest .FUN:4f604aa46882b368
## 22: 142b4176fe87d51d           file.size                   182
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2020-05-19 10:14:46
##  2: 2020-05-19 10:14:46
##  3: 2020-05-19 10:14:46
##  4: 2020-05-19 10:14:46
##  5: 2020-05-19 10:14:46
##  6: 2020-05-19 10:14:46
##  7: 2020-05-19 10:14:46
##  8: 2020-05-19 10:14:46
##  9: 2020-05-19 10:14:46
## 10: 2020-05-19 10:14:46
## 11: 2020-05-19 10:14:46
## 12: 2020-05-19 10:14:46
## 13: 2020-05-19 10:14:46
## 14: 2020-05-19 10:14:46
## 15: 2020-05-19 10:14:46
## 16: 2020-05-19 10:14:46
## 17: 2020-05-19 10:14:46
## 18: 2020-05-19 10:14:46
## 19: 2020-05-19 10:14:46
## 20: 2020-05-19 10:14:46
## 21: 2020-05-19 10:14:46
## 22: 2020-05-19 10:14:46
##             createdDate
clearCache(tmpDir, ask = FALSE)

Example 4: searching for multiple objects in the cache

# default userTags is "and" matching; for "or" matching use |
ranNumsA <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# show all objects (runif and rnorm in this case)
showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##              cacheId              tagKey              tagValue
##  1: 3aef38d1fc02aee5          objectName                     a
##  2: 3aef38d1fc02aee5            function                 runif
##  3: 3aef38d1fc02aee5               class               numeric
##  4: 3aef38d1fc02aee5         object.size                  1008
##  5: 3aef38d1fc02aee5            accessed   2020-05-19 10:14:46
##  6: 3aef38d1fc02aee5             inCloud                 FALSE
##  7: 3aef38d1fc02aee5          resultHash                      
##  8: 3aef38d1fc02aee5   elapsedTimeDigest     0.0005729198 secs
##  9: 3aef38d1fc02aee5 elapsedTimeFirstRun     1.716614e-05 secs
## 10: 3aef38d1fc02aee5      otherFunctions               saveRDS
## 11: 3aef38d1fc02aee5      otherFunctions               do.call
## 12: 3aef38d1fc02aee5      otherFunctions          process_file
## 13: 3aef38d1fc02aee5      otherFunctions         process_group
## 14: 3aef38d1fc02aee5      otherFunctions   process_group.block
## 15: 3aef38d1fc02aee5      otherFunctions            call_block
## 16: 3aef38d1fc02aee5      otherFunctions            block_exec
## 17: 3aef38d1fc02aee5      otherFunctions                in_dir
## 18: 3aef38d1fc02aee5      otherFunctions             timing_fn
## 19: 3aef38d1fc02aee5      otherFunctions                handle
## 20: 3aef38d1fc02aee5           preDigest    n:7eef4eae85fd9229
## 21: 3aef38d1fc02aee5           preDigest .FUN:881ec847b7161f3c
## 22: 3aef38d1fc02aee5           file.size                   167
## 23: f7bee22047b8d0c1          objectName                     b
## 24: f7bee22047b8d0c1            function                 rnorm
## 25: f7bee22047b8d0c1               class               numeric
## 26: f7bee22047b8d0c1         object.size                  1008
## 27: f7bee22047b8d0c1            accessed   2020-05-19 10:14:47
## 28: f7bee22047b8d0c1             inCloud                 FALSE
## 29: f7bee22047b8d0c1          resultHash                      
## 30: f7bee22047b8d0c1   elapsedTimeDigest     0.0009102821 secs
## 31: f7bee22047b8d0c1 elapsedTimeFirstRun     1.978874e-05 secs
## 32: f7bee22047b8d0c1      otherFunctions               saveRDS
## 33: f7bee22047b8d0c1      otherFunctions               do.call
## 34: f7bee22047b8d0c1      otherFunctions          process_file
## 35: f7bee22047b8d0c1      otherFunctions         process_group
## 36: f7bee22047b8d0c1      otherFunctions   process_group.block
## 37: f7bee22047b8d0c1      otherFunctions            call_block
## 38: f7bee22047b8d0c1      otherFunctions            block_exec
## 39: f7bee22047b8d0c1      otherFunctions                in_dir
## 40: f7bee22047b8d0c1      otherFunctions             timing_fn
## 41: f7bee22047b8d0c1      otherFunctions                handle
## 42: f7bee22047b8d0c1           preDigest    n:7eef4eae85fd9229
## 43: f7bee22047b8d0c1           preDigest .FUN:4f604aa46882b368
## 44: f7bee22047b8d0c1           file.size                   173
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2020-05-19 10:14:46
##  2: 2020-05-19 10:14:46
##  3: 2020-05-19 10:14:46
##  4: 2020-05-19 10:14:46
##  5: 2020-05-19 10:14:46
##  6: 2020-05-19 10:14:46
##  7: 2020-05-19 10:14:46
##  8: 2020-05-19 10:14:46
##  9: 2020-05-19 10:14:46
## 10: 2020-05-19 10:14:46
## 11: 2020-05-19 10:14:46
## 12: 2020-05-19 10:14:46
## 13: 2020-05-19 10:14:46
## 14: 2020-05-19 10:14:46
## 15: 2020-05-19 10:14:46
## 16: 2020-05-19 10:14:46
## 17: 2020-05-19 10:14:46
## 18: 2020-05-19 10:14:46
## 19: 2020-05-19 10:14:46
## 20: 2020-05-19 10:14:46
## 21: 2020-05-19 10:14:46
## 22: 2020-05-19 10:14:46
## 23: 2020-05-19 10:14:47
## 24: 2020-05-19 10:14:47
## 25: 2020-05-19 10:14:47
## 26: 2020-05-19 10:14:47
## 27: 2020-05-19 10:14:47
## 28: 2020-05-19 10:14:47
## 29: 2020-05-19 10:14:47
## 30: 2020-05-19 10:14:47
## 31: 2020-05-19 10:14:47
## 32: 2020-05-19 10:14:47
## 33: 2020-05-19 10:14:47
## 34: 2020-05-19 10:14:47
## 35: 2020-05-19 10:14:47
## 36: 2020-05-19 10:14:47
## 37: 2020-05-19 10:14:47
## 38: 2020-05-19 10:14:47
## 39: 2020-05-19 10:14:47
## 40: 2020-05-19 10:14:47
## 41: 2020-05-19 10:14:47
## 42: 2020-05-19 10:14:47
## 43: 2020-05-19 10:14:47
## 44: 2020-05-19 10:14:47
##             createdDate
# show objects that are both runif and rnorm
# (i.e., none in this case, because objecs are either or, not both)
showCache(tmpDir, userTags = c("runif", "rnorm")) ## empty
## Cache size:
##   Total (including Rasters): 0 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows and 4 cols): cacheId,tagKey,tagValue,createdDate
# show objects that are either runif or rnorm ("or" search)
showCache(tmpDir, userTags = "runif|rnorm")
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##              cacheId              tagKey              tagValue
##  1: 3aef38d1fc02aee5          objectName                     a
##  2: 3aef38d1fc02aee5            function                 runif
##  3: 3aef38d1fc02aee5               class               numeric
##  4: 3aef38d1fc02aee5         object.size                  1008
##  5: 3aef38d1fc02aee5            accessed   2020-05-19 10:14:46
##  6: 3aef38d1fc02aee5             inCloud                 FALSE
##  7: 3aef38d1fc02aee5          resultHash                      
##  8: 3aef38d1fc02aee5   elapsedTimeDigest     0.0005729198 secs
##  9: 3aef38d1fc02aee5 elapsedTimeFirstRun     1.716614e-05 secs
## 10: 3aef38d1fc02aee5      otherFunctions               saveRDS
## 11: 3aef38d1fc02aee5      otherFunctions               do.call
## 12: 3aef38d1fc02aee5      otherFunctions          process_file
## 13: 3aef38d1fc02aee5      otherFunctions         process_group
## 14: 3aef38d1fc02aee5      otherFunctions   process_group.block
## 15: 3aef38d1fc02aee5      otherFunctions            call_block
## 16: 3aef38d1fc02aee5      otherFunctions            block_exec
## 17: 3aef38d1fc02aee5      otherFunctions                in_dir
## 18: 3aef38d1fc02aee5      otherFunctions             timing_fn
## 19: 3aef38d1fc02aee5      otherFunctions                handle
## 20: 3aef38d1fc02aee5           preDigest    n:7eef4eae85fd9229
## 21: 3aef38d1fc02aee5           preDigest .FUN:881ec847b7161f3c
## 22: 3aef38d1fc02aee5           file.size                   167
## 23: f7bee22047b8d0c1          objectName                     b
## 24: f7bee22047b8d0c1            function                 rnorm
## 25: f7bee22047b8d0c1               class               numeric
## 26: f7bee22047b8d0c1         object.size                  1008
## 27: f7bee22047b8d0c1            accessed   2020-05-19 10:14:47
## 28: f7bee22047b8d0c1             inCloud                 FALSE
## 29: f7bee22047b8d0c1          resultHash                      
## 30: f7bee22047b8d0c1   elapsedTimeDigest     0.0009102821 secs
## 31: f7bee22047b8d0c1 elapsedTimeFirstRun     1.978874e-05 secs
## 32: f7bee22047b8d0c1      otherFunctions               saveRDS
## 33: f7bee22047b8d0c1      otherFunctions               do.call
## 34: f7bee22047b8d0c1      otherFunctions          process_file
## 35: f7bee22047b8d0c1      otherFunctions         process_group
## 36: f7bee22047b8d0c1      otherFunctions   process_group.block
## 37: f7bee22047b8d0c1      otherFunctions            call_block
## 38: f7bee22047b8d0c1      otherFunctions            block_exec
## 39: f7bee22047b8d0c1      otherFunctions                in_dir
## 40: f7bee22047b8d0c1      otherFunctions             timing_fn
## 41: f7bee22047b8d0c1      otherFunctions                handle
## 42: f7bee22047b8d0c1           preDigest    n:7eef4eae85fd9229
## 43: f7bee22047b8d0c1           preDigest .FUN:4f604aa46882b368
## 44: f7bee22047b8d0c1           file.size                   173
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2020-05-19 10:14:46
##  2: 2020-05-19 10:14:46
##  3: 2020-05-19 10:14:46
##  4: 2020-05-19 10:14:46
##  5: 2020-05-19 10:14:46
##  6: 2020-05-19 10:14:46
##  7: 2020-05-19 10:14:46
##  8: 2020-05-19 10:14:46
##  9: 2020-05-19 10:14:46
## 10: 2020-05-19 10:14:46
## 11: 2020-05-19 10:14:46
## 12: 2020-05-19 10:14:46
## 13: 2020-05-19 10:14:46
## 14: 2020-05-19 10:14:46
## 15: 2020-05-19 10:14:46
## 16: 2020-05-19 10:14:46
## 17: 2020-05-19 10:14:46
## 18: 2020-05-19 10:14:46
## 19: 2020-05-19 10:14:46
## 20: 2020-05-19 10:14:46
## 21: 2020-05-19 10:14:46
## 22: 2020-05-19 10:14:46
## 23: 2020-05-19 10:14:47
## 24: 2020-05-19 10:14:47
## 25: 2020-05-19 10:14:47
## 26: 2020-05-19 10:14:47
## 27: 2020-05-19 10:14:47
## 28: 2020-05-19 10:14:47
## 29: 2020-05-19 10:14:47
## 30: 2020-05-19 10:14:47
## 31: 2020-05-19 10:14:47
## 32: 2020-05-19 10:14:47
## 33: 2020-05-19 10:14:47
## 34: 2020-05-19 10:14:47
## 35: 2020-05-19 10:14:47
## 36: 2020-05-19 10:14:47
## 37: 2020-05-19 10:14:47
## 38: 2020-05-19 10:14:47
## 39: 2020-05-19 10:14:47
## 40: 2020-05-19 10:14:47
## 41: 2020-05-19 10:14:47
## 42: 2020-05-19 10:14:47
## 43: 2020-05-19 10:14:47
## 44: 2020-05-19 10:14:47
##             createdDate
# keep only objects that are either runif or rnorm ("or" search)
keepCache(tmpDir, userTags = "runif|rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##              cacheId              tagKey              tagValue
##  1: 3aef38d1fc02aee5          objectName                     a
##  2: 3aef38d1fc02aee5            function                 runif
##  3: 3aef38d1fc02aee5               class               numeric
##  4: 3aef38d1fc02aee5         object.size                  1008
##  5: 3aef38d1fc02aee5            accessed   2020-05-19 10:14:46
##  6: 3aef38d1fc02aee5             inCloud                 FALSE
##  7: 3aef38d1fc02aee5          resultHash                      
##  8: 3aef38d1fc02aee5   elapsedTimeDigest     0.0005729198 secs
##  9: 3aef38d1fc02aee5 elapsedTimeFirstRun     1.716614e-05 secs
## 10: 3aef38d1fc02aee5      otherFunctions               saveRDS
## 11: 3aef38d1fc02aee5      otherFunctions               do.call
## 12: 3aef38d1fc02aee5      otherFunctions          process_file
## 13: 3aef38d1fc02aee5      otherFunctions         process_group
## 14: 3aef38d1fc02aee5      otherFunctions   process_group.block
## 15: 3aef38d1fc02aee5      otherFunctions            call_block
## 16: 3aef38d1fc02aee5      otherFunctions            block_exec
## 17: 3aef38d1fc02aee5      otherFunctions                in_dir
## 18: 3aef38d1fc02aee5      otherFunctions             timing_fn
## 19: 3aef38d1fc02aee5      otherFunctions                handle
## 20: 3aef38d1fc02aee5           preDigest    n:7eef4eae85fd9229
## 21: 3aef38d1fc02aee5           preDigest .FUN:881ec847b7161f3c
## 22: 3aef38d1fc02aee5           file.size                   167
## 23: f7bee22047b8d0c1          objectName                     b
## 24: f7bee22047b8d0c1            function                 rnorm
## 25: f7bee22047b8d0c1               class               numeric
## 26: f7bee22047b8d0c1         object.size                  1008
## 27: f7bee22047b8d0c1            accessed   2020-05-19 10:14:47
## 28: f7bee22047b8d0c1             inCloud                 FALSE
## 29: f7bee22047b8d0c1          resultHash                      
## 30: f7bee22047b8d0c1   elapsedTimeDigest     0.0009102821 secs
## 31: f7bee22047b8d0c1 elapsedTimeFirstRun     1.978874e-05 secs
## 32: f7bee22047b8d0c1      otherFunctions               saveRDS
## 33: f7bee22047b8d0c1      otherFunctions               do.call
## 34: f7bee22047b8d0c1      otherFunctions          process_file
## 35: f7bee22047b8d0c1      otherFunctions         process_group
## 36: f7bee22047b8d0c1      otherFunctions   process_group.block
## 37: f7bee22047b8d0c1      otherFunctions            call_block
## 38: f7bee22047b8d0c1      otherFunctions            block_exec
## 39: f7bee22047b8d0c1      otherFunctions                in_dir
## 40: f7bee22047b8d0c1      otherFunctions             timing_fn
## 41: f7bee22047b8d0c1      otherFunctions                handle
## 42: f7bee22047b8d0c1           preDigest    n:7eef4eae85fd9229
## 43: f7bee22047b8d0c1           preDigest .FUN:4f604aa46882b368
## 44: f7bee22047b8d0c1           file.size                   173
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2020-05-19 10:14:46
##  2: 2020-05-19 10:14:46
##  3: 2020-05-19 10:14:46
##  4: 2020-05-19 10:14:46
##  5: 2020-05-19 10:14:46
##  6: 2020-05-19 10:14:46
##  7: 2020-05-19 10:14:46
##  8: 2020-05-19 10:14:46
##  9: 2020-05-19 10:14:46
## 10: 2020-05-19 10:14:46
## 11: 2020-05-19 10:14:46
## 12: 2020-05-19 10:14:46
## 13: 2020-05-19 10:14:46
## 14: 2020-05-19 10:14:46
## 15: 2020-05-19 10:14:46
## 16: 2020-05-19 10:14:46
## 17: 2020-05-19 10:14:46
## 18: 2020-05-19 10:14:46
## 19: 2020-05-19 10:14:46
## 20: 2020-05-19 10:14:46
## 21: 2020-05-19 10:14:46
## 22: 2020-05-19 10:14:46
## 23: 2020-05-19 10:14:47
## 24: 2020-05-19 10:14:47
## 25: 2020-05-19 10:14:47
## 26: 2020-05-19 10:14:47
## 27: 2020-05-19 10:14:47
## 28: 2020-05-19 10:14:47
## 29: 2020-05-19 10:14:47
## 30: 2020-05-19 10:14:47
## 31: 2020-05-19 10:14:47
## 32: 2020-05-19 10:14:47
## 33: 2020-05-19 10:14:47
## 34: 2020-05-19 10:14:47
## 35: 2020-05-19 10:14:47
## 36: 2020-05-19 10:14:47
## 37: 2020-05-19 10:14:47
## 38: 2020-05-19 10:14:47
## 39: 2020-05-19 10:14:47
## 40: 2020-05-19 10:14:47
## 41: 2020-05-19 10:14:47
## 42: 2020-05-19 10:14:47
## 43: 2020-05-19 10:14:47
## 44: 2020-05-19 10:14:47
##             createdDate
clearCache(tmpDir, ask = FALSE)

Example 5: using caching to speed up rerunning expensive computations

ras <- raster(extent(0, 5, 0, 5), res = 1,
              vals = sample(1:5, replace = TRUE, size = 25),
              crs = "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84")

# A slow operation, like GIS operation
notCached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  projectRaster(ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)

cached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  # using quote works also
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)

# second time is much faster
reRun <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)
##   ...(Object to retrieve (8fb0ea82871276bf.rds))
##      loaded cached result from previous projectRaster call,
# recovered cached version is same as non-cached version
all.equal(notCached, reRun) ## TRUE
## [1] TRUE

Nested Caching

Nested caching, which is when Caching of a function occurs inside an outer function, which is itself cached. This is a critical element to working within a reproducible work flow. It is not enough during development to cache flat code chunks, as there will be many levels of “slow” functions. Ideally, at all points in a development cycle, it should be possible to get to any line of code starting from the very initial steps, running through everything up to that point, in less that 1 second. If the workflow can be kept very fast like this, then there is a guarantee that it will work at any point.

##########################
## Nested Caching
# Make 2 functions
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean)
}
outer <- function(n) {
  Cache(inner, 0.1, cacheRepo = tmpdir2)
}

# make 2 different cache paths
tmpdir1 <- file.path(tempdir(), "first")
tmpdir2 <- file.path(tempdir(), "second")

# Run the Cache ... notOlderThan propagates to all 3 Cache calls,
#   but cacheRepo is tmpdir1 in top level Cache and all nested
#   Cache calls, unless individually overridden ... here inner
#   uses tmpdir2 repository
Cache(outer, n = 2, cacheRepo = tmpdir1, notOlderThan = Sys.time())
## [1]  2.419979 -1.067450  2.232404
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:a7af5367c13aba8f"
## attr(,"call")
## [1] ""
showCache(tmpdir1) # 2 function calls
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##              cacheId              tagKey              tagValue
##  1: 4ac2b7c0f42d1e46            function                 rnorm
##  2: 4ac2b7c0f42d1e46               class               numeric
##  3: 4ac2b7c0f42d1e46         object.size                  1008
##  4: 4ac2b7c0f42d1e46            accessed   2020-05-19 10:14:48
##  5: 4ac2b7c0f42d1e46             inCloud                 FALSE
##  6: 4ac2b7c0f42d1e46          resultHash                      
##  7: 4ac2b7c0f42d1e46   elapsedTimeDigest     0.0007126331 secs
##  8: 4ac2b7c0f42d1e46 elapsedTimeFirstRun     2.074242e-05 secs
##  9: 4ac2b7c0f42d1e46      otherFunctions               saveRDS
## 10: 4ac2b7c0f42d1e46      otherFunctions               do.call
## 11: 4ac2b7c0f42d1e46      otherFunctions          process_file
## 12: 4ac2b7c0f42d1e46      otherFunctions         process_group
## 13: 4ac2b7c0f42d1e46      otherFunctions   process_group.block
## 14: 4ac2b7c0f42d1e46      otherFunctions            call_block
## 15: 4ac2b7c0f42d1e46      otherFunctions            block_exec
## 16: 4ac2b7c0f42d1e46      otherFunctions                in_dir
## 17: 4ac2b7c0f42d1e46      otherFunctions             timing_fn
## 18: 4ac2b7c0f42d1e46      otherFunctions                handle
## 19: 4ac2b7c0f42d1e46           preDigest    n:7f12988bd88a0fb8
## 20: 4ac2b7c0f42d1e46           preDigest mean:22413394efd9f6a3
## 21: 4ac2b7c0f42d1e46           preDigest .FUN:4f604aa46882b368
## 22: 4ac2b7c0f42d1e46           file.size                   164
## 23: a7af5367c13aba8f            function                 outer
## 24: a7af5367c13aba8f               class               numeric
## 25: a7af5367c13aba8f         object.size                  1008
## 26: a7af5367c13aba8f            accessed   2020-05-19 10:14:48
## 27: a7af5367c13aba8f             inCloud                 FALSE
## 28: a7af5367c13aba8f          resultHash                      
## 29: a7af5367c13aba8f   elapsedTimeDigest      0.001113892 secs
## 30: a7af5367c13aba8f elapsedTimeFirstRun        0.5015917 secs
## 31: a7af5367c13aba8f      otherFunctions               saveRDS
## 32: a7af5367c13aba8f      otherFunctions               do.call
## 33: a7af5367c13aba8f      otherFunctions          process_file
## 34: a7af5367c13aba8f      otherFunctions         process_group
## 35: a7af5367c13aba8f      otherFunctions   process_group.block
## 36: a7af5367c13aba8f      otherFunctions            call_block
## 37: a7af5367c13aba8f      otherFunctions            block_exec
## 38: a7af5367c13aba8f      otherFunctions                in_dir
## 39: a7af5367c13aba8f      otherFunctions             timing_fn
## 40: a7af5367c13aba8f      otherFunctions                handle
## 41: a7af5367c13aba8f           preDigest    n:82dc709f2b91918a
## 42: a7af5367c13aba8f           preDigest .FUN:892a6afc47a63a90
## 43: a7af5367c13aba8f           file.size                   164
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2020-05-19 10:14:48
##  2: 2020-05-19 10:14:48
##  3: 2020-05-19 10:14:48
##  4: 2020-05-19 10:14:48
##  5: 2020-05-19 10:14:48
##  6: 2020-05-19 10:14:48
##  7: 2020-05-19 10:14:48
##  8: 2020-05-19 10:14:48
##  9: 2020-05-19 10:14:48
## 10: 2020-05-19 10:14:48
## 11: 2020-05-19 10:14:48
## 12: 2020-05-19 10:14:48
## 13: 2020-05-19 10:14:48
## 14: 2020-05-19 10:14:48
## 15: 2020-05-19 10:14:48
## 16: 2020-05-19 10:14:48
## 17: 2020-05-19 10:14:48
## 18: 2020-05-19 10:14:48
## 19: 2020-05-19 10:14:48
## 20: 2020-05-19 10:14:48
## 21: 2020-05-19 10:14:48
## 22: 2020-05-19 10:14:48
## 23: 2020-05-19 10:14:48
## 24: 2020-05-19 10:14:48
## 25: 2020-05-19 10:14:48
## 26: 2020-05-19 10:14:48
## 27: 2020-05-19 10:14:48
## 28: 2020-05-19 10:14:48
## 29: 2020-05-19 10:14:48
## 30: 2020-05-19 10:14:48
## 31: 2020-05-19 10:14:48
## 32: 2020-05-19 10:14:48
## 33: 2020-05-19 10:14:48
## 34: 2020-05-19 10:14:48
## 35: 2020-05-19 10:14:48
## 36: 2020-05-19 10:14:48
## 37: 2020-05-19 10:14:48
## 38: 2020-05-19 10:14:48
## 39: 2020-05-19 10:14:48
## 40: 2020-05-19 10:14:48
## 41: 2020-05-19 10:14:48
## 42: 2020-05-19 10:14:48
## 43: 2020-05-19 10:14:48
##             createdDate
showCache(tmpdir2) # 1 function call
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
##              cacheId              tagKey              tagValue
##  1: 33ceb4fb525fd08f            function                 inner
##  2: 33ceb4fb525fd08f               class               numeric
##  3: 33ceb4fb525fd08f         object.size                  1008
##  4: 33ceb4fb525fd08f            accessed   2020-05-19 10:14:48
##  5: 33ceb4fb525fd08f             inCloud                 FALSE
##  6: 33ceb4fb525fd08f          resultHash                      
##  7: 33ceb4fb525fd08f   elapsedTimeDigest      0.001278639 secs
##  8: 33ceb4fb525fd08f elapsedTimeFirstRun        0.2588439 secs
##  9: 33ceb4fb525fd08f      otherFunctions               saveRDS
## 10: 33ceb4fb525fd08f      otherFunctions               do.call
## 11: 33ceb4fb525fd08f      otherFunctions          process_file
## 12: 33ceb4fb525fd08f      otherFunctions         process_group
## 13: 33ceb4fb525fd08f      otherFunctions   process_group.block
## 14: 33ceb4fb525fd08f      otherFunctions            call_block
## 15: 33ceb4fb525fd08f      otherFunctions            block_exec
## 16: 33ceb4fb525fd08f      otherFunctions                in_dir
## 17: 33ceb4fb525fd08f      otherFunctions             timing_fn
## 18: 33ceb4fb525fd08f      otherFunctions                handle
## 19: 33ceb4fb525fd08f           preDigest mean:22413394efd9f6a3
## 20: 33ceb4fb525fd08f           preDigest .FUN:87e2c30917a34d25
## 21: 33ceb4fb525fd08f           file.size                   164
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2020-05-19 10:14:48
##  2: 2020-05-19 10:14:48
##  3: 2020-05-19 10:14:48
##  4: 2020-05-19 10:14:48
##  5: 2020-05-19 10:14:48
##  6: 2020-05-19 10:14:48
##  7: 2020-05-19 10:14:48
##  8: 2020-05-19 10:14:48
##  9: 2020-05-19 10:14:48
## 10: 2020-05-19 10:14:48
## 11: 2020-05-19 10:14:48
## 12: 2020-05-19 10:14:48
## 13: 2020-05-19 10:14:48
## 14: 2020-05-19 10:14:48
## 15: 2020-05-19 10:14:48
## 16: 2020-05-19 10:14:48
## 17: 2020-05-19 10:14:48
## 18: 2020-05-19 10:14:48
## 19: 2020-05-19 10:14:48
## 20: 2020-05-19 10:14:48
## 21: 2020-05-19 10:14:48
##             createdDate
# userTags get appended
# all items have the outer tag propagate, plus inner ones only have inner ones
clearCache(tmpdir1, ask = FALSE)
outerTag <- "outerTag"
innerTag <- "innerTag"
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean, notOlderThan = Sys.time() - 1e5, userTags = innerTag)
}
outer <- function(n) {
  Cache(inner, 0.1)
}
aa <- Cache(outer, n = 2, cacheRepo = tmpdir1, userTags = outerTag)
showCache(tmpdir1) # rnorm function has outerTag and innerTag, inner and outer only have outerTag
## Cache size:
##   Total (including Rasters): 756 bytes
##   Selected objects (not including Rasters): 756 bytes
##              cacheId              tagKey              tagValue
##  1: 4ac2b7c0f42d1e46            innerTag              innerTag
##  2: 4ac2b7c0f42d1e46            outerTag              outerTag
##  3: 4ac2b7c0f42d1e46            function                 rnorm
##  4: 4ac2b7c0f42d1e46               class               numeric
##  5: 4ac2b7c0f42d1e46         object.size                  1008
##  6: 4ac2b7c0f42d1e46            accessed   2020-05-19 10:14:48
##  7: 4ac2b7c0f42d1e46             inCloud                 FALSE
##  8: 4ac2b7c0f42d1e46          resultHash                      
##  9: 4ac2b7c0f42d1e46   elapsedTimeDigest     0.0008065701 secs
## 10: 4ac2b7c0f42d1e46 elapsedTimeFirstRun     2.193451e-05 secs
## 11: 4ac2b7c0f42d1e46      otherFunctions               saveRDS
## 12: 4ac2b7c0f42d1e46      otherFunctions               do.call
## 13: 4ac2b7c0f42d1e46      otherFunctions          process_file
## 14: 4ac2b7c0f42d1e46      otherFunctions         process_group
## 15: 4ac2b7c0f42d1e46      otherFunctions   process_group.block
## 16: 4ac2b7c0f42d1e46      otherFunctions            call_block
## 17: 4ac2b7c0f42d1e46      otherFunctions            block_exec
## 18: 4ac2b7c0f42d1e46      otherFunctions                in_dir
## 19: 4ac2b7c0f42d1e46      otherFunctions             timing_fn
## 20: 4ac2b7c0f42d1e46      otherFunctions                handle
## 21: 4ac2b7c0f42d1e46           preDigest    n:7f12988bd88a0fb8
## 22: 4ac2b7c0f42d1e46           preDigest mean:22413394efd9f6a3
## 23: 4ac2b7c0f42d1e46           preDigest .FUN:4f604aa46882b368
## 24: 4ac2b7c0f42d1e46           file.size                   165
## 25: b06af03d5a73dc7d            outerTag              outerTag
## 26: b06af03d5a73dc7d            function                 inner
## 27: b06af03d5a73dc7d               class               numeric
## 28: b06af03d5a73dc7d         object.size                  1008
## 29: b06af03d5a73dc7d            accessed   2020-05-19 10:14:49
## 30: b06af03d5a73dc7d             inCloud                 FALSE
## 31: b06af03d5a73dc7d          resultHash                      
## 32: b06af03d5a73dc7d   elapsedTimeDigest     0.0005898476 secs
## 33: b06af03d5a73dc7d elapsedTimeFirstRun        0.2709534 secs
## 34: b06af03d5a73dc7d      otherFunctions               saveRDS
## 35: b06af03d5a73dc7d      otherFunctions               do.call
## 36: b06af03d5a73dc7d      otherFunctions          process_file
## 37: b06af03d5a73dc7d      otherFunctions         process_group
## 38: b06af03d5a73dc7d      otherFunctions   process_group.block
## 39: b06af03d5a73dc7d      otherFunctions            call_block
## 40: b06af03d5a73dc7d      otherFunctions            block_exec
## 41: b06af03d5a73dc7d      otherFunctions                in_dir
## 42: b06af03d5a73dc7d      otherFunctions             timing_fn
## 43: b06af03d5a73dc7d      otherFunctions                handle
## 44: b06af03d5a73dc7d           preDigest mean:22413394efd9f6a3
## 45: b06af03d5a73dc7d           preDigest .FUN:7ad10bc1ae528d8c
## 46: b06af03d5a73dc7d           file.size                   165
## 47: 88a34e1d033329e5            outerTag              outerTag
## 48: 88a34e1d033329e5            function                 outer
## 49: 88a34e1d033329e5               class               numeric
## 50: 88a34e1d033329e5         object.size                  1008
## 51: 88a34e1d033329e5            accessed   2020-05-19 10:14:49
## 52: 88a34e1d033329e5             inCloud                 FALSE
## 53: 88a34e1d033329e5          resultHash                      
## 54: 88a34e1d033329e5   elapsedTimeDigest     0.0005862713 secs
## 55: 88a34e1d033329e5 elapsedTimeFirstRun        0.5081649 secs
## 56: 88a34e1d033329e5      otherFunctions               saveRDS
## 57: 88a34e1d033329e5      otherFunctions               do.call
## 58: 88a34e1d033329e5      otherFunctions          process_file
## 59: 88a34e1d033329e5      otherFunctions         process_group
## 60: 88a34e1d033329e5      otherFunctions   process_group.block
## 61: 88a34e1d033329e5      otherFunctions            call_block
## 62: 88a34e1d033329e5      otherFunctions            block_exec
## 63: 88a34e1d033329e5      otherFunctions                in_dir
## 64: 88a34e1d033329e5      otherFunctions             timing_fn
## 65: 88a34e1d033329e5      otherFunctions                handle
## 66: 88a34e1d033329e5           preDigest    n:82dc709f2b91918a
## 67: 88a34e1d033329e5           preDigest .FUN:5f06fb5fbffe9e3b
## 68: 88a34e1d033329e5           file.size                   165
##              cacheId              tagKey              tagValue
##             createdDate
##  1: 2020-05-19 10:14:48
##  2: 2020-05-19 10:14:48
##  3: 2020-05-19 10:14:48
##  4: 2020-05-19 10:14:48
##  5: 2020-05-19 10:14:48
##  6: 2020-05-19 10:14:48
##  7: 2020-05-19 10:14:48
##  8: 2020-05-19 10:14:48
##  9: 2020-05-19 10:14:48
## 10: 2020-05-19 10:14:48
## 11: 2020-05-19 10:14:48
## 12: 2020-05-19 10:14:48
## 13: 2020-05-19 10:14:48
## 14: 2020-05-19 10:14:48
## 15: 2020-05-19 10:14:48
## 16: 2020-05-19 10:14:48
## 17: 2020-05-19 10:14:48
## 18: 2020-05-19 10:14:48
## 19: 2020-05-19 10:14:48
## 20: 2020-05-19 10:14:48
## 21: 2020-05-19 10:14:48
## 22: 2020-05-19 10:14:48
## 23: 2020-05-19 10:14:48
## 24: 2020-05-19 10:14:48
## 25: 2020-05-19 10:14:49
## 26: 2020-05-19 10:14:49
## 27: 2020-05-19 10:14:49
## 28: 2020-05-19 10:14:49
## 29: 2020-05-19 10:14:49
## 30: 2020-05-19 10:14:49
## 31: 2020-05-19 10:14:49
## 32: 2020-05-19 10:14:49
## 33: 2020-05-19 10:14:49
## 34: 2020-05-19 10:14:49
## 35: 2020-05-19 10:14:49
## 36: 2020-05-19 10:14:49
## 37: 2020-05-19 10:14:49
## 38: 2020-05-19 10:14:49
## 39: 2020-05-19 10:14:49
## 40: 2020-05-19 10:14:49
## 41: 2020-05-19 10:14:49
## 42: 2020-05-19 10:14:49
## 43: 2020-05-19 10:14:49
## 44: 2020-05-19 10:14:49
## 45: 2020-05-19 10:14:49
## 46: 2020-05-19 10:14:49
## 47: 2020-05-19 10:14:49
## 48: 2020-05-19 10:14:49
## 49: 2020-05-19 10:14:49
## 50: 2020-05-19 10:14:49
## 51: 2020-05-19 10:14:49
## 52: 2020-05-19 10:14:49
## 53: 2020-05-19 10:14:49
## 54: 2020-05-19 10:14:49
## 55: 2020-05-19 10:14:49
## 56: 2020-05-19 10:14:49
## 57: 2020-05-19 10:14:49
## 58: 2020-05-19 10:14:49
## 59: 2020-05-19 10:14:49
## 60: 2020-05-19 10:14:49
## 61: 2020-05-19 10:14:49
## 62: 2020-05-19 10:14:49
## 63: 2020-05-19 10:14:49
## 64: 2020-05-19 10:14:49
## 65: 2020-05-19 10:14:49
## 66: 2020-05-19 10:14:49
## 67: 2020-05-19 10:14:49
## 68: 2020-05-19 10:14:49
##             createdDate

cacheId

Sometimes, it is not absolutely desirable to maintain the work flow intact because changes that are irrelevant to the analysis, such as changing messages sent to a user, may be changed, without a desire to rerun functions. The cacheId argument is for this. Once a piece of code is run, then the cacheId can be manually extracted (it is reported at the end of a Cache call) and manually placed in the code, passed in as, say, cacheId = "ad184ce64541972b50afd8e7b75f821b".

### cacheId
set.seed(1)
Cache(rnorm, 1, cacheRepo = tmpdir1)
## [1] -0.6264538
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:7072c305d8c69df0"
## attr(,"call")
## [1] ""
# manually look at output attribute which shows cacheId: 7072c305d8c69df0
Cache(rnorm, 1, cacheRepo = tmpdir1, cacheId = "7072c305d8c69df0") # same value
## cacheId is same as calculated hash
##   ...(Object to retrieve (7072c305d8c69df0.rds))
##      loaded cached result from previous rnorm call,
## [1] -0.6264538
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] FALSE
## 
## attr(,"tags")
## [1] "cacheId:7072c305d8c69df0"
## attr(,"call")
## [1] ""
# override even with different inputs:
Cache(rnorm, 2, cacheRepo = tmpdir1, cacheId = "7072c305d8c69df0")
## cacheId is not same as calculated hash. Manually searching for cacheId:7072c305d8c69df0
##   ...(Object to retrieve (7072c305d8c69df0.rds))
##      loaded cached result from previous rnorm call,
## [1] -0.6264538
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] FALSE
## 
## attr(,"tags")
## [1] "cacheId:7072c305d8c69df0"
## attr(,"call")
## [1] ""

Working with the Cache manually

Since the cache is simply a DBI data table (of an SQLite database by default). In addition, there are several helpers in the reproducible package, including showCache, keepCache and clearCache that may be useful. Also, one can access cached items manually (rather than simply rerunning the same Cache function again).

# As of reproducible version 1.0, there is a new backend directly using DBI
mapHash <- unique(showCache(tmpDir, userTags = "projectRaster")$cacheId)
## Cache size:
##   Total (including Rasters): 3.3 Kb
##   Selected objects (not including Rasters): 3.3 Kb
map <- loadFromCache(mapHash[1], cachePath = tmpDir)
plot(map)

## cleanup
unlink(dirname(tmpDir), recursive = TRUE)

Reproducible Workflow

In general, we feel that a liberal use of Cache will make a re-usable and reproducible work flow. shiny apps can be made, taking advantage of Cache. Indeed, much of the difficulty in managing data sets and saving them for future use, can be accommodated by caching.

Nested Caching

Cache individual functions

Cache(<functionName>, <other arguments>)

This will allow fine scale control of individual function calls.