Introduction to Cache
Eliot J. B. McIntire
November 09 2022
Source:vignettes/Intro-to-Cache.Rmd
Intro-to-Cache.RmdReproducible workflow
As part of a reproducible workflow, caching of function calls, code
chunks, and other elements of a project is a critical component. The
objective of a reproducible workflow is is likely that an entire work
flow from raw data to publication, decision support, report writing,
presentation building etc., could be built and be reproducible anywhere,
on any computer, operating system, with any starting conditions, on
demand. The reproducible::Cache function is built to work
with any R function.
Differences with other approaches
Cache users DBI as a backend, with key
functions, dbReadTable, dbRemoveTable,
dbSendQuery, dbSendStatement,
dbCreateTable and dbAppendTable. These can all
be accessed via Cache, showCache,
clearCache, and keepCache. It is optimized for
speed of transactions, using fastdigest::fastdigest on R
memory objects and digest::digest on files. The main
function is superficially similar to archivist::cache,
which uses digest::digest in all cases to determine whether
the arguments are identical in subsequent iterations. It also but does
many things that make standard caching with
digest::digest don’t work reliably between systems. For
these, the function .robustDigest is introduced to make
caching transferable between systems. This is relevant for file paths,
environments, parallel clusters, functions (which are contained within
an environment), and many others (e.g., see ?.robustDigest
for methods). Cache also adds important elements like
automated tagging and the option to retrieve disk-cached values via
stashed objects in memory using memoise::memoise. This
means that running Cache 1, 2, and 3 times on the same
function will get progressively faster. This can be extremely useful for
web apps built with, say shiny.
Function-level caching
Any function can be cached using:
Cache(FUN = functionName, ...).
This will be a slight change to a function call, such as:
projectRaster(raster, crs = crs(newRaster)) to
Cache(projectRaster, raster, crs = crs(newRaster)).
This is particularly useful for expensive operations.
## Loading required package: sp
library(reproducible)
tmpDir <- file.path(tempdir(), "reproducible_examples", "Cache")
checkPath(tmpDir, create = TRUE)## [1] "/tmp/RtmpbyTFl0/reproducible_examples/Cache"
ras <- raster(extent(0, 1000, 0, 1000), vals = 1:1e6, res = 1)
crs(ras) <- "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +datum=WGS84"
newCRS <- "+init=epsg:4326" # A longlat crs
# No Cache
system.time(suppressWarnings(map1 <- projectRaster(ras, crs = newCRS))) # Warnings due to new PROJ## user system elapsed
## 4.613 0.305 4.919
# Try with memoise for this example -- for many simple cases, memoising will not be faster
opts <- options("reproducible.useMemoise" = TRUE)
# With Cache -- a little slower the first time because saving to disk
system.time(suppressWarnings(map1 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir,
notOlderThan = Sys.time())))## user system elapsed
## 4.475 0.394 4.877
# vastly faster the second time
system.time(map2 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))## ...(Object to retrieve (c9370bd403e2a7b3.rds) is large: 6.5 Mb)## loaded memoised result from previous projectRaster call## user system elapsed
## 0.389 0.001 0.394
# may be faster the third time because of memoise; but this example is too simple to show
system.time(map3 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))## ...(Object to retrieve (c9370bd403e2a7b3.rds) is large: 6.5 Mb)
## loaded memoised result from previous projectRaster call## user system elapsed
## 0.042 0.000 0.045## [1] TRUE
all.equal(map1, map3) # TRUE## [1] TRUECaching examples
Basic use
##
## Attaching package: 'magrittr'## The following object is masked from 'package:raster':
##
## extract
try(clearCache(tmpDir, ask = FALSE), silent = TRUE) # just to make sure it is clear
ranNumsA <- Cache(rnorm, 10, 16, cacheRepo = tmpDir)
# All same
ranNumsB <- Cache(rnorm, 10, 16, cacheRepo = tmpDir) # recovers cached copy## ...(Object to retrieve (f11fb1a2880f8060.rds))## loaded cached result from previous rnorm call## ...(Object to retrieve (f11fb1a2880f8060.rds))
## loaded cached result from previous rnorm call
# Any minor change makes it different
ranNumsE <- Cache(rnorm, 10, 6, cacheRepo = tmpDir) # differentExample 1: Basic cache use with tags
ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")
showCache(tmpDir, userTags = c("objectName"))## Cache size: ## Total (including Rasters): 504 bytes## Selected objects (not including Rasters): 504 bytes## cacheId tagKey tagValue
## 1: 3aef38d1fc02aee5 objectName b
## 2: 3aef38d1fc02aee5 function runif
## 3: 3aef38d1fc02aee5 class numeric
## 4: 3aef38d1fc02aee5 object.size 1008
## 5: 3aef38d1fc02aee5 accessed 2022-11-09 15:47:27
## 6: 3aef38d1fc02aee5 inCloud FALSE
## 7: 3aef38d1fc02aee5 resultHash
## 8: 3aef38d1fc02aee5 elapsedTimeDigest 0.0005917549 secs
## 9: 3aef38d1fc02aee5 elapsedTimeFirstRun 3.933907e-05 secs
## 10: 3aef38d1fc02aee5 otherFunctions saveRDS
## 11: 3aef38d1fc02aee5 otherFunctions do.call
## 12: 3aef38d1fc02aee5 otherFunctions process_file
## 13: 3aef38d1fc02aee5 otherFunctions process_group
## 14: 3aef38d1fc02aee5 otherFunctions process_group.block
## 15: 3aef38d1fc02aee5 otherFunctions call_block
## 16: 3aef38d1fc02aee5 otherFunctions block_exec
## 17: 3aef38d1fc02aee5 otherFunctions eng_r
## 18: 3aef38d1fc02aee5 otherFunctions in_input_dir
## 19: 3aef38d1fc02aee5 otherFunctions in_dir
## 20: 3aef38d1fc02aee5 otherFunctions timing_fn
## 21: 3aef38d1fc02aee5 otherFunctions handle
## 22: 3aef38d1fc02aee5 preDigest n:7eef4eae85fd9229
## 23: 3aef38d1fc02aee5 preDigest .FUN:881ec847b7161f3c
## 24: 3aef38d1fc02aee5 file.size 168
## 25: f7bee22047b8d0c1 objectName a
## 26: f7bee22047b8d0c1 function rnorm
## 27: f7bee22047b8d0c1 class numeric
## 28: f7bee22047b8d0c1 object.size 1008
## 29: f7bee22047b8d0c1 accessed 2022-11-09 15:47:27
## 30: f7bee22047b8d0c1 inCloud FALSE
## 31: f7bee22047b8d0c1 resultHash
## 32: f7bee22047b8d0c1 elapsedTimeDigest 0.0004713535 secs
## 33: f7bee22047b8d0c1 elapsedTimeFirstRun 4.172325e-05 secs
## 34: f7bee22047b8d0c1 otherFunctions saveRDS
## 35: f7bee22047b8d0c1 otherFunctions do.call
## 36: f7bee22047b8d0c1 otherFunctions process_file
## 37: f7bee22047b8d0c1 otherFunctions process_group
## 38: f7bee22047b8d0c1 otherFunctions process_group.block
## 39: f7bee22047b8d0c1 otherFunctions call_block
## 40: f7bee22047b8d0c1 otherFunctions block_exec
## 41: f7bee22047b8d0c1 otherFunctions eng_r
## 42: f7bee22047b8d0c1 otherFunctions in_input_dir
## 43: f7bee22047b8d0c1 otherFunctions in_dir
## 44: f7bee22047b8d0c1 otherFunctions timing_fn
## 45: f7bee22047b8d0c1 otherFunctions handle
## 46: f7bee22047b8d0c1 preDigest n:7eef4eae85fd9229
## 47: f7bee22047b8d0c1 preDigest .FUN:4f604aa46882b368
## 48: f7bee22047b8d0c1 file.size 174
## cacheId tagKey tagValue
## createdDate
## 1: 2022-11-09 15:47:27
## 2: 2022-11-09 15:47:27
## 3: 2022-11-09 15:47:27
## 4: 2022-11-09 15:47:27
## 5: 2022-11-09 15:47:27
## 6: 2022-11-09 15:47:27
## 7: 2022-11-09 15:47:27
## 8: 2022-11-09 15:47:27
## 9: 2022-11-09 15:47:27
## 10: 2022-11-09 15:47:27
## 11: 2022-11-09 15:47:27
## 12: 2022-11-09 15:47:27
## 13: 2022-11-09 15:47:27
## 14: 2022-11-09 15:47:27
## 15: 2022-11-09 15:47:27
## 16: 2022-11-09 15:47:27
## 17: 2022-11-09 15:47:27
## 18: 2022-11-09 15:47:27
## 19: 2022-11-09 15:47:27
## 20: 2022-11-09 15:47:27
## 21: 2022-11-09 15:47:27
## 22: 2022-11-09 15:47:27
## 23: 2022-11-09 15:47:27
## 24: 2022-11-09 15:47:27
## 25: 2022-11-09 15:47:27
## 26: 2022-11-09 15:47:27
## 27: 2022-11-09 15:47:27
## 28: 2022-11-09 15:47:27
## 29: 2022-11-09 15:47:27
## 30: 2022-11-09 15:47:27
## 31: 2022-11-09 15:47:27
## 32: 2022-11-09 15:47:27
## 33: 2022-11-09 15:47:27
## 34: 2022-11-09 15:47:27
## 35: 2022-11-09 15:47:27
## 36: 2022-11-09 15:47:27
## 37: 2022-11-09 15:47:27
## 38: 2022-11-09 15:47:27
## 39: 2022-11-09 15:47:27
## 40: 2022-11-09 15:47:27
## 41: 2022-11-09 15:47:27
## 42: 2022-11-09 15:47:27
## 43: 2022-11-09 15:47:27
## 44: 2022-11-09 15:47:27
## 45: 2022-11-09 15:47:27
## 46: 2022-11-09 15:47:27
## 47: 2022-11-09 15:47:27
## 48: 2022-11-09 15:47:27
## createdDate## Cache size: ## Total (including Rasters): 252 bytes## Selected objects (not including Rasters): 252 bytes## cacheId tagKey tagValue
## 1: f7bee22047b8d0c1 objectName a
## 2: f7bee22047b8d0c1 function rnorm
## 3: f7bee22047b8d0c1 class numeric
## 4: f7bee22047b8d0c1 object.size 1008
## 5: f7bee22047b8d0c1 accessed 2022-11-09 15:47:27
## 6: f7bee22047b8d0c1 inCloud FALSE
## 7: f7bee22047b8d0c1 resultHash
## 8: f7bee22047b8d0c1 elapsedTimeDigest 0.0004713535 secs
## 9: f7bee22047b8d0c1 elapsedTimeFirstRun 4.172325e-05 secs
## 10: f7bee22047b8d0c1 otherFunctions saveRDS
## 11: f7bee22047b8d0c1 otherFunctions do.call
## 12: f7bee22047b8d0c1 otherFunctions process_file
## 13: f7bee22047b8d0c1 otherFunctions process_group
## 14: f7bee22047b8d0c1 otherFunctions process_group.block
## 15: f7bee22047b8d0c1 otherFunctions call_block
## 16: f7bee22047b8d0c1 otherFunctions block_exec
## 17: f7bee22047b8d0c1 otherFunctions eng_r
## 18: f7bee22047b8d0c1 otherFunctions in_input_dir
## 19: f7bee22047b8d0c1 otherFunctions in_dir
## 20: f7bee22047b8d0c1 otherFunctions timing_fn
## 21: f7bee22047b8d0c1 otherFunctions handle
## 22: f7bee22047b8d0c1 preDigest n:7eef4eae85fd9229
## 23: f7bee22047b8d0c1 preDigest .FUN:4f604aa46882b368
## 24: f7bee22047b8d0c1 file.size 174
## cacheId tagKey tagValue
## createdDate
## 1: 2022-11-09 15:47:27
## 2: 2022-11-09 15:47:27
## 3: 2022-11-09 15:47:27
## 4: 2022-11-09 15:47:27
## 5: 2022-11-09 15:47:27
## 6: 2022-11-09 15:47:27
## 7: 2022-11-09 15:47:27
## 8: 2022-11-09 15:47:27
## 9: 2022-11-09 15:47:27
## 10: 2022-11-09 15:47:27
## 11: 2022-11-09 15:47:27
## 12: 2022-11-09 15:47:27
## 13: 2022-11-09 15:47:27
## 14: 2022-11-09 15:47:27
## 15: 2022-11-09 15:47:27
## 16: 2022-11-09 15:47:27
## 17: 2022-11-09 15:47:27
## 18: 2022-11-09 15:47:27
## 19: 2022-11-09 15:47:27
## 20: 2022-11-09 15:47:27
## 21: 2022-11-09 15:47:27
## 22: 2022-11-09 15:47:27
## 23: 2022-11-09 15:47:27
## 24: 2022-11-09 15:47:27
## createdDate## Cache size: ## Total (including Rasters): 252 bytes## Selected objects (not including Rasters): 252 bytes## cacheId tagKey tagValue
## 1: 3aef38d1fc02aee5 objectName b
## 2: 3aef38d1fc02aee5 function runif
## 3: 3aef38d1fc02aee5 class numeric
## 4: 3aef38d1fc02aee5 object.size 1008
## 5: 3aef38d1fc02aee5 accessed 2022-11-09 15:47:27
## 6: 3aef38d1fc02aee5 inCloud FALSE
## 7: 3aef38d1fc02aee5 resultHash
## 8: 3aef38d1fc02aee5 elapsedTimeDigest 0.0005917549 secs
## 9: 3aef38d1fc02aee5 elapsedTimeFirstRun 3.933907e-05 secs
## 10: 3aef38d1fc02aee5 otherFunctions saveRDS
## 11: 3aef38d1fc02aee5 otherFunctions do.call
## 12: 3aef38d1fc02aee5 otherFunctions process_file
## 13: 3aef38d1fc02aee5 otherFunctions process_group
## 14: 3aef38d1fc02aee5 otherFunctions process_group.block
## 15: 3aef38d1fc02aee5 otherFunctions call_block
## 16: 3aef38d1fc02aee5 otherFunctions block_exec
## 17: 3aef38d1fc02aee5 otherFunctions eng_r
## 18: 3aef38d1fc02aee5 otherFunctions in_input_dir
## 19: 3aef38d1fc02aee5 otherFunctions in_dir
## 20: 3aef38d1fc02aee5 otherFunctions timing_fn
## 21: 3aef38d1fc02aee5 otherFunctions handle
## 22: 3aef38d1fc02aee5 preDigest n:7eef4eae85fd9229
## 23: 3aef38d1fc02aee5 preDigest .FUN:881ec847b7161f3c
## 24: 3aef38d1fc02aee5 file.size 168
## cacheId tagKey tagValue
## createdDate
## 1: 2022-11-09 15:47:27
## 2: 2022-11-09 15:47:27
## 3: 2022-11-09 15:47:27
## 4: 2022-11-09 15:47:27
## 5: 2022-11-09 15:47:27
## 6: 2022-11-09 15:47:27
## 7: 2022-11-09 15:47:27
## 8: 2022-11-09 15:47:27
## 9: 2022-11-09 15:47:27
## 10: 2022-11-09 15:47:27
## 11: 2022-11-09 15:47:27
## 12: 2022-11-09 15:47:27
## 13: 2022-11-09 15:47:27
## 14: 2022-11-09 15:47:27
## 15: 2022-11-09 15:47:27
## 16: 2022-11-09 15:47:27
## 17: 2022-11-09 15:47:27
## 18: 2022-11-09 15:47:27
## 19: 2022-11-09 15:47:27
## 20: 2022-11-09 15:47:27
## 21: 2022-11-09 15:47:27
## 22: 2022-11-09 15:47:27
## 23: 2022-11-09 15:47:27
## 24: 2022-11-09 15:47:27
## createdDate
clearCache(tmpDir, userTags = c("runif"), ask = FALSE) # remove only cached objects made during runif call## Loading required namespace: googledrive## Cache size: ## Total (including Rasters): 252 bytes## Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) # only those made during rnorm call## Cache size: ## Total (including Rasters): 804 bytes## Selected objects (not including Rasters): 804 bytes## cacheId tagKey tagValue
## 1: f11fb1a2880f8060 function rnorm
## 2: f11fb1a2880f8060 class numeric
## 3: f11fb1a2880f8060 object.size 1104
## 4: f11fb1a2880f8060 accessed 2022-11-09 15:47:27
## 5: f11fb1a2880f8060 inCloud FALSE
## 6: f11fb1a2880f8060 resultHash
## 7: f11fb1a2880f8060 elapsedTimeDigest 0.0005757809 secs
## 8: f11fb1a2880f8060 elapsedTimeFirstRun 6.29425e-05 secs
## 9: f11fb1a2880f8060 otherFunctions saveRDS
## 10: f11fb1a2880f8060 otherFunctions do.call
## 11: f11fb1a2880f8060 otherFunctions process_file
## 12: f11fb1a2880f8060 otherFunctions process_group
## 13: f11fb1a2880f8060 otherFunctions process_group.block
## 14: f11fb1a2880f8060 otherFunctions call_block
## 15: f11fb1a2880f8060 otherFunctions block_exec
## 16: f11fb1a2880f8060 otherFunctions eng_r
## 17: f11fb1a2880f8060 otherFunctions in_input_dir
## 18: f11fb1a2880f8060 otherFunctions in_dir
## 19: f11fb1a2880f8060 otherFunctions timing_fn
## 20: f11fb1a2880f8060 otherFunctions handle
## 21: f11fb1a2880f8060 preDigest n:c5775c3b366fb719
## 22: f11fb1a2880f8060 preDigest mean:15620f138033a66c
## 23: f11fb1a2880f8060 preDigest .FUN:4f604aa46882b368
## 24: f11fb1a2880f8060 file.size 224
## 25: f11fb1a2880f8060 accessed 2022-11-09 15:47:27
## 26: f11fb1a2880f8060 elapsedTimeLoad 0.003323555 secs
## 27: f11fb1a2880f8060 accessed 2022-11-09 15:47:27
## 28: c16f0e2c57596069 function rnorm
## 29: c16f0e2c57596069 class numeric
## 30: c16f0e2c57596069 object.size 1104
## 31: c16f0e2c57596069 accessed 2022-11-09 15:47:27
## 32: c16f0e2c57596069 inCloud FALSE
## 33: c16f0e2c57596069 resultHash
## 34: c16f0e2c57596069 elapsedTimeDigest 0.0005404949 secs
## 35: c16f0e2c57596069 elapsedTimeFirstRun 4.220009e-05 secs
## 36: c16f0e2c57596069 otherFunctions saveRDS
## 37: c16f0e2c57596069 otherFunctions do.call
## 38: c16f0e2c57596069 otherFunctions process_file
## 39: c16f0e2c57596069 otherFunctions process_group
## 40: c16f0e2c57596069 otherFunctions process_group.block
## 41: c16f0e2c57596069 otherFunctions call_block
## 42: c16f0e2c57596069 otherFunctions block_exec
## 43: c16f0e2c57596069 otherFunctions eng_r
## 44: c16f0e2c57596069 otherFunctions in_input_dir
## 45: c16f0e2c57596069 otherFunctions in_dir
## 46: c16f0e2c57596069 otherFunctions timing_fn
## 47: c16f0e2c57596069 otherFunctions handle
## 48: c16f0e2c57596069 preDigest n:c5775c3b366fb719
## 49: c16f0e2c57596069 preDigest mean:152602b8ff81e5bb
## 50: c16f0e2c57596069 preDigest .FUN:4f604aa46882b368
## 51: c16f0e2c57596069 file.size 223
## 52: f7bee22047b8d0c1 objectName a
## 53: f7bee22047b8d0c1 function rnorm
## 54: f7bee22047b8d0c1 class numeric
## 55: f7bee22047b8d0c1 object.size 1008
## 56: f7bee22047b8d0c1 accessed 2022-11-09 15:47:27
## 57: f7bee22047b8d0c1 inCloud FALSE
## 58: f7bee22047b8d0c1 resultHash
## 59: f7bee22047b8d0c1 elapsedTimeDigest 0.0004713535 secs
## 60: f7bee22047b8d0c1 elapsedTimeFirstRun 4.172325e-05 secs
## 61: f7bee22047b8d0c1 otherFunctions saveRDS
## 62: f7bee22047b8d0c1 otherFunctions do.call
## 63: f7bee22047b8d0c1 otherFunctions process_file
## 64: f7bee22047b8d0c1 otherFunctions process_group
## 65: f7bee22047b8d0c1 otherFunctions process_group.block
## 66: f7bee22047b8d0c1 otherFunctions call_block
## 67: f7bee22047b8d0c1 otherFunctions block_exec
## 68: f7bee22047b8d0c1 otherFunctions eng_r
## 69: f7bee22047b8d0c1 otherFunctions in_input_dir
## 70: f7bee22047b8d0c1 otherFunctions in_dir
## 71: f7bee22047b8d0c1 otherFunctions timing_fn
## 72: f7bee22047b8d0c1 otherFunctions handle
## 73: f7bee22047b8d0c1 preDigest n:7eef4eae85fd9229
## 74: f7bee22047b8d0c1 preDigest .FUN:4f604aa46882b368
## 75: f7bee22047b8d0c1 file.size 174
## cacheId tagKey tagValue
## createdDate
## 1: 2022-11-09 15:47:27
## 2: 2022-11-09 15:47:27
## 3: 2022-11-09 15:47:27
## 4: 2022-11-09 15:47:27
## 5: 2022-11-09 15:47:27
## 6: 2022-11-09 15:47:27
## 7: 2022-11-09 15:47:27
## 8: 2022-11-09 15:47:27
## 9: 2022-11-09 15:47:27
## 10: 2022-11-09 15:47:27
## 11: 2022-11-09 15:47:27
## 12: 2022-11-09 15:47:27
## 13: 2022-11-09 15:47:27
## 14: 2022-11-09 15:47:27
## 15: 2022-11-09 15:47:27
## 16: 2022-11-09 15:47:27
## 17: 2022-11-09 15:47:27
## 18: 2022-11-09 15:47:27
## 19: 2022-11-09 15:47:27
## 20: 2022-11-09 15:47:27
## 21: 2022-11-09 15:47:27
## 22: 2022-11-09 15:47:27
## 23: 2022-11-09 15:47:27
## 24: 2022-11-09 15:47:27
## 25: 2022-11-09 15:47:27
## 26: 2022-11-09 15:47:27
## 27: 2022-11-09 15:47:27
## 28: 2022-11-09 15:47:27
## 29: 2022-11-09 15:47:27
## 30: 2022-11-09 15:47:27
## 31: 2022-11-09 15:47:27
## 32: 2022-11-09 15:47:27
## 33: 2022-11-09 15:47:27
## 34: 2022-11-09 15:47:27
## 35: 2022-11-09 15:47:27
## 36: 2022-11-09 15:47:27
## 37: 2022-11-09 15:47:27
## 38: 2022-11-09 15:47:27
## 39: 2022-11-09 15:47:27
## 40: 2022-11-09 15:47:27
## 41: 2022-11-09 15:47:27
## 42: 2022-11-09 15:47:27
## 43: 2022-11-09 15:47:27
## 44: 2022-11-09 15:47:27
## 45: 2022-11-09 15:47:27
## 46: 2022-11-09 15:47:27
## 47: 2022-11-09 15:47:27
## 48: 2022-11-09 15:47:27
## 49: 2022-11-09 15:47:27
## 50: 2022-11-09 15:47:27
## 51: 2022-11-09 15:47:27
## 52: 2022-11-09 15:47:27
## 53: 2022-11-09 15:47:27
## 54: 2022-11-09 15:47:27
## 55: 2022-11-09 15:47:27
## 56: 2022-11-09 15:47:27
## 57: 2022-11-09 15:47:27
## 58: 2022-11-09 15:47:27
## 59: 2022-11-09 15:47:27
## 60: 2022-11-09 15:47:27
## 61: 2022-11-09 15:47:27
## 62: 2022-11-09 15:47:27
## 63: 2022-11-09 15:47:27
## 64: 2022-11-09 15:47:27
## 65: 2022-11-09 15:47:27
## 66: 2022-11-09 15:47:27
## 67: 2022-11-09 15:47:27
## 68: 2022-11-09 15:47:27
## 69: 2022-11-09 15:47:27
## 70: 2022-11-09 15:47:27
## 71: 2022-11-09 15:47:27
## 72: 2022-11-09 15:47:27
## 73: 2022-11-09 15:47:27
## 74: 2022-11-09 15:47:27
## 75: 2022-11-09 15:47:27
## createdDate
clearCache(tmpDir, ask = FALSE)Example 2: using the “accessed” tag
ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")
# access it again, from Cache
Sys.sleep(1)
ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")## ...(Object to retrieve (f7bee22047b8d0c1.rds))## loaded cached result from previous rnorm call
wholeCache <- showCache(tmpDir)## Cache size: ## Total (including Rasters): 504 bytes## Selected objects (not including Rasters): 504 bytes
# keep only items accessed "recently" (i.e., only objectName:a)
onlyRecentlyAccessed <- showCache(tmpDir, userTags = max(wholeCache[tagKey == "accessed"]$tagValue))## Cache size: ## Total (including Rasters): 252 bytes## Selected objects (not including Rasters): 252 bytes
# inverse join with 2 data.tables ... using: a[!b]
# i.e., return all of wholeCache that was not recently accessed
# Note: the two different ways to access -- old way with "artifact" will be deprecated
toRemove <- unique(wholeCache[!onlyRecentlyAccessed, on = "cacheId"], by = "cacheId")$cacheId
clearCache(tmpDir, toRemove, ask = FALSE) # remove ones not recently accessed## Cache size: ## Total (including Rasters): 252 bytes## Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) # still has more recently accessed## Cache size: ## Total (including Rasters): 252 bytes## Selected objects (not including Rasters): 252 bytes## cacheId tagKey tagValue
## 1: f7bee22047b8d0c1 objectName a
## 2: f7bee22047b8d0c1 function rnorm
## 3: f7bee22047b8d0c1 class numeric
## 4: f7bee22047b8d0c1 object.size 1008
## 5: f7bee22047b8d0c1 accessed 2022-11-09 15:47:28
## 6: f7bee22047b8d0c1 inCloud FALSE
## 7: f7bee22047b8d0c1 resultHash
## 8: f7bee22047b8d0c1 elapsedTimeDigest 0.0005419254 secs
## 9: f7bee22047b8d0c1 elapsedTimeFirstRun 4.935265e-05 secs
## 10: f7bee22047b8d0c1 otherFunctions saveRDS
## 11: f7bee22047b8d0c1 otherFunctions do.call
## 12: f7bee22047b8d0c1 otherFunctions process_file
## 13: f7bee22047b8d0c1 otherFunctions process_group
## 14: f7bee22047b8d0c1 otherFunctions process_group.block
## 15: f7bee22047b8d0c1 otherFunctions call_block
## 16: f7bee22047b8d0c1 otherFunctions block_exec
## 17: f7bee22047b8d0c1 otherFunctions eng_r
## 18: f7bee22047b8d0c1 otherFunctions in_input_dir
## 19: f7bee22047b8d0c1 otherFunctions in_dir
## 20: f7bee22047b8d0c1 otherFunctions timing_fn
## 21: f7bee22047b8d0c1 otherFunctions handle
## 22: f7bee22047b8d0c1 preDigest n:7eef4eae85fd9229
## 23: f7bee22047b8d0c1 preDigest .FUN:4f604aa46882b368
## 24: f7bee22047b8d0c1 file.size 174
## 25: f7bee22047b8d0c1 accessed 2022-11-09 15:47:29
## 26: f7bee22047b8d0c1 elapsedTimeLoad 0.004569054 secs
## cacheId tagKey tagValue
## createdDate
## 1: 2022-11-09 15:47:28
## 2: 2022-11-09 15:47:28
## 3: 2022-11-09 15:47:28
## 4: 2022-11-09 15:47:28
## 5: 2022-11-09 15:47:28
## 6: 2022-11-09 15:47:28
## 7: 2022-11-09 15:47:28
## 8: 2022-11-09 15:47:28
## 9: 2022-11-09 15:47:28
## 10: 2022-11-09 15:47:28
## 11: 2022-11-09 15:47:28
## 12: 2022-11-09 15:47:28
## 13: 2022-11-09 15:47:28
## 14: 2022-11-09 15:47:28
## 15: 2022-11-09 15:47:28
## 16: 2022-11-09 15:47:28
## 17: 2022-11-09 15:47:28
## 18: 2022-11-09 15:47:28
## 19: 2022-11-09 15:47:28
## 20: 2022-11-09 15:47:28
## 21: 2022-11-09 15:47:28
## 22: 2022-11-09 15:47:28
## 23: 2022-11-09 15:47:28
## 24: 2022-11-09 15:47:28
## 25: 2022-11-09 15:47:29
## 26: 2022-11-09 15:47:29
## createdDateExample 3: using keepCache
ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")## ...(Object to retrieve (f7bee22047b8d0c1.rds))## loaded cached result from previous rnorm call
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")
# keep only those cached items from the last 24 hours
oneDay <- 60 * 60 * 24
keepCache(tmpDir, after = Sys.time() - oneDay, ask = FALSE)## Cache size: ## Total (including Rasters): 504 bytes## Selected objects (not including Rasters): 504 bytes## cacheId tagKey tagValue
## 1: f7bee22047b8d0c1 objectName a
## 2: f7bee22047b8d0c1 function rnorm
## 3: f7bee22047b8d0c1 class numeric
## 4: f7bee22047b8d0c1 object.size 1008
## 5: f7bee22047b8d0c1 accessed 2022-11-09 15:47:28
## 6: f7bee22047b8d0c1 inCloud FALSE
## 7: f7bee22047b8d0c1 resultHash
## 8: f7bee22047b8d0c1 elapsedTimeDigest 0.0005419254 secs
## 9: f7bee22047b8d0c1 elapsedTimeFirstRun 4.935265e-05 secs
## 10: f7bee22047b8d0c1 otherFunctions saveRDS
## 11: f7bee22047b8d0c1 otherFunctions do.call
## 12: f7bee22047b8d0c1 otherFunctions process_file
## 13: f7bee22047b8d0c1 otherFunctions process_group
## 14: f7bee22047b8d0c1 otherFunctions process_group.block
## 15: f7bee22047b8d0c1 otherFunctions call_block
## 16: f7bee22047b8d0c1 otherFunctions block_exec
## 17: f7bee22047b8d0c1 otherFunctions eng_r
## 18: f7bee22047b8d0c1 otherFunctions in_input_dir
## 19: f7bee22047b8d0c1 otherFunctions in_dir
## 20: f7bee22047b8d0c1 otherFunctions timing_fn
## 21: f7bee22047b8d0c1 otherFunctions handle
## 22: f7bee22047b8d0c1 preDigest n:7eef4eae85fd9229
## 23: f7bee22047b8d0c1 preDigest .FUN:4f604aa46882b368
## 24: f7bee22047b8d0c1 file.size 174
## 25: f7bee22047b8d0c1 accessed 2022-11-09 15:47:29
## 26: f7bee22047b8d0c1 elapsedTimeLoad 0.003910303 secs
## 27: f7bee22047b8d0c1 accessed 2022-11-09 15:47:29
## 28: 3aef38d1fc02aee5 objectName b
## 29: 3aef38d1fc02aee5 function runif
## 30: 3aef38d1fc02aee5 class numeric
## 31: 3aef38d1fc02aee5 object.size 1008
## 32: 3aef38d1fc02aee5 accessed 2022-11-09 15:47:29
## 33: 3aef38d1fc02aee5 inCloud FALSE
## 34: 3aef38d1fc02aee5 resultHash
## 35: 3aef38d1fc02aee5 elapsedTimeDigest 0.0004780293 secs
## 36: 3aef38d1fc02aee5 elapsedTimeFirstRun 4.029274e-05 secs
## 37: 3aef38d1fc02aee5 otherFunctions saveRDS
## 38: 3aef38d1fc02aee5 otherFunctions do.call
## 39: 3aef38d1fc02aee5 otherFunctions process_file
## 40: 3aef38d1fc02aee5 otherFunctions process_group
## 41: 3aef38d1fc02aee5 otherFunctions process_group.block
## 42: 3aef38d1fc02aee5 otherFunctions call_block
## 43: 3aef38d1fc02aee5 otherFunctions block_exec
## 44: 3aef38d1fc02aee5 otherFunctions eng_r
## 45: 3aef38d1fc02aee5 otherFunctions in_input_dir
## 46: 3aef38d1fc02aee5 otherFunctions in_dir
## 47: 3aef38d1fc02aee5 otherFunctions timing_fn
## 48: 3aef38d1fc02aee5 otherFunctions handle
## 49: 3aef38d1fc02aee5 preDigest n:7eef4eae85fd9229
## 50: 3aef38d1fc02aee5 preDigest .FUN:881ec847b7161f3c
## 51: 3aef38d1fc02aee5 file.size 169
## cacheId tagKey tagValue
## createdDate
## 1: 2022-11-09 15:47:28
## 2: 2022-11-09 15:47:28
## 3: 2022-11-09 15:47:28
## 4: 2022-11-09 15:47:28
## 5: 2022-11-09 15:47:28
## 6: 2022-11-09 15:47:28
## 7: 2022-11-09 15:47:28
## 8: 2022-11-09 15:47:28
## 9: 2022-11-09 15:47:28
## 10: 2022-11-09 15:47:28
## 11: 2022-11-09 15:47:28
## 12: 2022-11-09 15:47:28
## 13: 2022-11-09 15:47:28
## 14: 2022-11-09 15:47:28
## 15: 2022-11-09 15:47:28
## 16: 2022-11-09 15:47:28
## 17: 2022-11-09 15:47:28
## 18: 2022-11-09 15:47:28
## 19: 2022-11-09 15:47:28
## 20: 2022-11-09 15:47:28
## 21: 2022-11-09 15:47:28
## 22: 2022-11-09 15:47:28
## 23: 2022-11-09 15:47:28
## 24: 2022-11-09 15:47:28
## 25: 2022-11-09 15:47:29
## 26: 2022-11-09 15:47:29
## 27: 2022-11-09 15:47:29
## 28: 2022-11-09 15:47:29
## 29: 2022-11-09 15:47:29
## 30: 2022-11-09 15:47:29
## 31: 2022-11-09 15:47:29
## 32: 2022-11-09 15:47:29
## 33: 2022-11-09 15:47:29
## 34: 2022-11-09 15:47:29
## 35: 2022-11-09 15:47:29
## 36: 2022-11-09 15:47:29
## 37: 2022-11-09 15:47:29
## 38: 2022-11-09 15:47:29
## 39: 2022-11-09 15:47:29
## 40: 2022-11-09 15:47:29
## 41: 2022-11-09 15:47:29
## 42: 2022-11-09 15:47:29
## 43: 2022-11-09 15:47:29
## 44: 2022-11-09 15:47:29
## 45: 2022-11-09 15:47:29
## 46: 2022-11-09 15:47:29
## 47: 2022-11-09 15:47:29
## 48: 2022-11-09 15:47:29
## 49: 2022-11-09 15:47:29
## 50: 2022-11-09 15:47:29
## 51: 2022-11-09 15:47:29
## createdDate
# Keep all Cache items created with an rnorm() call
keepCache(tmpDir, userTags = "rnorm", ask = FALSE)## Cache size: ## Total (including Rasters): 252 bytes## Selected objects (not including Rasters): 252 bytes## cacheId tagKey tagValue
## 1: f7bee22047b8d0c1 objectName a
## 2: f7bee22047b8d0c1 function rnorm
## 3: f7bee22047b8d0c1 class numeric
## 4: f7bee22047b8d0c1 object.size 1008
## 5: f7bee22047b8d0c1 accessed 2022-11-09 15:47:28
## 6: f7bee22047b8d0c1 inCloud FALSE
## 7: f7bee22047b8d0c1 resultHash
## 8: f7bee22047b8d0c1 elapsedTimeDigest 0.0005419254 secs
## 9: f7bee22047b8d0c1 elapsedTimeFirstRun 4.935265e-05 secs
## 10: f7bee22047b8d0c1 otherFunctions saveRDS
## 11: f7bee22047b8d0c1 otherFunctions do.call
## 12: f7bee22047b8d0c1 otherFunctions process_file
## 13: f7bee22047b8d0c1 otherFunctions process_group
## 14: f7bee22047b8d0c1 otherFunctions process_group.block
## 15: f7bee22047b8d0c1 otherFunctions call_block
## 16: f7bee22047b8d0c1 otherFunctions block_exec
## 17: f7bee22047b8d0c1 otherFunctions eng_r
## 18: f7bee22047b8d0c1 otherFunctions in_input_dir
## 19: f7bee22047b8d0c1 otherFunctions in_dir
## 20: f7bee22047b8d0c1 otherFunctions timing_fn
## 21: f7bee22047b8d0c1 otherFunctions handle
## 22: f7bee22047b8d0c1 preDigest n:7eef4eae85fd9229
## 23: f7bee22047b8d0c1 preDigest .FUN:4f604aa46882b368
## 24: f7bee22047b8d0c1 file.size 174
## 25: f7bee22047b8d0c1 accessed 2022-11-09 15:47:29
## 26: f7bee22047b8d0c1 elapsedTimeLoad 0.003910303 secs
## 27: f7bee22047b8d0c1 accessed 2022-11-09 15:47:29
## cacheId tagKey tagValue
## createdDate
## 1: 2022-11-09 15:47:28
## 2: 2022-11-09 15:47:28
## 3: 2022-11-09 15:47:28
## 4: 2022-11-09 15:47:28
## 5: 2022-11-09 15:47:28
## 6: 2022-11-09 15:47:28
## 7: 2022-11-09 15:47:28
## 8: 2022-11-09 15:47:28
## 9: 2022-11-09 15:47:28
## 10: 2022-11-09 15:47:28
## 11: 2022-11-09 15:47:28
## 12: 2022-11-09 15:47:28
## 13: 2022-11-09 15:47:28
## 14: 2022-11-09 15:47:28
## 15: 2022-11-09 15:47:28
## 16: 2022-11-09 15:47:28
## 17: 2022-11-09 15:47:28
## 18: 2022-11-09 15:47:28
## 19: 2022-11-09 15:47:28
## 20: 2022-11-09 15:47:28
## 21: 2022-11-09 15:47:28
## 22: 2022-11-09 15:47:28
## 23: 2022-11-09 15:47:28
## 24: 2022-11-09 15:47:28
## 25: 2022-11-09 15:47:29
## 26: 2022-11-09 15:47:29
## 27: 2022-11-09 15:47:29
## createdDate
# Remove all Cache items that happened within a rnorm() call
clearCache(tmpDir, userTags = "rnorm", ask = FALSE)## Cache size: ## Total (including Rasters): 252 bytes## Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) ## empty## Cache size: ## Total (including Rasters): 0 bytes## Selected objects (not including Rasters): 0 bytes## Empty data.table (0 rows and 4 cols): cacheId,tagKey,tagValue,createdDate
# Also, can set a time before caching happens and remove based on this
# --> a useful, simple way to control Cache
ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
startTime <- Sys.time()
Sys.sleep(1)
ranNumsB <- Cache(rnorm, 5, cacheRepo = tmpDir, userTags = "objectName:b")
keepCache(tmpDir, after = startTime, ask = FALSE) # keep only those newer than startTime## Cache size: ## Total (including Rasters): 256 bytes## Selected objects (not including Rasters): 256 bytes## cacheId tagKey tagValue
## 1: 142b4176fe87d51d objectName b
## 2: 142b4176fe87d51d function rnorm
## 3: 142b4176fe87d51d class numeric
## 4: 142b4176fe87d51d object.size 1024
## 5: 142b4176fe87d51d accessed 2022-11-09 15:47:30
## 6: 142b4176fe87d51d inCloud FALSE
## 7: 142b4176fe87d51d resultHash
## 8: 142b4176fe87d51d elapsedTimeDigest 0.0004806519 secs
## 9: 142b4176fe87d51d elapsedTimeFirstRun 4.172325e-05 secs
## 10: 142b4176fe87d51d otherFunctions saveRDS
## 11: 142b4176fe87d51d otherFunctions do.call
## 12: 142b4176fe87d51d otherFunctions process_file
## 13: 142b4176fe87d51d otherFunctions process_group
## 14: 142b4176fe87d51d otherFunctions process_group.block
## 15: 142b4176fe87d51d otherFunctions call_block
## 16: 142b4176fe87d51d otherFunctions block_exec
## 17: 142b4176fe87d51d otherFunctions eng_r
## 18: 142b4176fe87d51d otherFunctions in_input_dir
## 19: 142b4176fe87d51d otherFunctions in_dir
## 20: 142b4176fe87d51d otherFunctions timing_fn
## 21: 142b4176fe87d51d otherFunctions handle
## 22: 142b4176fe87d51d preDigest n:a4f076b3db622faf
## 23: 142b4176fe87d51d preDigest .FUN:4f604aa46882b368
## 24: 142b4176fe87d51d file.size 183
## cacheId tagKey tagValue
## createdDate
## 1: 2022-11-09 15:47:30
## 2: 2022-11-09 15:47:30
## 3: 2022-11-09 15:47:30
## 4: 2022-11-09 15:47:30
## 5: 2022-11-09 15:47:30
## 6: 2022-11-09 15:47:30
## 7: 2022-11-09 15:47:30
## 8: 2022-11-09 15:47:30
## 9: 2022-11-09 15:47:30
## 10: 2022-11-09 15:47:30
## 11: 2022-11-09 15:47:30
## 12: 2022-11-09 15:47:30
## 13: 2022-11-09 15:47:30
## 14: 2022-11-09 15:47:30
## 15: 2022-11-09 15:47:30
## 16: 2022-11-09 15:47:30
## 17: 2022-11-09 15:47:30
## 18: 2022-11-09 15:47:30
## 19: 2022-11-09 15:47:30
## 20: 2022-11-09 15:47:30
## 21: 2022-11-09 15:47:30
## 22: 2022-11-09 15:47:30
## 23: 2022-11-09 15:47:30
## 24: 2022-11-09 15:47:30
## createdDate
clearCache(tmpDir, ask = FALSE)Example 4: searching for multiple objects in the cache
# default userTags is "and" matching; for "or" matching use |
ranNumsA <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:b")
# show all objects (runif and rnorm in this case)
showCache(tmpDir)## Cache size: ## Total (including Rasters): 504 bytes## Selected objects (not including Rasters): 504 bytes## cacheId tagKey tagValue
## 1: 3aef38d1fc02aee5 objectName a
## 2: 3aef38d1fc02aee5 function runif
## 3: 3aef38d1fc02aee5 class numeric
## 4: 3aef38d1fc02aee5 object.size 1008
## 5: 3aef38d1fc02aee5 accessed 2022-11-09 15:47:30
## 6: 3aef38d1fc02aee5 inCloud FALSE
## 7: 3aef38d1fc02aee5 resultHash
## 8: 3aef38d1fc02aee5 elapsedTimeDigest 0.0004980564 secs
## 9: 3aef38d1fc02aee5 elapsedTimeFirstRun 4.1008e-05 secs
## 10: 3aef38d1fc02aee5 otherFunctions saveRDS
## 11: 3aef38d1fc02aee5 otherFunctions do.call
## 12: 3aef38d1fc02aee5 otherFunctions process_file
## 13: 3aef38d1fc02aee5 otherFunctions process_group
## 14: 3aef38d1fc02aee5 otherFunctions process_group.block
## 15: 3aef38d1fc02aee5 otherFunctions call_block
## 16: 3aef38d1fc02aee5 otherFunctions block_exec
## 17: 3aef38d1fc02aee5 otherFunctions eng_r
## 18: 3aef38d1fc02aee5 otherFunctions in_input_dir
## 19: 3aef38d1fc02aee5 otherFunctions in_dir
## 20: 3aef38d1fc02aee5 otherFunctions timing_fn
## 21: 3aef38d1fc02aee5 otherFunctions handle
## 22: 3aef38d1fc02aee5 preDigest n:7eef4eae85fd9229
## 23: 3aef38d1fc02aee5 preDigest .FUN:881ec847b7161f3c
## 24: 3aef38d1fc02aee5 file.size 168
## 25: f7bee22047b8d0c1 objectName b
## 26: f7bee22047b8d0c1 function rnorm
## 27: f7bee22047b8d0c1 class numeric
## 28: f7bee22047b8d0c1 object.size 1008
## 29: f7bee22047b8d0c1 accessed 2022-11-09 15:47:31
## 30: f7bee22047b8d0c1 inCloud FALSE
## 31: f7bee22047b8d0c1 resultHash
## 32: f7bee22047b8d0c1 elapsedTimeDigest 0.0004622936 secs
## 33: f7bee22047b8d0c1 elapsedTimeFirstRun 4.076958e-05 secs
## 34: f7bee22047b8d0c1 otherFunctions saveRDS
## 35: f7bee22047b8d0c1 otherFunctions do.call
## 36: f7bee22047b8d0c1 otherFunctions process_file
## 37: f7bee22047b8d0c1 otherFunctions process_group
## 38: f7bee22047b8d0c1 otherFunctions process_group.block
## 39: f7bee22047b8d0c1 otherFunctions call_block
## 40: f7bee22047b8d0c1 otherFunctions block_exec
## 41: f7bee22047b8d0c1 otherFunctions eng_r
## 42: f7bee22047b8d0c1 otherFunctions in_input_dir
## 43: f7bee22047b8d0c1 otherFunctions in_dir
## 44: f7bee22047b8d0c1 otherFunctions timing_fn
## 45: f7bee22047b8d0c1 otherFunctions handle
## 46: f7bee22047b8d0c1 preDigest n:7eef4eae85fd9229
## 47: f7bee22047b8d0c1 preDigest .FUN:4f604aa46882b368
## 48: f7bee22047b8d0c1 file.size 174
## cacheId tagKey tagValue
## createdDate
## 1: 2022-11-09 15:47:30
## 2: 2022-11-09 15:47:30
## 3: 2022-11-09 15:47:30
## 4: 2022-11-09 15:47:30
## 5: 2022-11-09 15:47:30
## 6: 2022-11-09 15:47:30
## 7: 2022-11-09 15:47:30
## 8: 2022-11-09 15:47:30
## 9: 2022-11-09 15:47:30
## 10: 2022-11-09 15:47:30
## 11: 2022-11-09 15:47:30
## 12: 2022-11-09 15:47:30
## 13: 2022-11-09 15:47:30
## 14: 2022-11-09 15:47:30
## 15: 2022-11-09 15:47:30
## 16: 2022-11-09 15:47:30
## 17: 2022-11-09 15:47:30
## 18: 2022-11-09 15:47:30
## 19: 2022-11-09 15:47:30
## 20: 2022-11-09 15:47:30
## 21: 2022-11-09 15:47:30
## 22: 2022-11-09 15:47:30
## 23: 2022-11-09 15:47:30
## 24: 2022-11-09 15:47:30
## 25: 2022-11-09 15:47:31
## 26: 2022-11-09 15:47:31
## 27: 2022-11-09 15:47:31
## 28: 2022-11-09 15:47:31
## 29: 2022-11-09 15:47:31
## 30: 2022-11-09 15:47:31
## 31: 2022-11-09 15:47:31
## 32: 2022-11-09 15:47:31
## 33: 2022-11-09 15:47:31
## 34: 2022-11-09 15:47:31
## 35: 2022-11-09 15:47:31
## 36: 2022-11-09 15:47:31
## 37: 2022-11-09 15:47:31
## 38: 2022-11-09 15:47:31
## 39: 2022-11-09 15:47:31
## 40: 2022-11-09 15:47:31
## 41: 2022-11-09 15:47:31
## 42: 2022-11-09 15:47:31
## 43: 2022-11-09 15:47:31
## 44: 2022-11-09 15:47:31
## 45: 2022-11-09 15:47:31
## 46: 2022-11-09 15:47:31
## 47: 2022-11-09 15:47:31
## 48: 2022-11-09 15:47:31
## createdDate
# show objects that are both runif and rnorm
# (i.e., none in this case, because objecs are either or, not both)
showCache(tmpDir, userTags = c("runif", "rnorm")) ## empty## Cache size: ## Total (including Rasters): 0 bytes## Selected objects (not including Rasters): 0 bytes## Empty data.table (0 rows and 4 cols): cacheId,tagKey,tagValue,createdDate
# show objects that are either runif or rnorm ("or" search)
showCache(tmpDir, userTags = "runif|rnorm")## Cache size: ## Total (including Rasters): 504 bytes## Selected objects (not including Rasters): 504 bytes## cacheId tagKey tagValue
## 1: 3aef38d1fc02aee5 objectName a
## 2: 3aef38d1fc02aee5 function runif
## 3: 3aef38d1fc02aee5 class numeric
## 4: 3aef38d1fc02aee5 object.size 1008
## 5: 3aef38d1fc02aee5 accessed 2022-11-09 15:47:30
## 6: 3aef38d1fc02aee5 inCloud FALSE
## 7: 3aef38d1fc02aee5 resultHash
## 8: 3aef38d1fc02aee5 elapsedTimeDigest 0.0004980564 secs
## 9: 3aef38d1fc02aee5 elapsedTimeFirstRun 4.1008e-05 secs
## 10: 3aef38d1fc02aee5 otherFunctions saveRDS
## 11: 3aef38d1fc02aee5 otherFunctions do.call
## 12: 3aef38d1fc02aee5 otherFunctions process_file
## 13: 3aef38d1fc02aee5 otherFunctions process_group
## 14: 3aef38d1fc02aee5 otherFunctions process_group.block
## 15: 3aef38d1fc02aee5 otherFunctions call_block
## 16: 3aef38d1fc02aee5 otherFunctions block_exec
## 17: 3aef38d1fc02aee5 otherFunctions eng_r
## 18: 3aef38d1fc02aee5 otherFunctions in_input_dir
## 19: 3aef38d1fc02aee5 otherFunctions in_dir
## 20: 3aef38d1fc02aee5 otherFunctions timing_fn
## 21: 3aef38d1fc02aee5 otherFunctions handle
## 22: 3aef38d1fc02aee5 preDigest n:7eef4eae85fd9229
## 23: 3aef38d1fc02aee5 preDigest .FUN:881ec847b7161f3c
## 24: 3aef38d1fc02aee5 file.size 168
## 25: f7bee22047b8d0c1 objectName b
## 26: f7bee22047b8d0c1 function rnorm
## 27: f7bee22047b8d0c1 class numeric
## 28: f7bee22047b8d0c1 object.size 1008
## 29: f7bee22047b8d0c1 accessed 2022-11-09 15:47:31
## 30: f7bee22047b8d0c1 inCloud FALSE
## 31: f7bee22047b8d0c1 resultHash
## 32: f7bee22047b8d0c1 elapsedTimeDigest 0.0004622936 secs
## 33: f7bee22047b8d0c1 elapsedTimeFirstRun 4.076958e-05 secs
## 34: f7bee22047b8d0c1 otherFunctions saveRDS
## 35: f7bee22047b8d0c1 otherFunctions do.call
## 36: f7bee22047b8d0c1 otherFunctions process_file
## 37: f7bee22047b8d0c1 otherFunctions process_group
## 38: f7bee22047b8d0c1 otherFunctions process_group.block
## 39: f7bee22047b8d0c1 otherFunctions call_block
## 40: f7bee22047b8d0c1 otherFunctions block_exec
## 41: f7bee22047b8d0c1 otherFunctions eng_r
## 42: f7bee22047b8d0c1 otherFunctions in_input_dir
## 43: f7bee22047b8d0c1 otherFunctions in_dir
## 44: f7bee22047b8d0c1 otherFunctions timing_fn
## 45: f7bee22047b8d0c1 otherFunctions handle
## 46: f7bee22047b8d0c1 preDigest n:7eef4eae85fd9229
## 47: f7bee22047b8d0c1 preDigest .FUN:4f604aa46882b368
## 48: f7bee22047b8d0c1 file.size 174
## cacheId tagKey tagValue
## createdDate
## 1: 2022-11-09 15:47:30
## 2: 2022-11-09 15:47:30
## 3: 2022-11-09 15:47:30
## 4: 2022-11-09 15:47:30
## 5: 2022-11-09 15:47:30
## 6: 2022-11-09 15:47:30
## 7: 2022-11-09 15:47:30
## 8: 2022-11-09 15:47:30
## 9: 2022-11-09 15:47:30
## 10: 2022-11-09 15:47:30
## 11: 2022-11-09 15:47:30
## 12: 2022-11-09 15:47:30
## 13: 2022-11-09 15:47:30
## 14: 2022-11-09 15:47:30
## 15: 2022-11-09 15:47:30
## 16: 2022-11-09 15:47:30
## 17: 2022-11-09 15:47:30
## 18: 2022-11-09 15:47:30
## 19: 2022-11-09 15:47:30
## 20: 2022-11-09 15:47:30
## 21: 2022-11-09 15:47:30
## 22: 2022-11-09 15:47:30
## 23: 2022-11-09 15:47:30
## 24: 2022-11-09 15:47:30
## 25: 2022-11-09 15:47:31
## 26: 2022-11-09 15:47:31
## 27: 2022-11-09 15:47:31
## 28: 2022-11-09 15:47:31
## 29: 2022-11-09 15:47:31
## 30: 2022-11-09 15:47:31
## 31: 2022-11-09 15:47:31
## 32: 2022-11-09 15:47:31
## 33: 2022-11-09 15:47:31
## 34: 2022-11-09 15:47:31
## 35: 2022-11-09 15:47:31
## 36: 2022-11-09 15:47:31
## 37: 2022-11-09 15:47:31
## 38: 2022-11-09 15:47:31
## 39: 2022-11-09 15:47:31
## 40: 2022-11-09 15:47:31
## 41: 2022-11-09 15:47:31
## 42: 2022-11-09 15:47:31
## 43: 2022-11-09 15:47:31
## 44: 2022-11-09 15:47:31
## 45: 2022-11-09 15:47:31
## 46: 2022-11-09 15:47:31
## 47: 2022-11-09 15:47:31
## 48: 2022-11-09 15:47:31
## createdDate
# keep only objects that are either runif or rnorm ("or" search)
keepCache(tmpDir, userTags = "runif|rnorm", ask = FALSE)## Cache size: ## Total (including Rasters): 504 bytes## Selected objects (not including Rasters): 504 bytes## cacheId tagKey tagValue
## 1: 3aef38d1fc02aee5 objectName a
## 2: 3aef38d1fc02aee5 function runif
## 3: 3aef38d1fc02aee5 class numeric
## 4: 3aef38d1fc02aee5 object.size 1008
## 5: 3aef38d1fc02aee5 accessed 2022-11-09 15:47:30
## 6: 3aef38d1fc02aee5 inCloud FALSE
## 7: 3aef38d1fc02aee5 resultHash
## 8: 3aef38d1fc02aee5 elapsedTimeDigest 0.0004980564 secs
## 9: 3aef38d1fc02aee5 elapsedTimeFirstRun 4.1008e-05 secs
## 10: 3aef38d1fc02aee5 otherFunctions saveRDS
## 11: 3aef38d1fc02aee5 otherFunctions do.call
## 12: 3aef38d1fc02aee5 otherFunctions process_file
## 13: 3aef38d1fc02aee5 otherFunctions process_group
## 14: 3aef38d1fc02aee5 otherFunctions process_group.block
## 15: 3aef38d1fc02aee5 otherFunctions call_block
## 16: 3aef38d1fc02aee5 otherFunctions block_exec
## 17: 3aef38d1fc02aee5 otherFunctions eng_r
## 18: 3aef38d1fc02aee5 otherFunctions in_input_dir
## 19: 3aef38d1fc02aee5 otherFunctions in_dir
## 20: 3aef38d1fc02aee5 otherFunctions timing_fn
## 21: 3aef38d1fc02aee5 otherFunctions handle
## 22: 3aef38d1fc02aee5 preDigest n:7eef4eae85fd9229
## 23: 3aef38d1fc02aee5 preDigest .FUN:881ec847b7161f3c
## 24: 3aef38d1fc02aee5 file.size 168
## 25: f7bee22047b8d0c1 objectName b
## 26: f7bee22047b8d0c1 function rnorm
## 27: f7bee22047b8d0c1 class numeric
## 28: f7bee22047b8d0c1 object.size 1008
## 29: f7bee22047b8d0c1 accessed 2022-11-09 15:47:31
## 30: f7bee22047b8d0c1 inCloud FALSE
## 31: f7bee22047b8d0c1 resultHash
## 32: f7bee22047b8d0c1 elapsedTimeDigest 0.0004622936 secs
## 33: f7bee22047b8d0c1 elapsedTimeFirstRun 4.076958e-05 secs
## 34: f7bee22047b8d0c1 otherFunctions saveRDS
## 35: f7bee22047b8d0c1 otherFunctions do.call
## 36: f7bee22047b8d0c1 otherFunctions process_file
## 37: f7bee22047b8d0c1 otherFunctions process_group
## 38: f7bee22047b8d0c1 otherFunctions process_group.block
## 39: f7bee22047b8d0c1 otherFunctions call_block
## 40: f7bee22047b8d0c1 otherFunctions block_exec
## 41: f7bee22047b8d0c1 otherFunctions eng_r
## 42: f7bee22047b8d0c1 otherFunctions in_input_dir
## 43: f7bee22047b8d0c1 otherFunctions in_dir
## 44: f7bee22047b8d0c1 otherFunctions timing_fn
## 45: f7bee22047b8d0c1 otherFunctions handle
## 46: f7bee22047b8d0c1 preDigest n:7eef4eae85fd9229
## 47: f7bee22047b8d0c1 preDigest .FUN:4f604aa46882b368
## 48: f7bee22047b8d0c1 file.size 174
## cacheId tagKey tagValue
## createdDate
## 1: 2022-11-09 15:47:30
## 2: 2022-11-09 15:47:30
## 3: 2022-11-09 15:47:30
## 4: 2022-11-09 15:47:30
## 5: 2022-11-09 15:47:30
## 6: 2022-11-09 15:47:30
## 7: 2022-11-09 15:47:30
## 8: 2022-11-09 15:47:30
## 9: 2022-11-09 15:47:30
## 10: 2022-11-09 15:47:30
## 11: 2022-11-09 15:47:30
## 12: 2022-11-09 15:47:30
## 13: 2022-11-09 15:47:30
## 14: 2022-11-09 15:47:30
## 15: 2022-11-09 15:47:30
## 16: 2022-11-09 15:47:30
## 17: 2022-11-09 15:47:30
## 18: 2022-11-09 15:47:30
## 19: 2022-11-09 15:47:30
## 20: 2022-11-09 15:47:30
## 21: 2022-11-09 15:47:30
## 22: 2022-11-09 15:47:30
## 23: 2022-11-09 15:47:30
## 24: 2022-11-09 15:47:30
## 25: 2022-11-09 15:47:31
## 26: 2022-11-09 15:47:31
## 27: 2022-11-09 15:47:31
## 28: 2022-11-09 15:47:31
## 29: 2022-11-09 15:47:31
## 30: 2022-11-09 15:47:31
## 31: 2022-11-09 15:47:31
## 32: 2022-11-09 15:47:31
## 33: 2022-11-09 15:47:31
## 34: 2022-11-09 15:47:31
## 35: 2022-11-09 15:47:31
## 36: 2022-11-09 15:47:31
## 37: 2022-11-09 15:47:31
## 38: 2022-11-09 15:47:31
## 39: 2022-11-09 15:47:31
## 40: 2022-11-09 15:47:31
## 41: 2022-11-09 15:47:31
## 42: 2022-11-09 15:47:31
## 43: 2022-11-09 15:47:31
## 44: 2022-11-09 15:47:31
## 45: 2022-11-09 15:47:31
## 46: 2022-11-09 15:47:31
## 47: 2022-11-09 15:47:31
## 48: 2022-11-09 15:47:31
## createdDate
clearCache(tmpDir, ask = FALSE)Example 5: using caching to speed up rerunning expensive computations
ras <- raster(extent(0, 5, 0, 5), res = 1,
vals = sample(1:5, replace = TRUE, size = 25),
crs = "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84")
# A slow operation, like GIS operation
notCached <- suppressWarnings(
# project raster generates warnings when run non-interactively
projectRaster(ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)
cached <- suppressWarnings(
# project raster generates warnings when run non-interactively
# using quote works also
Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)
# second time is much faster
reRun <- suppressWarnings(
# project raster generates warnings when run non-interactively
Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)## ...(Object to retrieve (03dc12ab39aa3710.rds))## loaded cached result from previous projectRaster call
# recovered cached version is same as non-cached version
all.equal(notCached, reRun) ## TRUE## [1] TRUENested Caching
Nested caching, which is when Caching of a function occurs inside an outer function, which is itself cached. This is a critical element to working within a reproducible work flow. It is not enough during development to cache flat code chunks, as there will be many levels of “slow” functions. Ideally, at all points in a development cycle, it should be possible to get to any line of code starting from the very initial steps, running through everything up to that point, in less than a few seconds. If the workflow can be kept very fast like this, then there is a guarantee that it will work at any point.
##########################
## Nested Caching
# Make 2 functions
inner <- function(mean) {
d <- 1
Cache(rnorm, n = 3, mean = mean)
}
outer <- function(n) {
Cache(inner, 0.1, cacheRepo = tmpdir2)
}
# make 2 different cache paths
tmpdir1 <- file.path(tempdir(), "first")
tmpdir2 <- file.path(tempdir(), "second")
# Run the Cache ... notOlderThan propagates to all 3 Cache calls,
# but cacheRepo is tmpdir1 in top level Cache and all nested
# Cache calls, unless individually overridden ... here inner
# uses tmpdir2 repository
Cache(outer, n = 2, cacheRepo = tmpdir1, notOlderThan = Sys.time())## [1] 0.86106044 -1.13559552 -0.04837833
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
##
## attr(,"tags")
## [1] "cacheId:a7af5367c13aba8f"
## attr(,"call")
## [1] ""
showCache(tmpdir1) # 2 function calls## Cache size: ## Total (including Rasters): 504 bytes## Selected objects (not including Rasters): 504 bytes## cacheId tagKey tagValue
## 1: 4ac2b7c0f42d1e46 function rnorm
## 2: 4ac2b7c0f42d1e46 class numeric
## 3: 4ac2b7c0f42d1e46 object.size 1008
## 4: 4ac2b7c0f42d1e46 accessed 2022-11-09 15:47:31
## 5: 4ac2b7c0f42d1e46 inCloud FALSE
## 6: 4ac2b7c0f42d1e46 resultHash
## 7: 4ac2b7c0f42d1e46 elapsedTimeDigest 0.0005071163 secs
## 8: 4ac2b7c0f42d1e46 elapsedTimeFirstRun 4.029274e-05 secs
## 9: 4ac2b7c0f42d1e46 otherFunctions saveRDS
## 10: 4ac2b7c0f42d1e46 otherFunctions do.call
## 11: 4ac2b7c0f42d1e46 otherFunctions process_file
## 12: 4ac2b7c0f42d1e46 otherFunctions process_group
## 13: 4ac2b7c0f42d1e46 otherFunctions process_group.block
## 14: 4ac2b7c0f42d1e46 otherFunctions call_block
## 15: 4ac2b7c0f42d1e46 otherFunctions block_exec
## 16: 4ac2b7c0f42d1e46 otherFunctions eng_r
## 17: 4ac2b7c0f42d1e46 otherFunctions in_input_dir
## 18: 4ac2b7c0f42d1e46 otherFunctions in_dir
## 19: 4ac2b7c0f42d1e46 otherFunctions timing_fn
## 20: 4ac2b7c0f42d1e46 otherFunctions handle
## 21: 4ac2b7c0f42d1e46 preDigest n:7f12988bd88a0fb8
## 22: 4ac2b7c0f42d1e46 preDigest mean:22413394efd9f6a3
## 23: 4ac2b7c0f42d1e46 preDigest .FUN:4f604aa46882b368
## 24: 4ac2b7c0f42d1e46 file.size 165
## 25: a7af5367c13aba8f function outer
## 26: a7af5367c13aba8f class numeric
## 27: a7af5367c13aba8f object.size 1008
## 28: a7af5367c13aba8f accessed 2022-11-09 15:47:31
## 29: a7af5367c13aba8f inCloud FALSE
## 30: a7af5367c13aba8f resultHash
## 31: a7af5367c13aba8f elapsedTimeDigest 0.0005381107 secs
## 32: a7af5367c13aba8f elapsedTimeFirstRun 0.06152153 secs
## 33: a7af5367c13aba8f otherFunctions saveRDS
## 34: a7af5367c13aba8f otherFunctions do.call
## 35: a7af5367c13aba8f otherFunctions process_file
## 36: a7af5367c13aba8f otherFunctions process_group
## 37: a7af5367c13aba8f otherFunctions process_group.block
## 38: a7af5367c13aba8f otherFunctions call_block
## 39: a7af5367c13aba8f otherFunctions block_exec
## 40: a7af5367c13aba8f otherFunctions eng_r
## 41: a7af5367c13aba8f otherFunctions in_input_dir
## 42: a7af5367c13aba8f otherFunctions in_dir
## 43: a7af5367c13aba8f otherFunctions timing_fn
## 44: a7af5367c13aba8f otherFunctions handle
## 45: a7af5367c13aba8f preDigest n:82dc709f2b91918a
## 46: a7af5367c13aba8f preDigest .FUN:892a6afc47a63a90
## 47: a7af5367c13aba8f file.size 165
## cacheId tagKey tagValue
## createdDate
## 1: 2022-11-09 15:47:31
## 2: 2022-11-09 15:47:31
## 3: 2022-11-09 15:47:31
## 4: 2022-11-09 15:47:31
## 5: 2022-11-09 15:47:31
## 6: 2022-11-09 15:47:31
## 7: 2022-11-09 15:47:31
## 8: 2022-11-09 15:47:31
## 9: 2022-11-09 15:47:31
## 10: 2022-11-09 15:47:31
## 11: 2022-11-09 15:47:31
## 12: 2022-11-09 15:47:31
## 13: 2022-11-09 15:47:31
## 14: 2022-11-09 15:47:31
## 15: 2022-11-09 15:47:31
## 16: 2022-11-09 15:47:31
## 17: 2022-11-09 15:47:31
## 18: 2022-11-09 15:47:31
## 19: 2022-11-09 15:47:31
## 20: 2022-11-09 15:47:31
## 21: 2022-11-09 15:47:31
## 22: 2022-11-09 15:47:31
## 23: 2022-11-09 15:47:31
## 24: 2022-11-09 15:47:31
## 25: 2022-11-09 15:47:31
## 26: 2022-11-09 15:47:31
## 27: 2022-11-09 15:47:31
## 28: 2022-11-09 15:47:31
## 29: 2022-11-09 15:47:31
## 30: 2022-11-09 15:47:31
## 31: 2022-11-09 15:47:31
## 32: 2022-11-09 15:47:31
## 33: 2022-11-09 15:47:31
## 34: 2022-11-09 15:47:31
## 35: 2022-11-09 15:47:31
## 36: 2022-11-09 15:47:31
## 37: 2022-11-09 15:47:31
## 38: 2022-11-09 15:47:31
## 39: 2022-11-09 15:47:31
## 40: 2022-11-09 15:47:31
## 41: 2022-11-09 15:47:31
## 42: 2022-11-09 15:47:31
## 43: 2022-11-09 15:47:31
## 44: 2022-11-09 15:47:31
## 45: 2022-11-09 15:47:31
## 46: 2022-11-09 15:47:31
## 47: 2022-11-09 15:47:31
## createdDate
showCache(tmpdir2) # 1 function call## Cache size: ## Total (including Rasters): 252 bytes## Selected objects (not including Rasters): 252 bytes## cacheId tagKey tagValue
## 1: 33ceb4fb525fd08f function inner
## 2: 33ceb4fb525fd08f class numeric
## 3: 33ceb4fb525fd08f object.size 1008
## 4: 33ceb4fb525fd08f accessed 2022-11-09 15:47:31
## 5: 33ceb4fb525fd08f inCloud FALSE
## 6: 33ceb4fb525fd08f resultHash
## 7: 33ceb4fb525fd08f elapsedTimeDigest 0.0005793571 secs
## 8: 33ceb4fb525fd08f elapsedTimeFirstRun 0.02184701 secs
## 9: 33ceb4fb525fd08f otherFunctions saveRDS
## 10: 33ceb4fb525fd08f otherFunctions do.call
## 11: 33ceb4fb525fd08f otherFunctions process_file
## 12: 33ceb4fb525fd08f otherFunctions process_group
## 13: 33ceb4fb525fd08f otherFunctions process_group.block
## 14: 33ceb4fb525fd08f otherFunctions call_block
## 15: 33ceb4fb525fd08f otherFunctions block_exec
## 16: 33ceb4fb525fd08f otherFunctions eng_r
## 17: 33ceb4fb525fd08f otherFunctions in_input_dir
## 18: 33ceb4fb525fd08f otherFunctions in_dir
## 19: 33ceb4fb525fd08f otherFunctions timing_fn
## 20: 33ceb4fb525fd08f otherFunctions handle
## 21: 33ceb4fb525fd08f preDigest mean:22413394efd9f6a3
## 22: 33ceb4fb525fd08f preDigest .FUN:87e2c30917a34d25
## 23: 33ceb4fb525fd08f file.size 165
## cacheId tagKey tagValue
## createdDate
## 1: 2022-11-09 15:47:31
## 2: 2022-11-09 15:47:31
## 3: 2022-11-09 15:47:31
## 4: 2022-11-09 15:47:31
## 5: 2022-11-09 15:47:31
## 6: 2022-11-09 15:47:31
## 7: 2022-11-09 15:47:31
## 8: 2022-11-09 15:47:31
## 9: 2022-11-09 15:47:31
## 10: 2022-11-09 15:47:31
## 11: 2022-11-09 15:47:31
## 12: 2022-11-09 15:47:31
## 13: 2022-11-09 15:47:31
## 14: 2022-11-09 15:47:31
## 15: 2022-11-09 15:47:31
## 16: 2022-11-09 15:47:31
## 17: 2022-11-09 15:47:31
## 18: 2022-11-09 15:47:31
## 19: 2022-11-09 15:47:31
## 20: 2022-11-09 15:47:31
## 21: 2022-11-09 15:47:31
## 22: 2022-11-09 15:47:31
## 23: 2022-11-09 15:47:31
## createdDate
# userTags get appended
# all items have the outer tag propagate, plus inner ones only have inner ones
clearCache(tmpdir1, ask = FALSE)
outerTag <- "outerTag"
innerTag <- "innerTag"
inner <- function(mean) {
d <- 1
Cache(rnorm, n = 3, mean = mean, notOlderThan = Sys.time() - 1e5, userTags = innerTag)
}
outer <- function(n) {
Cache(inner, 0.1)
}
aa <- Cache(outer, n = 2, cacheRepo = tmpdir1, userTags = outerTag)
showCache(tmpdir1) # rnorm function has outerTag and innerTag, inner and outer only have outerTag## Cache size: ## Total (including Rasters): 756 bytes## Selected objects (not including Rasters): 756 bytes## cacheId tagKey tagValue
## 1: 4ac2b7c0f42d1e46 innerTag innerTag
## 2: 4ac2b7c0f42d1e46 outerTag outerTag
## 3: 4ac2b7c0f42d1e46 function rnorm
## 4: 4ac2b7c0f42d1e46 class numeric
## 5: 4ac2b7c0f42d1e46 object.size 1008
## 6: 4ac2b7c0f42d1e46 accessed 2022-11-09 15:47:31
## 7: 4ac2b7c0f42d1e46 inCloud FALSE
## 8: 4ac2b7c0f42d1e46 resultHash
## 9: 4ac2b7c0f42d1e46 elapsedTimeDigest 0.0005271435 secs
## 10: 4ac2b7c0f42d1e46 elapsedTimeFirstRun 8.201599e-05 secs
## 11: 4ac2b7c0f42d1e46 otherFunctions saveRDS
## 12: 4ac2b7c0f42d1e46 otherFunctions do.call
## 13: 4ac2b7c0f42d1e46 otherFunctions process_file
## 14: 4ac2b7c0f42d1e46 otherFunctions process_group
## 15: 4ac2b7c0f42d1e46 otherFunctions process_group.block
## 16: 4ac2b7c0f42d1e46 otherFunctions call_block
## 17: 4ac2b7c0f42d1e46 otherFunctions block_exec
## 18: 4ac2b7c0f42d1e46 otherFunctions eng_r
## 19: 4ac2b7c0f42d1e46 otherFunctions in_input_dir
## 20: 4ac2b7c0f42d1e46 otherFunctions in_dir
## 21: 4ac2b7c0f42d1e46 otherFunctions timing_fn
## 22: 4ac2b7c0f42d1e46 otherFunctions handle
## 23: 4ac2b7c0f42d1e46 preDigest n:7f12988bd88a0fb8
## 24: 4ac2b7c0f42d1e46 preDigest mean:22413394efd9f6a3
## 25: 4ac2b7c0f42d1e46 preDigest .FUN:4f604aa46882b368
## 26: 4ac2b7c0f42d1e46 file.size 165
## 27: b06af03d5a73dc7d outerTag outerTag
## 28: b06af03d5a73dc7d function inner
## 29: b06af03d5a73dc7d class numeric
## 30: b06af03d5a73dc7d object.size 1008
## 31: b06af03d5a73dc7d accessed 2022-11-09 15:47:31
## 32: b06af03d5a73dc7d inCloud FALSE
## 33: b06af03d5a73dc7d resultHash
## 34: b06af03d5a73dc7d elapsedTimeDigest 0.0004880428 secs
## 35: b06af03d5a73dc7d elapsedTimeFirstRun 0.02314568 secs
## 36: b06af03d5a73dc7d otherFunctions saveRDS
## 37: b06af03d5a73dc7d otherFunctions do.call
## 38: b06af03d5a73dc7d otherFunctions process_file
## 39: b06af03d5a73dc7d otherFunctions process_group
## 40: b06af03d5a73dc7d otherFunctions process_group.block
## 41: b06af03d5a73dc7d otherFunctions call_block
## 42: b06af03d5a73dc7d otherFunctions block_exec
## 43: b06af03d5a73dc7d otherFunctions eng_r
## 44: b06af03d5a73dc7d otherFunctions in_input_dir
## 45: b06af03d5a73dc7d otherFunctions in_dir
## 46: b06af03d5a73dc7d otherFunctions timing_fn
## 47: b06af03d5a73dc7d otherFunctions handle
## 48: b06af03d5a73dc7d preDigest mean:22413394efd9f6a3
## 49: b06af03d5a73dc7d preDigest .FUN:7ad10bc1ae528d8c
## 50: b06af03d5a73dc7d file.size 165
## 51: 88a34e1d033329e5 outerTag outerTag
## 52: 88a34e1d033329e5 function outer
## 53: 88a34e1d033329e5 class numeric
## 54: 88a34e1d033329e5 object.size 1008
## 55: 88a34e1d033329e5 accessed 2022-11-09 15:47:31
## 56: 88a34e1d033329e5 inCloud FALSE
## 57: 88a34e1d033329e5 resultHash
## 58: 88a34e1d033329e5 elapsedTimeDigest 0.0004944801 secs
## 59: 88a34e1d033329e5 elapsedTimeFirstRun 0.04455709 secs
## 60: 88a34e1d033329e5 otherFunctions saveRDS
## 61: 88a34e1d033329e5 otherFunctions do.call
## 62: 88a34e1d033329e5 otherFunctions process_file
## 63: 88a34e1d033329e5 otherFunctions process_group
## 64: 88a34e1d033329e5 otherFunctions process_group.block
## 65: 88a34e1d033329e5 otherFunctions call_block
## 66: 88a34e1d033329e5 otherFunctions block_exec
## 67: 88a34e1d033329e5 otherFunctions eng_r
## 68: 88a34e1d033329e5 otherFunctions in_input_dir
## 69: 88a34e1d033329e5 otherFunctions in_dir
## 70: 88a34e1d033329e5 otherFunctions timing_fn
## 71: 88a34e1d033329e5 otherFunctions handle
## 72: 88a34e1d033329e5 preDigest n:82dc709f2b91918a
## 73: 88a34e1d033329e5 preDigest .FUN:5f06fb5fbffe9e3b
## 74: 88a34e1d033329e5 file.size 165
## cacheId tagKey tagValue
## createdDate
## 1: 2022-11-09 15:47:31
## 2: 2022-11-09 15:47:31
## 3: 2022-11-09 15:47:31
## 4: 2022-11-09 15:47:31
## 5: 2022-11-09 15:47:31
## 6: 2022-11-09 15:47:31
## 7: 2022-11-09 15:47:31
## 8: 2022-11-09 15:47:31
## 9: 2022-11-09 15:47:31
## 10: 2022-11-09 15:47:31
## 11: 2022-11-09 15:47:31
## 12: 2022-11-09 15:47:31
## 13: 2022-11-09 15:47:31
## 14: 2022-11-09 15:47:31
## 15: 2022-11-09 15:47:31
## 16: 2022-11-09 15:47:31
## 17: 2022-11-09 15:47:31
## 18: 2022-11-09 15:47:31
## 19: 2022-11-09 15:47:31
## 20: 2022-11-09 15:47:31
## 21: 2022-11-09 15:47:31
## 22: 2022-11-09 15:47:31
## 23: 2022-11-09 15:47:31
## 24: 2022-11-09 15:47:31
## 25: 2022-11-09 15:47:31
## 26: 2022-11-09 15:47:31
## 27: 2022-11-09 15:47:31
## 28: 2022-11-09 15:47:31
## 29: 2022-11-09 15:47:31
## 30: 2022-11-09 15:47:31
## 31: 2022-11-09 15:47:31
## 32: 2022-11-09 15:47:31
## 33: 2022-11-09 15:47:31
## 34: 2022-11-09 15:47:31
## 35: 2022-11-09 15:47:31
## 36: 2022-11-09 15:47:31
## 37: 2022-11-09 15:47:31
## 38: 2022-11-09 15:47:31
## 39: 2022-11-09 15:47:31
## 40: 2022-11-09 15:47:31
## 41: 2022-11-09 15:47:31
## 42: 2022-11-09 15:47:31
## 43: 2022-11-09 15:47:31
## 44: 2022-11-09 15:47:31
## 45: 2022-11-09 15:47:31
## 46: 2022-11-09 15:47:31
## 47: 2022-11-09 15:47:31
## 48: 2022-11-09 15:47:31
## 49: 2022-11-09 15:47:31
## 50: 2022-11-09 15:47:31
## 51: 2022-11-09 15:47:31
## 52: 2022-11-09 15:47:31
## 53: 2022-11-09 15:47:31
## 54: 2022-11-09 15:47:31
## 55: 2022-11-09 15:47:31
## 56: 2022-11-09 15:47:31
## 57: 2022-11-09 15:47:31
## 58: 2022-11-09 15:47:31
## 59: 2022-11-09 15:47:31
## 60: 2022-11-09 15:47:31
## 61: 2022-11-09 15:47:31
## 62: 2022-11-09 15:47:31
## 63: 2022-11-09 15:47:31
## 64: 2022-11-09 15:47:31
## 65: 2022-11-09 15:47:31
## 66: 2022-11-09 15:47:31
## 67: 2022-11-09 15:47:31
## 68: 2022-11-09 15:47:31
## 69: 2022-11-09 15:47:31
## 70: 2022-11-09 15:47:31
## 71: 2022-11-09 15:47:31
## 72: 2022-11-09 15:47:31
## 73: 2022-11-09 15:47:31
## 74: 2022-11-09 15:47:31
## createdDatecacheId
Sometimes, it is not absolutely desirable to maintain the work flow
intact because changes that are irrelevant to the analysis, such as
changing messages sent to a user, may be changed, without a desire to
rerun functions. The cacheId argument is for this. Once a
piece of code is run, then the cacheId can be manually
extracted (it is reported at the end of a Cache call) and manually
placed in the code, passed in as, say,
cacheId = "ad184ce64541972b50afd8e7b75f821b".
## [1] -0.6264538
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
##
## attr(,"tags")
## [1] "cacheId:7072c305d8c69df0"
## attr(,"call")
## [1] ""
# manually look at output attribute which shows cacheId: 7072c305d8c69df0
Cache(rnorm, 1, cacheRepo = tmpdir1, cacheId = "7072c305d8c69df0") # same value## cacheId is same as calculated hash## ...(Object to retrieve (7072c305d8c69df0.rds))## loaded cached result from previous rnorm call## [1] -0.6264538
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] FALSE
##
## attr(,"tags")
## [1] "cacheId:7072c305d8c69df0"
## attr(,"call")
## [1] ""
# override even with different inputs:
Cache(rnorm, 2, cacheRepo = tmpdir1, cacheId = "7072c305d8c69df0")## cacheId is not same as calculated hash. Manually searching for cacheId:7072c305d8c69df0## ...(Object to retrieve (7072c305d8c69df0.rds))## loaded cached result from previous rnorm call## [1] -0.6264538
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] FALSE
##
## attr(,"tags")
## [1] "cacheId:7072c305d8c69df0"
## attr(,"call")
## [1] ""Working with the Cache manually
Since the cache is simply a DBI data table (of an SQLite
database by default). In addition, there are several helpers in the
reproducible package, including showCache,
keepCache and clearCache that may be useful.
Also, one can access cached items manually (rather than simply rerunning
the same Cache function again).
# As of reproducible version 1.0, there is a new backend directly using DBI
mapHash <- unique(showCache(tmpDir, userTags = "projectRaster")$cacheId)## Cache size: ## Total (including Rasters): 3.4 Kb## Selected objects (not including Rasters): 3.4 Kb
map <- loadFromCache(mapHash[1], cachePath = tmpDir)
plot(map)