vignettes/Intro-to-Cache.Rmd
Intro-to-Cache.Rmd
As part of a reproducible workflow, caching of function calls, code
chunks, and other elements of a project is a critical component. The
objective of a reproducible workflow is is likely that an entire work
flow from raw data to publication, decision support, report writing,
presentation building etc., could be built and be reproducible anywhere,
on any computer, operating system, with any starting conditions, on
demand. The reproducible::Cache
function is built to work
with any R function.
Cache
users DBI
as a backend, with key
functions, dbReadTable
, dbRemoveTable
,
dbSendQuery
, dbSendStatement
,
dbCreateTable
and dbAppendTable
. These can all
be accessed via Cache
, showCache
,
clearCache
, and keepCache
. It is optimized for
speed of transactions, using fastdigest::fastdigest
on R
memory objects and digest::digest
on files. The main
function is superficially similar to archivist::cache
,
which uses digest::digest
in all cases to determine whether
the arguments are identical in subsequent iterations. It also but does
many things that make standard caching with
digest::digest
don’t work reliably between systems. For
these, the function .robustDigest
is introduced to make
caching transferable between systems. This is relevant for file paths,
environments, parallel clusters, functions (which are contained within
an environment), and many others (e.g., see ?.robustDigest
for methods). Cache
also adds important elements like
automated tagging and the option to retrieve disk-cached values via
stashed objects in memory using memoise::memoise
. This
means that running Cache
1, 2, and 3 times on the same
function will get progressively faster. This can be extremely useful for
web apps built with, say shiny
.
Any function can be cached by wrapping Cache
around the
function call, or by using base pipe |>
or using:
Cache(FUN = functionName, ...)
This will be a slight change to a function call, such as:
terra::project(raster, crs = terra::crs(newRaster))
to
Cache(terra::project(raster, crs = terra::crs(newRaster)))
or
Cache(terra::project, raster, crs = terra::crs(newRaster))
or with the pipe
terra::project(raster, crs = terra::crs(newRaster)) |> Cache()
This is particularly useful for expensive operations.
##
## Attaching package: 'data.table'
## The following object is masked from 'package:terra':
##
## shift
tmpDir <- file.path(tempdir(), "reproducible_examples", "Cache")
dir.create(tmpDir, recursive = TRUE)
ras <- terra::rast(terra::ext(0, 300, 0, 300), vals = 1:9e4, res = 1)
terra::crs(ras) <- "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +datum=WGS84"
newCRS <- "+init=epsg:4326" # A longlat crs
# No Cache
system.time(suppressWarnings(map1 <- terra::project(ras, newCRS))) # Warnings due to new PROJ
## user system elapsed
## 0.037 0.004 0.041
# Try with memoise for this example -- for many simple cases, memoising will not be faster
opts <- options("reproducible.useMemoise" = TRUE)
# With Cache -- a little slower the first time because saving to disk
system.time({
suppressWarnings({
map1 <- Cache(terra::project, ras, newCRS, cachePath = tmpDir, notOlderThan = Sys.time())
})
})
## user system elapsed
## 0.278 0.004 0.288
# faster the second time; improvement depends on size of object and time to run function
system.time({
map2 <- Cache(terra::project, ras, newCRS, cachePath = tmpDir)
})
## ...(Object to retrieve (fn: terra::project, 80665208e4c10be7.rds))
## loaded memoised result from previous terra::project call
## user system elapsed
## 0.070 0.000 0.072
## [1] "Attributes: < Names: 2 string mismatches >"
## [2] "Attributes: < Length mismatch: comparison on first 3 components >"
## [3] "Attributes: < Component \".Cache\": Component \"newCache\": 1 element mismatch >"
## [4] "Attributes: < Component 2: Attributes: < target is NULL, current is list > >"
## [5] "Attributes: < Component 2: 1 string mismatch >"
## [6] "Attributes: < Component 3: Modes: character, S4 >"
## [7] "Attributes: < Component 3: names for current but not for target >"
## [8] "Attributes: < Component 3: Attributes: < Names: 1 string mismatch > >"
## [9] "Attributes: < Component 3: Attributes: < Length mismatch: comparison on first 1 components > >"
## [10] "Attributes: < Component 3: Attributes: < Component 1: Modes: character, environment > >"
## [11] "Attributes: < Component 3: Attributes: < Component 1: Lengths: 1, 29 > >"
## [12] "Attributes: < Component 3: Attributes: < Component 1: names for current but not for target > >"
## [13] "Attributes: < Component 3: Attributes: < Component 1: target is character, current is environment > >"
## [14] "Attributes: < Component 3: target is character, current is Rcpp_SpatRaster >"
try(clearCache(tmpDir, ask = FALSE), silent = TRUE) # just to make sure it is clear
ranNumsA <- Cache(rnorm, 10, 16, cachePath = tmpDir)
# All same
ranNumsB <- Cache(rnorm, 10, 16, cachePath = tmpDir) # recovers cached copy
## ...(Object to retrieve (fn: rnorm, 4fca280cda001fc9.rds))
## loaded cached result from previous rnorm call
## ...(Object to retrieve (fn: quote, 4fca280cda001fc9.rds))
## loaded cached result from previous quote call
## ...(Object to retrieve (fn: rnorm, 4fca280cda001fc9.rds))
## loaded cached result from previous rnorm call
## ...(Object to retrieve (fn: rnorm, 4fca280cda001fc9.rds))
## loaded cached result from previous rnorm call
# Any minor change makes it different
ranNumsE <- Cache(rnorm, 10, 6, cachePath = tmpDir) # different
ranNumsA <- Cache(rnorm, 4, cachePath = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif(4), cachePath = tmpDir, userTags = "objectName:b")
showCache(tmpDir, userTags = c("objectName"))
## Cache size:
## Total (including Rasters): 504 bytes
## Selected objects (not including Rasters): 504 bytes
## cacheId tagKey tagValue
## 1: ad0ea27476c50b66 objectName a
## 2: ad0ea27476c50b66 function rnorm
## 3: ad0ea27476c50b66 class numeric
## 4: ad0ea27476c50b66 object.size 1008
## 5: ad0ea27476c50b66 accessed 2023-11-22 17:00:37.800424
## 6: ad0ea27476c50b66 inCloud FALSE
## 7: ad0ea27476c50b66 fromDisk FALSE
## 8: ad0ea27476c50b66 resultHash
## 9: ad0ea27476c50b66 elapsedTimeDigest 0.002403259 secs
## 10: ad0ea27476c50b66 elapsedTimeFirstRun 2.670288e-05 secs
## 11: ad0ea27476c50b66 otherFunctions saveRDS
## 12: ad0ea27476c50b66 otherFunctions do.call
## 13: ad0ea27476c50b66 otherFunctions process_file
## 14: ad0ea27476c50b66 otherFunctions handle_error
## 15: ad0ea27476c50b66 otherFunctions process_group
## 16: ad0ea27476c50b66 otherFunctions process_group.block
## 17: ad0ea27476c50b66 otherFunctions call_block
## 18: ad0ea27476c50b66 otherFunctions block_exec
## 19: ad0ea27476c50b66 otherFunctions eng_r
## 20: ad0ea27476c50b66 otherFunctions in_input_dir
## 21: ad0ea27476c50b66 otherFunctions in_dir
## 22: ad0ea27476c50b66 otherFunctions timing_fn
## 23: ad0ea27476c50b66 otherFunctions handle
## 24: ad0ea27476c50b66 preDigest n:7eef4eae85fd9229
## 25: ad0ea27476c50b66 preDigest mean:c40c00762a0dac94
## 26: ad0ea27476c50b66 preDigest sd:853b1797f54b229c
## 27: ad0ea27476c50b66 preDigest .FUN:4f604aa46882b368
## 28: deaa37372f85861b objectName b
## 29: deaa37372f85861b function runif
## 30: deaa37372f85861b class numeric
## 31: deaa37372f85861b object.size 1008
## 32: deaa37372f85861b accessed 2023-11-22 17:00:37.818458
## 33: deaa37372f85861b inCloud FALSE
## 34: deaa37372f85861b fromDisk FALSE
## 35: deaa37372f85861b resultHash
## 36: deaa37372f85861b elapsedTimeDigest 0.002296686 secs
## 37: deaa37372f85861b elapsedTimeFirstRun 3.027916e-05 secs
## 38: deaa37372f85861b otherFunctions saveRDS
## 39: deaa37372f85861b otherFunctions do.call
## 40: deaa37372f85861b otherFunctions process_file
## 41: deaa37372f85861b otherFunctions handle_error
## 42: deaa37372f85861b otherFunctions process_group
## 43: deaa37372f85861b otherFunctions process_group.block
## 44: deaa37372f85861b otherFunctions call_block
## 45: deaa37372f85861b otherFunctions block_exec
## 46: deaa37372f85861b otherFunctions eng_r
## 47: deaa37372f85861b otherFunctions in_input_dir
## 48: deaa37372f85861b otherFunctions in_dir
## 49: deaa37372f85861b otherFunctions timing_fn
## 50: deaa37372f85861b otherFunctions handle
## 51: deaa37372f85861b preDigest n:7eef4eae85fd9229
## 52: deaa37372f85861b preDigest min:c40c00762a0dac94
## 53: deaa37372f85861b preDigest max:853b1797f54b229c
## 54: deaa37372f85861b preDigest .FUN:881ec847b7161f3c
## cacheId tagKey tagValue
## createdDate
## 1: 2023-11-22 17:00:37.80095
## 2: 2023-11-22 17:00:37.80095
## 3: 2023-11-22 17:00:37.80095
## 4: 2023-11-22 17:00:37.80095
## 5: 2023-11-22 17:00:37.80095
## 6: 2023-11-22 17:00:37.80095
## 7: 2023-11-22 17:00:37.80095
## 8: 2023-11-22 17:00:37.80095
## 9: 2023-11-22 17:00:37.80095
## 10: 2023-11-22 17:00:37.80095
## 11: 2023-11-22 17:00:37.80095
## 12: 2023-11-22 17:00:37.80095
## 13: 2023-11-22 17:00:37.80095
## 14: 2023-11-22 17:00:37.80095
## 15: 2023-11-22 17:00:37.80095
## 16: 2023-11-22 17:00:37.80095
## 17: 2023-11-22 17:00:37.80095
## 18: 2023-11-22 17:00:37.80095
## 19: 2023-11-22 17:00:37.80095
## 20: 2023-11-22 17:00:37.80095
## 21: 2023-11-22 17:00:37.80095
## 22: 2023-11-22 17:00:37.80095
## 23: 2023-11-22 17:00:37.80095
## 24: 2023-11-22 17:00:37.80095
## 25: 2023-11-22 17:00:37.80095
## 26: 2023-11-22 17:00:37.80095
## 27: 2023-11-22 17:00:37.80095
## 28: 2023-11-22 17:00:37.81898
## 29: 2023-11-22 17:00:37.81898
## 30: 2023-11-22 17:00:37.81898
## 31: 2023-11-22 17:00:37.81898
## 32: 2023-11-22 17:00:37.81898
## 33: 2023-11-22 17:00:37.81898
## 34: 2023-11-22 17:00:37.81898
## 35: 2023-11-22 17:00:37.81898
## 36: 2023-11-22 17:00:37.81898
## 37: 2023-11-22 17:00:37.81898
## 38: 2023-11-22 17:00:37.81898
## 39: 2023-11-22 17:00:37.81898
## 40: 2023-11-22 17:00:37.81898
## 41: 2023-11-22 17:00:37.81898
## 42: 2023-11-22 17:00:37.81898
## 43: 2023-11-22 17:00:37.81898
## 44: 2023-11-22 17:00:37.81898
## 45: 2023-11-22 17:00:37.81898
## 46: 2023-11-22 17:00:37.81898
## 47: 2023-11-22 17:00:37.81898
## 48: 2023-11-22 17:00:37.81898
## 49: 2023-11-22 17:00:37.81898
## 50: 2023-11-22 17:00:37.81898
## 51: 2023-11-22 17:00:37.81898
## 52: 2023-11-22 17:00:37.81898
## 53: 2023-11-22 17:00:37.81898
## 54: 2023-11-22 17:00:37.81898
## createdDate
## Cache size:
## Total (including Rasters): 252 bytes
## Selected objects (not including Rasters): 252 bytes
## cacheId tagKey tagValue
## 1: ad0ea27476c50b66 objectName a
## 2: ad0ea27476c50b66 function rnorm
## 3: ad0ea27476c50b66 class numeric
## 4: ad0ea27476c50b66 object.size 1008
## 5: ad0ea27476c50b66 accessed 2023-11-22 17:00:37.800424
## 6: ad0ea27476c50b66 inCloud FALSE
## 7: ad0ea27476c50b66 fromDisk FALSE
## 8: ad0ea27476c50b66 resultHash
## 9: ad0ea27476c50b66 elapsedTimeDigest 0.002403259 secs
## 10: ad0ea27476c50b66 elapsedTimeFirstRun 2.670288e-05 secs
## 11: ad0ea27476c50b66 otherFunctions saveRDS
## 12: ad0ea27476c50b66 otherFunctions do.call
## 13: ad0ea27476c50b66 otherFunctions process_file
## 14: ad0ea27476c50b66 otherFunctions handle_error
## 15: ad0ea27476c50b66 otherFunctions process_group
## 16: ad0ea27476c50b66 otherFunctions process_group.block
## 17: ad0ea27476c50b66 otherFunctions call_block
## 18: ad0ea27476c50b66 otherFunctions block_exec
## 19: ad0ea27476c50b66 otherFunctions eng_r
## 20: ad0ea27476c50b66 otherFunctions in_input_dir
## 21: ad0ea27476c50b66 otherFunctions in_dir
## 22: ad0ea27476c50b66 otherFunctions timing_fn
## 23: ad0ea27476c50b66 otherFunctions handle
## 24: ad0ea27476c50b66 preDigest n:7eef4eae85fd9229
## 25: ad0ea27476c50b66 preDigest mean:c40c00762a0dac94
## 26: ad0ea27476c50b66 preDigest sd:853b1797f54b229c
## 27: ad0ea27476c50b66 preDigest .FUN:4f604aa46882b368
## cacheId tagKey tagValue
## createdDate
## 1: 2023-11-22 17:00:37.80095
## 2: 2023-11-22 17:00:37.80095
## 3: 2023-11-22 17:00:37.80095
## 4: 2023-11-22 17:00:37.80095
## 5: 2023-11-22 17:00:37.80095
## 6: 2023-11-22 17:00:37.80095
## 7: 2023-11-22 17:00:37.80095
## 8: 2023-11-22 17:00:37.80095
## 9: 2023-11-22 17:00:37.80095
## 10: 2023-11-22 17:00:37.80095
## 11: 2023-11-22 17:00:37.80095
## 12: 2023-11-22 17:00:37.80095
## 13: 2023-11-22 17:00:37.80095
## 14: 2023-11-22 17:00:37.80095
## 15: 2023-11-22 17:00:37.80095
## 16: 2023-11-22 17:00:37.80095
## 17: 2023-11-22 17:00:37.80095
## 18: 2023-11-22 17:00:37.80095
## 19: 2023-11-22 17:00:37.80095
## 20: 2023-11-22 17:00:37.80095
## 21: 2023-11-22 17:00:37.80095
## 22: 2023-11-22 17:00:37.80095
## 23: 2023-11-22 17:00:37.80095
## 24: 2023-11-22 17:00:37.80095
## 25: 2023-11-22 17:00:37.80095
## 26: 2023-11-22 17:00:37.80095
## 27: 2023-11-22 17:00:37.80095
## createdDate
## Cache size:
## Total (including Rasters): 252 bytes
## Selected objects (not including Rasters): 252 bytes
## cacheId tagKey tagValue
## 1: deaa37372f85861b objectName b
## 2: deaa37372f85861b function runif
## 3: deaa37372f85861b class numeric
## 4: deaa37372f85861b object.size 1008
## 5: deaa37372f85861b accessed 2023-11-22 17:00:37.818458
## 6: deaa37372f85861b inCloud FALSE
## 7: deaa37372f85861b fromDisk FALSE
## 8: deaa37372f85861b resultHash
## 9: deaa37372f85861b elapsedTimeDigest 0.002296686 secs
## 10: deaa37372f85861b elapsedTimeFirstRun 3.027916e-05 secs
## 11: deaa37372f85861b otherFunctions saveRDS
## 12: deaa37372f85861b otherFunctions do.call
## 13: deaa37372f85861b otherFunctions process_file
## 14: deaa37372f85861b otherFunctions handle_error
## 15: deaa37372f85861b otherFunctions process_group
## 16: deaa37372f85861b otherFunctions process_group.block
## 17: deaa37372f85861b otherFunctions call_block
## 18: deaa37372f85861b otherFunctions block_exec
## 19: deaa37372f85861b otherFunctions eng_r
## 20: deaa37372f85861b otherFunctions in_input_dir
## 21: deaa37372f85861b otherFunctions in_dir
## 22: deaa37372f85861b otherFunctions timing_fn
## 23: deaa37372f85861b otherFunctions handle
## 24: deaa37372f85861b preDigest n:7eef4eae85fd9229
## 25: deaa37372f85861b preDigest min:c40c00762a0dac94
## 26: deaa37372f85861b preDigest max:853b1797f54b229c
## 27: deaa37372f85861b preDigest .FUN:881ec847b7161f3c
## cacheId tagKey tagValue
## createdDate
## 1: 2023-11-22 17:00:37.81898
## 2: 2023-11-22 17:00:37.81898
## 3: 2023-11-22 17:00:37.81898
## 4: 2023-11-22 17:00:37.81898
## 5: 2023-11-22 17:00:37.81898
## 6: 2023-11-22 17:00:37.81898
## 7: 2023-11-22 17:00:37.81898
## 8: 2023-11-22 17:00:37.81898
## 9: 2023-11-22 17:00:37.81898
## 10: 2023-11-22 17:00:37.81898
## 11: 2023-11-22 17:00:37.81898
## 12: 2023-11-22 17:00:37.81898
## 13: 2023-11-22 17:00:37.81898
## 14: 2023-11-22 17:00:37.81898
## 15: 2023-11-22 17:00:37.81898
## 16: 2023-11-22 17:00:37.81898
## 17: 2023-11-22 17:00:37.81898
## 18: 2023-11-22 17:00:37.81898
## 19: 2023-11-22 17:00:37.81898
## 20: 2023-11-22 17:00:37.81898
## 21: 2023-11-22 17:00:37.81898
## 22: 2023-11-22 17:00:37.81898
## 23: 2023-11-22 17:00:37.81898
## 24: 2023-11-22 17:00:37.81898
## 25: 2023-11-22 17:00:37.81898
## 26: 2023-11-22 17:00:37.81898
## 27: 2023-11-22 17:00:37.81898
## createdDate
clearCache(tmpDir, userTags = c("runif"), ask = FALSE) # remove only cached objects made during runif call
## Cache size:
## Total (including Rasters): 252 bytes
## Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) # all
## Cache size:
## Total (including Rasters): 804 bytes
## Selected objects (not including Rasters): 804 bytes
## cacheId tagKey tagValue
## 1: 4fca280cda001fc9 function rnorm
## 2: 4fca280cda001fc9 class numeric
## 3: 4fca280cda001fc9 object.size 1104
## 4: 4fca280cda001fc9 accessed 2023-11-22 17:00:37.642897
## 5: 4fca280cda001fc9 inCloud FALSE
## 6: 4fca280cda001fc9 fromDisk FALSE
## 7: 4fca280cda001fc9 resultHash
## 8: 4fca280cda001fc9 elapsedTimeDigest 0.002775669 secs
## 9: 4fca280cda001fc9 elapsedTimeFirstRun 4.029274e-05 secs
## 10: 4fca280cda001fc9 otherFunctions saveRDS
## 11: 4fca280cda001fc9 otherFunctions do.call
## 12: 4fca280cda001fc9 otherFunctions process_file
## 13: 4fca280cda001fc9 otherFunctions handle_error
## 14: 4fca280cda001fc9 otherFunctions process_group
## 15: 4fca280cda001fc9 otherFunctions process_group.block
## 16: 4fca280cda001fc9 otherFunctions call_block
## 17: 4fca280cda001fc9 otherFunctions block_exec
## 18: 4fca280cda001fc9 otherFunctions eng_r
## 19: 4fca280cda001fc9 otherFunctions in_input_dir
## 20: 4fca280cda001fc9 otherFunctions in_dir
## 21: 4fca280cda001fc9 otherFunctions timing_fn
## 22: 4fca280cda001fc9 otherFunctions handle
## 23: 4fca280cda001fc9 preDigest n:c5775c3b366fb719
## 24: 4fca280cda001fc9 preDigest mean:15620f138033a66c
## 25: 4fca280cda001fc9 preDigest sd:853b1797f54b229c
## 26: 4fca280cda001fc9 preDigest .FUN:4f604aa46882b368
## 27: 4fca280cda001fc9 accessed 2023-11-22 17:00:37.662573
## 28: 4fca280cda001fc9 elapsedTimeLoad 0.002296209 secs
## 29: 4fca280cda001fc9 accessed 2023-11-22 17:00:37.676569
## 30: 4fca280cda001fc9 accessed 2023-11-22 17:00:37.689539
## 31: 4fca280cda001fc9 accessed 2023-11-22 17:00:37.702141
## 32: 5efb386c43c29ce1 function rnorm
## 33: 5efb386c43c29ce1 class numeric
## 34: 5efb386c43c29ce1 object.size 1104
## 35: 5efb386c43c29ce1 accessed 2023-11-22 17:00:37.714624
## 36: 5efb386c43c29ce1 inCloud FALSE
## 37: 5efb386c43c29ce1 fromDisk FALSE
## 38: 5efb386c43c29ce1 resultHash
## 39: 5efb386c43c29ce1 elapsedTimeDigest 0.001955271 secs
## 40: 5efb386c43c29ce1 elapsedTimeFirstRun 2.765656e-05 secs
## 41: 5efb386c43c29ce1 otherFunctions saveRDS
## 42: 5efb386c43c29ce1 otherFunctions do.call
## 43: 5efb386c43c29ce1 otherFunctions process_file
## 44: 5efb386c43c29ce1 otherFunctions handle_error
## 45: 5efb386c43c29ce1 otherFunctions process_group
## 46: 5efb386c43c29ce1 otherFunctions process_group.block
## 47: 5efb386c43c29ce1 otherFunctions call_block
## 48: 5efb386c43c29ce1 otherFunctions block_exec
## 49: 5efb386c43c29ce1 otherFunctions eng_r
## 50: 5efb386c43c29ce1 otherFunctions in_input_dir
## 51: 5efb386c43c29ce1 otherFunctions in_dir
## 52: 5efb386c43c29ce1 otherFunctions timing_fn
## 53: 5efb386c43c29ce1 otherFunctions handle
## 54: 5efb386c43c29ce1 preDigest n:c5775c3b366fb719
## 55: 5efb386c43c29ce1 preDigest mean:152602b8ff81e5bb
## 56: 5efb386c43c29ce1 preDigest sd:853b1797f54b229c
## 57: 5efb386c43c29ce1 preDigest .FUN:4f604aa46882b368
## 58: ad0ea27476c50b66 objectName a
## 59: ad0ea27476c50b66 function rnorm
## 60: ad0ea27476c50b66 class numeric
## 61: ad0ea27476c50b66 object.size 1008
## 62: ad0ea27476c50b66 accessed 2023-11-22 17:00:37.800424
## 63: ad0ea27476c50b66 inCloud FALSE
## 64: ad0ea27476c50b66 fromDisk FALSE
## 65: ad0ea27476c50b66 resultHash
## 66: ad0ea27476c50b66 elapsedTimeDigest 0.002403259 secs
## 67: ad0ea27476c50b66 elapsedTimeFirstRun 2.670288e-05 secs
## 68: ad0ea27476c50b66 otherFunctions saveRDS
## 69: ad0ea27476c50b66 otherFunctions do.call
## 70: ad0ea27476c50b66 otherFunctions process_file
## 71: ad0ea27476c50b66 otherFunctions handle_error
## 72: ad0ea27476c50b66 otherFunctions process_group
## 73: ad0ea27476c50b66 otherFunctions process_group.block
## 74: ad0ea27476c50b66 otherFunctions call_block
## 75: ad0ea27476c50b66 otherFunctions block_exec
## 76: ad0ea27476c50b66 otherFunctions eng_r
## 77: ad0ea27476c50b66 otherFunctions in_input_dir
## 78: ad0ea27476c50b66 otherFunctions in_dir
## 79: ad0ea27476c50b66 otherFunctions timing_fn
## 80: ad0ea27476c50b66 otherFunctions handle
## 81: ad0ea27476c50b66 preDigest n:7eef4eae85fd9229
## 82: ad0ea27476c50b66 preDigest mean:c40c00762a0dac94
## 83: ad0ea27476c50b66 preDigest sd:853b1797f54b229c
## 84: ad0ea27476c50b66 preDigest .FUN:4f604aa46882b368
## cacheId tagKey tagValue
## createdDate
## 1: 2023-11-22 17:00:37.643383
## 2: 2023-11-22 17:00:37.643383
## 3: 2023-11-22 17:00:37.643383
## 4: 2023-11-22 17:00:37.643383
## 5: 2023-11-22 17:00:37.643383
## 6: 2023-11-22 17:00:37.643383
## 7: 2023-11-22 17:00:37.643383
## 8: 2023-11-22 17:00:37.643383
## 9: 2023-11-22 17:00:37.643383
## 10: 2023-11-22 17:00:37.643383
## 11: 2023-11-22 17:00:37.643383
## 12: 2023-11-22 17:00:37.643383
## 13: 2023-11-22 17:00:37.643383
## 14: 2023-11-22 17:00:37.643383
## 15: 2023-11-22 17:00:37.643383
## 16: 2023-11-22 17:00:37.643383
## 17: 2023-11-22 17:00:37.643383
## 18: 2023-11-22 17:00:37.643383
## 19: 2023-11-22 17:00:37.643383
## 20: 2023-11-22 17:00:37.643383
## 21: 2023-11-22 17:00:37.643383
## 22: 2023-11-22 17:00:37.643383
## 23: 2023-11-22 17:00:37.643383
## 24: 2023-11-22 17:00:37.643383
## 25: 2023-11-22 17:00:37.643383
## 26: 2023-11-22 17:00:37.643383
## 27: 2023-11-22 17:00:37.662573
## 28: 2023-11-22 17:00:37.665095
## 29: 2023-11-22 17:00:37.676569
## 30: 2023-11-22 17:00:37.689539
## 31: 2023-11-22 17:00:37.702141
## 32: 2023-11-22 17:00:37.715126
## 33: 2023-11-22 17:00:37.715126
## 34: 2023-11-22 17:00:37.715126
## 35: 2023-11-22 17:00:37.715126
## 36: 2023-11-22 17:00:37.715126
## 37: 2023-11-22 17:00:37.715126
## 38: 2023-11-22 17:00:37.715126
## 39: 2023-11-22 17:00:37.715126
## 40: 2023-11-22 17:00:37.715126
## 41: 2023-11-22 17:00:37.715126
## 42: 2023-11-22 17:00:37.715126
## 43: 2023-11-22 17:00:37.715126
## 44: 2023-11-22 17:00:37.715126
## 45: 2023-11-22 17:00:37.715126
## 46: 2023-11-22 17:00:37.715126
## 47: 2023-11-22 17:00:37.715126
## 48: 2023-11-22 17:00:37.715126
## 49: 2023-11-22 17:00:37.715126
## 50: 2023-11-22 17:00:37.715126
## 51: 2023-11-22 17:00:37.715126
## 52: 2023-11-22 17:00:37.715126
## 53: 2023-11-22 17:00:37.715126
## 54: 2023-11-22 17:00:37.715126
## 55: 2023-11-22 17:00:37.715126
## 56: 2023-11-22 17:00:37.715126
## 57: 2023-11-22 17:00:37.715126
## 58: 2023-11-22 17:00:37.80095
## 59: 2023-11-22 17:00:37.80095
## 60: 2023-11-22 17:00:37.80095
## 61: 2023-11-22 17:00:37.80095
## 62: 2023-11-22 17:00:37.80095
## 63: 2023-11-22 17:00:37.80095
## 64: 2023-11-22 17:00:37.80095
## 65: 2023-11-22 17:00:37.80095
## 66: 2023-11-22 17:00:37.80095
## 67: 2023-11-22 17:00:37.80095
## 68: 2023-11-22 17:00:37.80095
## 69: 2023-11-22 17:00:37.80095
## 70: 2023-11-22 17:00:37.80095
## 71: 2023-11-22 17:00:37.80095
## 72: 2023-11-22 17:00:37.80095
## 73: 2023-11-22 17:00:37.80095
## 74: 2023-11-22 17:00:37.80095
## 75: 2023-11-22 17:00:37.80095
## 76: 2023-11-22 17:00:37.80095
## 77: 2023-11-22 17:00:37.80095
## 78: 2023-11-22 17:00:37.80095
## 79: 2023-11-22 17:00:37.80095
## 80: 2023-11-22 17:00:37.80095
## 81: 2023-11-22 17:00:37.80095
## 82: 2023-11-22 17:00:37.80095
## 83: 2023-11-22 17:00:37.80095
## 84: 2023-11-22 17:00:37.80095
## createdDate
clearCache(tmpDir, ask = FALSE)
ranNumsA <- Cache(rnorm, 4, cachePath = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cachePath = tmpDir, userTags = "objectName:b")
# access it again, from Cache
Sys.sleep(1)
ranNumsA <- Cache(rnorm, 4, cachePath = tmpDir, userTags = "objectName:a")
## ...(Object to retrieve (fn: rnorm, ad0ea27476c50b66.rds))
## loaded cached result from previous rnorm call
wholeCache <- showCache(tmpDir)
## Cache size:
## Total (including Rasters): 504 bytes
## Selected objects (not including Rasters): 504 bytes
# keep only items accessed "recently" (i.e., only objectName:a)
onlyRecentlyAccessed <- showCache(tmpDir, userTags = max(wholeCache[tagKey == "accessed"]$tagValue))
## Cache size:
## Total (including Rasters): 252 bytes
## Selected objects (not including Rasters): 252 bytes
# inverse join with 2 data.tables ... using: a[!b]
# i.e., return all of wholeCache that was not recently accessed
# Note: the two different ways to access -- old way with "artifact" will be deprecated
toRemove <- unique(wholeCache[!onlyRecentlyAccessed, on = "cacheId"], by = "cacheId")$cacheId
clearCache(tmpDir, toRemove, ask = FALSE) # remove ones not recently accessed
## Cache size:
## Total (including Rasters): 252 bytes
## Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) # still has more recently accessed
## Cache size:
## Total (including Rasters): 252 bytes
## Selected objects (not including Rasters): 252 bytes
## cacheId tagKey tagValue
## 1: ad0ea27476c50b66 objectName a
## 2: ad0ea27476c50b66 function rnorm
## 3: ad0ea27476c50b66 class numeric
## 4: ad0ea27476c50b66 object.size 1008
## 5: ad0ea27476c50b66 accessed 2023-11-22 17:00:37.981493
## 6: ad0ea27476c50b66 inCloud FALSE
## 7: ad0ea27476c50b66 fromDisk FALSE
## 8: ad0ea27476c50b66 resultHash
## 9: ad0ea27476c50b66 elapsedTimeDigest 0.002360582 secs
## 10: ad0ea27476c50b66 elapsedTimeFirstRun 3.004074e-05 secs
## 11: ad0ea27476c50b66 otherFunctions saveRDS
## 12: ad0ea27476c50b66 otherFunctions do.call
## 13: ad0ea27476c50b66 otherFunctions process_file
## 14: ad0ea27476c50b66 otherFunctions handle_error
## 15: ad0ea27476c50b66 otherFunctions process_group
## 16: ad0ea27476c50b66 otherFunctions process_group.block
## 17: ad0ea27476c50b66 otherFunctions call_block
## 18: ad0ea27476c50b66 otherFunctions block_exec
## 19: ad0ea27476c50b66 otherFunctions eng_r
## 20: ad0ea27476c50b66 otherFunctions in_input_dir
## 21: ad0ea27476c50b66 otherFunctions in_dir
## 22: ad0ea27476c50b66 otherFunctions timing_fn
## 23: ad0ea27476c50b66 otherFunctions handle
## 24: ad0ea27476c50b66 preDigest n:7eef4eae85fd9229
## 25: ad0ea27476c50b66 preDigest mean:c40c00762a0dac94
## 26: ad0ea27476c50b66 preDigest sd:853b1797f54b229c
## 27: ad0ea27476c50b66 preDigest .FUN:4f604aa46882b368
## 28: ad0ea27476c50b66 accessed 2023-11-22 17:00:39.016507
## 29: ad0ea27476c50b66 elapsedTimeLoad 0.002650976 secs
## cacheId tagKey tagValue
## createdDate
## 1: 2023-11-22 17:00:37.982011
## 2: 2023-11-22 17:00:37.982011
## 3: 2023-11-22 17:00:37.982011
## 4: 2023-11-22 17:00:37.982011
## 5: 2023-11-22 17:00:37.982011
## 6: 2023-11-22 17:00:37.982011
## 7: 2023-11-22 17:00:37.982011
## 8: 2023-11-22 17:00:37.982011
## 9: 2023-11-22 17:00:37.982011
## 10: 2023-11-22 17:00:37.982011
## 11: 2023-11-22 17:00:37.982011
## 12: 2023-11-22 17:00:37.982011
## 13: 2023-11-22 17:00:37.982011
## 14: 2023-11-22 17:00:37.982011
## 15: 2023-11-22 17:00:37.982011
## 16: 2023-11-22 17:00:37.982011
## 17: 2023-11-22 17:00:37.982011
## 18: 2023-11-22 17:00:37.982011
## 19: 2023-11-22 17:00:37.982011
## 20: 2023-11-22 17:00:37.982011
## 21: 2023-11-22 17:00:37.982011
## 22: 2023-11-22 17:00:37.982011
## 23: 2023-11-22 17:00:37.982011
## 24: 2023-11-22 17:00:37.982011
## 25: 2023-11-22 17:00:37.982011
## 26: 2023-11-22 17:00:37.982011
## 27: 2023-11-22 17:00:37.982011
## 28: 2023-11-22 17:00:39.016507
## 29: 2023-11-22 17:00:39.019306
## createdDate
keepCache
does the same as previous example, but more
simply.
ranNumsA <- Cache(rnorm, 4, cachePath = tmpDir, userTags = "objectName:a")
## ...(Object to retrieve (fn: rnorm, ad0ea27476c50b66.rds))
## loaded cached result from previous rnorm call
ranNumsB <- Cache(runif(4), cachePath = tmpDir, userTags = "objectName:b")
# keep only those cached items from the last 24 hours
oneDay <- 60 * 60 * 24
keepCache(tmpDir, after = Sys.time() - oneDay, ask = FALSE)
## Cache size:
## Total (including Rasters): 504 bytes
## Selected objects (not including Rasters): 504 bytes
## cacheId tagKey tagValue
## 1: ad0ea27476c50b66 objectName a
## 2: ad0ea27476c50b66 function rnorm
## 3: ad0ea27476c50b66 class numeric
## 4: ad0ea27476c50b66 object.size 1008
## 5: ad0ea27476c50b66 accessed 2023-11-22 17:00:37.981493
## 6: ad0ea27476c50b66 inCloud FALSE
## 7: ad0ea27476c50b66 fromDisk FALSE
## 8: ad0ea27476c50b66 resultHash
## 9: ad0ea27476c50b66 elapsedTimeDigest 0.002360582 secs
## 10: ad0ea27476c50b66 elapsedTimeFirstRun 3.004074e-05 secs
## 11: ad0ea27476c50b66 otherFunctions saveRDS
## 12: ad0ea27476c50b66 otherFunctions do.call
## 13: ad0ea27476c50b66 otherFunctions process_file
## 14: ad0ea27476c50b66 otherFunctions handle_error
## 15: ad0ea27476c50b66 otherFunctions process_group
## 16: ad0ea27476c50b66 otherFunctions process_group.block
## 17: ad0ea27476c50b66 otherFunctions call_block
## 18: ad0ea27476c50b66 otherFunctions block_exec
## 19: ad0ea27476c50b66 otherFunctions eng_r
## 20: ad0ea27476c50b66 otherFunctions in_input_dir
## 21: ad0ea27476c50b66 otherFunctions in_dir
## 22: ad0ea27476c50b66 otherFunctions timing_fn
## 23: ad0ea27476c50b66 otherFunctions handle
## 24: ad0ea27476c50b66 preDigest n:7eef4eae85fd9229
## 25: ad0ea27476c50b66 preDigest mean:c40c00762a0dac94
## 26: ad0ea27476c50b66 preDigest sd:853b1797f54b229c
## 27: ad0ea27476c50b66 preDigest .FUN:4f604aa46882b368
## 28: ad0ea27476c50b66 accessed 2023-11-22 17:00:39.016507
## 29: ad0ea27476c50b66 elapsedTimeLoad 0.003867865 secs
## 30: ad0ea27476c50b66 accessed 2023-11-22 17:00:39.144076
## 31: deaa37372f85861b objectName b
## 32: deaa37372f85861b function runif
## 33: deaa37372f85861b class numeric
## 34: deaa37372f85861b object.size 1008
## 35: deaa37372f85861b accessed 2023-11-22 17:00:39.158666
## 36: deaa37372f85861b inCloud FALSE
## 37: deaa37372f85861b fromDisk FALSE
## 38: deaa37372f85861b resultHash
## 39: deaa37372f85861b elapsedTimeDigest 0.002058983 secs
## 40: deaa37372f85861b elapsedTimeFirstRun 2.551079e-05 secs
## 41: deaa37372f85861b otherFunctions saveRDS
## 42: deaa37372f85861b otherFunctions do.call
## 43: deaa37372f85861b otherFunctions process_file
## 44: deaa37372f85861b otherFunctions handle_error
## 45: deaa37372f85861b otherFunctions process_group
## 46: deaa37372f85861b otherFunctions process_group.block
## 47: deaa37372f85861b otherFunctions call_block
## 48: deaa37372f85861b otherFunctions block_exec
## 49: deaa37372f85861b otherFunctions eng_r
## 50: deaa37372f85861b otherFunctions in_input_dir
## 51: deaa37372f85861b otherFunctions in_dir
## 52: deaa37372f85861b otherFunctions timing_fn
## 53: deaa37372f85861b otherFunctions handle
## 54: deaa37372f85861b preDigest n:7eef4eae85fd9229
## 55: deaa37372f85861b preDigest min:c40c00762a0dac94
## 56: deaa37372f85861b preDigest max:853b1797f54b229c
## 57: deaa37372f85861b preDigest .FUN:881ec847b7161f3c
## cacheId tagKey tagValue
## createdDate
## 1: 2023-11-22 17:00:37.982011
## 2: 2023-11-22 17:00:37.982011
## 3: 2023-11-22 17:00:37.982011
## 4: 2023-11-22 17:00:37.982011
## 5: 2023-11-22 17:00:37.982011
## 6: 2023-11-22 17:00:37.982011
## 7: 2023-11-22 17:00:37.982011
## 8: 2023-11-22 17:00:37.982011
## 9: 2023-11-22 17:00:37.982011
## 10: 2023-11-22 17:00:37.982011
## 11: 2023-11-22 17:00:37.982011
## 12: 2023-11-22 17:00:37.982011
## 13: 2023-11-22 17:00:37.982011
## 14: 2023-11-22 17:00:37.982011
## 15: 2023-11-22 17:00:37.982011
## 16: 2023-11-22 17:00:37.982011
## 17: 2023-11-22 17:00:37.982011
## 18: 2023-11-22 17:00:37.982011
## 19: 2023-11-22 17:00:37.982011
## 20: 2023-11-22 17:00:37.982011
## 21: 2023-11-22 17:00:37.982011
## 22: 2023-11-22 17:00:37.982011
## 23: 2023-11-22 17:00:37.982011
## 24: 2023-11-22 17:00:37.982011
## 25: 2023-11-22 17:00:37.982011
## 26: 2023-11-22 17:00:37.982011
## 27: 2023-11-22 17:00:37.982011
## 28: 2023-11-22 17:00:39.016507
## 29: 2023-11-22 17:00:39.019306
## 30: 2023-11-22 17:00:39.144076
## 31: 2023-11-22 17:00:39.159136
## 32: 2023-11-22 17:00:39.159136
## 33: 2023-11-22 17:00:39.159136
## 34: 2023-11-22 17:00:39.159136
## 35: 2023-11-22 17:00:39.159136
## 36: 2023-11-22 17:00:39.159136
## 37: 2023-11-22 17:00:39.159136
## 38: 2023-11-22 17:00:39.159136
## 39: 2023-11-22 17:00:39.159136
## 40: 2023-11-22 17:00:39.159136
## 41: 2023-11-22 17:00:39.159136
## 42: 2023-11-22 17:00:39.159136
## 43: 2023-11-22 17:00:39.159136
## 44: 2023-11-22 17:00:39.159136
## 45: 2023-11-22 17:00:39.159136
## 46: 2023-11-22 17:00:39.159136
## 47: 2023-11-22 17:00:39.159136
## 48: 2023-11-22 17:00:39.159136
## 49: 2023-11-22 17:00:39.159136
## 50: 2023-11-22 17:00:39.159136
## 51: 2023-11-22 17:00:39.159136
## 52: 2023-11-22 17:00:39.159136
## 53: 2023-11-22 17:00:39.159136
## 54: 2023-11-22 17:00:39.159136
## 55: 2023-11-22 17:00:39.159136
## 56: 2023-11-22 17:00:39.159136
## 57: 2023-11-22 17:00:39.159136
## createdDate
# Keep all Cache items created with an rnorm() call
keepCache(tmpDir, userTags = "rnorm", ask = FALSE)
## Cache size:
## Total (including Rasters): 252 bytes
## Selected objects (not including Rasters): 252 bytes
## Cache size:
## Total (including Rasters): 252 bytes
## Selected objects (not including Rasters): 252 bytes
## cacheId tagKey tagValue
## 1: ad0ea27476c50b66 objectName a
## 2: ad0ea27476c50b66 function rnorm
## 3: ad0ea27476c50b66 class numeric
## 4: ad0ea27476c50b66 object.size 1008
## 5: ad0ea27476c50b66 accessed 2023-11-22 17:00:37.981493
## 6: ad0ea27476c50b66 inCloud FALSE
## 7: ad0ea27476c50b66 fromDisk FALSE
## 8: ad0ea27476c50b66 resultHash
## 9: ad0ea27476c50b66 elapsedTimeDigest 0.002360582 secs
## 10: ad0ea27476c50b66 elapsedTimeFirstRun 3.004074e-05 secs
## 11: ad0ea27476c50b66 otherFunctions saveRDS
## 12: ad0ea27476c50b66 otherFunctions do.call
## 13: ad0ea27476c50b66 otherFunctions process_file
## 14: ad0ea27476c50b66 otherFunctions handle_error
## 15: ad0ea27476c50b66 otherFunctions process_group
## 16: ad0ea27476c50b66 otherFunctions process_group.block
## 17: ad0ea27476c50b66 otherFunctions call_block
## 18: ad0ea27476c50b66 otherFunctions block_exec
## 19: ad0ea27476c50b66 otherFunctions eng_r
## 20: ad0ea27476c50b66 otherFunctions in_input_dir
## 21: ad0ea27476c50b66 otherFunctions in_dir
## 22: ad0ea27476c50b66 otherFunctions timing_fn
## 23: ad0ea27476c50b66 otherFunctions handle
## 24: ad0ea27476c50b66 preDigest n:7eef4eae85fd9229
## 25: ad0ea27476c50b66 preDigest mean:c40c00762a0dac94
## 26: ad0ea27476c50b66 preDigest sd:853b1797f54b229c
## 27: ad0ea27476c50b66 preDigest .FUN:4f604aa46882b368
## 28: ad0ea27476c50b66 accessed 2023-11-22 17:00:39.016507
## 29: ad0ea27476c50b66 elapsedTimeLoad 0.003867865 secs
## 30: ad0ea27476c50b66 accessed 2023-11-22 17:00:39.144076
## cacheId tagKey tagValue
## createdDate
## 1: 2023-11-22 17:00:37.982011
## 2: 2023-11-22 17:00:37.982011
## 3: 2023-11-22 17:00:37.982011
## 4: 2023-11-22 17:00:37.982011
## 5: 2023-11-22 17:00:37.982011
## 6: 2023-11-22 17:00:37.982011
## 7: 2023-11-22 17:00:37.982011
## 8: 2023-11-22 17:00:37.982011
## 9: 2023-11-22 17:00:37.982011
## 10: 2023-11-22 17:00:37.982011
## 11: 2023-11-22 17:00:37.982011
## 12: 2023-11-22 17:00:37.982011
## 13: 2023-11-22 17:00:37.982011
## 14: 2023-11-22 17:00:37.982011
## 15: 2023-11-22 17:00:37.982011
## 16: 2023-11-22 17:00:37.982011
## 17: 2023-11-22 17:00:37.982011
## 18: 2023-11-22 17:00:37.982011
## 19: 2023-11-22 17:00:37.982011
## 20: 2023-11-22 17:00:37.982011
## 21: 2023-11-22 17:00:37.982011
## 22: 2023-11-22 17:00:37.982011
## 23: 2023-11-22 17:00:37.982011
## 24: 2023-11-22 17:00:37.982011
## 25: 2023-11-22 17:00:37.982011
## 26: 2023-11-22 17:00:37.982011
## 27: 2023-11-22 17:00:37.982011
## 28: 2023-11-22 17:00:39.016507
## 29: 2023-11-22 17:00:39.019306
## 30: 2023-11-22 17:00:39.144076
## createdDate
showCache(tmpDir)
## Cache size:
## Total (including Rasters): 252 bytes
## Selected objects (not including Rasters): 252 bytes
## cacheId tagKey tagValue
## 1: ad0ea27476c50b66 objectName a
## 2: ad0ea27476c50b66 function rnorm
## 3: ad0ea27476c50b66 class numeric
## 4: ad0ea27476c50b66 object.size 1008
## 5: ad0ea27476c50b66 accessed 2023-11-22 17:00:37.981493
## 6: ad0ea27476c50b66 inCloud FALSE
## 7: ad0ea27476c50b66 fromDisk FALSE
## 8: ad0ea27476c50b66 resultHash
## 9: ad0ea27476c50b66 elapsedTimeDigest 0.002360582 secs
## 10: ad0ea27476c50b66 elapsedTimeFirstRun 3.004074e-05 secs
## 11: ad0ea27476c50b66 otherFunctions saveRDS
## 12: ad0ea27476c50b66 otherFunctions do.call
## 13: ad0ea27476c50b66 otherFunctions process_file
## 14: ad0ea27476c50b66 otherFunctions handle_error
## 15: ad0ea27476c50b66 otherFunctions process_group
## 16: ad0ea27476c50b66 otherFunctions process_group.block
## 17: ad0ea27476c50b66 otherFunctions call_block
## 18: ad0ea27476c50b66 otherFunctions block_exec
## 19: ad0ea27476c50b66 otherFunctions eng_r
## 20: ad0ea27476c50b66 otherFunctions in_input_dir
## 21: ad0ea27476c50b66 otherFunctions in_dir
## 22: ad0ea27476c50b66 otherFunctions timing_fn
## 23: ad0ea27476c50b66 otherFunctions handle
## 24: ad0ea27476c50b66 preDigest n:7eef4eae85fd9229
## 25: ad0ea27476c50b66 preDigest mean:c40c00762a0dac94
## 26: ad0ea27476c50b66 preDigest sd:853b1797f54b229c
## 27: ad0ea27476c50b66 preDigest .FUN:4f604aa46882b368
## 28: ad0ea27476c50b66 accessed 2023-11-22 17:00:39.016507
## 29: ad0ea27476c50b66 elapsedTimeLoad 0.003867865 secs
## 30: ad0ea27476c50b66 accessed 2023-11-22 17:00:39.144076
## cacheId tagKey tagValue
## createdDate
## 1: 2023-11-22 17:00:37.982011
## 2: 2023-11-22 17:00:37.982011
## 3: 2023-11-22 17:00:37.982011
## 4: 2023-11-22 17:00:37.982011
## 5: 2023-11-22 17:00:37.982011
## 6: 2023-11-22 17:00:37.982011
## 7: 2023-11-22 17:00:37.982011
## 8: 2023-11-22 17:00:37.982011
## 9: 2023-11-22 17:00:37.982011
## 10: 2023-11-22 17:00:37.982011
## 11: 2023-11-22 17:00:37.982011
## 12: 2023-11-22 17:00:37.982011
## 13: 2023-11-22 17:00:37.982011
## 14: 2023-11-22 17:00:37.982011
## 15: 2023-11-22 17:00:37.982011
## 16: 2023-11-22 17:00:37.982011
## 17: 2023-11-22 17:00:37.982011
## 18: 2023-11-22 17:00:37.982011
## 19: 2023-11-22 17:00:37.982011
## 20: 2023-11-22 17:00:37.982011
## 21: 2023-11-22 17:00:37.982011
## 22: 2023-11-22 17:00:37.982011
## 23: 2023-11-22 17:00:37.982011
## 24: 2023-11-22 17:00:37.982011
## 25: 2023-11-22 17:00:37.982011
## 26: 2023-11-22 17:00:37.982011
## 27: 2023-11-22 17:00:37.982011
## 28: 2023-11-22 17:00:39.016507
## 29: 2023-11-22 17:00:39.019306
## 30: 2023-11-22 17:00:39.144076
## createdDate
# Remove all Cache items that happened within a rnorm() call
clearCache(tmpDir, userTags = "rnorm", ask = FALSE)
## Cache size:
## Total (including Rasters): 252 bytes
## Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) ## empty
## Cache size:
## Total (including Rasters): 0 bytes
## Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows and 4 cols): cacheId,tagKey,tagValue,createdDate
# Also, can set a time before caching happens and remove based on this
# --> a useful, simple way to control Cache
ranNumsA <- Cache(rnorm, 4, cachePath = tmpDir, userTags = "objectName:a")
startTime <- Sys.time()
Sys.sleep(1)
ranNumsB <- Cache(rnorm, 5, cachePath = tmpDir, userTags = "objectName:b")
keepCache(tmpDir, after = startTime, ask = FALSE) # keep only those newer than startTime
## Cache size:
## Total (including Rasters): 256 bytes
## Selected objects (not including Rasters): 256 bytes
## Cache size:
## Total (including Rasters): 252 bytes
## Selected objects (not including Rasters): 252 bytes
## cacheId tagKey tagValue
## 1: ccacbf62081a42b4 objectName b
## 2: ccacbf62081a42b4 function rnorm
## 3: ccacbf62081a42b4 class numeric
## 4: ccacbf62081a42b4 object.size 1024
## 5: ccacbf62081a42b4 accessed 2023-11-22 17:00:40.260866
## 6: ccacbf62081a42b4 inCloud FALSE
## 7: ccacbf62081a42b4 fromDisk FALSE
## 8: ccacbf62081a42b4 resultHash
## 9: ccacbf62081a42b4 elapsedTimeDigest 0.002124786 secs
## 10: ccacbf62081a42b4 elapsedTimeFirstRun 2.574921e-05 secs
## 11: ccacbf62081a42b4 otherFunctions saveRDS
## 12: ccacbf62081a42b4 otherFunctions do.call
## 13: ccacbf62081a42b4 otherFunctions process_file
## 14: ccacbf62081a42b4 otherFunctions handle_error
## 15: ccacbf62081a42b4 otherFunctions process_group
## 16: ccacbf62081a42b4 otherFunctions process_group.block
## 17: ccacbf62081a42b4 otherFunctions call_block
## 18: ccacbf62081a42b4 otherFunctions block_exec
## 19: ccacbf62081a42b4 otherFunctions eng_r
## 20: ccacbf62081a42b4 otherFunctions in_input_dir
## 21: ccacbf62081a42b4 otherFunctions in_dir
## 22: ccacbf62081a42b4 otherFunctions timing_fn
## 23: ccacbf62081a42b4 otherFunctions handle
## 24: ccacbf62081a42b4 preDigest n:a4f076b3db622faf
## 25: ccacbf62081a42b4 preDigest mean:c40c00762a0dac94
## 26: ccacbf62081a42b4 preDigest sd:853b1797f54b229c
## 27: ccacbf62081a42b4 preDigest .FUN:4f604aa46882b368
## cacheId tagKey tagValue
## createdDate
## 1: 2023-11-22 17:00:40.261334
## 2: 2023-11-22 17:00:40.261334
## 3: 2023-11-22 17:00:40.261334
## 4: 2023-11-22 17:00:40.261334
## 5: 2023-11-22 17:00:40.261334
## 6: 2023-11-22 17:00:40.261334
## 7: 2023-11-22 17:00:40.261334
## 8: 2023-11-22 17:00:40.261334
## 9: 2023-11-22 17:00:40.261334
## 10: 2023-11-22 17:00:40.261334
## 11: 2023-11-22 17:00:40.261334
## 12: 2023-11-22 17:00:40.261334
## 13: 2023-11-22 17:00:40.261334
## 14: 2023-11-22 17:00:40.261334
## 15: 2023-11-22 17:00:40.261334
## 16: 2023-11-22 17:00:40.261334
## 17: 2023-11-22 17:00:40.261334
## 18: 2023-11-22 17:00:40.261334
## 19: 2023-11-22 17:00:40.261334
## 20: 2023-11-22 17:00:40.261334
## 21: 2023-11-22 17:00:40.261334
## 22: 2023-11-22 17:00:40.261334
## 23: 2023-11-22 17:00:40.261334
## 24: 2023-11-22 17:00:40.261334
## 25: 2023-11-22 17:00:40.261334
## 26: 2023-11-22 17:00:40.261334
## 27: 2023-11-22 17:00:40.261334
## createdDate
clearCache(tmpDir, ask = FALSE)
# default userTags is "and" matching; for "or" matching use |
ranNumsA <- Cache(runif, 4, cachePath = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(rnorm, 4, cachePath = tmpDir, userTags = "objectName:b")
# show all objects (runif and rnorm in this case)
showCache(tmpDir)
## Cache size:
## Total (including Rasters): 504 bytes
## Selected objects (not including Rasters): 504 bytes
## cacheId tagKey tagValue
## 1: ad0ea27476c50b66 objectName b
## 2: ad0ea27476c50b66 function rnorm
## 3: ad0ea27476c50b66 class numeric
## 4: ad0ea27476c50b66 object.size 1008
## 5: ad0ea27476c50b66 accessed 2023-11-22 17:00:40.418544
## 6: ad0ea27476c50b66 inCloud FALSE
## 7: ad0ea27476c50b66 fromDisk FALSE
## 8: ad0ea27476c50b66 resultHash
## 9: ad0ea27476c50b66 elapsedTimeDigest 0.001995087 secs
## 10: ad0ea27476c50b66 elapsedTimeFirstRun 2.527237e-05 secs
## 11: ad0ea27476c50b66 otherFunctions saveRDS
## 12: ad0ea27476c50b66 otherFunctions do.call
## 13: ad0ea27476c50b66 otherFunctions process_file
## 14: ad0ea27476c50b66 otherFunctions handle_error
## 15: ad0ea27476c50b66 otherFunctions process_group
## 16: ad0ea27476c50b66 otherFunctions process_group.block
## 17: ad0ea27476c50b66 otherFunctions call_block
## 18: ad0ea27476c50b66 otherFunctions block_exec
## 19: ad0ea27476c50b66 otherFunctions eng_r
## 20: ad0ea27476c50b66 otherFunctions in_input_dir
## 21: ad0ea27476c50b66 otherFunctions in_dir
## 22: ad0ea27476c50b66 otherFunctions timing_fn
## 23: ad0ea27476c50b66 otherFunctions handle
## 24: ad0ea27476c50b66 preDigest n:7eef4eae85fd9229
## 25: ad0ea27476c50b66 preDigest mean:c40c00762a0dac94
## 26: ad0ea27476c50b66 preDigest sd:853b1797f54b229c
## 27: ad0ea27476c50b66 preDigest .FUN:4f604aa46882b368
## 28: deaa37372f85861b objectName a
## 29: deaa37372f85861b function runif
## 30: deaa37372f85861b class numeric
## 31: deaa37372f85861b object.size 1008
## 32: deaa37372f85861b accessed 2023-11-22 17:00:40.402388
## 33: deaa37372f85861b inCloud FALSE
## 34: deaa37372f85861b fromDisk FALSE
## 35: deaa37372f85861b resultHash
## 36: deaa37372f85861b elapsedTimeDigest 0.002233267 secs
## 37: deaa37372f85861b elapsedTimeFirstRun 2.551079e-05 secs
## 38: deaa37372f85861b otherFunctions saveRDS
## 39: deaa37372f85861b otherFunctions do.call
## 40: deaa37372f85861b otherFunctions process_file
## 41: deaa37372f85861b otherFunctions handle_error
## 42: deaa37372f85861b otherFunctions process_group
## 43: deaa37372f85861b otherFunctions process_group.block
## 44: deaa37372f85861b otherFunctions call_block
## 45: deaa37372f85861b otherFunctions block_exec
## 46: deaa37372f85861b otherFunctions eng_r
## 47: deaa37372f85861b otherFunctions in_input_dir
## 48: deaa37372f85861b otherFunctions in_dir
## 49: deaa37372f85861b otherFunctions timing_fn
## 50: deaa37372f85861b otherFunctions handle
## 51: deaa37372f85861b preDigest n:7eef4eae85fd9229
## 52: deaa37372f85861b preDigest min:c40c00762a0dac94
## 53: deaa37372f85861b preDigest max:853b1797f54b229c
## 54: deaa37372f85861b preDigest .FUN:881ec847b7161f3c
## cacheId tagKey tagValue
## createdDate
## 1: 2023-11-22 17:00:40.419014
## 2: 2023-11-22 17:00:40.419014
## 3: 2023-11-22 17:00:40.419014
## 4: 2023-11-22 17:00:40.419014
## 5: 2023-11-22 17:00:40.419014
## 6: 2023-11-22 17:00:40.419014
## 7: 2023-11-22 17:00:40.419014
## 8: 2023-11-22 17:00:40.419014
## 9: 2023-11-22 17:00:40.419014
## 10: 2023-11-22 17:00:40.419014
## 11: 2023-11-22 17:00:40.419014
## 12: 2023-11-22 17:00:40.419014
## 13: 2023-11-22 17:00:40.419014
## 14: 2023-11-22 17:00:40.419014
## 15: 2023-11-22 17:00:40.419014
## 16: 2023-11-22 17:00:40.419014
## 17: 2023-11-22 17:00:40.419014
## 18: 2023-11-22 17:00:40.419014
## 19: 2023-11-22 17:00:40.419014
## 20: 2023-11-22 17:00:40.419014
## 21: 2023-11-22 17:00:40.419014
## 22: 2023-11-22 17:00:40.419014
## 23: 2023-11-22 17:00:40.419014
## 24: 2023-11-22 17:00:40.419014
## 25: 2023-11-22 17:00:40.419014
## 26: 2023-11-22 17:00:40.419014
## 27: 2023-11-22 17:00:40.419014
## 28: 2023-11-22 17:00:40.40289
## 29: 2023-11-22 17:00:40.40289
## 30: 2023-11-22 17:00:40.40289
## 31: 2023-11-22 17:00:40.40289
## 32: 2023-11-22 17:00:40.40289
## 33: 2023-11-22 17:00:40.40289
## 34: 2023-11-22 17:00:40.40289
## 35: 2023-11-22 17:00:40.40289
## 36: 2023-11-22 17:00:40.40289
## 37: 2023-11-22 17:00:40.40289
## 38: 2023-11-22 17:00:40.40289
## 39: 2023-11-22 17:00:40.40289
## 40: 2023-11-22 17:00:40.40289
## 41: 2023-11-22 17:00:40.40289
## 42: 2023-11-22 17:00:40.40289
## 43: 2023-11-22 17:00:40.40289
## 44: 2023-11-22 17:00:40.40289
## 45: 2023-11-22 17:00:40.40289
## 46: 2023-11-22 17:00:40.40289
## 47: 2023-11-22 17:00:40.40289
## 48: 2023-11-22 17:00:40.40289
## 49: 2023-11-22 17:00:40.40289
## 50: 2023-11-22 17:00:40.40289
## 51: 2023-11-22 17:00:40.40289
## 52: 2023-11-22 17:00:40.40289
## 53: 2023-11-22 17:00:40.40289
## 54: 2023-11-22 17:00:40.40289
## createdDate
# show objects that are both runif and rnorm
# (i.e., none in this case, because objecs are either or, not both)
showCache(tmpDir, userTags = c("runif", "rnorm")) ## empty
## Cache size:
## Total (including Rasters): 0 bytes
## Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows and 4 cols): cacheId,tagKey,tagValue,createdDate
# show objects that are either runif or rnorm ("or" search)
showCache(tmpDir, userTags = "runif|rnorm")
## Cache size:
## Total (including Rasters): 504 bytes
## Selected objects (not including Rasters): 504 bytes
## cacheId tagKey tagValue
## 1: ad0ea27476c50b66 objectName b
## 2: ad0ea27476c50b66 function rnorm
## 3: ad0ea27476c50b66 class numeric
## 4: ad0ea27476c50b66 object.size 1008
## 5: ad0ea27476c50b66 accessed 2023-11-22 17:00:40.418544
## 6: ad0ea27476c50b66 inCloud FALSE
## 7: ad0ea27476c50b66 fromDisk FALSE
## 8: ad0ea27476c50b66 resultHash
## 9: ad0ea27476c50b66 elapsedTimeDigest 0.001995087 secs
## 10: ad0ea27476c50b66 elapsedTimeFirstRun 2.527237e-05 secs
## 11: ad0ea27476c50b66 otherFunctions saveRDS
## 12: ad0ea27476c50b66 otherFunctions do.call
## 13: ad0ea27476c50b66 otherFunctions process_file
## 14: ad0ea27476c50b66 otherFunctions handle_error
## 15: ad0ea27476c50b66 otherFunctions process_group
## 16: ad0ea27476c50b66 otherFunctions process_group.block
## 17: ad0ea27476c50b66 otherFunctions call_block
## 18: ad0ea27476c50b66 otherFunctions block_exec
## 19: ad0ea27476c50b66 otherFunctions eng_r
## 20: ad0ea27476c50b66 otherFunctions in_input_dir
## 21: ad0ea27476c50b66 otherFunctions in_dir
## 22: ad0ea27476c50b66 otherFunctions timing_fn
## 23: ad0ea27476c50b66 otherFunctions handle
## 24: ad0ea27476c50b66 preDigest n:7eef4eae85fd9229
## 25: ad0ea27476c50b66 preDigest mean:c40c00762a0dac94
## 26: ad0ea27476c50b66 preDigest sd:853b1797f54b229c
## 27: ad0ea27476c50b66 preDigest .FUN:4f604aa46882b368
## 28: deaa37372f85861b objectName a
## 29: deaa37372f85861b function runif
## 30: deaa37372f85861b class numeric
## 31: deaa37372f85861b object.size 1008
## 32: deaa37372f85861b accessed 2023-11-22 17:00:40.402388
## 33: deaa37372f85861b inCloud FALSE
## 34: deaa37372f85861b fromDisk FALSE
## 35: deaa37372f85861b resultHash
## 36: deaa37372f85861b elapsedTimeDigest 0.002233267 secs
## 37: deaa37372f85861b elapsedTimeFirstRun 2.551079e-05 secs
## 38: deaa37372f85861b otherFunctions saveRDS
## 39: deaa37372f85861b otherFunctions do.call
## 40: deaa37372f85861b otherFunctions process_file
## 41: deaa37372f85861b otherFunctions handle_error
## 42: deaa37372f85861b otherFunctions process_group
## 43: deaa37372f85861b otherFunctions process_group.block
## 44: deaa37372f85861b otherFunctions call_block
## 45: deaa37372f85861b otherFunctions block_exec
## 46: deaa37372f85861b otherFunctions eng_r
## 47: deaa37372f85861b otherFunctions in_input_dir
## 48: deaa37372f85861b otherFunctions in_dir
## 49: deaa37372f85861b otherFunctions timing_fn
## 50: deaa37372f85861b otherFunctions handle
## 51: deaa37372f85861b preDigest n:7eef4eae85fd9229
## 52: deaa37372f85861b preDigest min:c40c00762a0dac94
## 53: deaa37372f85861b preDigest max:853b1797f54b229c
## 54: deaa37372f85861b preDigest .FUN:881ec847b7161f3c
## cacheId tagKey tagValue
## createdDate
## 1: 2023-11-22 17:00:40.419014
## 2: 2023-11-22 17:00:40.419014
## 3: 2023-11-22 17:00:40.419014
## 4: 2023-11-22 17:00:40.419014
## 5: 2023-11-22 17:00:40.419014
## 6: 2023-11-22 17:00:40.419014
## 7: 2023-11-22 17:00:40.419014
## 8: 2023-11-22 17:00:40.419014
## 9: 2023-11-22 17:00:40.419014
## 10: 2023-11-22 17:00:40.419014
## 11: 2023-11-22 17:00:40.419014
## 12: 2023-11-22 17:00:40.419014
## 13: 2023-11-22 17:00:40.419014
## 14: 2023-11-22 17:00:40.419014
## 15: 2023-11-22 17:00:40.419014
## 16: 2023-11-22 17:00:40.419014
## 17: 2023-11-22 17:00:40.419014
## 18: 2023-11-22 17:00:40.419014
## 19: 2023-11-22 17:00:40.419014
## 20: 2023-11-22 17:00:40.419014
## 21: 2023-11-22 17:00:40.419014
## 22: 2023-11-22 17:00:40.419014
## 23: 2023-11-22 17:00:40.419014
## 24: 2023-11-22 17:00:40.419014
## 25: 2023-11-22 17:00:40.419014
## 26: 2023-11-22 17:00:40.419014
## 27: 2023-11-22 17:00:40.419014
## 28: 2023-11-22 17:00:40.40289
## 29: 2023-11-22 17:00:40.40289
## 30: 2023-11-22 17:00:40.40289
## 31: 2023-11-22 17:00:40.40289
## 32: 2023-11-22 17:00:40.40289
## 33: 2023-11-22 17:00:40.40289
## 34: 2023-11-22 17:00:40.40289
## 35: 2023-11-22 17:00:40.40289
## 36: 2023-11-22 17:00:40.40289
## 37: 2023-11-22 17:00:40.40289
## 38: 2023-11-22 17:00:40.40289
## 39: 2023-11-22 17:00:40.40289
## 40: 2023-11-22 17:00:40.40289
## 41: 2023-11-22 17:00:40.40289
## 42: 2023-11-22 17:00:40.40289
## 43: 2023-11-22 17:00:40.40289
## 44: 2023-11-22 17:00:40.40289
## 45: 2023-11-22 17:00:40.40289
## 46: 2023-11-22 17:00:40.40289
## 47: 2023-11-22 17:00:40.40289
## 48: 2023-11-22 17:00:40.40289
## 49: 2023-11-22 17:00:40.40289
## 50: 2023-11-22 17:00:40.40289
## 51: 2023-11-22 17:00:40.40289
## 52: 2023-11-22 17:00:40.40289
## 53: 2023-11-22 17:00:40.40289
## 54: 2023-11-22 17:00:40.40289
## createdDate
# keep only objects that are either runif or rnorm ("or" search)
keepCache(tmpDir, userTags = "runif|rnorm", ask = FALSE)
## Cache size:
## Total (including Rasters): 504 bytes
## Selected objects (not including Rasters): 504 bytes
## cacheId tagKey tagValue
## 1: ad0ea27476c50b66 objectName b
## 2: ad0ea27476c50b66 function rnorm
## 3: ad0ea27476c50b66 class numeric
## 4: ad0ea27476c50b66 object.size 1008
## 5: ad0ea27476c50b66 accessed 2023-11-22 17:00:40.418544
## 6: ad0ea27476c50b66 inCloud FALSE
## 7: ad0ea27476c50b66 fromDisk FALSE
## 8: ad0ea27476c50b66 resultHash
## 9: ad0ea27476c50b66 elapsedTimeDigest 0.001995087 secs
## 10: ad0ea27476c50b66 elapsedTimeFirstRun 2.527237e-05 secs
## 11: ad0ea27476c50b66 otherFunctions saveRDS
## 12: ad0ea27476c50b66 otherFunctions do.call
## 13: ad0ea27476c50b66 otherFunctions process_file
## 14: ad0ea27476c50b66 otherFunctions handle_error
## 15: ad0ea27476c50b66 otherFunctions process_group
## 16: ad0ea27476c50b66 otherFunctions process_group.block
## 17: ad0ea27476c50b66 otherFunctions call_block
## 18: ad0ea27476c50b66 otherFunctions block_exec
## 19: ad0ea27476c50b66 otherFunctions eng_r
## 20: ad0ea27476c50b66 otherFunctions in_input_dir
## 21: ad0ea27476c50b66 otherFunctions in_dir
## 22: ad0ea27476c50b66 otherFunctions timing_fn
## 23: ad0ea27476c50b66 otherFunctions handle
## 24: ad0ea27476c50b66 preDigest n:7eef4eae85fd9229
## 25: ad0ea27476c50b66 preDigest mean:c40c00762a0dac94
## 26: ad0ea27476c50b66 preDigest sd:853b1797f54b229c
## 27: ad0ea27476c50b66 preDigest .FUN:4f604aa46882b368
## 28: deaa37372f85861b objectName a
## 29: deaa37372f85861b function runif
## 30: deaa37372f85861b class numeric
## 31: deaa37372f85861b object.size 1008
## 32: deaa37372f85861b accessed 2023-11-22 17:00:40.402388
## 33: deaa37372f85861b inCloud FALSE
## 34: deaa37372f85861b fromDisk FALSE
## 35: deaa37372f85861b resultHash
## 36: deaa37372f85861b elapsedTimeDigest 0.002233267 secs
## 37: deaa37372f85861b elapsedTimeFirstRun 2.551079e-05 secs
## 38: deaa37372f85861b otherFunctions saveRDS
## 39: deaa37372f85861b otherFunctions do.call
## 40: deaa37372f85861b otherFunctions process_file
## 41: deaa37372f85861b otherFunctions handle_error
## 42: deaa37372f85861b otherFunctions process_group
## 43: deaa37372f85861b otherFunctions process_group.block
## 44: deaa37372f85861b otherFunctions call_block
## 45: deaa37372f85861b otherFunctions block_exec
## 46: deaa37372f85861b otherFunctions eng_r
## 47: deaa37372f85861b otherFunctions in_input_dir
## 48: deaa37372f85861b otherFunctions in_dir
## 49: deaa37372f85861b otherFunctions timing_fn
## 50: deaa37372f85861b otherFunctions handle
## 51: deaa37372f85861b preDigest n:7eef4eae85fd9229
## 52: deaa37372f85861b preDigest min:c40c00762a0dac94
## 53: deaa37372f85861b preDigest max:853b1797f54b229c
## 54: deaa37372f85861b preDigest .FUN:881ec847b7161f3c
## cacheId tagKey tagValue
## createdDate
## 1: 2023-11-22 17:00:40.419014
## 2: 2023-11-22 17:00:40.419014
## 3: 2023-11-22 17:00:40.419014
## 4: 2023-11-22 17:00:40.419014
## 5: 2023-11-22 17:00:40.419014
## 6: 2023-11-22 17:00:40.419014
## 7: 2023-11-22 17:00:40.419014
## 8: 2023-11-22 17:00:40.419014
## 9: 2023-11-22 17:00:40.419014
## 10: 2023-11-22 17:00:40.419014
## 11: 2023-11-22 17:00:40.419014
## 12: 2023-11-22 17:00:40.419014
## 13: 2023-11-22 17:00:40.419014
## 14: 2023-11-22 17:00:40.419014
## 15: 2023-11-22 17:00:40.419014
## 16: 2023-11-22 17:00:40.419014
## 17: 2023-11-22 17:00:40.419014
## 18: 2023-11-22 17:00:40.419014
## 19: 2023-11-22 17:00:40.419014
## 20: 2023-11-22 17:00:40.419014
## 21: 2023-11-22 17:00:40.419014
## 22: 2023-11-22 17:00:40.419014
## 23: 2023-11-22 17:00:40.419014
## 24: 2023-11-22 17:00:40.419014
## 25: 2023-11-22 17:00:40.419014
## 26: 2023-11-22 17:00:40.419014
## 27: 2023-11-22 17:00:40.419014
## 28: 2023-11-22 17:00:40.40289
## 29: 2023-11-22 17:00:40.40289
## 30: 2023-11-22 17:00:40.40289
## 31: 2023-11-22 17:00:40.40289
## 32: 2023-11-22 17:00:40.40289
## 33: 2023-11-22 17:00:40.40289
## 34: 2023-11-22 17:00:40.40289
## 35: 2023-11-22 17:00:40.40289
## 36: 2023-11-22 17:00:40.40289
## 37: 2023-11-22 17:00:40.40289
## 38: 2023-11-22 17:00:40.40289
## 39: 2023-11-22 17:00:40.40289
## 40: 2023-11-22 17:00:40.40289
## 41: 2023-11-22 17:00:40.40289
## 42: 2023-11-22 17:00:40.40289
## 43: 2023-11-22 17:00:40.40289
## 44: 2023-11-22 17:00:40.40289
## 45: 2023-11-22 17:00:40.40289
## 46: 2023-11-22 17:00:40.40289
## 47: 2023-11-22 17:00:40.40289
## 48: 2023-11-22 17:00:40.40289
## 49: 2023-11-22 17:00:40.40289
## 50: 2023-11-22 17:00:40.40289
## 51: 2023-11-22 17:00:40.40289
## 52: 2023-11-22 17:00:40.40289
## 53: 2023-11-22 17:00:40.40289
## 54: 2023-11-22 17:00:40.40289
## createdDate
clearCache(tmpDir, ask = FALSE)
ras <- terra::rast(terra::ext(0, 5, 0, 5),
res = 1,
vals = sample(1:5, replace = TRUE, size = 25),
crs = "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84"
)
rasCRS <- terra::crs(ras)
# A slow operation, like GIS operation
notCached <- suppressWarnings(
# project raster generates warnings when run non-interactively
terra::project(ras, rasCRS, res = 5)
)
cached <- suppressWarnings(
# project raster generates warnings when run non-interactively
# using quote works also
Cache(terra::project, ras, rasCRS, res = 5, cachePath = tmpDir)
)
# second time is much faster
reRun <- suppressWarnings(
# project raster generates warnings when run non-interactively
Cache(terra::project, ras, rasCRS, res = 5, cachePath = tmpDir)
)
## ...(Object to retrieve (fn: terra::project, d93242f1dda475b7.rds))
## loaded cached result from previous terra::project call
# recovered cached version is same as non-cached version
all.equal(notCached, reRun, check.attributes = FALSE) ## TRUE
## [1] "Attributes: < Names: 2 string mismatches >"
## [2] "Attributes: < Length mismatch: comparison on first 2 components >"
## [3] "Attributes: < Component 1: Modes: character, list >"
## [4] "Attributes: < Component 1: names for current but not for target >"
## [5] "Attributes: < Component 1: Attributes: < names for target but not for current > >"
## [6] "Attributes: < Component 1: Attributes: < Length mismatch: comparison on first 0 components > >"
## [7] "Attributes: < Component 1: target is character, current is list >"
## [8] "Attributes: < Component 2: 'current' is not an envRefClass >"
Nested caching, which is when Caching of a function occurs inside an outer function, which is itself cached. This is a critical element to working within a reproducible work flow. It is not enough during development to cache flat code chunks, as there will be many levels of “slow” functions. Ideally, at all points in a development cycle, it should be possible to get to any line of code starting from the very initial steps, running through everything up to that point, in less than a few seconds. If the workflow can be kept very fast like this, then there is a guarantee that it will work at any point.
##########################
## Nested Caching
# Make 2 functions
inner <- function(mean) {
d <- 1
Cache(rnorm, n = 3, mean = mean)
}
outer <- function(n) {
Cache(inner, 0.1, cachePath = tmpdir2)
}
# make 2 different cache paths
tmpdir1 <- file.path(tempdir(), "first")
tmpdir2 <- file.path(tempdir(), "second")
# Run the Cache ... notOlderThan propagates to all 3 Cache calls,
# but cachePath is tmpdir1 in top level Cache and all nested
# Cache calls, unless individually overridden ... here inner
# uses tmpdir2 repository
Cache(outer, n = 2, cachePath = tmpdir1, notOlderThan = Sys.time())
## No cachePath supplied and getOption('reproducible.cachePath') is inside a temporary directory;
## this will not persist across R sessions.
## [1] 0.3555315 -1.4052696 1.1279295
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
##
## attr(,"tags")
## [1] "cacheId:dffc8e3bf8c23b6e"
## attr(,"call")
## [1] ""
showCache(tmpdir1) # 2 function calls
## Cache size:
## Total (including Rasters): 252 bytes
## Selected objects (not including Rasters): 252 bytes
## cacheId tagKey tagValue
## 1: dffc8e3bf8c23b6e function outer
## 2: dffc8e3bf8c23b6e class numeric
## 3: dffc8e3bf8c23b6e object.size 1008
## 4: dffc8e3bf8c23b6e accessed 2023-11-22 17:00:40.831893
## 5: dffc8e3bf8c23b6e inCloud FALSE
## 6: dffc8e3bf8c23b6e fromDisk FALSE
## 7: dffc8e3bf8c23b6e resultHash
## 8: dffc8e3bf8c23b6e elapsedTimeDigest 0.002130985 secs
## 9: dffc8e3bf8c23b6e elapsedTimeFirstRun 0.04881787 secs
## 10: dffc8e3bf8c23b6e otherFunctions saveRDS
## 11: dffc8e3bf8c23b6e otherFunctions do.call
## 12: dffc8e3bf8c23b6e otherFunctions process_file
## 13: dffc8e3bf8c23b6e otherFunctions handle_error
## 14: dffc8e3bf8c23b6e otherFunctions process_group
## 15: dffc8e3bf8c23b6e otherFunctions process_group.block
## 16: dffc8e3bf8c23b6e otherFunctions call_block
## 17: dffc8e3bf8c23b6e otherFunctions block_exec
## 18: dffc8e3bf8c23b6e otherFunctions eng_r
## 19: dffc8e3bf8c23b6e otherFunctions in_input_dir
## 20: dffc8e3bf8c23b6e otherFunctions in_dir
## 21: dffc8e3bf8c23b6e otherFunctions timing_fn
## 22: dffc8e3bf8c23b6e otherFunctions handle
## 23: dffc8e3bf8c23b6e preDigest n:82dc709f2b91918a
## 24: dffc8e3bf8c23b6e preDigest .FUN:b89dd186387954a0
## cacheId tagKey tagValue
## createdDate
## 1: 2023-11-22 17:00:40.832381
## 2: 2023-11-22 17:00:40.832381
## 3: 2023-11-22 17:00:40.832381
## 4: 2023-11-22 17:00:40.832381
## 5: 2023-11-22 17:00:40.832381
## 6: 2023-11-22 17:00:40.832381
## 7: 2023-11-22 17:00:40.832381
## 8: 2023-11-22 17:00:40.832381
## 9: 2023-11-22 17:00:40.832381
## 10: 2023-11-22 17:00:40.832381
## 11: 2023-11-22 17:00:40.832381
## 12: 2023-11-22 17:00:40.832381
## 13: 2023-11-22 17:00:40.832381
## 14: 2023-11-22 17:00:40.832381
## 15: 2023-11-22 17:00:40.832381
## 16: 2023-11-22 17:00:40.832381
## 17: 2023-11-22 17:00:40.832381
## 18: 2023-11-22 17:00:40.832381
## 19: 2023-11-22 17:00:40.832381
## 20: 2023-11-22 17:00:40.832381
## 21: 2023-11-22 17:00:40.832381
## 22: 2023-11-22 17:00:40.832381
## 23: 2023-11-22 17:00:40.832381
## 24: 2023-11-22 17:00:40.832381
## createdDate
showCache(tmpdir2) # 1 function call
## Cache size:
## Total (including Rasters): 252 bytes
## Selected objects (not including Rasters): 252 bytes
## cacheId tagKey tagValue
## 1: 33ceb4fb525fd08f function inner
## 2: 33ceb4fb525fd08f class numeric
## 3: 33ceb4fb525fd08f object.size 1008
## 4: 33ceb4fb525fd08f accessed 2023-11-22 17:00:40.823194
## 5: 33ceb4fb525fd08f inCloud FALSE
## 6: 33ceb4fb525fd08f fromDisk FALSE
## 7: 33ceb4fb525fd08f resultHash
## 8: 33ceb4fb525fd08f elapsedTimeDigest 0.001929998 secs
## 9: 33ceb4fb525fd08f elapsedTimeFirstRun 0.02427864 secs
## 10: 33ceb4fb525fd08f otherFunctions saveRDS
## 11: 33ceb4fb525fd08f otherFunctions do.call
## 12: 33ceb4fb525fd08f otherFunctions process_file
## 13: 33ceb4fb525fd08f otherFunctions handle_error
## 14: 33ceb4fb525fd08f otherFunctions process_group
## 15: 33ceb4fb525fd08f otherFunctions process_group.block
## 16: 33ceb4fb525fd08f otherFunctions call_block
## 17: 33ceb4fb525fd08f otherFunctions block_exec
## 18: 33ceb4fb525fd08f otherFunctions eng_r
## 19: 33ceb4fb525fd08f otherFunctions in_input_dir
## 20: 33ceb4fb525fd08f otherFunctions in_dir
## 21: 33ceb4fb525fd08f otherFunctions timing_fn
## 22: 33ceb4fb525fd08f otherFunctions handle
## 23: 33ceb4fb525fd08f otherFunctions out
## 24: 33ceb4fb525fd08f preDigest mean:22413394efd9f6a3
## 25: 33ceb4fb525fd08f preDigest .FUN:87e2c30917a34d25
## cacheId tagKey tagValue
## createdDate
## 1: 2023-11-22 17:00:40.823672
## 2: 2023-11-22 17:00:40.823672
## 3: 2023-11-22 17:00:40.823672
## 4: 2023-11-22 17:00:40.823672
## 5: 2023-11-22 17:00:40.823672
## 6: 2023-11-22 17:00:40.823672
## 7: 2023-11-22 17:00:40.823672
## 8: 2023-11-22 17:00:40.823672
## 9: 2023-11-22 17:00:40.823672
## 10: 2023-11-22 17:00:40.823672
## 11: 2023-11-22 17:00:40.823672
## 12: 2023-11-22 17:00:40.823672
## 13: 2023-11-22 17:00:40.823672
## 14: 2023-11-22 17:00:40.823672
## 15: 2023-11-22 17:00:40.823672
## 16: 2023-11-22 17:00:40.823672
## 17: 2023-11-22 17:00:40.823672
## 18: 2023-11-22 17:00:40.823672
## 19: 2023-11-22 17:00:40.823672
## 20: 2023-11-22 17:00:40.823672
## 21: 2023-11-22 17:00:40.823672
## 22: 2023-11-22 17:00:40.823672
## 23: 2023-11-22 17:00:40.823672
## 24: 2023-11-22 17:00:40.823672
## 25: 2023-11-22 17:00:40.823672
## createdDate
# userTags get appended
# all items have the outer tag propagate, plus inner ones only have inner ones
clearCache(tmpdir1, ask = FALSE)
outerTag <- "outerTag"
innerTag <- "innerTag"
inner <- function(mean) {
d <- 1
Cache(rnorm, n = 3, mean = mean, notOlderThan = Sys.time() - 1e5, userTags = innerTag)
}
outer <- function(n) {
Cache(inner, 0.1)
}
aa <- Cache(outer, n = 2, cachePath = tmpdir1, userTags = outerTag)
## No cachePath supplied and getOption('reproducible.cachePath') is inside a temporary directory;
## this will not persist across R sessions.
## No cachePath supplied and getOption('reproducible.cachePath') is inside a temporary directory;
## this will not persist across R sessions.
## ...(Object to retrieve (fn: rnorm, efa1ccee79a31d4c.rds))
## loaded cached result from previous rnorm call
showCache(tmpdir1) # rnorm function has outerTag and innerTag, inner and outer only have outerTag
## Cache size:
## Total (including Rasters): 252 bytes
## Selected objects (not including Rasters): 252 bytes
## cacheId tagKey tagValue
## 1: 88a34e1d033329e5 outerTag outerTag
## 2: 88a34e1d033329e5 function outer
## 3: 88a34e1d033329e5 class numeric
## 4: 88a34e1d033329e5 object.size 1008
## 5: 88a34e1d033329e5 accessed 2023-11-22 17:00:40.912106
## 6: 88a34e1d033329e5 inCloud FALSE
## 7: 88a34e1d033329e5 fromDisk FALSE
## 8: 88a34e1d033329e5 resultHash
## 9: 88a34e1d033329e5 elapsedTimeDigest 0.001890421 secs
## 10: 88a34e1d033329e5 elapsedTimeFirstRun 0.03134584 secs
## 11: 88a34e1d033329e5 otherFunctions saveRDS
## 12: 88a34e1d033329e5 otherFunctions do.call
## 13: 88a34e1d033329e5 otherFunctions process_file
## 14: 88a34e1d033329e5 otherFunctions handle_error
## 15: 88a34e1d033329e5 otherFunctions process_group
## 16: 88a34e1d033329e5 otherFunctions process_group.block
## 17: 88a34e1d033329e5 otherFunctions call_block
## 18: 88a34e1d033329e5 otherFunctions block_exec
## 19: 88a34e1d033329e5 otherFunctions eng_r
## 20: 88a34e1d033329e5 otherFunctions in_input_dir
## 21: 88a34e1d033329e5 otherFunctions in_dir
## 22: 88a34e1d033329e5 otherFunctions timing_fn
## 23: 88a34e1d033329e5 otherFunctions handle
## 24: 88a34e1d033329e5 preDigest n:82dc709f2b91918a
## 25: 88a34e1d033329e5 preDigest .FUN:5f06fb5fbffe9e3b
## cacheId tagKey tagValue
## createdDate
## 1: 2023-11-22 17:00:40.912607
## 2: 2023-11-22 17:00:40.912607
## 3: 2023-11-22 17:00:40.912607
## 4: 2023-11-22 17:00:40.912607
## 5: 2023-11-22 17:00:40.912607
## 6: 2023-11-22 17:00:40.912607
## 7: 2023-11-22 17:00:40.912607
## 8: 2023-11-22 17:00:40.912607
## 9: 2023-11-22 17:00:40.912607
## 10: 2023-11-22 17:00:40.912607
## 11: 2023-11-22 17:00:40.912607
## 12: 2023-11-22 17:00:40.912607
## 13: 2023-11-22 17:00:40.912607
## 14: 2023-11-22 17:00:40.912607
## 15: 2023-11-22 17:00:40.912607
## 16: 2023-11-22 17:00:40.912607
## 17: 2023-11-22 17:00:40.912607
## 18: 2023-11-22 17:00:40.912607
## 19: 2023-11-22 17:00:40.912607
## 20: 2023-11-22 17:00:40.912607
## 21: 2023-11-22 17:00:40.912607
## 22: 2023-11-22 17:00:40.912607
## 23: 2023-11-22 17:00:40.912607
## 24: 2023-11-22 17:00:40.912607
## 25: 2023-11-22 17:00:40.912607
## createdDate
Sometimes, it is not absolutely desirable to maintain the work flow
intact because changes that are irrelevant to the analysis, such as
changing messages sent to a user, may be changed, without a desire to
rerun functions. The cacheId
argument is for this. Once a
piece of code is run, then the cacheId
can be manually
extracted (it is reported at the end of a Cache call) and manually
placed in the code, passed in as, say,
cacheId = "ad184ce64541972b50afd8e7b75f821b"
.
## [1] -0.6264538
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
##
## attr(,"tags")
## [1] "cacheId:422bae4ed2f770cc"
## attr(,"call")
## [1] ""
# manually look at output attribute which shows cacheId: 7072c305d8c69df0
Cache(rnorm, 1, cachePath = tmpdir1, cacheId = "422bae4ed2f770cc") # same value
## cacheId is same as calculated hash
## ...(Object to retrieve (fn: rnorm, 422bae4ed2f770cc.rds))
## loaded cached result from previous rnorm call
## [1] -0.6264538
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] FALSE
##
## attr(,"tags")
## [1] "cacheId:422bae4ed2f770cc"
## attr(,"call")
## [1] ""
# override even with different inputs:
Cache(rnorm, 2, cachePath = tmpdir1, cacheId = "422bae4ed2f770cc")
## cacheId is not same as calculated hash. Manually searching for cacheId:422bae4ed2f770cc
## [1] 0.7383247 0.5757814
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
##
## attr(,"tags")
## [1] "cacheId:422bae4ed2f770cc"
## attr(,"call")
## [1] ""
Since the cache is simply a DBI
data table (of an SQLite
database by default). In addition, there are several helpers in the
reproducible
package, including showCache
,
keepCache
and clearCache
that may be useful.
Also, one can access cached items manually (rather than simply rerunning
the same Cache
function again).
# As of reproducible version 1.0, there is a new backend directly using DBI
mapHash <- unique(showCache(tmpDir, userTags = "project")$cacheId)
## Cache size:
## Total (including Rasters): 774 bytes
## Selected objects (not including Rasters): 774 bytes
map <- loadFromCache(mapHash[1], cachePath = tmpDir)
## loaded cached result from previous call
terra::plot(map)
By default, caching relies on a sqlite database for it’s backend.
While this works in many situations, there are some important
limitations of using sqlite for caching, including 1) speed; 2)
concurrent transactions; 3) sharing database across machines or
projects. Fortunately, Cache
makes use of DBI
package and thus supports several database backends, including mysql and
postgresql.
See https://github.com/PredictiveEcology/SpaDES/wiki/Using-alternate-database-backends-for-Cache for further information on configuring these additional backends.