Introduction to Cache
Eliot J. B. McIntire
August 17 2020
Source:vignettes/Intro-to-Cache.Rmd
Intro-to-Cache.RmdReproducible workflow
As part of a reproducible workflow, caching of function calls, code chunks, and other elements of a project is a critical component. The objective of a reproducible workflow is is likely that an entire work flow from raw data to publication, decision support, report writing, presentation building etc., could be built and be reproducible anywhere, on any computer, operating system, with any starting conditions, on demand. The reproducible::Cache function is built to work with any R function.
Differences with other approaches
Cache users DBI as a backend, with key functions, dbReadTable, dbRemoveTable, dbSendQuery, dbSendStatement, dbCreateTable and dbAppendTable. These can all be accessed via Cache, showCache, clearCache, and keepCache. It is optimized for speed of transactions, using fastdigest::fastdigest on R memory objects and digest::digest on files. The main function is superficially similar to archivist::cache, which uses digest::digest in all cases to determine whether the arguments are identical in subsequent iterations. It also but does many things that make standard caching with digest::digest don’t work reliably between systems. For these, the function .robustDigest is introduced to make caching transferable between systems. This is relevant for file paths, environments, parallel clusters, functions (which are contained within an environment), and many others (e.g., see ?.robustDigest for methods). Cache also adds important elements like automated tagging and the option to retrieve disk-cached values via stashed objects in memory using memoise::memoise. This means that running Cache 1, 2, and 3 times on the same function will get progressively faster. This can be extremely useful for web apps built with, say shiny.
Function-level caching
Any function can be cached using: Cache(FUN = functionName, ...).
This will be a slight change to a function call, such as: projectRaster(raster, crs = crs(newRaster)) to Cache(projectRaster, raster, crs = crs(newRaster)).
This is particularly useful for expensive operations.
library(raster)
## Loading required package: splibrary(reproducible) tmpDir <- file.path(tempdir(), "reproducible_examples", "Cache") checkPath(tmpDir, create = TRUE)
## [1] "C:/Users/EMCINT~1.L-V/AppData/Local/Temp/Rtmp0UIDzZ/reproducible_examples/Cache"ras <- raster(extent(0, 1000, 0, 1000), vals = 1:1e6, res = 1) crs(ras) <- "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +datum=WGS84" newCRS <- "+init=epsg:4326" # A longlat crs # No Cache system.time(suppressWarnings(map1 <- projectRaster(ras, crs = newCRS))) # Warnings due to new PROJ
## user system elapsed
## 2.54 0.35 3.06# Try with memoise for this example -- for many simple cases, memoising will not be faster opts <- options("reproducible.useMemoise" = TRUE) # With Cache -- a little slower the first time because saving to disk system.time(suppressWarnings(map1 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir, notOlderThan = Sys.time())))
## user system elapsed
## 3.27 0.46 4.05# vastly faster the second time system.time(map2 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
## ...(Object to retrieve (f2fc4c7fa19afa06.rds) is large: 6.5 Mb)## loaded memoised result from previous projectRaster call,## user system elapsed
## 0.36 0.09 0.47# may be faster the third time because of memoise; but this example is too simple to show system.time(map3 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
## ...(Object to retrieve (f2fc4c7fa19afa06.rds) is large: 6.5 Mb)
## loaded memoised result from previous projectRaster call,## user system elapsed
## 0.48 0.11 0.75## [1] TRUEall.equal(map1, map3) # TRUE
## [1] TRUECaching examples
Basic use
##
## Attaching package: 'magrittr'## The following object is masked from 'package:raster':
##
## extracttry(clearCache(tmpDir, ask = FALSE), silent = TRUE) # just to make sure it is clear ranNumsA <- Cache(rnorm, 10, 16, cacheRepo = tmpDir) # All same ranNumsB <- Cache(rnorm, 10, 16, cacheRepo = tmpDir) # recovers cached copy
## ...(Object to retrieve (f11fb1a2880f8060.rds))## loaded cached result from previous rnorm call,## ...(Object to retrieve (f11fb1a2880f8060.rds))## loaded cached result from previous 'rnorm' pipe sequence call,## ...(Object to retrieve (f11fb1a2880f8060.rds))## loaded cached result from previous rnorm call,Example 2: using the “accessed” tag
ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a") ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b") # access it again, from Cache Sys.sleep(1) ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
## ...(Object to retrieve (f7bee22047b8d0c1.rds))## loaded cached result from previous rnorm call,wholeCache <- showCache(tmpDir)
## Cache size:## Total (including Rasters): 504 bytes## Selected objects (not including Rasters): 504 bytes# keep only items accessed "recently" (i.e., only objectName:a) onlyRecentlyAccessed <- showCache(tmpDir, userTags = max(wholeCache[tagKey == "accessed"]$tagValue))
## Cache size:## Total (including Rasters): 252 bytes## Selected objects (not including Rasters): 252 bytes# inverse join with 2 data.tables ... using: a[!b] # i.e., return all of wholeCache that was not recently accessed # Note: the two different ways to access -- old way with "artifact" will be deprecated toRemove <- unique(wholeCache[!onlyRecentlyAccessed, on = "cacheId"], by = "cacheId")$cacheId clearCache(tmpDir, toRemove, ask = FALSE) # remove ones not recently accessed
## Cache size:## Total (including Rasters): 252 bytes## Selected objects (not including Rasters): 252 bytesshowCache(tmpDir) # still has more recently accessed
## Cache size:## Total (including Rasters): 252 bytes## Selected objects (not including Rasters): 252 bytes## cacheId tagKey tagValue
## 1: f7bee22047b8d0c1 objectName a
## 2: f7bee22047b8d0c1 function rnorm
## 3: f7bee22047b8d0c1 class numeric
## 4: f7bee22047b8d0c1 object.size 1008
## 5: f7bee22047b8d0c1 accessed 2020-08-17 13:09:06
## 6: f7bee22047b8d0c1 inCloud FALSE
## 7: f7bee22047b8d0c1 resultHash
## 8: f7bee22047b8d0c1 elapsedTimeDigest 0.002508879 secs
## 9: f7bee22047b8d0c1 elapsedTimeFirstRun 0 secs
## 10: f7bee22047b8d0c1 otherFunctions saveRDS
## 11: f7bee22047b8d0c1 otherFunctions do.call
## 12: f7bee22047b8d0c1 otherFunctions process_file
## 13: f7bee22047b8d0c1 otherFunctions process_group
## 14: f7bee22047b8d0c1 otherFunctions process_group.block
## 15: f7bee22047b8d0c1 otherFunctions call_block
## 16: f7bee22047b8d0c1 otherFunctions block_exec
## 17: f7bee22047b8d0c1 otherFunctions in_dir
## 18: f7bee22047b8d0c1 otherFunctions timing_fn
## 19: f7bee22047b8d0c1 otherFunctions handle
## 20: f7bee22047b8d0c1 preDigest n:7eef4eae85fd9229
## 21: f7bee22047b8d0c1 preDigest .FUN:4f604aa46882b368
## 22: f7bee22047b8d0c1 file.size 174
## 23: f7bee22047b8d0c1 accessed 2020-08-17 13:09:08
## 24: f7bee22047b8d0c1 elapsedTimeLoad 0.009024143 secs
## cacheId tagKey tagValue
## createdDate
## 1: 2020-08-17 13:09:06
## 2: 2020-08-17 13:09:06
## 3: 2020-08-17 13:09:06
## 4: 2020-08-17 13:09:06
## 5: 2020-08-17 13:09:06
## 6: 2020-08-17 13:09:06
## 7: 2020-08-17 13:09:06
## 8: 2020-08-17 13:09:06
## 9: 2020-08-17 13:09:06
## 10: 2020-08-17 13:09:06
## 11: 2020-08-17 13:09:06
## 12: 2020-08-17 13:09:06
## 13: 2020-08-17 13:09:06
## 14: 2020-08-17 13:09:06
## 15: 2020-08-17 13:09:06
## 16: 2020-08-17 13:09:06
## 17: 2020-08-17 13:09:06
## 18: 2020-08-17 13:09:06
## 19: 2020-08-17 13:09:06
## 20: 2020-08-17 13:09:06
## 21: 2020-08-17 13:09:06
## 22: 2020-08-17 13:09:06
## 23: 2020-08-17 13:09:08
## 24: 2020-08-17 13:09:08
## createdDateExample 3: using keepCache
ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
## ...(Object to retrieve (f7bee22047b8d0c1.rds))## loaded cached result from previous rnorm call,ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b") # keep only those cached items from the last 24 hours oneDay <- 60 * 60 * 24 keepCache(tmpDir, after = Sys.time() - oneDay, ask = FALSE)
## Cache size:## Total (including Rasters): 504 bytes## Selected objects (not including Rasters): 504 bytes## cacheId tagKey tagValue
## 1: f7bee22047b8d0c1 objectName a
## 2: f7bee22047b8d0c1 function rnorm
## 3: f7bee22047b8d0c1 class numeric
## 4: f7bee22047b8d0c1 object.size 1008
## 5: f7bee22047b8d0c1 accessed 2020-08-17 13:09:06
## 6: f7bee22047b8d0c1 inCloud FALSE
## 7: f7bee22047b8d0c1 resultHash
## 8: f7bee22047b8d0c1 elapsedTimeDigest 0.002508879 secs
## 9: f7bee22047b8d0c1 elapsedTimeFirstRun 0 secs
## 10: f7bee22047b8d0c1 otherFunctions saveRDS
## 11: f7bee22047b8d0c1 otherFunctions do.call
## 12: f7bee22047b8d0c1 otherFunctions process_file
## 13: f7bee22047b8d0c1 otherFunctions process_group
## 14: f7bee22047b8d0c1 otherFunctions process_group.block
## 15: f7bee22047b8d0c1 otherFunctions call_block
## 16: f7bee22047b8d0c1 otherFunctions block_exec
## 17: f7bee22047b8d0c1 otherFunctions in_dir
## 18: f7bee22047b8d0c1 otherFunctions timing_fn
## 19: f7bee22047b8d0c1 otherFunctions handle
## 20: f7bee22047b8d0c1 preDigest n:7eef4eae85fd9229
## 21: f7bee22047b8d0c1 preDigest .FUN:4f604aa46882b368
## 22: f7bee22047b8d0c1 file.size 174
## 23: f7bee22047b8d0c1 accessed 2020-08-17 13:09:08
## 24: f7bee22047b8d0c1 elapsedTimeLoad 0.01203299 secs
## 25: f7bee22047b8d0c1 accessed 2020-08-17 13:09:08
## 26: 3aef38d1fc02aee5 objectName b
## 27: 3aef38d1fc02aee5 function runif
## 28: 3aef38d1fc02aee5 class numeric
## 29: 3aef38d1fc02aee5 object.size 1008
## 30: 3aef38d1fc02aee5 accessed 2020-08-17 13:09:08
## 31: 3aef38d1fc02aee5 inCloud FALSE
## 32: 3aef38d1fc02aee5 resultHash
## 33: 3aef38d1fc02aee5 elapsedTimeDigest 0.001002073 secs
## 34: 3aef38d1fc02aee5 elapsedTimeFirstRun 0 secs
## 35: 3aef38d1fc02aee5 otherFunctions saveRDS
## 36: 3aef38d1fc02aee5 otherFunctions do.call
## 37: 3aef38d1fc02aee5 otherFunctions process_file
## 38: 3aef38d1fc02aee5 otherFunctions process_group
## 39: 3aef38d1fc02aee5 otherFunctions process_group.block
## 40: 3aef38d1fc02aee5 otherFunctions call_block
## 41: 3aef38d1fc02aee5 otherFunctions block_exec
## 42: 3aef38d1fc02aee5 otherFunctions in_dir
## 43: 3aef38d1fc02aee5 otherFunctions timing_fn
## 44: 3aef38d1fc02aee5 otherFunctions handle
## 45: 3aef38d1fc02aee5 preDigest n:7eef4eae85fd9229
## 46: 3aef38d1fc02aee5 preDigest .FUN:881ec847b7161f3c
## 47: 3aef38d1fc02aee5 file.size 169
## cacheId tagKey tagValue
## createdDate
## 1: 2020-08-17 13:09:06
## 2: 2020-08-17 13:09:06
## 3: 2020-08-17 13:09:06
## 4: 2020-08-17 13:09:06
## 5: 2020-08-17 13:09:06
## 6: 2020-08-17 13:09:06
## 7: 2020-08-17 13:09:06
## 8: 2020-08-17 13:09:06
## 9: 2020-08-17 13:09:06
## 10: 2020-08-17 13:09:06
## 11: 2020-08-17 13:09:06
## 12: 2020-08-17 13:09:06
## 13: 2020-08-17 13:09:06
## 14: 2020-08-17 13:09:06
## 15: 2020-08-17 13:09:06
## 16: 2020-08-17 13:09:06
## 17: 2020-08-17 13:09:06
## 18: 2020-08-17 13:09:06
## 19: 2020-08-17 13:09:06
## 20: 2020-08-17 13:09:06
## 21: 2020-08-17 13:09:06
## 22: 2020-08-17 13:09:06
## 23: 2020-08-17 13:09:08
## 24: 2020-08-17 13:09:08
## 25: 2020-08-17 13:09:08
## 26: 2020-08-17 13:09:08
## 27: 2020-08-17 13:09:08
## 28: 2020-08-17 13:09:08
## 29: 2020-08-17 13:09:08
## 30: 2020-08-17 13:09:08
## 31: 2020-08-17 13:09:08
## 32: 2020-08-17 13:09:08
## 33: 2020-08-17 13:09:08
## 34: 2020-08-17 13:09:08
## 35: 2020-08-17 13:09:08
## 36: 2020-08-17 13:09:08
## 37: 2020-08-17 13:09:08
## 38: 2020-08-17 13:09:08
## 39: 2020-08-17 13:09:08
## 40: 2020-08-17 13:09:08
## 41: 2020-08-17 13:09:08
## 42: 2020-08-17 13:09:08
## 43: 2020-08-17 13:09:08
## 44: 2020-08-17 13:09:08
## 45: 2020-08-17 13:09:08
## 46: 2020-08-17 13:09:08
## 47: 2020-08-17 13:09:08
## createdDate# Keep all Cache items created with an rnorm() call keepCache(tmpDir, userTags = "rnorm", ask = FALSE)
## Cache size:## Total (including Rasters): 252 bytes## Selected objects (not including Rasters): 252 bytes## cacheId tagKey tagValue
## 1: f7bee22047b8d0c1 objectName a
## 2: f7bee22047b8d0c1 function rnorm
## 3: f7bee22047b8d0c1 class numeric
## 4: f7bee22047b8d0c1 object.size 1008
## 5: f7bee22047b8d0c1 accessed 2020-08-17 13:09:06
## 6: f7bee22047b8d0c1 inCloud FALSE
## 7: f7bee22047b8d0c1 resultHash
## 8: f7bee22047b8d0c1 elapsedTimeDigest 0.002508879 secs
## 9: f7bee22047b8d0c1 elapsedTimeFirstRun 0 secs
## 10: f7bee22047b8d0c1 otherFunctions saveRDS
## 11: f7bee22047b8d0c1 otherFunctions do.call
## 12: f7bee22047b8d0c1 otherFunctions process_file
## 13: f7bee22047b8d0c1 otherFunctions process_group
## 14: f7bee22047b8d0c1 otherFunctions process_group.block
## 15: f7bee22047b8d0c1 otherFunctions call_block
## 16: f7bee22047b8d0c1 otherFunctions block_exec
## 17: f7bee22047b8d0c1 otherFunctions in_dir
## 18: f7bee22047b8d0c1 otherFunctions timing_fn
## 19: f7bee22047b8d0c1 otherFunctions handle
## 20: f7bee22047b8d0c1 preDigest n:7eef4eae85fd9229
## 21: f7bee22047b8d0c1 preDigest .FUN:4f604aa46882b368
## 22: f7bee22047b8d0c1 file.size 174
## 23: f7bee22047b8d0c1 accessed 2020-08-17 13:09:08
## 24: f7bee22047b8d0c1 elapsedTimeLoad 0.01203299 secs
## 25: f7bee22047b8d0c1 accessed 2020-08-17 13:09:08
## cacheId tagKey tagValue
## createdDate
## 1: 2020-08-17 13:09:06
## 2: 2020-08-17 13:09:06
## 3: 2020-08-17 13:09:06
## 4: 2020-08-17 13:09:06
## 5: 2020-08-17 13:09:06
## 6: 2020-08-17 13:09:06
## 7: 2020-08-17 13:09:06
## 8: 2020-08-17 13:09:06
## 9: 2020-08-17 13:09:06
## 10: 2020-08-17 13:09:06
## 11: 2020-08-17 13:09:06
## 12: 2020-08-17 13:09:06
## 13: 2020-08-17 13:09:06
## 14: 2020-08-17 13:09:06
## 15: 2020-08-17 13:09:06
## 16: 2020-08-17 13:09:06
## 17: 2020-08-17 13:09:06
## 18: 2020-08-17 13:09:06
## 19: 2020-08-17 13:09:06
## 20: 2020-08-17 13:09:06
## 21: 2020-08-17 13:09:06
## 22: 2020-08-17 13:09:06
## 23: 2020-08-17 13:09:08
## 24: 2020-08-17 13:09:08
## 25: 2020-08-17 13:09:08
## createdDate# Remove all Cache items that happened within a rnorm() call clearCache(tmpDir, userTags = "rnorm", ask = FALSE)
## Cache size:## Total (including Rasters): 252 bytes## Selected objects (not including Rasters): 252 bytesshowCache(tmpDir) ## empty
## Cache size:## Total (including Rasters): 0 bytes## Selected objects (not including Rasters): 0 bytes## Empty data.table (0 rows and 4 cols): cacheId,tagKey,tagValue,createdDate# Also, can set a time before caching happens and remove based on this # --> a useful, simple way to control Cache ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a") startTime <- Sys.time() Sys.sleep(1) ranNumsB <- Cache(rnorm, 5, cacheRepo = tmpDir, userTags = "objectName:b") keepCache(tmpDir, after = startTime, ask = FALSE) # keep only those newer than startTime
## Cache size:## Total (including Rasters): 256 bytes## Selected objects (not including Rasters): 256 bytes## cacheId tagKey tagValue
## 1: 142b4176fe87d51d objectName b
## 2: 142b4176fe87d51d function rnorm
## 3: 142b4176fe87d51d class numeric
## 4: 142b4176fe87d51d object.size 1024
## 5: 142b4176fe87d51d accessed 2020-08-17 13:09:09
## 6: 142b4176fe87d51d inCloud FALSE
## 7: 142b4176fe87d51d resultHash
## 8: 142b4176fe87d51d elapsedTimeDigest 0.003006935 secs
## 9: 142b4176fe87d51d elapsedTimeFirstRun 0 secs
## 10: 142b4176fe87d51d otherFunctions saveRDS
## 11: 142b4176fe87d51d otherFunctions do.call
## 12: 142b4176fe87d51d otherFunctions process_file
## 13: 142b4176fe87d51d otherFunctions process_group
## 14: 142b4176fe87d51d otherFunctions process_group.block
## 15: 142b4176fe87d51d otherFunctions call_block
## 16: 142b4176fe87d51d otherFunctions block_exec
## 17: 142b4176fe87d51d otherFunctions in_dir
## 18: 142b4176fe87d51d otherFunctions timing_fn
## 19: 142b4176fe87d51d otherFunctions handle
## 20: 142b4176fe87d51d preDigest n:a4f076b3db622faf
## 21: 142b4176fe87d51d preDigest .FUN:4f604aa46882b368
## 22: 142b4176fe87d51d file.size 184
## cacheId tagKey tagValue
## createdDate
## 1: 2020-08-17 13:09:09
## 2: 2020-08-17 13:09:09
## 3: 2020-08-17 13:09:09
## 4: 2020-08-17 13:09:09
## 5: 2020-08-17 13:09:09
## 6: 2020-08-17 13:09:09
## 7: 2020-08-17 13:09:09
## 8: 2020-08-17 13:09:09
## 9: 2020-08-17 13:09:09
## 10: 2020-08-17 13:09:09
## 11: 2020-08-17 13:09:09
## 12: 2020-08-17 13:09:09
## 13: 2020-08-17 13:09:09
## 14: 2020-08-17 13:09:09
## 15: 2020-08-17 13:09:09
## 16: 2020-08-17 13:09:09
## 17: 2020-08-17 13:09:09
## 18: 2020-08-17 13:09:09
## 19: 2020-08-17 13:09:09
## 20: 2020-08-17 13:09:09
## 21: 2020-08-17 13:09:09
## 22: 2020-08-17 13:09:09
## createdDateclearCache(tmpDir, ask = FALSE)
Example 4: searching for multiple objects in the cache
# default userTags is "and" matching; for "or" matching use | ranNumsA <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:a") ranNumsB <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:b") # show all objects (runif and rnorm in this case) showCache(tmpDir)
## Cache size:## Total (including Rasters): 504 bytes## Selected objects (not including Rasters): 504 bytes## cacheId tagKey tagValue
## 1: 3aef38d1fc02aee5 objectName a
## 2: 3aef38d1fc02aee5 function runif
## 3: 3aef38d1fc02aee5 class numeric
## 4: 3aef38d1fc02aee5 object.size 1008
## 5: 3aef38d1fc02aee5 accessed 2020-08-17 13:09:10
## 6: 3aef38d1fc02aee5 inCloud FALSE
## 7: 3aef38d1fc02aee5 resultHash
## 8: 3aef38d1fc02aee5 elapsedTimeDigest 0.004007101 secs
## 9: 3aef38d1fc02aee5 elapsedTimeFirstRun 0 secs
## 10: 3aef38d1fc02aee5 otherFunctions saveRDS
## 11: 3aef38d1fc02aee5 otherFunctions do.call
## 12: 3aef38d1fc02aee5 otherFunctions process_file
## 13: 3aef38d1fc02aee5 otherFunctions process_group
## 14: 3aef38d1fc02aee5 otherFunctions process_group.block
## 15: 3aef38d1fc02aee5 otherFunctions call_block
## 16: 3aef38d1fc02aee5 otherFunctions block_exec
## 17: 3aef38d1fc02aee5 otherFunctions in_dir
## 18: 3aef38d1fc02aee5 otherFunctions timing_fn
## 19: 3aef38d1fc02aee5 otherFunctions handle
## 20: 3aef38d1fc02aee5 preDigest n:7eef4eae85fd9229
## 21: 3aef38d1fc02aee5 preDigest .FUN:881ec847b7161f3c
## 22: 3aef38d1fc02aee5 file.size 167
## 23: f7bee22047b8d0c1 objectName b
## 24: f7bee22047b8d0c1 function rnorm
## 25: f7bee22047b8d0c1 class numeric
## 26: f7bee22047b8d0c1 object.size 1008
## 27: f7bee22047b8d0c1 accessed 2020-08-17 13:09:10
## 28: f7bee22047b8d0c1 inCloud FALSE
## 29: f7bee22047b8d0c1 resultHash
## 30: f7bee22047b8d0c1 elapsedTimeDigest 0.002006054 secs
## 31: f7bee22047b8d0c1 elapsedTimeFirstRun 0 secs
## 32: f7bee22047b8d0c1 otherFunctions saveRDS
## 33: f7bee22047b8d0c1 otherFunctions do.call
## 34: f7bee22047b8d0c1 otherFunctions process_file
## 35: f7bee22047b8d0c1 otherFunctions process_group
## 36: f7bee22047b8d0c1 otherFunctions process_group.block
## 37: f7bee22047b8d0c1 otherFunctions call_block
## 38: f7bee22047b8d0c1 otherFunctions block_exec
## 39: f7bee22047b8d0c1 otherFunctions in_dir
## 40: f7bee22047b8d0c1 otherFunctions timing_fn
## 41: f7bee22047b8d0c1 otherFunctions handle
## 42: f7bee22047b8d0c1 preDigest n:7eef4eae85fd9229
## 43: f7bee22047b8d0c1 preDigest .FUN:4f604aa46882b368
## 44: f7bee22047b8d0c1 file.size 174
## cacheId tagKey tagValue
## createdDate
## 1: 2020-08-17 13:09:10
## 2: 2020-08-17 13:09:10
## 3: 2020-08-17 13:09:10
## 4: 2020-08-17 13:09:10
## 5: 2020-08-17 13:09:10
## 6: 2020-08-17 13:09:10
## 7: 2020-08-17 13:09:10
## 8: 2020-08-17 13:09:10
## 9: 2020-08-17 13:09:10
## 10: 2020-08-17 13:09:10
## 11: 2020-08-17 13:09:10
## 12: 2020-08-17 13:09:10
## 13: 2020-08-17 13:09:10
## 14: 2020-08-17 13:09:10
## 15: 2020-08-17 13:09:10
## 16: 2020-08-17 13:09:10
## 17: 2020-08-17 13:09:10
## 18: 2020-08-17 13:09:10
## 19: 2020-08-17 13:09:10
## 20: 2020-08-17 13:09:10
## 21: 2020-08-17 13:09:10
## 22: 2020-08-17 13:09:10
## 23: 2020-08-17 13:09:10
## 24: 2020-08-17 13:09:10
## 25: 2020-08-17 13:09:10
## 26: 2020-08-17 13:09:10
## 27: 2020-08-17 13:09:10
## 28: 2020-08-17 13:09:10
## 29: 2020-08-17 13:09:10
## 30: 2020-08-17 13:09:10
## 31: 2020-08-17 13:09:10
## 32: 2020-08-17 13:09:10
## 33: 2020-08-17 13:09:10
## 34: 2020-08-17 13:09:10
## 35: 2020-08-17 13:09:10
## 36: 2020-08-17 13:09:10
## 37: 2020-08-17 13:09:10
## 38: 2020-08-17 13:09:10
## 39: 2020-08-17 13:09:10
## 40: 2020-08-17 13:09:10
## 41: 2020-08-17 13:09:10
## 42: 2020-08-17 13:09:10
## 43: 2020-08-17 13:09:10
## 44: 2020-08-17 13:09:10
## createdDate# show objects that are both runif and rnorm # (i.e., none in this case, because objecs are either or, not both) showCache(tmpDir, userTags = c("runif", "rnorm")) ## empty
## Cache size:## Total (including Rasters): 0 bytes## Selected objects (not including Rasters): 0 bytes## Empty data.table (0 rows and 4 cols): cacheId,tagKey,tagValue,createdDate# show objects that are either runif or rnorm ("or" search) showCache(tmpDir, userTags = "runif|rnorm")
## Cache size:## Total (including Rasters): 504 bytes## Selected objects (not including Rasters): 504 bytes## cacheId tagKey tagValue
## 1: 3aef38d1fc02aee5 objectName a
## 2: 3aef38d1fc02aee5 function runif
## 3: 3aef38d1fc02aee5 class numeric
## 4: 3aef38d1fc02aee5 object.size 1008
## 5: 3aef38d1fc02aee5 accessed 2020-08-17 13:09:10
## 6: 3aef38d1fc02aee5 inCloud FALSE
## 7: 3aef38d1fc02aee5 resultHash
## 8: 3aef38d1fc02aee5 elapsedTimeDigest 0.004007101 secs
## 9: 3aef38d1fc02aee5 elapsedTimeFirstRun 0 secs
## 10: 3aef38d1fc02aee5 otherFunctions saveRDS
## 11: 3aef38d1fc02aee5 otherFunctions do.call
## 12: 3aef38d1fc02aee5 otherFunctions process_file
## 13: 3aef38d1fc02aee5 otherFunctions process_group
## 14: 3aef38d1fc02aee5 otherFunctions process_group.block
## 15: 3aef38d1fc02aee5 otherFunctions call_block
## 16: 3aef38d1fc02aee5 otherFunctions block_exec
## 17: 3aef38d1fc02aee5 otherFunctions in_dir
## 18: 3aef38d1fc02aee5 otherFunctions timing_fn
## 19: 3aef38d1fc02aee5 otherFunctions handle
## 20: 3aef38d1fc02aee5 preDigest n:7eef4eae85fd9229
## 21: 3aef38d1fc02aee5 preDigest .FUN:881ec847b7161f3c
## 22: 3aef38d1fc02aee5 file.size 167
## 23: f7bee22047b8d0c1 objectName b
## 24: f7bee22047b8d0c1 function rnorm
## 25: f7bee22047b8d0c1 class numeric
## 26: f7bee22047b8d0c1 object.size 1008
## 27: f7bee22047b8d0c1 accessed 2020-08-17 13:09:10
## 28: f7bee22047b8d0c1 inCloud FALSE
## 29: f7bee22047b8d0c1 resultHash
## 30: f7bee22047b8d0c1 elapsedTimeDigest 0.002006054 secs
## 31: f7bee22047b8d0c1 elapsedTimeFirstRun 0 secs
## 32: f7bee22047b8d0c1 otherFunctions saveRDS
## 33: f7bee22047b8d0c1 otherFunctions do.call
## 34: f7bee22047b8d0c1 otherFunctions process_file
## 35: f7bee22047b8d0c1 otherFunctions process_group
## 36: f7bee22047b8d0c1 otherFunctions process_group.block
## 37: f7bee22047b8d0c1 otherFunctions call_block
## 38: f7bee22047b8d0c1 otherFunctions block_exec
## 39: f7bee22047b8d0c1 otherFunctions in_dir
## 40: f7bee22047b8d0c1 otherFunctions timing_fn
## 41: f7bee22047b8d0c1 otherFunctions handle
## 42: f7bee22047b8d0c1 preDigest n:7eef4eae85fd9229
## 43: f7bee22047b8d0c1 preDigest .FUN:4f604aa46882b368
## 44: f7bee22047b8d0c1 file.size 174
## cacheId tagKey tagValue
## createdDate
## 1: 2020-08-17 13:09:10
## 2: 2020-08-17 13:09:10
## 3: 2020-08-17 13:09:10
## 4: 2020-08-17 13:09:10
## 5: 2020-08-17 13:09:10
## 6: 2020-08-17 13:09:10
## 7: 2020-08-17 13:09:10
## 8: 2020-08-17 13:09:10
## 9: 2020-08-17 13:09:10
## 10: 2020-08-17 13:09:10
## 11: 2020-08-17 13:09:10
## 12: 2020-08-17 13:09:10
## 13: 2020-08-17 13:09:10
## 14: 2020-08-17 13:09:10
## 15: 2020-08-17 13:09:10
## 16: 2020-08-17 13:09:10
## 17: 2020-08-17 13:09:10
## 18: 2020-08-17 13:09:10
## 19: 2020-08-17 13:09:10
## 20: 2020-08-17 13:09:10
## 21: 2020-08-17 13:09:10
## 22: 2020-08-17 13:09:10
## 23: 2020-08-17 13:09:10
## 24: 2020-08-17 13:09:10
## 25: 2020-08-17 13:09:10
## 26: 2020-08-17 13:09:10
## 27: 2020-08-17 13:09:10
## 28: 2020-08-17 13:09:10
## 29: 2020-08-17 13:09:10
## 30: 2020-08-17 13:09:10
## 31: 2020-08-17 13:09:10
## 32: 2020-08-17 13:09:10
## 33: 2020-08-17 13:09:10
## 34: 2020-08-17 13:09:10
## 35: 2020-08-17 13:09:10
## 36: 2020-08-17 13:09:10
## 37: 2020-08-17 13:09:10
## 38: 2020-08-17 13:09:10
## 39: 2020-08-17 13:09:10
## 40: 2020-08-17 13:09:10
## 41: 2020-08-17 13:09:10
## 42: 2020-08-17 13:09:10
## 43: 2020-08-17 13:09:10
## 44: 2020-08-17 13:09:10
## createdDate# keep only objects that are either runif or rnorm ("or" search) keepCache(tmpDir, userTags = "runif|rnorm", ask = FALSE)
## Cache size:## Total (including Rasters): 504 bytes## Selected objects (not including Rasters): 504 bytes## cacheId tagKey tagValue
## 1: 3aef38d1fc02aee5 objectName a
## 2: 3aef38d1fc02aee5 function runif
## 3: 3aef38d1fc02aee5 class numeric
## 4: 3aef38d1fc02aee5 object.size 1008
## 5: 3aef38d1fc02aee5 accessed 2020-08-17 13:09:10
## 6: 3aef38d1fc02aee5 inCloud FALSE
## 7: 3aef38d1fc02aee5 resultHash
## 8: 3aef38d1fc02aee5 elapsedTimeDigest 0.004007101 secs
## 9: 3aef38d1fc02aee5 elapsedTimeFirstRun 0 secs
## 10: 3aef38d1fc02aee5 otherFunctions saveRDS
## 11: 3aef38d1fc02aee5 otherFunctions do.call
## 12: 3aef38d1fc02aee5 otherFunctions process_file
## 13: 3aef38d1fc02aee5 otherFunctions process_group
## 14: 3aef38d1fc02aee5 otherFunctions process_group.block
## 15: 3aef38d1fc02aee5 otherFunctions call_block
## 16: 3aef38d1fc02aee5 otherFunctions block_exec
## 17: 3aef38d1fc02aee5 otherFunctions in_dir
## 18: 3aef38d1fc02aee5 otherFunctions timing_fn
## 19: 3aef38d1fc02aee5 otherFunctions handle
## 20: 3aef38d1fc02aee5 preDigest n:7eef4eae85fd9229
## 21: 3aef38d1fc02aee5 preDigest .FUN:881ec847b7161f3c
## 22: 3aef38d1fc02aee5 file.size 167
## 23: f7bee22047b8d0c1 objectName b
## 24: f7bee22047b8d0c1 function rnorm
## 25: f7bee22047b8d0c1 class numeric
## 26: f7bee22047b8d0c1 object.size 1008
## 27: f7bee22047b8d0c1 accessed 2020-08-17 13:09:10
## 28: f7bee22047b8d0c1 inCloud FALSE
## 29: f7bee22047b8d0c1 resultHash
## 30: f7bee22047b8d0c1 elapsedTimeDigest 0.002006054 secs
## 31: f7bee22047b8d0c1 elapsedTimeFirstRun 0 secs
## 32: f7bee22047b8d0c1 otherFunctions saveRDS
## 33: f7bee22047b8d0c1 otherFunctions do.call
## 34: f7bee22047b8d0c1 otherFunctions process_file
## 35: f7bee22047b8d0c1 otherFunctions process_group
## 36: f7bee22047b8d0c1 otherFunctions process_group.block
## 37: f7bee22047b8d0c1 otherFunctions call_block
## 38: f7bee22047b8d0c1 otherFunctions block_exec
## 39: f7bee22047b8d0c1 otherFunctions in_dir
## 40: f7bee22047b8d0c1 otherFunctions timing_fn
## 41: f7bee22047b8d0c1 otherFunctions handle
## 42: f7bee22047b8d0c1 preDigest n:7eef4eae85fd9229
## 43: f7bee22047b8d0c1 preDigest .FUN:4f604aa46882b368
## 44: f7bee22047b8d0c1 file.size 174
## cacheId tagKey tagValue
## createdDate
## 1: 2020-08-17 13:09:10
## 2: 2020-08-17 13:09:10
## 3: 2020-08-17 13:09:10
## 4: 2020-08-17 13:09:10
## 5: 2020-08-17 13:09:10
## 6: 2020-08-17 13:09:10
## 7: 2020-08-17 13:09:10
## 8: 2020-08-17 13:09:10
## 9: 2020-08-17 13:09:10
## 10: 2020-08-17 13:09:10
## 11: 2020-08-17 13:09:10
## 12: 2020-08-17 13:09:10
## 13: 2020-08-17 13:09:10
## 14: 2020-08-17 13:09:10
## 15: 2020-08-17 13:09:10
## 16: 2020-08-17 13:09:10
## 17: 2020-08-17 13:09:10
## 18: 2020-08-17 13:09:10
## 19: 2020-08-17 13:09:10
## 20: 2020-08-17 13:09:10
## 21: 2020-08-17 13:09:10
## 22: 2020-08-17 13:09:10
## 23: 2020-08-17 13:09:10
## 24: 2020-08-17 13:09:10
## 25: 2020-08-17 13:09:10
## 26: 2020-08-17 13:09:10
## 27: 2020-08-17 13:09:10
## 28: 2020-08-17 13:09:10
## 29: 2020-08-17 13:09:10
## 30: 2020-08-17 13:09:10
## 31: 2020-08-17 13:09:10
## 32: 2020-08-17 13:09:10
## 33: 2020-08-17 13:09:10
## 34: 2020-08-17 13:09:10
## 35: 2020-08-17 13:09:10
## 36: 2020-08-17 13:09:10
## 37: 2020-08-17 13:09:10
## 38: 2020-08-17 13:09:10
## 39: 2020-08-17 13:09:10
## 40: 2020-08-17 13:09:10
## 41: 2020-08-17 13:09:10
## 42: 2020-08-17 13:09:10
## 43: 2020-08-17 13:09:10
## 44: 2020-08-17 13:09:10
## createdDateclearCache(tmpDir, ask = FALSE)
Example 5: using caching to speed up rerunning expensive computations
ras <- raster(extent(0, 5, 0, 5), res = 1, vals = sample(1:5, replace = TRUE, size = 25), crs = "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84") # A slow operation, like GIS operation notCached <- suppressWarnings( # project raster generates warnings when run non-interactively projectRaster(ras, crs = crs(ras), res = 5, cacheRepo = tmpDir) ) cached <- suppressWarnings( # project raster generates warnings when run non-interactively # using quote works also Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir) ) # second time is much faster reRun <- suppressWarnings( # project raster generates warnings when run non-interactively Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir) )
## ...(Object to retrieve (7aa653e36bb42d74.rds))## loaded cached result from previous projectRaster call,# recovered cached version is same as non-cached version all.equal(notCached, reRun) ## TRUE
## [1] TRUENested Caching
Nested caching, which is when Caching of a function occurs inside an outer function, which is itself cached. This is a critical element to working within a reproducible work flow. It is not enough during development to cache flat code chunks, as there will be many levels of “slow” functions. Ideally, at all points in a development cycle, it should be possible to get to any line of code starting from the very initial steps, running through everything up to that point, in less than a few seconds. If the workflow can be kept very fast like this, then there is a guarantee that it will work at any point.
########################## ## Nested Caching # Make 2 functions inner <- function(mean) { d <- 1 Cache(rnorm, n = 3, mean = mean) } outer <- function(n) { Cache(inner, 0.1, cacheRepo = tmpdir2) } # make 2 different cache paths tmpdir1 <- file.path(tempdir(), "first") tmpdir2 <- file.path(tempdir(), "second") # Run the Cache ... notOlderThan propagates to all 3 Cache calls, # but cacheRepo is tmpdir1 in top level Cache and all nested # Cache calls, unless individually overridden ... here inner # uses tmpdir2 repository Cache(outer, n = 2, cacheRepo = tmpdir1, notOlderThan = Sys.time())
## [1] -1.0656387 -1.7851212 0.6830708
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
##
## attr(,"tags")
## [1] "cacheId:a7af5367c13aba8f"
## attr(,"call")
## [1] ""showCache(tmpdir1) # 2 function calls
## Cache size:## Total (including Rasters): 504 bytes## Selected objects (not including Rasters): 504 bytes## cacheId tagKey tagValue
## 1: 4ac2b7c0f42d1e46 function rnorm
## 2: 4ac2b7c0f42d1e46 class numeric
## 3: 4ac2b7c0f42d1e46 object.size 1008
## 4: 4ac2b7c0f42d1e46 accessed 2020-08-17 13:09:12
## 5: 4ac2b7c0f42d1e46 inCloud FALSE
## 6: 4ac2b7c0f42d1e46 resultHash
## 7: 4ac2b7c0f42d1e46 elapsedTimeDigest 0.003006935 secs
## 8: 4ac2b7c0f42d1e46 elapsedTimeFirstRun 0 secs
## 9: 4ac2b7c0f42d1e46 otherFunctions saveRDS
## 10: 4ac2b7c0f42d1e46 otherFunctions do.call
## 11: 4ac2b7c0f42d1e46 otherFunctions process_file
## 12: 4ac2b7c0f42d1e46 otherFunctions process_group
## 13: 4ac2b7c0f42d1e46 otherFunctions process_group.block
## 14: 4ac2b7c0f42d1e46 otherFunctions call_block
## 15: 4ac2b7c0f42d1e46 otherFunctions block_exec
## 16: 4ac2b7c0f42d1e46 otherFunctions in_dir
## 17: 4ac2b7c0f42d1e46 otherFunctions timing_fn
## 18: 4ac2b7c0f42d1e46 otherFunctions handle
## 19: 4ac2b7c0f42d1e46 preDigest n:7f12988bd88a0fb8
## 20: 4ac2b7c0f42d1e46 preDigest mean:22413394efd9f6a3
## 21: 4ac2b7c0f42d1e46 preDigest .FUN:4f604aa46882b368
## 22: 4ac2b7c0f42d1e46 file.size 166
## 23: a7af5367c13aba8f function outer
## 24: a7af5367c13aba8f class numeric
## 25: a7af5367c13aba8f object.size 1008
## 26: a7af5367c13aba8f accessed 2020-08-17 13:09:12
## 27: a7af5367c13aba8f inCloud FALSE
## 28: a7af5367c13aba8f resultHash
## 29: a7af5367c13aba8f elapsedTimeDigest 0.003009081 secs
## 30: a7af5367c13aba8f elapsedTimeFirstRun 0.2862711 secs
## 31: a7af5367c13aba8f otherFunctions saveRDS
## 32: a7af5367c13aba8f otherFunctions do.call
## 33: a7af5367c13aba8f otherFunctions process_file
## 34: a7af5367c13aba8f otherFunctions process_group
## 35: a7af5367c13aba8f otherFunctions process_group.block
## 36: a7af5367c13aba8f otherFunctions call_block
## 37: a7af5367c13aba8f otherFunctions block_exec
## 38: a7af5367c13aba8f otherFunctions in_dir
## 39: a7af5367c13aba8f otherFunctions timing_fn
## 40: a7af5367c13aba8f otherFunctions handle
## 41: a7af5367c13aba8f preDigest n:82dc709f2b91918a
## 42: a7af5367c13aba8f preDigest .FUN:892a6afc47a63a90
## 43: a7af5367c13aba8f file.size 166
## cacheId tagKey tagValue
## createdDate
## 1: 2020-08-17 13:09:12
## 2: 2020-08-17 13:09:12
## 3: 2020-08-17 13:09:12
## 4: 2020-08-17 13:09:12
## 5: 2020-08-17 13:09:12
## 6: 2020-08-17 13:09:12
## 7: 2020-08-17 13:09:12
## 8: 2020-08-17 13:09:12
## 9: 2020-08-17 13:09:12
## 10: 2020-08-17 13:09:12
## 11: 2020-08-17 13:09:12
## 12: 2020-08-17 13:09:12
## 13: 2020-08-17 13:09:12
## 14: 2020-08-17 13:09:12
## 15: 2020-08-17 13:09:12
## 16: 2020-08-17 13:09:12
## 17: 2020-08-17 13:09:12
## 18: 2020-08-17 13:09:12
## 19: 2020-08-17 13:09:12
## 20: 2020-08-17 13:09:12
## 21: 2020-08-17 13:09:12
## 22: 2020-08-17 13:09:12
## 23: 2020-08-17 13:09:12
## 24: 2020-08-17 13:09:12
## 25: 2020-08-17 13:09:12
## 26: 2020-08-17 13:09:12
## 27: 2020-08-17 13:09:12
## 28: 2020-08-17 13:09:12
## 29: 2020-08-17 13:09:12
## 30: 2020-08-17 13:09:12
## 31: 2020-08-17 13:09:12
## 32: 2020-08-17 13:09:12
## 33: 2020-08-17 13:09:12
## 34: 2020-08-17 13:09:12
## 35: 2020-08-17 13:09:12
## 36: 2020-08-17 13:09:12
## 37: 2020-08-17 13:09:12
## 38: 2020-08-17 13:09:12
## 39: 2020-08-17 13:09:12
## 40: 2020-08-17 13:09:12
## 41: 2020-08-17 13:09:12
## 42: 2020-08-17 13:09:12
## 43: 2020-08-17 13:09:12
## createdDateshowCache(tmpdir2) # 1 function call
## Cache size:## Total (including Rasters): 252 bytes## Selected objects (not including Rasters): 252 bytes## cacheId tagKey tagValue
## 1: 33ceb4fb525fd08f function inner
## 2: 33ceb4fb525fd08f class numeric
## 3: 33ceb4fb525fd08f object.size 1008
## 4: 33ceb4fb525fd08f accessed 2020-08-17 13:09:12
## 5: 33ceb4fb525fd08f inCloud FALSE
## 6: 33ceb4fb525fd08f resultHash
## 7: 33ceb4fb525fd08f elapsedTimeDigest 0.003006935 secs
## 8: 33ceb4fb525fd08f elapsedTimeFirstRun 0.1178169 secs
## 9: 33ceb4fb525fd08f otherFunctions saveRDS
## 10: 33ceb4fb525fd08f otherFunctions do.call
## 11: 33ceb4fb525fd08f otherFunctions process_file
## 12: 33ceb4fb525fd08f otherFunctions process_group
## 13: 33ceb4fb525fd08f otherFunctions process_group.block
## 14: 33ceb4fb525fd08f otherFunctions call_block
## 15: 33ceb4fb525fd08f otherFunctions block_exec
## 16: 33ceb4fb525fd08f otherFunctions in_dir
## 17: 33ceb4fb525fd08f otherFunctions timing_fn
## 18: 33ceb4fb525fd08f otherFunctions handle
## 19: 33ceb4fb525fd08f preDigest mean:22413394efd9f6a3
## 20: 33ceb4fb525fd08f preDigest .FUN:87e2c30917a34d25
## 21: 33ceb4fb525fd08f file.size 166
## cacheId tagKey tagValue
## createdDate
## 1: 2020-08-17 13:09:12
## 2: 2020-08-17 13:09:12
## 3: 2020-08-17 13:09:12
## 4: 2020-08-17 13:09:12
## 5: 2020-08-17 13:09:12
## 6: 2020-08-17 13:09:12
## 7: 2020-08-17 13:09:12
## 8: 2020-08-17 13:09:12
## 9: 2020-08-17 13:09:12
## 10: 2020-08-17 13:09:12
## 11: 2020-08-17 13:09:12
## 12: 2020-08-17 13:09:12
## 13: 2020-08-17 13:09:12
## 14: 2020-08-17 13:09:12
## 15: 2020-08-17 13:09:12
## 16: 2020-08-17 13:09:12
## 17: 2020-08-17 13:09:12
## 18: 2020-08-17 13:09:12
## 19: 2020-08-17 13:09:12
## 20: 2020-08-17 13:09:12
## 21: 2020-08-17 13:09:12
## createdDate# userTags get appended # all items have the outer tag propagate, plus inner ones only have inner ones clearCache(tmpdir1, ask = FALSE) outerTag <- "outerTag" innerTag <- "innerTag" inner <- function(mean) { d <- 1 Cache(rnorm, n = 3, mean = mean, notOlderThan = Sys.time() - 1e5, userTags = innerTag) } outer <- function(n) { Cache(inner, 0.1) } aa <- Cache(outer, n = 2, cacheRepo = tmpdir1, userTags = outerTag) showCache(tmpdir1) # rnorm function has outerTag and innerTag, inner and outer only have outerTag
## Cache size:## Total (including Rasters): 756 bytes## Selected objects (not including Rasters): 756 bytes## cacheId tagKey tagValue
## 1: 4ac2b7c0f42d1e46 innerTag innerTag
## 2: 4ac2b7c0f42d1e46 outerTag outerTag
## 3: 4ac2b7c0f42d1e46 function rnorm
## 4: 4ac2b7c0f42d1e46 class numeric
## 5: 4ac2b7c0f42d1e46 object.size 1008
## 6: 4ac2b7c0f42d1e46 accessed 2020-08-17 13:09:13
## 7: 4ac2b7c0f42d1e46 inCloud FALSE
## 8: 4ac2b7c0f42d1e46 resultHash
## 9: 4ac2b7c0f42d1e46 elapsedTimeDigest 0.005012989 secs
## 10: 4ac2b7c0f42d1e46 elapsedTimeFirstRun 0 secs
## 11: 4ac2b7c0f42d1e46 otherFunctions saveRDS
## 12: 4ac2b7c0f42d1e46 otherFunctions do.call
## 13: 4ac2b7c0f42d1e46 otherFunctions process_file
## 14: 4ac2b7c0f42d1e46 otherFunctions process_group
## 15: 4ac2b7c0f42d1e46 otherFunctions process_group.block
## 16: 4ac2b7c0f42d1e46 otherFunctions call_block
## 17: 4ac2b7c0f42d1e46 otherFunctions block_exec
## 18: 4ac2b7c0f42d1e46 otherFunctions in_dir
## 19: 4ac2b7c0f42d1e46 otherFunctions timing_fn
## 20: 4ac2b7c0f42d1e46 otherFunctions handle
## 21: 4ac2b7c0f42d1e46 preDigest n:7f12988bd88a0fb8
## 22: 4ac2b7c0f42d1e46 preDigest mean:22413394efd9f6a3
## 23: 4ac2b7c0f42d1e46 preDigest .FUN:4f604aa46882b368
## 24: 4ac2b7c0f42d1e46 file.size 166
## 25: b06af03d5a73dc7d outerTag outerTag
## 26: b06af03d5a73dc7d function inner
## 27: b06af03d5a73dc7d class numeric
## 28: b06af03d5a73dc7d object.size 1008
## 29: b06af03d5a73dc7d accessed 2020-08-17 13:09:13
## 30: b06af03d5a73dc7d inCloud FALSE
## 31: b06af03d5a73dc7d resultHash
## 32: b06af03d5a73dc7d elapsedTimeDigest 0.006016016 secs
## 33: b06af03d5a73dc7d elapsedTimeFirstRun 0.1183181 secs
## 34: b06af03d5a73dc7d otherFunctions saveRDS
## 35: b06af03d5a73dc7d otherFunctions do.call
## 36: b06af03d5a73dc7d otherFunctions process_file
## 37: b06af03d5a73dc7d otherFunctions process_group
## 38: b06af03d5a73dc7d otherFunctions process_group.block
## 39: b06af03d5a73dc7d otherFunctions call_block
## 40: b06af03d5a73dc7d otherFunctions block_exec
## 41: b06af03d5a73dc7d otherFunctions in_dir
## 42: b06af03d5a73dc7d otherFunctions timing_fn
## 43: b06af03d5a73dc7d otherFunctions handle
## 44: b06af03d5a73dc7d preDigest mean:22413394efd9f6a3
## 45: b06af03d5a73dc7d preDigest .FUN:7ad10bc1ae528d8c
## 46: b06af03d5a73dc7d file.size 166
## 47: 88a34e1d033329e5 outerTag outerTag
## 48: 88a34e1d033329e5 function outer
## 49: 88a34e1d033329e5 class numeric
## 50: 88a34e1d033329e5 object.size 1008
## 51: 88a34e1d033329e5 accessed 2020-08-17 13:09:13
## 52: 88a34e1d033329e5 inCloud FALSE
## 53: 88a34e1d033329e5 resultHash
## 54: 88a34e1d033329e5 elapsedTimeDigest 0.006518126 secs
## 55: 88a34e1d033329e5 elapsedTimeFirstRun 0.2205951 secs
## 56: 88a34e1d033329e5 otherFunctions saveRDS
## 57: 88a34e1d033329e5 otherFunctions do.call
## 58: 88a34e1d033329e5 otherFunctions process_file
## 59: 88a34e1d033329e5 otherFunctions process_group
## 60: 88a34e1d033329e5 otherFunctions process_group.block
## 61: 88a34e1d033329e5 otherFunctions call_block
## 62: 88a34e1d033329e5 otherFunctions block_exec
## 63: 88a34e1d033329e5 otherFunctions in_dir
## 64: 88a34e1d033329e5 otherFunctions timing_fn
## 65: 88a34e1d033329e5 otherFunctions handle
## 66: 88a34e1d033329e5 preDigest n:82dc709f2b91918a
## 67: 88a34e1d033329e5 preDigest .FUN:5f06fb5fbffe9e3b
## 68: 88a34e1d033329e5 file.size 166
## cacheId tagKey tagValue
## createdDate
## 1: 2020-08-17 13:09:13
## 2: 2020-08-17 13:09:13
## 3: 2020-08-17 13:09:13
## 4: 2020-08-17 13:09:13
## 5: 2020-08-17 13:09:13
## 6: 2020-08-17 13:09:13
## 7: 2020-08-17 13:09:13
## 8: 2020-08-17 13:09:13
## 9: 2020-08-17 13:09:13
## 10: 2020-08-17 13:09:13
## 11: 2020-08-17 13:09:13
## 12: 2020-08-17 13:09:13
## 13: 2020-08-17 13:09:13
## 14: 2020-08-17 13:09:13
## 15: 2020-08-17 13:09:13
## 16: 2020-08-17 13:09:13
## 17: 2020-08-17 13:09:13
## 18: 2020-08-17 13:09:13
## 19: 2020-08-17 13:09:13
## 20: 2020-08-17 13:09:13
## 21: 2020-08-17 13:09:13
## 22: 2020-08-17 13:09:13
## 23: 2020-08-17 13:09:13
## 24: 2020-08-17 13:09:13
## 25: 2020-08-17 13:09:13
## 26: 2020-08-17 13:09:13
## 27: 2020-08-17 13:09:13
## 28: 2020-08-17 13:09:13
## 29: 2020-08-17 13:09:13
## 30: 2020-08-17 13:09:13
## 31: 2020-08-17 13:09:13
## 32: 2020-08-17 13:09:13
## 33: 2020-08-17 13:09:13
## 34: 2020-08-17 13:09:13
## 35: 2020-08-17 13:09:13
## 36: 2020-08-17 13:09:13
## 37: 2020-08-17 13:09:13
## 38: 2020-08-17 13:09:13
## 39: 2020-08-17 13:09:13
## 40: 2020-08-17 13:09:13
## 41: 2020-08-17 13:09:13
## 42: 2020-08-17 13:09:13
## 43: 2020-08-17 13:09:13
## 44: 2020-08-17 13:09:13
## 45: 2020-08-17 13:09:13
## 46: 2020-08-17 13:09:13
## 47: 2020-08-17 13:09:13
## 48: 2020-08-17 13:09:13
## 49: 2020-08-17 13:09:13
## 50: 2020-08-17 13:09:13
## 51: 2020-08-17 13:09:13
## 52: 2020-08-17 13:09:13
## 53: 2020-08-17 13:09:13
## 54: 2020-08-17 13:09:13
## 55: 2020-08-17 13:09:13
## 56: 2020-08-17 13:09:13
## 57: 2020-08-17 13:09:13
## 58: 2020-08-17 13:09:13
## 59: 2020-08-17 13:09:13
## 60: 2020-08-17 13:09:13
## 61: 2020-08-17 13:09:13
## 62: 2020-08-17 13:09:13
## 63: 2020-08-17 13:09:13
## 64: 2020-08-17 13:09:13
## 65: 2020-08-17 13:09:13
## 66: 2020-08-17 13:09:13
## 67: 2020-08-17 13:09:13
## 68: 2020-08-17 13:09:13
## createdDatecacheId
Sometimes, it is not absolutely desirable to maintain the work flow intact because changes that are irrelevant to the analysis, such as changing messages sent to a user, may be changed, without a desire to rerun functions. The cacheId argument is for this. Once a piece of code is run, then the cacheId can be manually extracted (it is reported at the end of a Cache call) and manually placed in the code, passed in as, say, cacheId = "ad184ce64541972b50afd8e7b75f821b".
## [1] -0.6264538
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
##
## attr(,"tags")
## [1] "cacheId:7072c305d8c69df0"
## attr(,"call")
## [1] ""# manually look at output attribute which shows cacheId: 7072c305d8c69df0 Cache(rnorm, 1, cacheRepo = tmpdir1, cacheId = "7072c305d8c69df0") # same value
## cacheId is same as calculated hash## ...(Object to retrieve (7072c305d8c69df0.rds))## loaded cached result from previous rnorm call,## [1] -0.6264538
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] FALSE
##
## attr(,"tags")
## [1] "cacheId:7072c305d8c69df0"
## attr(,"call")
## [1] ""# override even with different inputs: Cache(rnorm, 2, cacheRepo = tmpdir1, cacheId = "7072c305d8c69df0")
## cacheId is not same as calculated hash. Manually searching for cacheId:7072c305d8c69df0## ...(Object to retrieve (7072c305d8c69df0.rds))## loaded cached result from previous rnorm call,## [1] -0.6264538
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] FALSE
##
## attr(,"tags")
## [1] "cacheId:7072c305d8c69df0"
## attr(,"call")
## [1] ""Working with the Cache manually
Since the cache is simply a DBI data table (of an SQLite database by default). In addition, there are several helpers in the reproducible package, including showCache, keepCache and clearCache that may be useful. Also, one can access cached items manually (rather than simply rerunning the same Cache function again).
# As of reproducible version 1.0, there is a new backend directly using DBI mapHash <- unique(showCache(tmpDir, userTags = "projectRaster")$cacheId)
## Cache size:## Total (including Rasters): 3.8 Kb## Selected objects (not including Rasters): 3.8 Kbmap <- loadFromCache(mapHash[1], cachePath = tmpDir) plot(map)
