prospectorQualtrics Dashboard

This is the dashboard for the Qualtrics Prospector.

When to run it:

While the schedule says, “if I am alive, I will run a check for new responses every X minutes”, the actual execution of the job is determined by SCRONTAB. This instead says, “wake up this process every Y time of day by running this script.”

To add this to SCRONTAB, paste the code generated by cronR to your SCRONTAB file:

library(cronR)
library(here) # makes it easier to deal with path issues
here() starts at /n/holylabs/cgolden_lab/Lab/frontier/works/prospectors/prospectorQualtrics
cmd <- cron_rscript(
  here("run.R"), 
  workdir = here(),
  rscript_log = here("pipelines/logs/orchestrator.log"),
  log_append = TRUE,
  log_timestamp = TRUE
)
cron_cmd <- cron_add(command = cmd, frequency = 'daily', at = "09:00", id = 'prospectorQualtrics', dry_run=TRUE, ask=FALSE)
Adding cronjob:
---------------

## cronR job
## id:   prospectorQualtrics
## tags: 
## desc: 
0 9 * * * cd '/n/holylabs/cgolden_lab/Lab/frontier/works/prospectors/prospectorQualtrics' && /n/home03/ttapera/software/spack/opt/spack/linux-icelake/r-4.5.1-munmhwpvanxog4vieqta3cvvdaarsali/rlib/R/bin/Rscript '/n/holylabs/cgolden_lab/Lab/frontier/works/prospectors/prospectorQualtrics/run.R'  >> '/n/holylabs/cgolden_lab/Lab/frontier/works/prospectors/prospectorQualtrics/pipelines/logs/orchestrator.log' 2>&1

With some SCRONTAB config we’ll need:

library(glue)
mail_user <- Sys.getenv("SLACK_EMAIL")
git_cmd <- glue('git diff --quiet || (git commit -a -m "Maestro run" && git push)')
glue(
'#SCRON -J Maestro-prospectorQualtrics', 
'#SCRON -p hsph', 
'#SCRON --mem=1G', 
'#SCRON -t 00:10:00', 
'#SCRON -o {here("pipelines/logs/cron.out")}', 
'#SCRON -e {here("pipelines/logs/cron.err")}', 
'#SCRON --open-mode=append',
'#SCRON --mail-type=FAIL,END', 
'#SCRON --mail-user={mail_user}', 
'{cron_cmd$min} {cron_cmd$hour} {cron_cmd$day_of_month} {cron_cmd$month} {cron_cmd$day_of_week} {cron_cmd$command} && {git_cmd}',
.sep = "\n")
#SCRON -J Maestro-prospectorQualtrics
#SCRON -p hsph
#SCRON --mem=1G
#SCRON -t 00:10:00
#SCRON -o /n/holylabs/cgolden_lab/Lab/frontier/works/prospectors/prospectorQualtrics/pipelines/logs/cron.out
#SCRON -e /n/holylabs/cgolden_lab/Lab/frontier/works/prospectors/prospectorQualtrics/pipelines/logs/cron.err
#SCRON --open-mode=append
#SCRON --mail-type=FAIL,END
#SCRON --mail-user=
0 9 * * * cd '/n/holylabs/cgolden_lab/Lab/frontier/works/prospectors/prospectorQualtrics' && /n/home03/ttapera/software/spack/opt/spack/linux-icelake/r-4.5.1-munmhwpvanxog4vieqta3cvvdaarsali/rlib/R/bin/Rscript '/n/holylabs/cgolden_lab/Lab/frontier/works/prospectors/prospectorQualtrics/run.R'  >> '/n/holylabs/cgolden_lab/Lab/frontier/works/prospectors/prospectorQualtrics/pipelines/logs/orchestrator.log' 2>&1 && git diff --quiet || (git commit -a -m "Maestro run" && git push)

Run The Pipeline

The pipeline only needs the following:

library(maestro)

schedule <- build_schedule()
ℹ 1 script successfully parsed
run_schedule(
  schedule,
  orch_frequency = "1 day",
  log_to_file = here("pipelines/logs/maestro.log")
)

── [2026-07-20 22:45:39]
Running pipelines ▶ 
ℹ process_data_pipeline
[1] "Running the maestro pipeline..."
Test passed with 3 successes 🌈.

  |                                                                            
  |                                                                      |   0%
  |                                                                            
  |======================================================================| 100%
Test passed with 4 successes 🎉.
Test passed with 2 successes 😸.
Test passed with 1 success 😸.
✔ process_data_pipeline [14.4s]


── [2026-07-20 22:45:39]
Pipeline execution completed ■ | 14.44 sec elapsed 
✔ 1 success | ! 0 warnings | ✖ 0 errors | ◼ 1 total
────────────────────────────────────────────────────────────────────────────────

── Next scheduled pipelines ❯ 
Pipe name | Next scheduled run
• process_data_pipeline | 2026-07-22

── Maestro Schedule with 1 pipeline:  
• Success
glue("Last rendered: {Sys.time()}")
Last rendered: 2026-07-20 18:45:53.721078
sessionInfo()
R version 4.5.1 (2025-06-13)
Platform: x86_64-pc-linux-gnu
Running under: Rocky Linux 8.10 (Green Obsidian)

Matrix products: default
BLAS/LAPACK: /n/home03/ttapera/software/spack/opt/spack/linux-icelake/openblas-0.3.33-n6tzifhikuaruph67cuwjcetvrnfmm2t/lib/libopenblas-r0.3.33.so;  LAPACK version 3.12.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

time zone: America/New_York
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] logger_0.4.2        testthat_3.3.2      googlesheets4_1.1.2
 [4] googledrive_2.1.2   qualtRics_3.2.2     tidyr_1.3.2        
 [7] stringr_1.6.0       dplyr_1.2.1         maestro_1.1.1      
[10] glue_1.8.1          here_1.0.2          cronR_0.6.5        

loaded via a namespace (and not attached):
 [1] xfun_0.58         httr2_1.2.2       htmlwidgets_1.6.4 insight_1.5.1    
 [5] box_1.2.2         gargle_1.6.1      tzdb_0.5.0        vctrs_0.7.3      
 [9] tools_4.5.1       generics_0.1.4    parallel_4.5.1    curl_7.1.0       
[13] tibble_3.3.1      pkgconfig_2.0.3   R.oo_1.27.1       desc_1.4.3       
[17] lifecycle_1.0.5   compiler_4.5.1    tictoc_1.2.1      brio_1.1.5       
[21] janitor_2.2.1     snakecase_0.11.1  htmltools_0.5.9   yaml_2.3.12      
[25] pillar_1.11.1     crayon_1.5.3      R.utils_2.13.0    openssl_2.4.2    
[29] tidyselect_1.2.1  sjlabelled_1.2.0  digest_0.6.39     stringi_1.8.7    
[33] rematch2_2.1.2    purrr_1.2.2       rprojroot_2.1.1   fastmap_1.2.0    
[37] archive_1.1.13    cli_3.6.6         magrittr_2.0.5    readr_2.2.0      
[41] withr_3.0.2       waldo_0.6.2       rappdirs_0.3.4    bit64_4.8.2      
[45] lubridate_1.9.5   timechange_0.4.0  roxygen2_8.0.0    rmarkdown_2.31   
[49] httr_1.4.8        bit_4.6.0         otel_0.2.0        cellranger_1.1.0 
[53] askpass_1.2.1     R.methodsS3_1.8.2 hms_1.1.4         evaluate_1.0.5   
[57] knitr_1.51        rlang_1.2.0       xml2_1.5.2        pkgload_1.5.2    
[61] vroom_1.7.1       jsonlite_2.0.0    R6_2.6.1          fs_2.1.0