Getting Started¶
Install Redis¶
To get started with cronster
, install Redis
on your computer
(macOS & Homebrew in this example):
$ brew install redis
$ brew services start redis
Digital Ocean has a great tutorial on how install Redis on Ubuntu.
Run the crawler¶
Assuming Redis
is running on your machine in default configuration
and you want to crawl from ~/projects
, run the following command:
$ cronster_crawler -r ~/projects
The full help output explains the other options of the CLI application.
It is possible to change certain parameters should Redis
on a different
host/port or should you want to adjust the interval between crawls:
Usage: cronster_crawler [OPTIONS]
Options:
-r, --root TEXT Crawling root, default: the current working directory
-h, --cache-host TEXT Cache host, default: localhost
-p, --cache-port INTEGER Cache port, default: 6379 (Redis default)
-i, --interval INTEGER Crawling interval, default: 2 seconds
--help Show this message and exit.
Run the scheduler¶
Assuming Redis
is running on your machine in default configuration,
run the following command:
$ cronster_scheduler
The full help output explains the other options of the CLI application.
It is possible to change Redis
-related parameters should Redis
on a
different host or port:
Usage: cronster_scheduler [OPTIONS]
Options:
-h, --cache-host TEXT Cache host, default: localhost
-p, --cache-port INTEGER Cache port, default: 6379 (Redis default)
--help Show this message and exit.
Write a crontab
file¶
Write the following YAML
file to ~/projects/crontab
(or any other location
anywhere in the hierarchy under your root
location):
test_job:
cmd: echo "Hello, World!"
schedule: "*/5 * * * *"
You should see the job being picked up by the crawler on the next crawl and should see the scheduler run the job every five minutes.