Step 2: Deployment

doctr-versions-menu

The Doctr Versions Menu package includes a doctr-versions-menu executable. This executable should be invoked when deploying the documentation on Travis, through doctr deploy’s --command flag.

As the explicit purpose of the Doctr Versions Menu is to enable documentation for multiple versions of a package at the same time, you’ll likely want to invoke doctr deploy also with the --no-required-master and --build-tags options.

For example, your .travis.yml file might include the following for deploying previously built documentation:

if [ ! -z "$TRAVIS_TAG" ]; then DEPLOY_DIR="$TRAVIS_TAG"; else DEPLOY_DIR="$TRAVIS_BRANCH"; fi

doctr deploy --command=doctr-versions-menu --no-require-master --build-tags "$DEPLOY_DIR"

See the deploy-section of the Doctr Versions Menu’s doctr_build.sh script (which is sourced from .travis.yml) for a more detailed example.

The main purpose of the doctr-versions-menu command is to generate the versions.json file that the Sphinx extension relies on, in the root of the gh-pages branch.

Deployment with Github Actions

As of December 2020, Travis ended their support for open source software. Thus, it is no longer possible to use Doctr as originally intended and as described throughout this documentation. The recommended alternative for all continous integration (including building documentation) is to use Github Actions. Doctr may add support for Github Actions in the future. In the meantime, for basic use cases (deploying the documentation to the gh-pages branch of the same repository), it is quite straightforward to implement the deployment “by hand”. A possible workflow is the following:

  • Build the documentation, zip the resulting html and upload it as an artifact

  • In a second job within the same workflow:

    • Check out the gh-pages branch

    • Download the artifact generated by the first job and unpack it to a temporary location

    • Synchronize the unpacked html files with the appropriate subfolder within the gh-pages branch

    • Run the doctr-versions-menu command

    • Commit and push the changes.

What makes this easy within Github Actions, is that the action is automatically authenticated to upload/download artifacts and to have push-access to the underlying repository. Thus, Doctr’s main feature of managing the deploy key for Travis becomes obsolete.

An example workflow file illustrates this in more detail.

If you are not using Travis/Doctr, you probably also want to change the menu title, by putting e.g.

doctr_versions_menu_conf = {
    'menu_title': 'Docs'  # instead of 'Doctr'
}

in your conf.py file.

Command line options

doctr-versions-menu

Usage

doctr-versions-menu [OPTIONS]

Options

--version

Show the version and exit.

--debug

enable debug logging [env var DOCTR_VERSIONS_MENU_DEBUG]

-o, --outfile <OUTFILE>

File to which to write json data [env var DOCTR_VERSIONS_MENU_OUTFILE; default: versions.json]

--default-branch <DEFAULT_BRANCH>

The name or possible names of the default branch for the project. If the project has no public releases, the default index.html should forward to the first available default branch. Traditionally, most projects have used ‘master’ as the default branch. More recently, the name ‘main’ has become standard. [env var DOCTR_VERSIONS_MENU_DEFAULT_BRANCH; default: master, main]

--versions <SPEC>

Specification of versions to be included in the menu, from oldest/lowest priority to newest/highest priority. The newest/highest priority items will be shown first. See the online documentation for the SPEC syntax. [env var DOCTR_VERSIONS_MENU_VERSIONS; default: (<branches> != <default-branch>), <releases>, <default-branch>]

--latest <SPEC>

Specification of which version is considered the “latest public release”. If it exists, the main index.html should forward to this version, and warnings e.g. for “outdated” versions should link to it. See the online documentation for the SPEC syntax. [env var DOCTR_VERSIONS_MENU_LATEST; default: (<public-releases>)[-1]]

--warning <NAME SPEC>

Define a warning. The NAME is a lowercase label that will appear in the warnings data in versions.json and maybe be picked up by the javascript rendering warning in the HTML output. The SPEC is a folder specification for all folders that should show the warning. See the online documentation for the syntax of SPEC. The SPEC should give given as a quoted string. This option may be given multiple times.If specified via an environment variable, use the form "NAME: SPEC; NAME: SPEC; …". Any colons and semi-colons in NAME and SPEC must be escaped in this case. See the online documentation for details. [env var DOCTR_VERSIONS_MENU_WARNING]

--label <SPEC LABELTEMPLATE>

Set a template for labels in the versions menu. The LABELTEMPLATE applies to all folders matching the given SPEC. See the online documentation for the syntax of SPEC. The LABELTEMPLATE is rendered with Jinja, receiving the folder name. See the online documentation for details. This option may be given multiple times. If specified via an environment variables, use the form "SPEC: LABELTEMPLATE; SPEC: LABELTEMPLATE; …". Any colons and semi-colons in SPEC and LABELTEMPLATE must be escaped in this case. See the online documentation for details. [env var DOCTR_VERSIONS_MENU_LABEL]

--write-index-html, --no-write-index-html

Whether to write an index.html that forwards to the latest public release. In the config file, override this as write_index_html=False. [env var DOCTR_VERSIONS_MENU_WRITE_INDEX_HTML; default: True]

--write-versions-py, --no-write-versions-py

Whether to write a script versions.py to the root of the gh-pages branch for regenerating versions.json. This is useful for maintenance on the gh-pages branch, e.g., removing outdated version. [env var DOCTR_VERSIONS_MENU_WRITE_VERSIONS_PY; default: True]

--ensure-no-jekyll, --ignore-no-jekyll

Whether to check that a .nojekyll file exist and create it otherwise. In the config file, override this as ensure_no_jekyll=False. [env var DOCTR_VERSIONS_MENU_ENSURE_NO_JEKYLL; default: True]

--downloads-file <FILE>

The name of the file inside of each folder from which to read the download links. Each line in the file must be of the form [label]: url. To disable download links, use --no-downloads-file or set the environment variable to an empty string. [env var DOCTR_VERSIONS_MENU_DOWNLOADS_FILE; default: _downloads]

--no-downloads-file

Disable the downloads file. In the config file, use downloads_file = False. Or, using an environment variable, set DOCTR_VERSIONS_MENU_DOWNLOADS_FILE="".

--suffix-latest <suffix_latest>

Suffix for the label of the latest public release. This is used in addition to any label set via the --label option [env var DOCTR_VERSIONS_MENU_SUFFIX_LATEST; default:  (latest)]

-c, --config <config>

Read configuration from FILE. Each line in FILE should be of the form “variable = value” in Python syntax, with variable names corresponding to any long-form command line flag, e.g. ensure_no_jekyll = False. Defaults to doctr-versions-menu.conf [env var DOCTR_VERSIONS_MENU_CONFIG]

Debugging

If the doctr-versions-menu command behaves unexpectedly, set the environment variable

DOCTR_VERSIONS_MENU_DEBUG=true

in your .travis.yml file, see --debug.

Make sure to include the debug output when reporting bugs.

Default assumptions

You should not have to customize doctr-versions-menu (that is, provide command line options) provided you stick to the following sensible assumptions:

  • Releases should be tagged as e.g. v0.1.0 and deployed to a folder of the same name. That is, a lower case letter v followed by a PEP 440-compatible version identifier.

  • The master branch should be deployed to a folder master, respectively main to a folder main for projects that use “main” as the default branch name.

  • Any other branch for which documentation is to be deployed should go in a folder matching the branch name.

By default, the index.html file will forward to the documentation of the latest public release (excluding pre-releases such as v1.0.0-rc1), or to the default branch (main or master) if there have been no public releases. There is no support for an RTD-style “latest”/”stable” folder. This is by design: deep-linking to “latest” documentation is a bad practice, as such links easily become invalid when a new version is released.

Customization

If you do need to customize doctr-versions-menu’s behavior, there are three options:

  1. Set the various DOCTR_VERSIONS_MENU_* environment variables

  2. Place a configuration file doctr-versions-menu.conf in the root of the gh-pages branch

  3. Call the doctr-versions-menu executable with explicit command line options

DOCTR_VERSIONS_MENU environment variables

The recommended way to customize doctr-versions-menu it to set environment variables. Each of doctr-versions-menu’s command-line-options has an associated environment variable, as listed in the documentation of the Command line options. You can set these environment variables in your .travis.yml file.

Warning

It can be quite tricky to properly specify environment variables in the YAML format used for the travis.yml configuration file. This is due to YAML’s complicated string-quoting rules. For DOCTR_VERSIONS_MENU_VERSIONS (see --versions), DOCTR_VERSIONS_MENU_LABEL (see --label), and DOCTR_VERSIONS_MENU_WARNING (see --warning) in particular, you will likely have to triple-quote the values to avoid Travis inperpolating inside of them.

- DOCTR_VERSIONS_MENU_VERSIONS: '''(<branches> != (master, rtd-theme)), (<releases>)[:-1], rtd-theme, (<releases>)[-1], master'''
- DOCTR_VERSIONS_MENU_LABEL: '''rtd-theme: v0.2.0 (rtd-theme)'''
- DOCTR_VERSIONS_MENU_WARNING: '''unreleased: (<branches> != rtd-theme), <local-releases>'''

For the --warning and --label options which can occur multiple times on the command line and take two arguments, the equivalent specification with an environment variable must separate the two arguments with a colon, and multiple argument pairs with a semicolon. Consequently, colons and semicolons that occur inside the arguments must be escaped with a backslash.

doctr-versions-menu.conf configuration file

An alternative way to configure doctr-versions-menu is to place a configuration file doctr-versions-menu.conf in the root of the gh-pages branch. Compared to setting environment variables in the .travis.yml file, this has the drawback that the configuration of your documentation is not tracked as part of your main/master branch, but instead lives on the gh-pages branch. Thus, changes to the documentation may involve organizing work spread across multiple branches, which can get confusing.

The configuration file can contain definitions matching doctr-versions-menu’s Command line options, formatted according to Configobj’s unrepr mode.

Every long-form flag has a corresponding config file variable, obtained by replacing hyphens with underscores. For boolean flags, the variable name is derived from the first flag option.

For example, the settings

downloads_file = ".downloads"
ensure_no_jekyll = False

correspond to --downloads-file=.downloads and --ignore-no-jekyll.

See also this doctr-versions-menu.conf file, which illustrates some advanced usage.

Passing command line options

Lastly, you may also explicitly invoke the doctr-versions-menu executable with specific Command line options in your .travis.yml, respectively the doctr_build.sh script.

For example, you might have the following deploy command:

python -m doctr deploy --key-path docs/doctr_deploy_key.enc \
        --command='doctr-versions-menu --debug' \
        --built-docs docs/_build/html --no-require-master \
        --build-tags "$DEPLOY_DIR"

Note the quotes around --command’s argument. Generally, the use of environment variables is more readable and easier to maintain, so you should avoid using command line options as part of your automated builds.

Folders included in the menu

By default, the version menu lists the default branch (main or master, see --default-branch) first, then any releases from newest to oldest, and then any non-release branches in reverse-alphabetical order. Having the “newest” releases appear first matches the behavior of Read-the-Docs.

The folders that are listed in the versions menu and their order can be customized via the --versions flag (DOCTR_VERSIONS_MENU_VERSIONS environment variable, or versions in the config file). This receives a folder specification as an argument that specifies the folders to appear in the menu in reverse order (bottom/right to top/left).

To un-reverse the default order of folders in the menu, so that the newest versions and the default branch appear last, you would put

DOCTR_VERSIONS_MENU_VERSIONS: '''((<branches> not in <default-branch>), <releases>, <default-branch>)[::-1]'''

in the definitions of environment variables in your .travis.yml file.

Labels in the versions menu

By default, the label for each folder that appears in the menu is simply the name of the folder. The “latest public release”, identified by --latest (the latest public release by default), has “(latest)” appended. This can be customized with --suffix-latest (DOCTR_VERSIONS_MENU_SUFFIX_LATEST environment variable).

More generally, the --label option may be used to define label templates for specific groups of folders. The option can be given multiple times. Each --label receives two arguments, a folder specification for the folders to which the template should apply, and a Jinja-template-string that should receive the variable folder for rendering. For example,

doctr-versions-menu --label '<releases>'  "{{ folder | replace('v', '', 1) }}" --label master '{{ folder }} (latest dev branch)'

drops the initial v from the folder name of released versions (v1.0.01.0.0) and appends a label `` (latest dev branch)`` to the label for the master folder.

When specifying the labels via the DOCTR_VERSIONS_MENU_LABEL environment variable, the multiple --label options are combined into a single value, separated by semicolons, and the two arguments separated by a colon. For the above example, an appropriate definition in .travis.yml’s environment variable definitions would be

- DOCTR_VERSIONS_MENU_LABEL: '''<releases>: {{ folder | replace("v", "", 1) }}; master: {{ folder }} (latest dev branch)'''

Note the triple-quotes, which are required here (technically, the single outer quotes prevent YAML from performing any escapes inside the string except for transforming the two inner quotes into a single quote which then prevents the shell from interpolating inside the string).

Similarly, in a config file, the above options would be specified as:

label = '''[
    ('<releases>', "{{ folder | replace('v', '', 1) }}"),
    ('master', '{{ folder }} (latest dev branch)')
]'''

The triple-quotes are required for a multi-line entry.

Note

Read-the-Docs uses “latest” to refer to the latest development version (usually main/master) instead of the latest public release, and instead labels the latest public release as “stable”. You may adopt Read-the-Docs nomeclature with e.g.

--suffix-latest=" (stable)" --label master 'master (latest)'

or

--suffix-latest=" (stable)" --label master latest

Custom warning messages

By default, the Doctr Versions Menu plugin injects warnings in the rendered HTML files, within the following types of folders:

  • an ‘outdated’ warning for <releases> older than the latest public release (identified by --latest)

  • an ‘unreleased’ warning for <branches> (anything that is not a PEP 440-conforming release), or <local-releases> (typically not used)

  • a ‘prereleased’ warning for anything considered a pre-release by PEP 440, e.g. v1.0.0-rc1

Which folders are included in the above three categories can be modified via the --warning option. This options receives two arguments, a “warning label” string (the above ‘outdated’, ‘unreleased’, or ‘prereleased’), and a folder specification for the folders to which the warning should apply. The option can be given multiple times. An empty specification would disable the warning, e.g.

doctr-versions-menu --warning prereleased ''

to disable the warning message on pre-releases.

It is also possible to define entirely new warning labels using --warning. For example,

doctr-versions-menu --warning post '<post-releases>'

would define a warning ‘post’ for all post-releases.

The information about which folders should display which warnings is stored internally in the resulting versions.json file, in a dict ‘warnings’ the maps folder names to a list of warning labels.

To actually show this new custom warning, the doctr-versions-menu.js_t template would have to be modified to pick up on the ‘post’ label, see the instructions for the Customization of the doctr_versions_menu Sphinx extension.

Similarly to Labels in the versions menu, when configuring the warnings via the DOCTR_VERSIONS_MENU_WARNING environment variable, multiple --warning options are combined into a single value, separated by semicolons, and the warning label and folder specification separated by a colon.

For the above two options, you might include the following the definition of the environment variables in the .travis.yml file:

- DOCTR_VERSIONS_MENU_WARNING: '''post: <post-relases>; prereleased:'''

The triple-quotes are required to protect the string from both YAML and shell interpolation.

In the config file, the above options may be configured as e.g.:

warning = '''[
    ('post', '<post-releases>'),
    ('prereleased', ''),
]'''

The default settings (with the default --latest) correspond to:

warning = '''[
    ('outdated', '(<releases> < (<public-releases>)[-1])'),
    ('unreleased',  '<branches>, <local-releases>'),
    ('prereleased', '<pre-releases>'),
]'''

Note the triple-quotes required for a multi-line entry.

Customizing index.html

By default, doctr-versions-menu generates an index.html file in the root of the gh-pages branch that redirects to the current “default folder”. This is the folder for the most current public release (--latest), or the default branch (see --default-branch) if no public release exists.

The generated index.html file can be customized by placing an index.html_t Jinja template into the root of the gh-pages branch. This template will be rendered receiving a dict version_data containing the data in versions.json (see the versions.json Structure).

The default template is

<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="Refresh" content="0; url={{ version_data['latest'] | default(version_data['default-branch']) | default(version_data['folders'][0], true) }}" />
  </head>
  <body>
    <p>Go to the <a href="{{ version_data['latest'] | default(version_data['default-branch'])  | default(version_data['folders'][0], true) }}">default documentation</a>.</p>
  </body>
</html>

Alternatively, if you want a completely static index.html, you could also just add that file by hand and use --no-write-index-html (that is, DOCTR_VERSIONS_MENU_WRITE_INDEX_HTML=false as an environment variable or write_index_html=False in the doctr-versions-menu.conf configuration file).

Maintenance on gh-pages

Unless --no-write-versions-py is given or DOCTR_VERSIONS_MENU_WRITE_VERSIONS_PY=false is set, running doctr-versions-menu will generate a script versions.py in the root of the gh-pages branch that may be used to regenerate the versions.json file. This script is intended for manual maintenance on the gh-pages branch, that is, outside of the normal automatic Doctr-deployment through Travis. For example, you may occasionally want to remove folders for outdated branches or pre-releases from the gh-pages branch, or update existing download links.

After any such change, run the versions.py script before committing and pushing the gh-pages branch.

Remember that each folder on the gh-pages branch generally contains its own doctr-versions-menu.js script. Switching a project to a new major version of doctr-versions-menu, if that version changes the internal data structure of versions.json, may require updating the doctr-versions-menu.js script in existing folders by hand.