Rejuvenating Documenter

This is the submission for the 2019 Google Summer of Code (GSoC) project by @mortenpi, mentored by @pfitzeb.

Documentation, essential for any software project, needs to be as easy as possible to write and publish. For Julia packages, you can use the Documenter package, which automatically generates and publishes package manuals as a web page or a PDF.

It is important for Documenter to be modern and flexible. This project aims to revitalize Documenter, by upgrading the generated HTML front end and making it easier for documentation authors to customize. This will make sure that Documenter will keep meeting the needs of the community for years to come.

Documenter works by creating an abstract representation of a document in memory and then it can output it in different formats (HTML, LaTeX/PDF, and Markdown are built-in). The HTML output is the primary one, which allows users to create static websites with their package documentation.

So far everything about the HTML front end has been relatively monolithic: the styling a single hand-written CSS file, all the JavaScript a single, static file, etc. This made it quite difficult for users to customize their deployments. The aim of the project was to update the HTML front end and make it more customizable.

Features

While over the course of the project I made a multitude of changes to Documenter (see Statistics for a references to all the pull requests), there were essentially three main features that I implemented as part of the summer of code: the New HTML front end, the New assets APIs and a New doctesting API.

New HTML front end

The central goal of the project was to significantly update the HTML front end that Documenter generates for the static sites.

The idea was to base it on a CSS framework, so that we could offload as much of the styling, mobile experience, etc., work to a centrally developed library, hopefully reducing the maintenance burden. After a brief comparison of various frameworks, I decided to go with Bulma. Documenter now vendors in a version of Bulma Sass files and builds its theme and layout on top of that.

Without further ado, this is what the Julia manual has looked like until now:

And this is what it will look like once we switch it over to the new front end^[1]:

I kept the general layout the same but tried to rely as much as possible on the framework. I also tried to address various small usability issues with the layout — for example, the sidebar menu items are now collapsible (with the level where it collapsed configurable).

I slightly overestimated how much we could benefit from a framework. In the end, to make it look nice and to make sure that it works exactly how I need it to, I still had to manually write a lot of custom (S)CSS. Also, e.g. the handling of colors in Bulma was not quite as sophisticated as I would have liked.

However, a very neat feature of the new approach is that everything is now based on Sass. You can now much more easily make changes to the layout, styling and colors by changing a few variables. This makes it much easier to both maintain and extend the Documenter style files.

I also made it much easier to implement a dark theme. By clicking on the cogwheel on the top-right, you can access a small settings dialog, which includes a small colophon, but also allows the theme to be picked:

I was able to use the Bulmaswatch's Darkly theme almost without modifications. Having a dynamic theme selector on a static site was a challenge of its own, which I summarized in more detail in one of the weekly updates (Theme selector). When switching to the dark theme, the site looks as follows:

Overall, the front end demos have already received positive feedback and all the major changes have been integrated into Documenter's master branch. There are a few things that need to be wrapped up after the summer of code project.

• There is an open issue with a few small theme bugs that need to be sorted out before releasing the new front end (#1091).
• Once that is done, we can release Documenter 0.24 and switch the Julia manual over to the new front end (JuliaLang/julia#32839).
• Creation and use of user-developed themes should be documented. However, this would really benefit from people trying to create themes first.

New assets APIs

The second major goal of the project was to make it easier for the user to customize the JavaScript dependencies.

Most of the JavaScript code for the generate HTML site lives in a documenter.js file, which simply gets included with a <script> tag on each of the pages. So far, it has been a monolithic file that just gets copied over and there has been no way to customize its contents.

In the last few weeks of the project, I implemented a more dynamic and configurable approach. Internally, Documenter now generates the documenter.js file on the fly every time it builds the document (#1092; see W12: Require JS for more details). This allows us to easily implement configuration options for the various JavaScript libraries that we are including.

Specifically, it made it very easy to implement the following:

• The highlights.js integration can now be configured with additional languages. (#1094)
• The user can now choose whether to use KaTeX or MathJax, and pass configuration option to both directly in the make.jl script. #1097

Finally, while not related to the dynamic generation of documenter.js, I also added the possibility for the user to add remote dependencies (#1108). This, for example, can be used to include additional fonts from e.g. Google Fonts:

assets = [
  asset("https://fonts.googleapis.com/css?family=Nanum+Brush+Script&display=swap", class=:css)
]

One thing that is yet to do is having an ability for the user to add in their own custom scripts to documenter.js.

New doctesting API

Documenter 0.23, which I released half-way through the summer of code, included a new API for doctesting (the automatic verification that outputs of code snippets in the manual are up to date).

This was not initially planned to be a major part of the project, as I just had to finish off an almost year-old pull request (#774). However, it turned out to be a bit more work than initially planned and was developed a little bit further based on feedback (#1054).

Previously, in order to verify all the code snippets, you had to perform a full documentation build. As this also involves running all the @example etc. blocks, this can be quite time-consuming. So the initial goal was to allow checking the doctests without running all the other build stages by passing a special option to makedocs:

makedocs(..., doctest=:only)

Internally, this feature required a little bit of refactoring. Generally, all the at-blocks and doctests get executed sequentially on a page. Specifically, before we can reliably run a doctest, we need to make sure that we have evaluated all the metadata in at-meta blocks. For doctests in docstrings, this means that we first need to figure out where exactly the docstring ends up in the manual, which means running quite a large chunk of the Documenter-machinery.

To get around this, doctests in docstrings now run completely independently of at-docs blocks etc. Instead of setting the metadata with at-meta blocks, you attach it to a module, which will then apply to all the doctests defined in that module. This can be done using the functions in Documenter's DocMeta module.

DocMeta.setdocmeta!(MyPackage, :DocTestSetup, :(using MyPackage); recursive=true)

Once all of that was implemented, it became clear it would be very easy to also add a function for just running doctests, such that it would only run the testsets. Hence, doctest was born:

using Test, Documenter, MyPackage
@testset "MyPackage" begin
    ... # other tests & testsets
    doctest(MyPackage)
    ... # other tests & testsets
end

Currently, internally it just does a simple makedocs call with doctest=:only.

The doctest function actually returns a test set, so it is very easy to incorporate it into existing package test sets. In the future, we can hopefully record each doctest separately in the log.

Statistics

Based on GitHub contributors page of Documenter for the period of the project I added 28,024 SLOC and removed 2,981. This might sounds impressive.. until you realize that it also includes vendored-in Bulma dependency and compiled CSS files for the default themes. GitHub's "Code frequency" graph for Documenter will be interesting for the foreseeable future:

The pull requests are probably a better metric. During the project, I opened:

• 33 pull requests against Documenter.jl
• 3 pull requests against DocumenterTools
• 7 pull requests against the core Julia repository

Some of these are very minor and not all were directly related to the summer of code project (e.g. general maintenance work). I have pointed out all the major ones in the text above.

Only three pull requests are still open and they have to do with: (1) the theming API, (2) switching the Julia manual over to the new front end, and (3) an unrelated pull request about CI settings.