W00: Community bonding period

I spent the community bonding period trying to clarify in my head how exactly I want to approach the rewrite of the HTML front-end. I set out to answer the following questions:

1. Are there any lessons I can learn from static site generators (SSGs) and other languages' documentation generators?
2. What would be the most suitable JS/CSS framework for Documenter's HTML output?

JS & CSS frameworks

The aim of this GSoC project is to use a CSS framework for the Documenter's HTML front-end, as opposed to rolling your own styles. This should lead to an improved user experience with less development effort. But there is more than one to choose from, so the first step is to figure out which one to use.

I did quickly glance at "single-page frameworks", such as Angular and Vue. However, these frameworks produce quite JS-heavy sites and build step involves using a Node.js-based toolchain, complicating the process and introducing a whole technology stack that a potential or a user trying to debug a problem would have to be familiar with. Neither would be ideal for Documenter's use case.

Hence, using a more straightforward CSS framework ^[1] seems to be the way to go. Essentially, it means linking the necessary JS and CSS files to the output and making use of the framework provided UI components.

Out of the different frameworks, I ended up looking more closely at Bootstrap, Semantic UI, Foundation and Pure.css, as they should be the more popular ones. Foundation immediately felt too heavy and complex, so I did not really look at it very closely in the end. Pure.css on the other hand seemed too simple and basic — its library of UI components is pretty limited.

This leaves Bootstrap and Semantic UI. Both have a good library of UI components. Some require JS, but that is unavoidable if you want any interactivity. Bootstrap should be the most widely used one (also, anecdotally, whenever I see a framework being used for a simple site, it tends to be Bootstrap) and was the one suggested by @haampie in #626.

On the other hand, Semantic UI prides itself for concise HTML and intuitive JS APIs. It does seem to be true for the latter, as the JS API design follows a pretty uniform structure. As an example, consider setting the state of a progress bar in both Bootstrap and Semantic UI:

/* Semantic UI */
$('#myprogressbar').progress({percent: 22});
/* Bootstrap */
$('#myprogressbar > .progressbar').css('width', '22%').attr('aria-valuenow', 22);

Using frameworks. Just to give a taste of how the UI components could be used in Documenter, we can look at how a docstring block could be implemented using the standard card component in Bootstrap:

<article class="card">
  <h5 class="card-header"><code>foo(x::Integer)</code></h5>
  <div class="card-body">
    <p class="card-text">The function <code>foo</code> does things.</p>
  </div>
  <div class="card-footer text-right"><a href="#" class="card-link">Source</a></div>
</article>

which, without any custom styling, renders as follows

Now, some customization of the style would be necessary before it could actually be used. But in the theme, it should be possible to create a custom class for Documenter-specific UI elements with the Sass @extend feature, where we can easily override some stying, but also inherit all the framework stying for cards:

.docstring {
    @extend .card;
    /* customization */
}

The opening tag would then simply be <article class="docstring">.

I have not really made up my mind yet between Bootstrap and Semantic UI. So instead I have decided that I will try out both a bit more extensively. That is, during say the first week, when I am creating the mockup for the new front end, I'll do it with both frameworks and see which one seems to work better for Documenter.

Web Components

Another technology I explored briefly was Web Components. Essentially, they are custom HTML tags, rendering of which is done by custom JS code.

As an example, I was thinking that it might be possible to use a custom tag for docstrings

<jldoc-docstring>
  <jldoc-title>foo(::Integer)</jldoc-title>
  <article>
    <p>Foo does things.</p>
  </article>
  <jldoc-sourcelink href="https://github.com/..." />
</jldoc-docstring>

where we would also have to provide a JS class that does the actual rendering. A skeleton of the necessary JS would look like this:

class DocStringComponent extends HTMLElement {
  constructor() {/* implementation */}
}
window.customElements.define('jldoc-docstring', DocStringComponent);

However, by design, their styling is encapsulated, and as far as I can tell, it is not straightforward to modify their styling with a global CSS file.

They also rely heavily on JS, with the actual component generated dynamically by constructing a shadow DOM tree. So for docstrings it would mean that the actual rendered DOM tree would have to be generated client-side for each docstring on every page load, which feels unnecessary.

But it might still be worth keeping Web Components in mind for other types of small widgets such as a GitHub star counter etc.

SSGs and documentation generators

I spent some time looking through the documentation of generic static site generators (e.g. JuDoc, Jekyll, Hugo, MkDocs) and the documentation generation systems for other languages (e.g. Sphinx, Doxygen, rustdoc). My main goal was to see how they manage themes, but I also tried to keep an eye open for any other interesting features they might have.

First I discovered that general purpose static site generators were not actually that illuminating. They, in general, use a templating language to compile HTML files and just throw all the user-provided JS and CSS in there too. The SSG itself generally does not place any restrictions on what components etc. are available for the site — that is completely up to the author of the theme.

I would say that the primary purpose of Documenter is to compile docstrings, code snippets, plain text etc. into a single coherent abstract document. Abstract in the sense that the same Document can then be represented in several different concrete forms (e.g. HTML pages, a PDF file).

With that in mind, I do not think Documenter should provide a full SSG experience with generic templates etc. Rather, its HTML output should be relatively simple and straightforward, as long as it satisfies the needs of manual authors — i.e. more or less what it is doing right now.^[2] Heavy customization options are also a potential source of breakage for the pkg.julialang.org site.

When it comes to dedicated documentation generators (Sphinx, Doxygen, rustdoc), they all generate their own HTML and apply their own style. The styles do not rely on frameworks but are usually handwritten CSS. Theming is generally a matter of providing your own CSS file. Sphinx's themes are a bit more sophisticated and resemble more a generic static site generator, as you can also create your own HTML templates etc.

The conclusion I would draw here is that Documenter's current approach is sensible and in line with how other programming languages handle generating their documentation. What needs to be done is to streamline the existing experience, while also keeping DocumentationGenerator in mind.

Inter-document links. One feature of Sphinx that caught my eye was its intersphinx extension, which can handle link between separate manuals. Such a feature would be handy for Documenter, and actually perhaps not that hard to achieve with DocumentationGenerator. It would be good to take a closer look at how it exactly works in Sphinx.

So what now?

With the actual GSoC coding period upon us, my first task is to put together a mockup of the new front end using a framework. As I mentioned before, I will probably create two competing ones, one with Bootstrap and another with Semantic UI and see which one wins.