=================================== Nim DocGen Tools Guide =================================== :Author: Erik O'Leary :Version: |nimversion| .. contents:: Introduction ============ This document describes the `documentation generation tools`:idx: built into the `Nim compiler `_, which can generate HTML and JSON output from input .nim files and projects, as well as HTML and LaTeX from input RST (reStructuredText) files. The output documentation will include module dependencies (``import``), any top-level documentation comments (##), and exported symbols (*), including procedures, types, and variables. Documentation Comments ---------------------- Any comments which are preceded by a double-hash (##), are interpreted as documentation. Comments are parsed as RST (see `reference `_), providing Nim module authors the ability to easily generate richly formatted documentation with only their well-documented code. Example: .. code-block:: nim type Person* = object ## This type contains a description of a person name: string age: int Outputs:: Person* = object name: string age: int This type contains a description of a person Field documentation comments can be added to fields like so: .. code-block:: nim var numValues: int ## \ ## `numValues` stores the number of values Note that without the `*` following the name of the type, the documentation for this type would not be generated. Documentation will only be generated for *exported* types/procedures/etc. Nim file input ----------------- The following examples will generate documentation for the below contrived *Nim* module, aptly named 'sample.nim' sample.nim: .. code-block:: nim ## This module is a sample. import strutils proc helloWorld*(times: int) = ## Takes an integer and outputs ## as many "hello world!"s for i in 0 .. times-1: echo "hello world!" helloWorld(5) Document Types ============== HTML ---- Generation of HTML documents is done via both the ``doc`` and ``doc2`` commands. These command take either a single .nim file, outputting a single .html file with the same base filename, or multiple .nim files, outputting multiple .html files and, optionally, an index file. The ``doc`` command:: nim doc sample Partial Output:: ... proc helloWorld*(times: int) ... Output can be viewed in full here: `docgen_sample.html `_. The next command, called ``doc2``, is very similar to the ``doc`` command, but will be run after the compiler performs semantic checking on the input nim module(s), which allows it to process macros. The ``doc2`` command:: nim doc2 sample Partial Output:: ... proc helloWorld(times: int) {.raises: [], tags: [].} ... The full output can be seen here: `docgen_sample2.html `_. As you can see, the tool has extracted additional information provided to it by the compiler beyond what the ``doc`` command provides, such as pragmas attached implicitly by the compiler. This type of information is not available from looking at the AST (Abstract Syntax Tree) prior to semantic checking, as the ``doc`` command does. JSON ---- Generation of JSON documents is done via the ``jsondoc`` command. This command takes in a .nim file, and outputs a .json file with the same base filename. Note that this tool is built off of the ``doc`` command, and therefore is performed before semantic checking. The ``jsondoc`` command:: nim jsondoc sample Output:: [ { "comment": "This module is a sample." }, { "name": "helloWorld", "type": "skProc", "description": "Takes an integer and outputs as many "hello world!"s", "code": "proc helloWorld*(times: int)" } ] Related Options =============== Project switch -------------- :: nim doc2 --project filename.nim This will recursively generate documentation of all nim modules imported into the input module, including system modules. Be careful with this command, as it may end up sprinkling html files all over your filesystem! Index switch ------------ :: nim doc2 --index:on filename.nim This will generate an index of all the exported symbols in the input Nim module, and put it into a neighboring file with the extension of ``.idx``. The index file is line oriented (newlines have to be escaped). Each line represents a tab separated record of several columns, the first two mandatory, the rest optional. See the `Index (idx) file format`_ section for details. Once index files have been generated for one or more modules, the Nim compiler command ``buildIndex directory`` can be run to go over all the index files in the specified directory to generate a `theindex.html `_ file. See source switch ----------------- :: nim doc2 --docSeeSrcUrl:txt filename.nim When you pass the ``docSeeSrcUrl`` switch to docgen, after each documented item in your source code the hyper link *See source* will appear pointing to the implementation of that item on a GitHub repository. You can click the link to see the implementation of the item. If you want to reuse this feature in your own documentation you will have to modify ``config/nimdoc.cfg`` to contain a ``doc.item.seesrc`` value with a hyper link to your own code repository. As you will see by the comments in that file, the value ``txt`` passed on the command line will be used in the HTML template along others like ``$path`` and ``$line``. In the case of Nim's own documentation, the ``txt`` value is just a commit hash to append to a formatted URL to https://github.com/Araq/Nim. The ``tools/nimweb.nim`` helper queries the current git commit hash during doc generation, but since you might be working on an unpublished repository, it also allows specifying a ``githash`` value in ``web/website.ini`` to force a specific commit in the output. Other Input Formats =================== The *Nim compiler* also has support for RST (reStructuredText) files with the ``rst2html`` and ``rst2tex`` commands. Documents like this one are initially written in a dialect of RST which adds support for nim sourcecode highlighting with the ``.. code-block:: nim`` prefix. ``code-block`` also supports highlighting of C++ and some other c-like languages. Usage:: nim rst2html docgen.txt Output:: You're reading it! The input can be viewed here `docgen.txt `_. The ``rst2tex`` command is invoked identically to ``rst2html``, but outputs a .tex file instead of .html. HTML anchor generation ====================== When you run the ``rst2html`` command, all sections in the RST document will get an anchor you can hyper link to. Usually you can guess the anchor lower casing the section title and replacing spaces with dashes, and in any case you can get it from the table of contents. But when you run the ``doc`` or ``doc2`` commands to generate API documentation, some symbol get one or two anchors at the same time: a numerical identifier, or a plain name plus a complex name. The numerical identifier is just a random number. The number gets assigned according to the section and position of the symbol in the file being processed and you should not rely on it being constant: if you add or remove a symbol the numbers may shuffle around. The plain name of a symbol is a simplified version of its fully exported signature. Variables or constants have the same plain name symbol as their complex name. The plain name for procs, templates, and other callable types will be their unquoted value after removing parameters, return types and pragmas. The plain name allows short and nice linking of symbols which works unless you have a module with collisions due to overloading. If you hyper link a plain name symbol and there are other matches on the same HTML file, most browsers will go to the first one. To differentiate the rest, you will need to use the complex name. A complex name for a callable type is made up from several parts: (**plain symbol**)(**.type**),(**first param**)?(**,param type**)\* The first thing to note is that all callable types have at least a comma, even if they don't have any parameters. If there are parameters, they are represented by their types and will be comma separated. To the plain symbol a suffix may be added depending on the type of the callable: ------------- -------------- Callable type Suffix ------------- -------------- proc *empty string* macro ``.m`` method ``.e`` iterator ``.i`` template ``.t`` converter ``.c`` ------------- -------------- The relationship of type to suffix is made by the proc ``complexName`` in the ``compiler/docgen.nim`` file. Here are some examples of complex names for symbols in the `system module `_. * ``type SignedInt = int | int8 | int16 | int32 | int64`` **=>** `#SignedInt `_ * ``var globalRaiseHook: proc (e: ref E_Base): bool {.nimcall.}`` **=>** `#globalRaiseHook `_ * ``const NimVersion = "0.0.0"`` **=>** `#NimVersion `_ * ``proc getTotalMem(): int {.rtl, raises: [], tags: [].}`` **=>** `#getTotalMem, `_ * ``proc len[T](x: seq[T]): int {.magic: "LengthSeq", noSideEffect.}`` **=>** `#len,seq[T] `_ * ``iterator pairs[T](a: seq[T]): tuple[key: int, val: T] {.inline.}`` **=>** `#pairs.i,seq[T] `_ * ``template newException[](exceptn: typedesc; message: string): expr`` **=>** `#newException.t,typedesc,string `_ Index (idx) file format ======================= Files with the ``.idx`` extension are generated when you use the `Index switch`_ along with commands to generate documentation from source or text files. You can programatically generate indices with the `setIndexTerm() `_ and `writeIndexFile() `_ procs. The purpose of ``idx`` files is to hold the interesting symbols and their HTML references so they can be later concatenated into a big index file with `mergeIndexes() `_. This section documents the file format in detail. Index files are line oriented and tab separated (newline and tab characters have to be escaped). Each line represents a record with at least two fields, but can have up to four (additional columns are ignored). The content of these columns is: 1. Mandatory term being indexed. Terms can include quoting according to Nim's rules (eg. \`^\` like in `the actors module `_). 2. Base filename plus anchor hyper link (eg. ``algorithm.html#*,int,SortOrder``). 3. Optional human readable string to display as hyper link. If the value is not present or is the empty string, the hyper link will be rendered using the term. Prefix whitespace indicates that this entry is not for an API symbol but for a TOC entry. 4. Optional title or description of the hyper link. Browsers usually display this as a tooltip after hovering a moment over the hyper link. The index generation tools try to differentiate between documentation generated from ``.nim`` files and documentation generated from ``.txt`` or ``.rst`` files. The former are always closely related to source code and consist mainly of API entries. The latter are generic documents meant for human reading. To differentiate both types (documents and APIs), the index generator will add to the index of documents an entry with the title of the document. Since the title is the topmost element, it will be added with a second field containing just the filename without any HTML anchor. By convention this entry without anchor is the *title entry*, and since entries in the index file are added as they are scanned, the title entry will be the first line. The title for APIs is not present because it can be generated concatenating the name of the file to the word **Module**. Normal symbols are added to the index with surrounding whitespaces removed. An exception to this are table of content (TOC) entries. TOC entries are added to the index file with their third column having as much prefix spaces as their level is in the TOC (at least 1 character). The prefix whitespace helps to filter TOC entries from API or text symbols. This is important because the amount of spaces is used to replicate the hiearchy for document TOCs in the final index, and TOC entries found in ``.nim`` files are discarded. Additional resources ==================== `Nim Compiler User Guide `_ `RST Quick Reference `_ The output for HTML and LaTeX comes from the ``config/nimdoc.cfg`` and ``config/nimdoc.tex.cfg`` configuration files. You can add and modify these files to your project to change the look of docgen output. You can import the `packages/docutils/rstgen module `_ in your programs if you want to reuse the compiler's documentation generation procs.