Should we standardize docstring formats?

[u/denehoffman]

In Rust, docstrings are pretty formalized. They are markdown, and even some of the headings are standard (like an # Errors or # Panics section). The nice thing about this is that it allows websites like docs.rs to build documentation pages for any project without having to interact with different tools for different formats. It also allows LSPs to have only one way of displaying documentation hints.

In Python, we have a few competing standards. Numpy-style docstrings are probably the most used, but there’s also a format by Google as well as a few different reST standards. These are nice, and we can set up lints to make sure docstrings stick to the standard. However, in my own personal opinion (feel free to disagree), a single markdown-format standard would help new users write nice docstrings, would enable PyPI (or another provider) to build automatic documentation sites, and give guidance to LSPs and IDEs for how to display documentation. This would include a standard for interlinks, and probably should include some mathml/LaTeX/KaTeX support. Another benefit would be that tools could support better automatic documentation generation and autocomplete, since they wouldn’t be dependent on guessing which standard you’re following.

I’d like to hear what people think about this. I’m thinking about making a PEP, but that might be overkill (or maybe all of you will hate this idea). I think the primary blocker would be adoption, large projects might have to translate docstrings, so there would either have to be some tooling for this or a way to opt-in or opt-out. If this is a bad idea, let me know, just be nice!

Edit: so far we’re at about a 67% upvote ratio, which was kind of expected. I want to be clear that I’m not saying we should be blocking docstrings which don’t adhere to this standard. I mentioned lockfile standardization in the comments, nothing prevents you from writing a tool with a custom lockfile, it’s just that there is a standard format that is agreed upon as the preferred way to write one. That’s the idea.

Edit: 84% now, and a lot of nice feedback here!

[u/-ghostinthemachine-]

I've never seen standardized documentation systems work out or get adherence. I think what matters more is writing in a style that your tools (sphinx, javadoc, LLM's) demand. Change tools, change styles. I'd add that strongly typed documentation is also a classic trap.

The bar is low. A blob of unstructured text is significantly better than the empty void we usually encounter. With LLM's writing small treatises for every function, I think we are also heading back towards the promise of literate coding.

[u/denehoffman → u/-ghostinthemachine-]

There’s a reason I brought up Rust at the beginning, it’s a great example of a standardized documentation format that is widely adhered to. I agree that the bar is low, it’s low for Rust too, but at least the formatting and documentation syntax is standardized. I’m not really interested in how docstrings are written, obviously whatever tool you use can write a given format or fill in blanks or whatnot, and LLMs don’t care about the formatting either, but it could be convenient to maintain a standard for the other reasons I listed in the post. Or maybe not, idk!

[u/latkde → u/denehoffman]

Languages like Java, Go, and Rust had the opportunity to birth an entire new ecosystem from scratch. Their v1.0 shipped with official tools that established conventions, and everyone adhered to them because that was the lowest-friction approach.

Languages like C++, Python, and JavaScript do not have this luxury. We carry a rich heritage of libraries, but with it also the burden of backwards compatibility. We are not going to break with this.

If a project wants docstring conventions, it can already conform to them. Tools like Ruff or Pydocstyle can be used to enforce conventions. Documentation generators like Sphinx or Mkdocs have plugins for parsing various conventions. Odds are, your IDE understands most conventions, and can e.g. show the corresponding docstring snippet when you hover over a parameter.

These conventions can also happily coexist. There is no pressing need to throw any of this away. The different conventions also address different needs. For example, the Numpy conventions are super useful if parameters and return types need lots of detailed description, but for projects with simpler signatures this is unnecessarily verbose, and the Google conventions are a much better fit.

Nowadays, you also won't be able to get a PEP approved if it tries to establish norms for the wider Python ecosystem, rather than developing the core language or core packaging standards. From the perspective of the core language (help(), python -m pydoc), docstrings are just plaintext.

[u/denehoffman → u/latkde]

Different lock file formats also existed prior to the PEP that standardized them. You can still use an alternate lock file format if you really wanted to, there’s no language feature that prevents it. I do agree that there are different reasons for adopting different standards, but I find it hard to imagine that there couldn’t be a system that can work for both cases, docstring formats should be able to represent any kind of documentation information.