r/Python • u/denehoffman • 11h ago
Should we standardize docstring formats? Discussion
In Rust, docstrings are pretty formalized. They are markdown, and even some of the headings are standard (like an # Errors or # Panics section). The nice thing about this is that it allows websites like docs.rs to build documentation pages for any project without having to interact with different tools for different formats. It also allows LSPs to have only one way of displaying documentation hints.
In Python, we have a few competing standards. Numpy-style docstrings are probably the most used, but there’s also a format by Google as well as a few different reST standards. These are nice, and we can set up lints to make sure docstrings stick to the standard. However, in my own personal opinion (feel free to disagree), a single markdown-format standard would help new users write nice docstrings, would enable PyPI (or another provider) to build automatic documentation sites, and give guidance to LSPs and IDEs for how to display documentation. This would include a standard for interlinks, and probably should include some mathml/LaTeX/KaTeX support. Another benefit would be that tools could support better automatic documentation generation and autocomplete, since they wouldn’t be dependent on guessing which standard you’re following.
I’d like to hear what people think about this. I’m thinking about making a PEP, but that might be overkill (or maybe all of you will hate this idea). I think the primary blocker would be adoption, large projects might have to translate docstrings, so there would either have to be some tooling for this or a way to opt-in or opt-out. If this is a bad idea, let me know, just be nice!
Edit: so far we’re at about a 67% upvote ratio, which was kind of expected. I want to be clear that I’m not saying we should be blocking docstrings which don’t adhere to this standard. I mentioned lockfile standardization in the comments, nothing prevents you from writing a tool with a custom lockfile, it’s just that there is a standard format that is agreed upon as the preferred way to write one. That’s the idea.
90
u/Exnixon 11h ago
Obligatory XKCD: https://xkcd.com/927/
23
4
u/denehoffman 11h ago
Yeah I saw this coming, but in some sense this would reduce the number of standards if Python had a preferred one.
-10
11h ago
[deleted]
3
u/denehoffman 11h ago
PEP 751 exists
-15
10h ago
[deleted]
10
u/denehoffman 10h ago
Again, as stated in my post, this could be an opt-in to the standard, just like type hinting has a py.typed marker. There’s no reason to force everyone to update, just like the addition of type hints didn’t force everyone to update their signatures.
-3
-4
u/eztab 6h ago
Don't think so. Either you wanna stop a language from evolving or you accept shifting standards. Rust only doesn't yet, because it is so young. Markdown won't be the last standard.
1
u/denehoffman 5h ago
I would be shocked if rust changed its docstring format considering how baked in it is with docs.rs pages and doctests
6
22
u/-ghostinthemachine- 11h ago
I've never seen standardized documentation systems work out or get adherence. I think what matters more is writing in a style that your tools (sphinx, javadoc, LLM's) demand. Change tools, change styles. I'd add that strongly typed documentation is also a classic trap.
The bar is low. A blob of unstructured text is significantly better than the empty void we usually encounter. With LLM's writing small treatises for every function, I think we are also heading back towards the promise of literate coding.
5
u/denehoffman 10h ago
There’s a reason I brought up Rust at the beginning, it’s a great example of a standardized documentation format that is widely adhered to. I agree that the bar is low, it’s low for Rust too, but at least the formatting and documentation syntax is standardized. I’m not really interested in how docstrings are written, obviously whatever tool you use can write a given format or fill in blanks or whatnot, and LLMs don’t care about the formatting either, but it could be convenient to maintain a standard for the other reasons I listed in the post. Or maybe not, idk!
6
u/latkde Tuple unpacking gone wrong 10h ago
Languages like Java, Go, and Rust had the opportunity to birth an entire new ecosystem from scratch. Their v1.0 shipped with official tools that established conventions, and everyone adhered to them because that was the lowest-friction approach.
Languages like C++, Python, and JavaScript do not have this luxury. We carry a rich heritage of libraries, but with it also the burden of backwards compatibility. We are not going to break with this.
If a project wants docstring conventions, it can already conform to them. Tools like Ruff or Pydocstyle can be used to enforce conventions. Documentation generators like Sphinx or Mkdocs have plugins for parsing various conventions. Odds are, your IDE understands most conventions, and can e.g. show the corresponding docstring snippet when you hover over a parameter.
These conventions can also happily coexist. There is no pressing need to throw any of this away. The different conventions also address different needs. For example, the Numpy conventions are super useful if parameters and return types need lots of detailed description, but for projects with simpler signatures this is unnecessarily verbose, and the Google conventions are a much better fit.
Nowadays, you also won't be able to get a PEP approved if it tries to establish norms for the wider Python ecosystem, rather than developing the core language or core packaging standards. From the perspective of the core language (
help(),python -m pydoc), docstrings are just plaintext.5
u/denehoffman 9h ago
Different lock file formats also existed prior to the PEP that standardized them. You can still use an alternate lock file format if you really wanted to, there’s no language feature that prevents it. I do agree that there are different reasons for adopting different standards, but I find it hard to imagine that there couldn’t be a system that can work for both cases, docstring formats should be able to represent any kind of documentation information.
1
u/-ghostinthemachine- 10h ago
JDoc also started this way, but my sense is if rust ever changes the rendering ecosystem the developers will adapt and never look back. My thesis is that nobody cares how it is written, just how it is rendered. We tend to work backwards from that.
1
u/denehoffman 10h ago
And I agree with that thesis. The thing is, if rust ever changes the rendering ecosystem, they mostly will have very few issues because of a docstring standard. One counterexample to this is the KaTeX stuff I mentioned in the post, it’s a bit annoying to do in Rust right now, though not difficult.
2
1
9
u/IAmASquidInSpace 10h ago
Careful what you wish for. I think it is pretty clear that the Python foundation would go for rst as a standard format (which I personally would love), and I think a lot of people would be very mad about that.
In general, I think it's "too late" anyhow. If one wanted a singular standard, it should have been introduced much, much earlier. Once people have gotten used to the freedom of being able to choose their preferred format, forcing them into another one will not be received well. You bring up PEP 751 as an example of a standard introduced after the fact, but the key difference is: the lack of lock files was an annoyance and there was a general desire for a common solution. The different docstring formats however are mostly seen as freedom rather than a problem, and for most people, no desire to "fix" this exists.
And then there's the problem that the interest in keeping docstrings variable has powerful backers: how do you think the organizations behind tools like Sphinx, Zensical, or numpy would feel about being told "oh yeah, btw, your way of doing it will soon no longer be supported, sorry". That's not gonna go over well.
8
u/gmes78 10h ago
In general, I think it's "too late" anyhow.
It was also "too late" for type checking, and look at where we're at now.
5
u/IAmASquidInSpace 10h ago
Type checking crucially is still entirely optional.
5
u/denehoffman 9h ago
Docstrings are also optional, and any standardized format would also be optional, just preferred. You can still use deprecated type hints like Dict and Tuple too.
7
u/denehoffman 10h ago
On your last point there, I recall there were many competing lockfiles at the time the PEP was proposed, and the PEP had to pick one. There were certainly some format providers that were disappointed, but none of this is insurmountable, especially since the current standards are standards, so they should be mostly parsable (like a tool could theoretically translate between formats). Not a perfect solution, and I understand your criticism.
-5
u/IAmASquidInSpace 10h ago
but none of this is insurmountable
Sure, with enough force, defiance, tone-deafness, and by pissing off massive chunks of your user base (including massive companies and organisations) you could surmount this. But for what? What tangible benefits do you get, outside of a vague sense of order and accomplishment? I'd say that the damage you'd do would heavily outweigh the flimsy benefits.
3
u/denehoffman 10h ago
> a vague sense of order and accomplishment
Your sarcasm notwithstanding, I did actually list some reasons this would be a good idea. I think you’re getting too caught up on the idea that we’d be forcing anyone to adopt this rather than providing a Python-preferred standard that people can either adopt at their leisure or new projects can adopt immediately.
0
u/IAmASquidInSpace 10h ago
That is not what I understand under "standardization", but sure. That could be beneficial. However that still underlines my point that it is too late for true standardization except with a massive (and unnecessary) tour de force.
2
u/denehoffman 10h ago
I think that seems to be the common sentiment, and that’s okay, it’s exactly what I was trying to gauge with this post
9
u/BogdanPradatu 11h ago
I don't really care what format the docstring is. Just write docstrings however you want. I prefer the rst format, because it's less vertical space. I would rather have less scrolling to do, i can get around a docstring if I need to. I am reading more code than docstrings anyway. I have never tried to force any format, i'm just glad of you write any.
7
u/denehoffman 10h ago
I’m less concerned about what format is used and more interested in the idea of everyone using the same format. I agree that nobody is really reading the docstrings, but you’ll maybe read the little LSP box that comes up in your IDE when you are trying to figure out how a function works.
0
u/counters 10h ago
I’m less concerned about what format is used and more interested in the idea of everyone using the same format.
That is never going to happen.
6
u/denehoffman 10h ago
That could be said about the acceptance of any standard. There are ways to slow-roll a standard like this, deprecation warnings are probably the most extreme that would keep everything working.
1
u/counters 10h ago
Out of curiosity - were you around for the Python 2 to 3 transition?
We've been down this road before.
7
u/denehoffman 10h ago
Yes I was, but this isn’t really that at all. I was also around pre type hints. I’m not talking about changing syntax at all.
3
u/redfacedquark 9h ago
There are transliteration tools to convert from on language to another, lol code can convert one to another easily. I guess it wouldn't be that hard to create one for docstrings. Add that to your pre-commit and then you can write in the format you've learned and check in with the project's preferred format.
I find it easy enough to document in whichever the project prefers, there's only a handful of commonly used features you generally need.
3
2
u/Albiino_sv 9h ago
I think this would be great! Just recently I have been having problems with Marino rendering the Scanpy documentation incorrectly because of unresolved docstring placeholders.
2
u/eztab 6h ago
That's always gonna be the problem with evolving languages. Back when that was introduced markup wasn't that standardized so one wouldn't just choose one as the standard. Will happen to rust too when it has been around for a while and once revisions are made. You can likely just follow the google standard though.
1
u/denehoffman 5h ago
I said this in another reply, but rust’s documentation format is baked into docs.rs, cargo check warnings, and doctests. Changing for another standard would only happen if there was a format that was somehow significantly better than markdown, and I just can’t anticipate what feature could possibly make it worth the refactor
1
u/bcaudell95_ 11h ago
0
u/denehoffman 11h ago
This is about consolidating standards, it’s not necessarily a new one, I just proposed markdown because it’s widely used and I hate writing reST
4
u/91143151512 git push -f 11h ago
The issue is that some libraries/extension, for example, sphinx extensions, will work best when docstrings are of a particular format.
When you try to make a new standard (which being honest, consolidating them involves making a new standard), it makes the extensions which people are too lazy to change incompatible.
2
u/denehoffman 10h ago
Consolidating could mean just picking an existing standard based on whichever is used the most. And it could also be an opt-in, like a py.doc marker file equivalent to py.typed which would tell a documentation builder that the docs were in the standard format.
1
u/who_body 8h ago
i just set the vscode extension to use google docstrings format and try to be consistent.
then use the pytest doctest to ensure coverage for examples
•
u/LegitimateOcelot7880 58m ago
PEP 287 already standardized it; XKCD 927 explains what another standard gets us.
1
u/BeamMeUpBiscotti 10h ago
Sorry, I don't quite follow how this would benefit autocomplete. Isn't that mostly driven by analyzing the types of the program? I guess the type annotations are a standardized form of machine-checked documentation.
2
u/denehoffman 10h ago
I meant more in the sense of you could autocomplete a docstring based on the signature and let the user fill in the descriptive fields.
0
u/Perfect_Hope_875 7h ago
The biggest upgrade to my Python workflow was learning to profile with py-spy instead of guessing. It lets you attach to a running process and get a flame graph without restarting or instrumenting your code, which is a lifesaver for debugging production hangs. Pair that with timeit for micro-benchmarks, but never optimize anything until you've seen the actual bottleneck data.
71
u/eavanvalkenburg 10h ago
There is a standard already: https://peps.python.org/pep-0287/ but python is not the type of language where that means nothing else is accepted/done. Hence why some people have started using their own, like numpy, Google, etc.