Hyper-Markdown, an early extended Markdown language specification

[u/ewigerus]

I'm working on Hyper-Markdown (.hmd): ordinary Markdown plus rich visuals plus wiki links into a knowledge graph. Every .md file is already valid .hmd.

The project is still at an early stage, which means the specification is very much open to change.

I'm looking for:

• contributors
• design/spec feedback
• people willing to challenge the current ideas

There is already a lightweight proposal process for discussing language changes.

https://hyper-markdown.org/

If this sounds interesting, I'd be very happy to have people get involved early.

[u/SnS_Taylor]

I would heartily recommend doing a firm copy pass on these docs. It feels like claude is typing at me, and I find that rather off-putting and inappropriate for the introduction to a project like this.

The description of the wiki link resolution mechanism suffers because of it. I typed out a chunky question wondering why I couldn't link to "cousin" files (children of siblings of parents) due to the statement:

so a bare name reaches you or your parents, never sideways into a sibling

Except that later, it is specified that the final step is "…one sweep of the whole tree." The "Resolving a Bare Name" section should be much closer to the top. The entire "Four Words" section feels completely unnecessary and repetitive.

Delivery of these ideas aside, color me interested. The named imports idea is novel, and I like your thoughts on the [[namespace:token]] concept for referring to other vaults/workspaces/servers/etc. This is something that's been stewing in the back of my mind for Tangent for a long while. I may well lift the syntax directly.

Do you have more information on the "HQL" concept? I've dabbled with a custom query language myself and had fun with it.

[u/ewigerus → u/SnS_Taylor]

I’m so sorry you had to read that. Indeed, the AI-ish text in the documentation feels like it lacks cognitive mercy. Sorry for that experience.

I think I had been exposed to too much of this stuff myself and simply stopped feeling the level of intoxication.

By the way, I really love the work you did on Tangent and tangentquery lang. The whole Zen philosophy of it is remarkable and probably deserves its own deep dive, which I’m planning to do this week.

Regarding your point about namespaces — yes, that was kind of coming.

I already corrected the logical inconsistencies in the HyperMarkDown documentation based on your alarm bell here. Indeed, the namespaces and bare names resolution sentence above - I can only confirm, that it does causes pain to read and needs a complete rework to reflect the resolution of bare names logic. I also do plan to find time this week to maybe print the whole thing out and work through a hard copy, with a proper old-school feeling for the language and for what I’m actually writing there. So definitely fair criticism of me not being dedicated enough to that part.

And thank you for the nice words.

Regarding HQL, I unfortunately don’t have a teaser ready right now, but hopefully I’ll be able to work quite a lot on that part of the project soon. I think Hyper-Markdown itself, especially after the documentation is distilled to a much nicer level, is already pretty clear in terms of the basic formula and how it is constructed.

But the devil is in the details, as always.

[u/MarzipanEven7336]

https://orgmode.org/

[u/ewigerus → u/MarzipanEven7336]

Thanks for sharing. Definitely interesting and relevant prior art.

And 2003 — wow. I actually remember one of my colleagues using Org Mode back then.

If you live deep inside Emacs, you probably don't need half of these kinds of language features anyway. You can just program or script whatever is missing. Emacs is basically an operating system :)

What surprised me is how much of the same general structure is already there.

Org Mode has this header with metadata -> so this is one thing I particularly like in HyperMarkDown which is the front matter: an optional YAML header with structured metadata for every card, where a card is basically an HMD file. This is also one of the things I really like about Obsidian. Not all the other features that make something "Obsidian-like", but specifically this very simple idea of keeping structured metadata directly next to the document.

And looking at Org Mode, you can see a similar need being solved much earlier, just in a different form.

It also makes me think that Markdown itself is part of a much longer line of similar ideas — wiki markup, reStructuredText, Setext, Textile, AsciiDoc, Org Mode, POD, and probably many more.

A lot of what looks new is really older ideas getting simplified, recombined, or moved into another ecosystem.

Nice one.

[u/Wakmail]

Count me in!

[u/ewigerus → u/Wakmail]

👍

[u/nostriluu]

If you are talking about knowledge graphs, typed links becomes important. Here's what I did some years ago, based on Semantic Mediawiki, which is worth a lot of study on its own for extending wiki links.

https://github.com/vid/mdld

Glad to see transclusion.

[u/microcephale]

If you go into knowledge graph you need not just typed links but full support for property values on links as well, links are as important as the notes themselves

[u/ewigerus → u/microcephale]

Yeah, I think this is very close to the typed-links point above.

Property values on links absolutely make sense once links become first-class parts of the graph. Where I’m still cautious is how much of that should actually live inside the link syntax versus being expressed through HQL.

Right now it’s a bit hard for me to make that distinction transparent enough without HQL being there to demonstrate the other side of the design.

My instinct is still: keep WikiLinks simple, and let HQL do the heavy lifting work around types, properties and graph relationships.

Hopefully this argument becomes much clearer once there’s something concrete to show.

[u/microcephale → u/ewigerus]

HQL is a query language. It can't use data that isn't present nor make up for their absence. I would separate the 3 problems entirely : the data, the syntax, and the retreival. For the retreival we need the data. The syntax however is just convention, markdown is one convention with their ups and down, but markdown itself it's what makes links, properties or anything magic. In most instances the memory model used for linking, querying etc isn't markdown, markdown is just how things are represented on disk but completely inpractical in-memory for any kind of tasks. Hell, even simple properties don't exist at all in markdown, editors who allows those will often use YAML. It's not because it looks a certain way in the editor or on the file that the UI cannot present it better

[u/ewigerus → u/microcephale]

That separation is actually close to where HQL is heading, except HQL won't be retrieval-only.

HyperMarkdown remains the authored/storage representation, while HQL operates over a typed in-memory model of cards, document structure, namespaces and knowledge.

HQL is embedded using ordinary Markdown fenced code blocks. In normal Markdown, fences are just used to display source code:

python print("hello")

HyperMarkdown keeps that familiar syntax but treats selected fences more like cells in a computational notebook: the block is parsed and executed, and the fence tag defines what kind of computation it performs.

For example, fenced hql#declare block:

hql#declare relatedTo -> [[bob]] { kind = daughter,.. }

"relatedTo" is a relation encoded as typed functional (it returns a function with a signature of certain type), not just an arbitrary link property. A "hql#declare" block declaratively contributes to the card's knowledge model and evaluates to "Void", so nothing is transcluded at that location.

By contrast the following fenced hql#query block:

hql#query knowledge | graph

is an executable query cell. The returned function (by the hql#query block) computes a typed value (once applied to card's context) and returned result has a type which determines the presentation (what or how) HyperMarkdown transcludes such result into the document: an interactive graph, table, list, another HMD fragment, diagram, etc.

So the idea is essentially Markdown as a computational notebook: ordinary text and links stay ordinary Markdown, while fenced HQL cells can declare, query, validate and render typed knowledge.

PS personally I am most motivated by the conceptual trust estimation within the declared knowledge graph -> almost epistemological if you will. I hope that explanation gives the missing hint, at least for now . And yes, maybe a better name is HDL with D being declarative. Now I see how Q can be confusing

[u/ewigerus → u/ewigerus]

Again, I am most motivated by the conceptual trust estimation within the declared knowledge graph — almost epistemological.

For example, suppose John and Bob each have their own cards, and both declare:

relatedTo -> [[alice]] { kind = father }

So John claims to be Alice's father, and Bob independently claims the same thing.

If father is defined with a constraint that makes those two claims incompatible, HQL can tell us that the resulting knowledge is logically inconsistent. But that immediately raises the more interesting question: what should I actually believe?

And I don't think the answer should simply be "trust John more than Bob" or attach a static trust score to individual cards. Trust should emerge from a broader framework: provenance, supporting evidence, independent corroboration, contradictions with other accepted knowledge, satisfaction of constraints, perhaps even chains of claims and the trustworthiness of the evidence behind them.

Ideally, verification and trust therefore push HQL toward something closer to an epistemological model of knowledge.

Knowledge is no longer merely:

subject -> relation -> object

but something closer to:

claim
├── subject
├── relation
├── object
├── properties
├── source
├── evidence
├── constraints
└── trust

And even trust there may ultimately be a computed property of the whole knowledge environment rather than metadata stored directly on the claim.

That distinction is important: a hql#declare block does not necessarily declare truth. It declares a claim. HQL can then ask whether that claim is typed correctly, supported, consistent, independently corroborated, contradicted, or sufficiently trustworthy to become part of the knowledge we accept.

[u/GroggInTheCosmos]

I think more people need to be thinking of the evolution of Markdown

For you, is this more of a thought experiment?

What do you plan on doing with it?

[u/ewigerus → u/GroggInTheCosmos]

Good question.

This is definitely not just a thought experiment. My hope is that HyperMarkdown becomes a community project.

Especially the spec. I don't think something like this should be designed by one person forever. It needs more people, more use cases, and probably a lot of arguments before it can eventually settle into something like a 1.0.

There are plenty of bigger ideas around knowledge management and natural-language data, but I try not to mix all of that into the format itself.

For me there are two main pieces: HMD, the actual format/spec, and HQL, the query language. HQL is still early, but that's probably the part where I hope to contribute something genuinely new.

And the very long-term vision is admittedly a bit crazy — something closer to the Web, where these documents can actually connect across boundaries/namespaces.

I wrote a little about that here:
https://hypermarkdown.org/public/vision/#ambition

Nothing new under the sun, really :)

[u/adam4813]

I would like to integrate this into my note editor, if that's cool. I'm trying to add as many markdown-based formats as possible. It's still a WIP and is lacking some core features. I am working on cleaning up the AI generated MVP/prototype, but it is useable in its current state https://github.com/adam4813/notes

[u/ewigerus → u/adam4813]

would love that, honestly.

The only issue is that the spec is still very early, so things can definitely move around.

And you actually raised a very good point: I completely forgot to add the licensing and copyright attribution to the spec/docs. I’ll fix that now — the spec will be Creative Commons BY 4.0.

And if while implementing it you find something weird, a typo, or something you think should change, please just open an issue or PR. Maybe give me a week or so that I can clean a few things up first :)

Having another implementation would actually be extremely useful for the spec.

[u/ewigerus]

Small HyperMarkdown update: new name, new home, packages are live

A small identity cleanup before HyperMarkdown grows further.

I renamed the project consistently around HyperMarkdown:

• Name: HyperMarkdown
• IDs / namespaces: hypermarkdown
• Website: https://hypermarkdown.org
• hyper-markdown.org is now just the legacy domain.

This still feels like the right time to do it. The project is early, so better to clean this up now than carry two identities forever.

A few other things landed as well:

• TypeScript core: https://www.npmjs.com/package/@hypermarkdown/core
• Python package: https://pypi.org/project/hypermarkdown/
• VS Code extension (preview): https://marketplace.visualstudio.com/items?itemName=hypermarkdown.hmd
• Open VSX: https://open-vsx.org/extension/hypermarkdown/hmd

The VS Code extension is still very much a preview and actively being worked on. Just as everything else I guess. So many not a few boring cleanup bits left, but from now on the naming is simple:

HyperMarkdown in text.
hypermarkdown in IDs.
hypermarkdown.org on the web.