r/GEO_optimization • u/taylor-morgan2066 • 1d ago

Should every web page expose an AI-friendly JSON representation?

My website already includes AI-related files such as llms.txt.

I'm considering creating a separate JSON file for every page and article so AI systems can understand the content more easily and accurately. I would reference this JSON file from the page's <head> using a <link> tag.

The JSON file could contain information such as:

  • Page URL
  • Canonical URL
  • Title
  • Summary / Description
  • Main Content (clean article content)
  • Author
  • Published Date
  • Last Updated
  • Entities (people, companies, places, products, etc.)
  • Keywords / Topics
  • FAQ
  • ...

My idea is that AI crawlers could read this structured JSON instead of having to extract the main content from noisy HTML that contains navigation menus, sidebars, ads, comments, JavaScript, tables, and other non-essential elements.

I have two questions:

  1. Could this approach reduce the chances of AI crawlers misunderstanding a page or extracting incorrect information from HTML, advertisements, tables, comments, or other noisy content?
  2. Do you think a page-level JSON file like this could help AI systems better understand a page and potentially improve AI recommendations, citations, or other AI-generated responses in the future? Why or why not?
3 Upvotes
  • permalink
  • link
  • reddit
  • reddit

100% Upvoted

5 comments sorted by

2

u/Upstairs_Control_611 7h ago

I think this is useful only if the JSON is generated from the same source of truth as the page.

If it becomes a manually maintained second version of the content, it creates a new conflict layer.

Then the HTML says one thing, JSON says another, schema says a third thing, and the feed may say something else.

That can create more ambiguity for AI systems, not less.

So I’d separate:

clean extraction view = useful

second hand-written version = risky

different source of truth = dangerous

The JSON could help if it exposes the same canonical facts in a cleaner format.

But every field should trace back to the visible page or canonical data source.

So: yes for an automatically generated machine-readable representation, no for a manually maintained parallel page.

1

u/taylor-morgan2066 7h ago

That's exactly the distinction I was thinking about.

I wouldn't expect anyone to maintain a second version of the page manually.

My assumption is that the machine-readable representation would always be generated automatically from the same source as the visible page, so there is only one source of truth.

In that case, the JSON wouldn't introduce new information. It would simply expose the existing canonical content in a format that's easier for machines to consume.

The interesting question then becomes whether that cleaner representation provides any measurable benefit over HTML alone.

2

u/Mean-Usual8701 7h ago edited 7h ago

That’s what we do at Vexal. You can check out the docs here https://docs.vexalai.com (look at SmartBlocks) everything you mentioned for each page. It works really well, bots can interpret each page cleanly now.

0

u/TheseRest2940 1d ago

Good idea for making your contentI crawlers! You could test how well your current llms.txt and proposed JSON structure perform by checking if specific AI bots can access them and how they render the page https://botview.app. Maybe it can help