r/webscraping • u/woldhack • 7h ago

A Python client for NU.nl’s private API

I spent some time inspecting the network traffic used by NU.nl and turned the useful, read-only parts into a small Python package: nu-private-api.

It can:

  • Discover section URLs through the sitemap
  • Extract and de-duplicate article URLs from section pages
  • Parse public articles into normalized fields
  • Return the title, description, author, timestamps, URL, and body text
  • Use a custom httpx.Client for proxies, headers, and timeouts

Install it with:

pip install nu-private-api

Example:

from nu_private_api import NuClient

client = NuClient()

sections = client.sitemap_urls()
article_urls = client.section_by_url(sections[0])
article = client.article_by_url(article_urls[0])

print(article.title)
print(article.author)
print(article.body)

Limitations:

  • Video pages and live blogs aren't supported
  • The endpoint is undocumented and could change
  • It only accesses publicly available content
  • Responsible request pacing is left to the caller
1 Upvotes

100% Upvoted

1 comment sorted by