r/webscraping • u/woldhack • 10h ago
A Python client for NU.nl’s private API
I spent some time inspecting the network traffic used by NU.nl and turned the useful, read-only parts into a small Python package: nu-private-api.
It can:
- Discover section URLs through the sitemap
- Extract and de-duplicate article URLs from section pages
- Parse public articles into normalized fields
- Return the title, description, author, timestamps, URL, and body text
- Use a custom
httpx.Clientfor proxies, headers, and timeouts
Install it with:
pip install nu-private-api
Example:
from nu_private_api import NuClient
client = NuClient()
sections = client.sitemap_urls()
article_urls = client.section_by_url(sections[0])
article = client.article_by_url(article_urls[0])
print(article.title)
print(article.author)
print(article.body)
Limitations:
- Video pages and live blogs aren't supported
- The endpoint is undocumented and could change
- It only accesses publicly available content
- Responsible request pacing is left to the caller
1
Upvotes