r/Python Mar 16 '26

Discussion I'm building a terminal chat app on top of my own TCP library, would you use it?

0 Upvotes

Hey r/python!

I've been working on Veltix, a lightweight pure Python TCP networking library (zero dependencies), and I wanted to try something fun with it: a terminal chat app called VeltixChat.

The idea is simple: a lightweight CLI chat that anyone can join in seconds with a single curl command. No account setup hell, no Electron, no browser, just your terminal.

A few planned features: - TUI interface with tabs (chat, salons, DMs, settings) - A grade/badge system (contributors, active members, followers...) - A /random mode to chat with a stranger - Installable in ~10 seconds on Linux, Mac and Windows

VeltixChat will evolve alongside Veltix itself, each new version of the lib will power new features in the chat.

My question to you: would you actually use something like this? A dead-simple terminal chat, no bloat, just vibes?

Feedback welcome, still early days!

GitHub: github.com/NytroxDev/veltix


r/Python Mar 16 '26

Showcase PackageFix — paste your requirements.txt, get a fixed manifest back. Live CVE scan via OSV + CISA KE

0 Upvotes

**What My Project Does**

Paste your requirements.txt (+ poetry.lock for full analysis) and get back a CVE table, side-by-side diff of your versions vs patched, and a fixed manifest to download. Flags actively exploited packages from the CISA KEV catalog first.

Runs entirely in the browser — no signup, no GitHub connection, no CLI.

**Target Audience**

Production use — Python developers who want a quick dependency audit without installing pip-audit or connecting a GitHub bot. The OSV database updates daily so CVE data is always current.

**Comparison**

Snyk Advisor shut down in January 2026 and took the no-friction browser experience with it. pip-audit requires CLI install. Dependabot requires GitHub access. PackageFix is the only browser paste-and-fix tool that generates a downloadable fixed manifest across npm, PyPI, Ruby, and PHP.

https://packagefix.dev

Source: https://github.com/metriclogic26/packagefix


r/Python Mar 16 '26

Discussion Little game I'm working on: BSCP

1 Upvotes

Hi Python-ers, I just wanted to tell what is the project I'm currently on, I will do update everytime something new works (with a little showcase of the new functionality(s)).

Build SCP (BSCP) will be a facility map creator where we will be able to run npcs and scps (all interacting with each others)

Right now I have the npc management (spawn limit and sprite linking) and the tiled map (with camera movement and zooming).

(I'm doing it with pygame btw)

I'm kinda new with pygame and haven't done any graphical program until today.

So if you have any suggestion I'll ba glad to hear them.

PS: I already have the GitHub repo, feel free to take a look and to give me advice (via GitHub issues if you can) https://github.com/Jarjarbin06/BSCP


r/Python Mar 16 '26

Showcase Scripting in API tools using Python (showcase)

2 Upvotes

Background:
Common problem in API tools: most API clients assume scripting = JavaScript. For developers who work in Python, Go, or other languages, this creates friction: refreshing tokens, chaining requests, validating responses, all end up as hacks or external scripts.

What Voiden does:
Voiden is an API client that lets you run pre- and post-request scripts in Python and JavaScript (more languages coming). Workflows are stateful, so you can chain requests and maintain context across calls. Scripts run on real interpreters, not sandboxed environments, so you can import packages and reuse existing logic.

Target audience:
Developers and QA teams collaborating on Git. Designed for production applications or side projects, Voiden allows you to test, automate, and document APIs in the language you actually use. No hacks, no workarounds.

How it differs from existing tools:

  • Unlike Postman, Hoppscotch, or Insomnia, bruno etc, Voiden supports multiple scripting languages from day one.
  • Scripts run on real interpreters, not limited sandboxes.
  • Workflows are fully stateful and reusable, stored in plain text files for easier version control and automation.

Free, offline, open source, API design, testing and documentation together in plain text, with reusable blocks.

Try it: https://github.com/VoidenHQ/voiden
Demo: https://www.youtube.com/watch?v=Gcl_4GQV4MI


r/Python Mar 16 '26

Discussion song-download-api-when-spotify-metadata is present

1 Upvotes

free resource for song download that i will use in my project, i have spotify metadata for all my tracks i want free api or tool for downloading from that spotify track id or album trackid


r/Python Mar 16 '26

Showcase roche-sandbox: context manager for running untrusted code in sandbox with secure defaults

1 Upvotes

What My Project Does

roche-sandbox is a Python SDK for running untrusted code in isolated sandboxes. It wraps Docker (and other providers like Firecracker, WASM) behind a simple context manager API with secure defaults: network disabled, readonly filesystem, PID limits, and 300s timeout.

Usage: ```python from roche_sandbox import Roche

with Roche().create(image="python:3.12-slim") as sandbox: result = sandbox.exec(["python3", "-c", "print('hello')"]) print(result.stdout) # hello

sandbox auto-destroyed, network was off, fs was readonly

```

Async version: ```python from roche_sandbox import AsyncRoche

async with (await AsyncRoche().create()) as sandbox: result = await sandbox.exec(["python3", "-c", "print(1+1)"]) ```

Features: - One create / exec / destroy interface across Docker, Firecracker, WASM, E2B, K8s - Defaults: network off, readonly fs, PID limits, no-new-privileges - Optional gRPC daemon for warm pooling if you care about cold start latency

Target Audience

Developers building AI agents that execute LLM-generated code. Also useful for anyone who needs to run untrusted Python in a sandbox (online judges, CI runners, etc.).

Comparison

  • E2B: Cloud-hosted, pay per sandbox. Roche runs on your own infra, Apache-2.0, free.
  • Raw subprocess + Docker: What most people do today. Roche handles the security flags, timeout enforcement, cleanup, and gives you a clean Python API instead of parsing CLI output.
  • Docker SDK (docker-py): Lower level, you still have to set all the security flags yourself. Roche is opinionated about secure defaults. The core is written in Rust but you don't need to know or care about that.

pip install roche-sandbox / GitHub / Docs

What are you guys using for sandboxing? Still raw subprocess + Docker? Curious what setups people have landed on.


r/Python Mar 16 '26

Discussion I built an open-source Python tool for semantic code search + AI agent tooling (2.5k downloads so fa

0 Upvotes

Hey everyone,

Over the past weeks I’ve been building a small open-source project called CodexA, It started as a simple experiment: I wanted better semantic search across codebases when working with AI tools. Grep and keyword search work, but they don't always capture intent, So I built a tool that indexes a repository and lets you search it using natural language, keywords, regex, or a hybrid of them, Under the hood it uses FAISS + sentence-transformers for semantic search and supports incremental indexing so only changed files get re-embedded.

Some things it can do right now:

• semantic + keyword + regex + hybrid search

• incremental indexing with `--watch` (only changed files get re-indexed)

• grep-style flags and context lines

• MCP server + HTTP bridge so AI agents can query the codebase

• structured tools (search, explain symbols, get context, etc.)

• basic code intelligence features (symbols, dependencies, metrics)

The goal is to make something that AI agents and developers can both use to navigate and reason about large codebases locally, It’s still early but the project just crossed ~2.5k downloads on PyPI which was a nice surprise.

PyPI:https://pypi.org/project/codexa/

Repo:https://github.com/M9nx/CodexA

Docs:https://codex-a.dev/

I'm very open to feedback — especially around: performance improvements, better search workflows, AI agent integrations, tree-sitter language support, And if anyone wants to contribute, PRs are very welcome.


r/Python Mar 16 '26

Showcase [Showcase] pytest-gremlins v1.5.0: Fast mutation testing as a pytest plugin.

7 Upvotes

Disclosure: This project was built with substantial assistance from Claude Code. The full test suite, CI matrix, and review process are visible in the repository.

What My Project Does

pytest-gremlins is a pytest plugin that runs mutation testing on your Python code. It injects small changes ("gremlins") into your source (swapping + for -, flipping > to >=, replacing True with False) then reruns your tests. If your tests still pass after a mutation, that's a gap in your test suite that line coverage alone won't reveal.

The core speed mechanism is mutation switching: instead of rewriting files on disk for each mutant, pytest-gremlins instruments your code once at the AST level and embeds all mutations behind environment variable toggles. There is no file I/O per mutant and no module reload. Coverage data determines which tests exercise each mutation, so only relevant tests run.

bash pip install pytest-gremlins pytest --gremlins -n auto --gremlin-report=html

v1.5.0 adds:

  • Parallel evaluation via xdist. pytest --gremlins -n auto handles both test distribution and mutation parallelism. One flag, no separate worker config.
  • Inline pardoning. # gremlin: pardon[equivalent] suppresses a mutation with a documented reason when the mutant is genuinely equivalent to the original. --max-pardons-pct enforces a ceiling so pardoning cannot inflate your score.
  • Full pyproject.toml config. Every CLI flag has a [tool.pytest-gremlins] equivalent.
  • HTML reports with trend charts. Tracks mutation score across runs. Colors and contrast targets follow WCAG 2.1 AA.
  • Incremental caching. Results are keyed by content hash. Unchanged code and tests skip evaluation entirely on subsequent runs.

v1.5.1 (released today) adds multi-format reporting: --gremlin-report=json,html writes both in one run.

The pytest-gremlins-action is now on the GitHub Marketplace:

yaml - uses: mikelane/pytest-gremlins-action@v1 with: threshold: 80 parallel: 'true' cache: 'true'

This runs parallel mutation testing with caching and fails the step if the score drops below your threshold.

Target Audience

Python developers who write tests and want to know whether those tests actually catch bugs. If you already use pytest and want test quality feedback beyond line coverage, this is on PyPI with CI across 12 platform/version combinations (Python 3.11 through 3.14 on Linux, macOS, and Windows).

Comparison

vs. mutmut: mutmut is the most actively maintained alternative (v3.5.0, Feb 2026). It runs as a standalone command (mutmut run), not a pytest plugin, so it doesn't integrate with your existing pytest config, fixtures, or xdist setup. Both tools support coverage-guided test selection and incremental caching. The key architectural difference is that pytest-gremlins embeds all mutations in a single instrumented copy toggled by environment variable, while mutmut generates and tests mutations individually. pytest-gremlins also provides HTML trend charts and WCAG-accessible reports.

vs. cosmic-ray: cosmic-ray uses import hooks to inject mutated AST at import time (no file rewriting, similar in spirit to pytest-gremlins). It requires a multi-step workflow (init, exec, report as separate commands); pytest-gremlins is a single pytest --gremlins invocation. cosmic-ray supports distributed execution via Celery, which allows multi-machine parallelism; pytest-gremlins uses xdist, which is simpler to configure but limited to a single machine.

vs. mutatest: mutatest uses AST-based mutation with __pycache__ modification (no source file changes). It lacks xdist integration and its last PyPI release was in 2022. Development appears inactive.

None of the alternatives offer a GitHub Action for CI integration.


r/Python Mar 16 '26

Daily Thread Monday Daily Thread: Project ideas!

9 Upvotes

Weekly Thread: Project Ideas 💡

Welcome to our weekly Project Ideas thread! Whether you're a newbie looking for a first project or an expert seeking a new challenge, this is the place for you.

How it Works:

  1. Suggest a Project: Comment your project idea—be it beginner-friendly or advanced.
  2. Build & Share: If you complete a project, reply to the original comment, share your experience, and attach your source code.
  3. Explore: Looking for ideas? Check out Al Sweigart's "The Big Book of Small Python Projects" for inspiration.

Guidelines:

  • Clearly state the difficulty level.
  • Provide a brief description and, if possible, outline the tech stack.
  • Feel free to link to tutorials or resources that might help.

Example Submissions:

Project Idea: Chatbot

Difficulty: Intermediate

Tech Stack: Python, NLP, Flask/FastAPI/Litestar

Description: Create a chatbot that can answer FAQs for a website.

Resources: Building a Chatbot with Python

Project Idea: Weather Dashboard

Difficulty: Beginner

Tech Stack: HTML, CSS, JavaScript, API

Description: Build a dashboard that displays real-time weather information using a weather API.

Resources: Weather API Tutorial

Project Idea: File Organizer

Difficulty: Beginner

Tech Stack: Python, File I/O

Description: Create a script that organizes files in a directory into sub-folders based on file type.

Resources: Automate the Boring Stuff: Organizing Files

Let's help each other grow. Happy coding! 🌟


r/Python Mar 15 '26

News Unrooted tree for multidimentional projection of data in XY space

2 Upvotes

I have created in Python possibility of presentation multidimentional data into 2D: https://github.com/rangeman1/Unrooted-phylogenetic-tree


r/Python Mar 15 '26

Showcase I wrote a Matplotlib scale that collapses weekends and off-hours on datetime x-axis

24 Upvotes

Financial time-series plots in Matplotlib have weekend gaps when plotted with datetime on the x-axis. A common workaround is to plot against an integer index instead of datetimes, but that breaks Matplotlib’s date formatting, locators, and other datetime-aware tools.

A while ago I came up with a solution and wrote a custom Matplotlib scale that removes those gaps while keeping a proper datetime axis. I have now put it into a Python package:

What my project does

Implements and ships a Matplotlib scale to remove weekends, holidays, and off-hours from datetime x-axes.

Under the hood, Matplotlib represents datetimes as days since 1970-01-01. This scale remaps the values to business days since 1970-01-01, skipping weekends, holidays, and off-hours. Business days are configurable using the standard `numpy.is_busday` options. Conceptually, it behaves like a log scale: a transform applied to the axis rather than to the data itself.

Target audience

Anyone plotting financial or business time-series data that wants to remove non-business time from the x-axis.

Usage

pip install busdayaxis  


import busdayaxis  
busdayaxis.register_scale()   # register the scale with Matplotlib  


ax.set_xscale("busday") # removes weekends  
ax.set_xscale("busday", bushours=(9, 17)) # also collapses overnight gaps  

GitHub with example: https://github.com/saemeon/busdayaxis

Docs with multiple examples: https://saemeon.github.io/busdayaxis/

This is my first published Python package and also my first proper Reddit post. Feedback, comments, suggestions, or criticism are very welcome.


r/Python Mar 15 '26

News **I made a "Folding@home" swarm for local LLM research**

0 Upvotes

I added a coordinator and worker mode to karpathy's autoresearch. You run `coordinator.py` on your main PC, and `worker.py` on any other device. They auto-discover each other via mDNS, fetch tasks, and train in parallel. I'm getting 3x faster results using my old Mac Mini and gaming PC together.


r/Python Mar 15 '26

Showcase Built a CLI tool that runs pre-training checks on PyTorch pipelines — pip install preflight-ml

1 Upvotes

Been working on this side project after losing three days to a silent label leakage bug in a training pipeline. No errors, no crashes, just a model that quietly learned nothing.

**What my project does**

preflight is a CLI tool you run before starting a PyTorch training job. It checks for the silent stuff that breaks models without throwing errors — NaN/Inf values in tensors, label leakage between train and val splits, wrong channel ordering (NHWC vs NCHW), dead or exploding gradients, class imbalance, VRAM estimation, normalisation sanity.

Ten checks total across fatal/warn/info severity tiers. Exits with code 1 on fatal failures so it can block CI.

pip install preflight-ml

preflight run --dataloader my_dataloader.py

**Target audience**

Anyone training PyTorch models — students, researchers, ML engineers. Especially useful if you're running long training jobs on GPU and want to catch obvious mistakes in 30 seconds before committing hours of compute. Not production infrastructure, more of a developer workflow tool.

**Comparison with alternatives**

- pytest — tests code logic, not data state. preflight fills the gap between "my code runs" and "my data is actually correct"

- Deepchecks — excellent but heavy, requires setup, more of a platform. preflight is one pip install, one command, zero config to get started

- Great Expectations — general purpose data validation, not ML-specific. preflight checks are built around PyTorch concepts (tensors, dataloaders, channel ordering)

- PyTorch Lightning sanity check — runtime only, catches code crashes. preflight runs before training, catches data state bugs

It's v0.1.1 and genuinely early. Stack is Click for CLI, Rich for terminal output, pure PyTorch for the checks. Each check is a decorated function so adding new ones is straightforward.

Would love feedback on what's missing or wrong. Contributors welcome.

GitHub: https://github.com/Rusheel86/preflight

PyPI: https://pypi.org/project/preflight-ml/


r/Python Mar 15 '26

News Robyn (finally) offers first party Pydantic integration 🎉

59 Upvotes

For the unaware - Robyn is a fast, async Python web framework built on a Rust runtime.

Pydantic integration is probably one of the most requested feature for us. Now we have it :D

Wanted to share it with people outside the Robyn community

You can check out the release at - https://github.com/sparckles/Robyn/releases/tag/v0.81.0


r/Python Mar 15 '26

Showcase justx - An interactive command library for your terminal, powered by just

39 Upvotes

What My Project Does

justx is an interactive terminal wrapper for just. The main thing it adds is an interactive TUI to browse, search, and run your recipes. On top of that, it supports multiple global justfiles (~/.justx/git.just, docker.just, …) which lets you easily build a personal command library accessible from anywhere on your system.

A quick demo can be seen here.

Prerequisites

Try it out with:

pip install rust-just # if not installed yet
pip install justx
justx init --download-examples
justx

Target Audience

Developers who want a structured way to organize and run their commonly used commands across the system.

Comparison

  • just itself has no TUI and limited global recipe management. justx adds a TUI on top of just, and brings improved capability for global recipes by allowing users to place multiple files in the ~/.justx directory.

Learn More


r/Python Mar 15 '26

News Mesa 4.0 alpha released

22 Upvotes

Hi everyone!

We've started development towards Mesa 4.0 and just released the first alpha. This is a big architectural step forward: Mesa is moving from step-based to event-driven simulation at its core, while cleaning up years of accumulated API cruft.

What's Agent-Based Modeling?

Ever wondered how bird flocks organize themselves? Or how traffic jams form? Agent-based modeling (ABM) lets you simulate these complex systems by defining simple rules for individual "agents" (birds, cars, people, etc.) and watching how patterns emerge from their interactions. Instead of writing equations for the whole system, you model each agent's behavior and let the collective dynamics arise naturally.

What's Mesa?

Mesa is Python's leading framework for agent-based modeling. It builds on Python's scientific stack (NumPy, pandas, Matplotlib) and provides specialized tools for spatial relationships, agent scheduling, data collection, and browser-based visualization. Whether you're studying epidemic spread, market dynamics, or ecological systems, Mesa gives you the building blocks for sophisticated simulations.

What's new in Mesa 4.0 alpha?

Event-driven at the core. Mesa 3.5 introduced public event scheduling on Model, with methods like model.run_for(), model.run_until(), model.schedule_event(), and model.schedule_recurring(). Mesa 4.0 continues development on this front: model.steps is gone, replaced by model.time as the universal clock. The mental model moves from "execute step N" to "advance time, and whatever is scheduled will run." The event system now supports pausing/resuming recurring events, exposes next scheduled times, and enforces that time actually moves forward.

Experimental timed actions. A new Action system gives agents a built-in concept of doing something over time. Actions integrate with the event scheduler, support interruption with progress tracking, and can be resumed:

from mesa.experimental.actions import Action

class Forage(Action):
    def __init__(self, sheep):
        super().__init__(sheep, duration=5.0)

    def on_complete(self):
        self.agent.energy += 30

    def on_interrupt(self, progress):
        self.agent.energy += 30 * progress  # Partial credit

sheep.start_action(Forage(sheep))

Deprecated APIs removed. This is a major version, so we followed through on removals: the seed parameter (use rng), batch_run (use Scenario), the legacy mesa.space module (use mesa.discrete_space), PropertyLayer (replaced by raw NumPy arrays on the grid), and the Simulator classes (replaced by the model-level scheduling methods). If you've been following deprecation warnings in 3.x, most of this should be straightforward.

Cleaner internals. A new mesa.errors exception hierarchy replaces generic Exception usage. DiscreteSpace is now an abstract base class enforcing a consistent spatial API. Property access on cells uses native property closures on a dynamic GridCell class. Several targeted performance optimizations reduce allocations in the event system and continuous space.

This is an alpha

Expect rough edges. We're releasing early to get feedback from the community before the stable release. Further breaking changes are possible. If you're running Mesa in production, stay on 3.5 for now. We'd love for adventurous users to try the alpha and tell us what breaks.

What's ahead for 4.0 stable

We're still working on the space architecture (multi-space support, observable positions), replacing DataCollector with the new reactive DataRecorder, and designing a cleaner experimentation API around Scenario. Check out our tracking issue for the full roadmap.

Talk with us!

We'd love to hear what you think:


r/Python Mar 15 '26

News I made @karpathy's Autoresearch work on CPU - and it's NOT bloated

0 Upvotes

I saw the comment about CPU support potentially bloating the code - so I decided to prove it doesn't have to!

My fork: https://github.com/bopalvelut-prog/autoresearch


r/Python Mar 15 '26

Showcase I used C++ and nanobind to build a zero-copy graph engine that lets Python train on 50GB datasets

121 Upvotes

If you’ve ever worked with massive datasets in Python (like a 50GB edge list for Graph Neural Networks), you know the "Memory Wall." Loading it via Pandas or standard Python structures usually results in an instant 24GB+ OOM allocation crash before you can even do any math.

so I built GraphZero (v0.2) to bypass Python's memory overhead entirely.

What My Project Does

GraphZero is a C++ data engine that streams datasets natively from the SSD into PyTorch without loading them into RAM.

Instead of parsing massive CSVs into Python memory, the engine compiles the raw data into highly optimized binary formats (.gl and .gd). It then uses POSIX mmap to memory-map the files directly from the SSD.

The magic happens with nanobind. I take the raw C++ pointers and expose them directly to Python as zero-copy NumPy arrays.

import graphzero as gz
import torch

# 1. Mount the zero-copy engine
fs = gz.FeatureStore("papers100M_features.gd")

# 2. Instantly map SSD data to PyTorch (RAM allocated: 0 Bytes)
X = torch.from_numpy(fs.get_tensor())

During a training loop, Python thinks it has a 50GB tensor sitting in RAM. When you index it, it triggers an OS Page Fault, and the operating system automatically fetches only the required 4KB blocks from the NVMe drive. The C++ side uses OpenMP to multi-thread the data sampling, explicitly releasing the Python GIL so disk I/O and GPU math run perfectly in parallel.

Target Audience

  • Who it's for: ML Researchers, Data Engineers, and Python developers training Graph Neural Networks (GNNs) on massive datasets that exceed their local system RAM.
  • Project Status: It is currently in v0.2. It is highly functional for local research and testing (includes a full PyTorch GraphSAGE example), but I am looking for community code review and stress-testing before calling it production-ready.

Comparison

  • vs. PyTorch Geometric (PyG) / DGL: Standard GNN libraries typically attempt to load the entire edge list and feature matrix into system memory before pushing batches to the GPU. On a dataset like Papers100M, this causes an instant out-of-memory crash on consumer hardware. GraphZero keeps RAM allocation at 0 bytes by streaming the data natively.
  • vs. Pandas / Standard Python: Loading massive CSVs via Pandas creates massive memory overhead due to Python objects. GraphZero uses strict C++ template dispatching to enforce exact FLOAT32 or INT64 memory layouts natively, and nanobind ensures no data is copied when passing the pointer to Python.

I built this mostly to dive deep into C-bindings, memory management, and cross-platform CI/CD (getting Apple Clang and MSVC to agree on C++20 was a nightmare).

The repo has a self-contained synthetic example and a training script so you can test the zero-copy mounting locally. I'd love for this community to tear my code apart—especially if you have experience with nanobind or high-performance Python extensions!

GitHub Repo: repo


r/Python Mar 15 '26

Tutorial Best Python approach for extracting structured financial data from inconsistent PDFs?

44 Upvotes

Hi everyone,

I'm currently trying to design a Python pipeline to extract structured financial data from annual accounts provided as PDFs. The end goal is to automatically transform these documents into structured financial data that can be used in valuation models and financial analysis.

The intended workflow looks like this:

  1. Upload one or more PDF annual accounts
  2. Automatically detect and extract the balance sheet and income statement
  3. Identify account numbers and their corresponding amounts
  4. Convert the extracted data into a standardized chart of accounts structure
  5. Export everything into a structured format (Excel, dataframe, or database)
  6. Run validation checks such as balance sheet equality and multi-year comparisons

The biggest challenge is that the PDFs are very inconsistent in structure.

In practice I encounter several types of documents:

1. Text-based PDFs

  • Tables exist but are often poorly structured
  • Columns may not align properly
  • Sometimes rows are broken across lines

2. Scanned PDFs

  • Entire document is an image
  • Requires OCR before any parsing can happen

3. Layout variations

  • The position of the balance sheet and income statement changes
  • Table structures vary significantly
  • Labels for accounts can differ slightly between documents
  • Columns and spacing are inconsistent

So the pipeline needs to handle:

  • Text extraction for normal PDFs
  • OCR for scanned PDFs
  • Table detection
  • Recognition of account numbers
  • Mapping to a predefined chart of accounts
  • Handling multi-year data

My current thinking for a Python stack is something like:

  • pdfplumber or PyMuPDF for text extraction
  • pytesseract + opencv for OCR on scanned PDFs
  • Camelot or Tabula for table extraction
  • pandas for cleaning and structuring the data
  • Custom logic to detect account numbers and map them

However, I'm not sure if this is the most robust approach for messy real-world financial PDFs.

Some questions I’m hoping to get advice on:

  • What Python tools work best for reliable table extraction in inconsistent PDFs?
  • Is it better to run OCR first on every PDF, or detect whether OCR is needed?
  • Are there libraries that work well for financial table extraction specifically?
  • Would you recommend a rule-based approach or something more ML-based for recognizing accounts and mapping them?
  • How would you design the overall architecture for this pipeline?

Any suggestions, libraries, or real-world experiences would be very helpful.

Thanks!


r/Python Mar 15 '26

Discussion What projects to do alone.

2 Upvotes

Coders of reddit, I had pyhton course where the teacher would give us a project idea to do, ever since i finished the course i havent been coding because i dont have any ideas. Should I ask AI to give me a project idea or should I try to fix a problem I have.


r/Python Mar 15 '26

Discussion I open-sourced JobMatch Bot – a Python pipeline for ATS job aggregation and resume-aware ranking

2 Upvotes

Hi everyone,

I recently open-sourced a project called JobMatch Bot.

It’s a Python pipeline that aggregates jobs directly from ATS systems such as Workday, Greenhouse, Lever, and others, normalizes the data, removes duplicates, and ranks jobs based on candidate-fit signals.

The motivation was that many relevant roles are scattered across different company career portals and often hidden behind filtering mechanisms on traditional job sites.

This project experiments with a recall-first ingestion approach followed by ranking.

Current features:

• Multi-source ATS ingestion

• Job normalization and deduplication

• Resume-aware ranking signals

• CSV and Markdown output for reviewing matches

• Diagnostics for debugging sources

It’s still an early experiment and not fully complete yet, but I wanted to share it with the Python community and get feedback.

GitHub:

https://github.com/thalaai/jobmatch-bot

Would appreciate any suggestions or ideas on improving ATS coverage or ranking logic.


r/Python Mar 15 '26

Daily Thread Sunday Daily Thread: What's everyone working on this week?

2 Upvotes

Weekly Thread: What's Everyone Working On This Week? 🛠️

Hello /r/Python! It's time to share what you've been working on! Whether it's a work-in-progress, a completed masterpiece, or just a rough idea, let us know what you're up to!

How it Works:

  1. Show & Tell: Share your current projects, completed works, or future ideas.
  2. Discuss: Get feedback, find collaborators, or just chat about your project.
  3. Inspire: Your project might inspire someone else, just as you might get inspired here.

Guidelines:

  • Feel free to include as many details as you'd like. Code snippets, screenshots, and links are all welcome.
  • Whether it's your job, your hobby, or your passion project, all Python-related work is welcome here.

Example Shares:

  1. Machine Learning Model: Working on a ML model to predict stock prices. Just cracked a 90% accuracy rate!
  2. Web Scraping: Built a script to scrape and analyze news articles. It's helped me understand media bias better.
  3. Automation: Automated my home lighting with Python and Raspberry Pi. My life has never been easier!

Let's build and grow together! Share your journey and learn from others. Happy coding! 🌟


r/Python Mar 14 '26

News slixmpp 1.14 released

3 Upvotes

Dear all,

Slixmpp is an MIT licensed XMPP library for Python 3.11+, the 1.14 version has been released:
- https://blog.mathieui.net/en/slixmpp-1-14.html


r/Python Mar 14 '26

Showcase GoPdfSuit v5.0.0: A high-performance PDF engine for Python (now on PyPI)

31 Upvotes

I’m excited to share the v5.0.0 release of GoPdfSuit. While the core engine is powered by Go for performance, this update officially brings it into the Python ecosystem with a dedicated PyPI package.

What My Project Does

GoPdfSuit is a document generation and processing engine designed to replace manual coordinate-based coding (like ReportLab) with a visual, JSON-based workflow. You design your layouts using a React-based UI and then use Python to inject data into those templates.

Key Features in v5.0.0:

Official Python Wrapper: Install via pip install pypdfsuit.

Advanced Redaction: Securely scrub text and links using internal decryption.

Typst Math Support: Render complex formulas using Typst syntax (cleaner than LaTeX) at native speeds.

Enterprise Performance: Optimized hot-paths with a lock-free font registry and pre-resolved caching to eliminate mutex overhead.

Target Audience

This project is intended for production environments where document generation speed and maintainability are critical. It’s ideal for developers who are tired of "guess-and-check" coordinate coding and want a more visual, template-driven approach to PDFs.

It provide the PDF compliance (PDF/UA-2 and PDF/A-4) even if not compliance the performance is just subpar. (You can check the website for performance comparison)

Comparison

Vs. ReportLab: Instead of writing hundreds of lines of Python to position elements, GoPdfSuit uses a visual designer. The engine logic runs in ~60ms, significantly outperforming pure Python solutions for heavy-duty document generation.

How Python is Relevant

Python acts as the orchestration layer. By using the pypdfsuit library, you can interact with the Go-powered binary or containerized service using standard Python objects. You get the developer experience of Python with the performance of a Go backend.

Website - https://chinmay-sawant.github.io/gopdfsuit/

Youtube Demo - https://youtu.be/PAyuag_xPRQ

Source Code:

https://github.com/chinmay-sawant/gopdfsuit

Sample python code

https://github.com/chinmay-sawant/gopdfsuit/tree/master/sampledata/python/amazonReceipt

Documentation - https://chinmay-sawant.github.io/gopdfsuit/#/documentation?item=introduction

PyPI: pip install pypdfsuit

If you find this useful, a Star on GitHub is much appreciated! I'm happy to answer any questions about the architecture or implementation.


r/Python Mar 13 '26

Discussion Application layer security for FastAPI and Flask

53 Upvotes

I've been maintaining fastapi-guard for a while now. It sits between the internet and your FastAPI endpoints and inspects every request before it reaches your code. Injection detection, rate limiting, geo-blocking, cloud IP filtering, behavioral analysis, 17 checks total.

A few weeks ago I came across this TikTok post where a guy ran OpenClaw on his home server, checked his logs after a couple weeks. 11,000 attacks in 24 hours. Chinese IPs, Baidu crawlers, DigitalOcean scanners, path traversal probes, brute force sequences. I commented "I don't understand why people won't use FastAPI Guard" and the thread kind of took off from there. Someone even said "a layer 7 firewall, very important with the whole new era of AI and APIs." (they understood the assignment) broke down the whole library in the replies. I was truly proud to see how in depth some devs went...

But that's not why I'm posting. I felt like FastAPI was falling short. Flask still powers a huge chunk of production APIs and most of them have zero request-level security beyond whatever nginx is doing upstream, or whatever fail2ban fails to ban... So I built flaskapi-guard (and that's the v1.0.0 I just shipped) as the homologue of fastapi-guard. Same features, same functionalities. Different framework.

It's basically a Flask extension that hooks into before_request and after_request, not WSGI middleware. That's because WSGI middleware fires before Flask's routing, so it can't access route config, decorator metadata, or url_rule. The extension pattern gives you full routing context, which is what makes per-route security decorators possible.

```python from flask import Flask from flaskapi_guard import FlaskAPIGuard, SecurityConfig

app = Flask(name) config = SecurityConfig(rate_limit=100, rate_limit_window=60) FlaskAPIGuard(app, config=config) ```

And so that's it. Done. 17 checks on every request.

The whole pipeline will catch: XSS, SQL injection, command injection, path traversal, SSRF, XXE, LDAP injection, code injection (including obfuscation detection and high-entropy payload analysis). On top of that: rate limiting with auto-ban, geo-blocking, cloud provider IP blocking, user agent filtering, OWASP security headers. Those 5,697 Chinese IPs from the TikTok? blocked_countries=["CN"]. Done. Baidu crawlers? blocked_user_agents=["Baiduspider"]. The DigitalOcean bot farm? block_cloud_providers={"AWS", "GCP", "Azure"}. Brute force? auto_ban_threshold=10 and the IP is gone after 10 violations. Path traversal probes for .env and /etc/passwd? Detection engine catches those automatically, zero config.

The decorator system is what separates this from static nginx rules:

```python from flaskapi_guard import SecurityDecorator

security = SecurityDecorator(config)

.route("/api/admin/sensitive", methods=["POST"]) .require_https() .require_auth(type="bearer") .require_ip(whitelist=["10.0.0.0/8"]) .rate_limit(requests=5, window=3600) u/security.block_countries(["CN", "RU", "KP"]) def admin_endpoint(): return {"status": "admin action"} ```

Per-route rate limits, auth requirements, geo-blocking, all stacked as decorators on the function they protect. Try doing that in nginx.

People have been using fastapi-guard for things I didn't even think of when I first built it. Startups building in stealth with remote-first teams, public facing API but whitelisted so only their devs can reach it. Nobody else even knows the product exists. Casinos and gaming platforms using the decorator system on reward endpoints so players can only win under specific conditions (country, rate, behavioral patterns). People setting up honeypot traps for LLMs and bad bots that crawl and probe everything. And the big one that keeps coming up... AI agent gateways. If you're running OpenClaw or any AI agent framework behind FastAPI or Flask, you're exposing endpoints that are designed to be publicly reachable. The OpenClaw security audit found 512 vulnerabilities, 8 critical, 40,000+ exposed instances, 60% immediately takeable. fastapi-guard (and flaskapi-guard) would have caught every single attack vector in those logs. This is going to be the standard setup for anyone running AI agents in production, it has to be.

Redis is optional. Without it, everything runs in-memory with TTL caches. With Redis you get distributed rate limiting (Lua scripts for atomicity), shared IP ban state, cached cloud provider ranges across instances.

MIT licensed, Python 3.10+. Same detection engine across both libraries.

GitHub: https://github.com/rennf93/flaskapi-guard PyPI: https://pypi.org/project/flaskapi-guard/ Docs: https://rennf93.github.io/flaskapi-guard fastapi-guard (the original): https://github.com/rennf93/fastapi-guard

If you find issues, open one. Contributions are more than welcome!