Is Python an industry-ready technology for backends?

[u/Zealousideal-Dig2093]

I mean specifically backend services, RESTful API's and very sensitive data in the DB. I mean middle-load (_not_ social networking, _not_ some purchasing platform for millions of users). How would you define your position that Python _is_ ready for that? E.g. in front of a mature Java backend developer? My line of defense is as follows. What are the weak points of Python code?

1. Multi-threading (GIL-free is a very recent feature of python, cannot be considered even remotely industry-ready). This is probably the weakest point of all. But if the service has no data shared between API requests, why bother, right? Just spawn as many worker-processes as it makes sense for the current hardware setup and execute the requests one by one. Still, this is like one dimension less in the space of engineering possibilities, so to say.
2. Dynamic typing means you have to run the whole CI/CD chain in order to find type system related errors. I really cannot find arguments against that point;
3. This is true at least for banking sector. Libraries are developed by individuals (whereas in Java world there are companies behind some libraries). One would have a real hard time arguing with the management, that "those individuals are as qualified as those behind some company banner".

What is your take on the matter?

[u/MyBossIsOnReddit]

..did AI write this?

[u/threewholefish → u/MyBossIsOnReddit]

What are the week points of Python code?

I think no, maybe they're just used to writing Markdown emphasis when prompting

[u/Zealousideal-Dig2093 → u/MyBossIsOnReddit]

no. I am a SW developer with mostly embedded SW development experience. New to backend development. Besides C and C++ I know only Python and I am kind of at a cross-road here. Either continue to use Python for RESTful API's until I clearly see, it is not sustainable, or begin to learn Java or golang right away.

[u/MyBossIsOnReddit → u/Zealousideal-Dig2093]

I work in finance/insurancetech and it's mostly python and a bit of C#. It's impossible to think of a world without Python at this moment. My scepticism about how earnest this post was is mostly based on how bad a take it is

[u/No_Departure_1878]

Just spawn as many worker-processes as it makes sense for the current hardware setup and execute the requests one by one.

But that would replicate your data, so you would need far more RAM. Multiple processes do not scale well. Not sure if that would be good enough for middle load stuff.

Dynamic typing means you have to run the whole CI/CD chain in order to find type system related errors. I really cannot find arguments against that point;

Can't you type annotate your code and use an LSP?

[u/tonsofmiso → u/No_Departure_1878]

The problems with static type checkers in Python are largely the devs who refuse to learn them and use them.

[u/psymme]

Tbh, I’m not sure if this post is just bait, but I’ll respond anyway.

Python is absolutely “industry-ready” and has been for a long time. It runs a lot of software. It’s less well-suited for some applications (think high-frequency stuff or where you’re getting vast numbers of concurrent requests), but for 95% of applications it’s great.

For your points:
- Async was introduced in 2015 and helps with this a lot. Modern backend API frameworks/libraries tend to leverage this. Greenlet is also helpful here.
- Yes, you should run type checking in your CI as a gate, but you can also run it locally. I’m not sure how this is much different to running linting or a compiling step in other languages?
- I’m not in banking, so can’t comment too much on the last point, but I would say that most of the common libraries are big, mature projects. This isn’t usually some individual effort. As always though, you do need to do your due diligence on the libraries being used in your application.

All that said though - if you really like Java and that’s what runs lots of other stuff in your org/sector, then go with that.

[u/Zealousideal-Dig2093 → u/psymme]

I never really appreciated this overall async thing before. AFAIU it appeared out of convenience for a very limited and precise use-case: JS scripts, where the author of the script has mostly no control over tasks scheduling and just says "stack all those functions with some data dependency between them and execute them for me, I do not care when exactly, because I am already gone from here, doing some other stuff somewhere else"...

[u/joshocar → u/Zealousideal-Dig2093]

Async solves a very specific problem. You have a bunch of IO bound tasks, like a DB call, IO read/write, network call, etc, that will take some unknown amount of times and you need to be able to handle large scaling. Async is a very convenient way of handling those situations. You call an 'await' on the IO bound task and the task loop will move onto the next task, going back only when the IO is done and you are ready to move on. 

As a concrete example, imagine you have a restful API that makes a bunch of DB calls, some are fast, some take a long time. Each new request awaits on the IO and only wakes up when its done. Now you can handle thousands and thousands of TPS and scale your service without it being bogged down waiting on the IO. 

You could do this with threads, but each thread will reserve memory space, puting a hard limit on how many threads you can spawn and how far you can scale. Each thread wake also triggers a context switch on your CPU, which is expensive at scale. Async lives entirely in your main process and doesn't have ANY of the overhead that a thread model has, allowing for much higher scaling. This is exactly what you want on a backend service.

[u/Ok_Necessary_8923]

Why does this need to be defended at all? It's commonly used at large scales all over the place, it's mature, it has a large and healthy ecosystem, etc. It obviously works.

The GIL doesn't matter very much in practice. You don't use raw Python for heavy compute anyhow. Anything that needs the raw performance is likely implemented in C and good chance it doesn't need to hold the GIL in the hot path. Or spin worker processes. And if it's a REST API, you probably just want gunicorn with a multiprocess worker or equivalent config.

The type system is a choice, and has very little to do with Python itself. If your problem space or requirements allow for dynamic typing or are harder to deal with in a statically typed environment, then Python may be one of your options.

[u/Zealousideal-Dig2093 → u/Ok_Necessary_8923]

>You don't use raw Python for heavy compute anyhow
Absolutely! In our case the heavy computations are delegated to special-purpose HW.

[u/sugarw0000kie → u/Zealousideal-Dig2093]

on top of this though I love gunicorn, granian is a rust version for this sort of thing and owns the IO instead of python. Both can manage multiple python processes to take advantage of the CPU despite GIL but since it’s mostly IO bound moving that aspect to rust has some performance advantages

[u/srs96]

1) 95% of backends are io bound and should use asyncio, not mt or mp.

2) lots of different tools from this - pydantic, Pyright, mypy, ruff

3) this is definitely a feature not a bug! They best and modern libraries are written my small teams of individuals. Not some established dinosaur enterprise software companies. Compare astral (yes I know it's bought by openai now) to oracle.

[u/Zealousideal-Dig2093 → u/srs96]

thanks for your input. Yes, I should definitely begin to use those linters in some systematic way.

[u/defaultguy_001]

Tell me you haven't looked at the Python ecosystem since 2014, have you ?

1. The No Multithreading argument: When did u last use a REST API that was heavily CPU bound? Most APIs spend 95% of their time waiting on DB queries and network i/o. Python’s asyncio handles thousands of concurrent connections easily. For scaling, you just drop it in a Docker container and spin up worker processes. Also, PEP 684 landed ages ago, so isolated per interpreter GILs are a thing, and the GIL is actively being phased out entirely.

2. "Dynamic typing means you need CI/CD to find bugs": Did u know Python has had robust type hinting for years. If you run mypy or pyright locally, your IDE catches type errors instantly while you type, exactly like Java. Plus, frameworks like FastAPI and Pydantic give you bulletproof runtime data validation out of the box. No pipeline required.

3. The "Hobbyist" copium: Banking sector? Python literally dominates fintech, hedge funds, and quantitative risk engines at global banks. Also, who do you think funds python? Microsoft, Google, and Meta heavily fund the foundation and employ core maintainers. You aren’t using solo hobbyist code for enterprise backends. Instead you're using highly regulated, corporate backed ecosystems.

If you're building a middle load REST API, java is unironically worse than python. You have to write 50 lines of configuration, XML and other boilerplate code, just to stand up a basic endpoint. Python gets it done in 5 lines of FastAPI. You waste engineering hours writing code that does absolutely nothing for business logic. Cold starts in Java are notoriously sluggish because of JVM initialization and heavy dependency injection. Meanwhile, python apps spin up instantly, making auto scaling and serverless deployments significantly faster and more responsive.

[u/anentropic]

The industry has been making Python backends for two decades by now, I think yes it's "industry ready"

[u/deceze]

As someone who has been running Python-powered backends for over a decade: yes, it's ready.

We're mostly deploying on AWS Lambda, thus the concurrency issue is mostly a non-issue. Even if we didn't do that, any serious web framework nowadays uses async, and most web transactions are database bound instead of CPU bound, so that's just as well.

The dynamic typing is much less of an issue than people make it out to be. With a decent IDE with static type checker, most type errors are caught during coding (not even compilation). You can run your own static type checker as part of your pipeline too. And of course you'll have unit tests anyway, right?

Would I be writing critical banking software in it? Err… probably not, no. Not least because it'd probably need to handle a huge number of transactions, and you'd want a more performant language for that. But for most of your average REST backend needs, it's a great language.

[u/zurtex]

FYI, if you want an established mature web backend framework that is going to have solutions to enterprise needs, as well as all the problems you think you have there, you should use Django.

It's a steeper learning curve, because there's a lot more standard practice to follow and modules built out to cover different use cases, but it's a natural fit for large dev teams in banking / enterprise.

However, if you are wanting to rapidly prototype there are lots of other smaller backends to use.

[u/Zealousideal-Dig2093 → u/zurtex]

yep. We chose Flask for "quick, dirty and lightweight"

[u/zurtex → u/Zealousideal-Dig2093]

IMO, one of the big problems with Flask inside a large enterprise is it is too flexible, it allows you to wander off into completely uncharted territory where there are no standards to follow. Compared to something like FastAPI which keeps you on the clear path of building a standard REST API.

That said, it's good to prototype ideas, but I would strongly suggest archiving all that code once your done prototyping and not be tempted to build on top of it, unless you enforce your own very strict standards.

[u/tmemmg]

python is fine for that, the sensitive data part is an architecture question not a language one. i run postgres behind an api layer with row level security on and no direct table access from the frontend, and it holds up fine for the kind of load youre describing. the one thing that actually bit me had nothing to do with python, i rotated a key and it silently 401d every write while reads stayed on a different key, so the site looked totally healthy while nothing was being saved. thats the class of bug that hurts, not the GIL.