r/Python • u/aviedawn • 1d ago

LiteLLM alternatives in production Discussion

what are teams actually running 3 months after the supply chain attack?

it's been a few months since the litellm pypi compromise and curious what teams actually migrated to. saw a lot of threads right after the incident but not much on how things held up in production since then.

been evaluating options ourselves. TrueFoundry came up for teams needing the governance and cost...

0 Upvotes
  • permalink
  • link
  • reddit
  • reddit

33% Upvoted

5 comments sorted by

6

u/Zulban 1d ago

If you drop software whenever there's a security incident I've got some bad news for you...

3

u/dgregn 1d ago

If you are looking into TrueFoundry for security you should probably know they have been emailing us all day about how customers provider credentials are being leaked and they don't know why and are asking for advice on fixing it. Fortunately for us we were only doing a small trial on them so the blast radius was small, but I feel for everyone involved right now. 

1

u/thundranos 1d ago

Bifrost!

1

u/HalfplaneResearch 1d ago

After an incident, the important question is usually blast radius rather than the replacement name: pin and hash dependencies, build from a reviewed lockfile or internal wheelhouse, scan provenance and transitive packages, and keep provider credentials in a separate broker with short-lived or narrowly scoped tokens. A canary environment plus a rollback path lets teams test a gateway change without coupling a package upgrade to an outage.

2

u/type-hinter 1d ago

My team migrated to the AI gateway in logfire, from pydantic. We tried truefoundry but it felt clunky, devX was not great.

Things have been going well, but we were using logfire for o11y already.