Thanks for the first week, and an honest note about where we're at! #61
Noah-Tervalon-Nvidia
announced in
Announcements
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Hi, my name is Terve (pronounced turv), I'm one of the folks on the development
team for NVIDIA PAIR and you'll likely be seeing me a lot in the various spaces
concerning this project. I'm really excited about the response that PAIR has
gotten and even more excited about how we are going to move forward with this
project.
I’m the main community touch point for this project, but I can tell you the
whole dev team is really excited about what we're building here and you'll see
them around working on code and responding to PRs and issues.
We weren't expecting this
I'll be honest, we hoped people would like PAIR as much as we do, but did not foresee
this much support coming! As I'm writing this, PAIR is at around 1,200 stars, close to
200 forks, 33 open issues, and 25 open pull requests, and almost all of that showed
up in the first week. We're swamped, and it's the best problem we could have asked for.
What's gotten to us most is how much real work is in it. You've sent pull requests
adding vLLM, SGLang, and llama.cpp as engines. Routing for the Anthropic Messages
API. Container and Kubernetes deployment. Authenticated LAN access to the
compatibility endpoints. A fix for node cards that only showed the first GPU on a
multi-GPU box. The bug reports are just as good: mDNS answers going out from an
ephemeral source port, discovery falling over across macOS Internet Sharing, a
scanner cadence that could take a Mac down.
We're reading all of it. If we didn’t get back to you yet, that's just us being
swamped, not us ignoring you.
How we've been working, and what's changing
Issue #22 pointed out that there are no CI workflows in this repo, and that the
Makefile points at a pipeline file that isn't here. That's true, and I'd rather
explain it than quietly patch it.
When we put PAIR out, the plan was to keep most of the development inside NVIDIA's
internal systems and use GitHub to ship releases and hear what people thought and
occasionally merge in new code.
For the level of engagement we were expecting, that was a reasonable call. It
isn't the right one anymore. When people are sending us engine integrations and
platform fixes in the first week, development needs to happen where those people
are, and so do the checks you need to show your change is solid.
So we're moving our process into this repo. What that looks like exactly, the CI,
the review, how a change gets into a release, is what we're working through right
now. I'd rather show you each piece as it lands than post a plan and then walk it
back.
The develop branch
First piece is already here. We made a
developbranch, and that's wherecontributions go from now on.
developwill be our feature branch where all PRs go,while
mainwill be our stable release branch that we update with versioned releasesand a set of specific features we want to include in them.
Please open new pull requests against
developinstead ofmain.maintracksreleased source.
developis where community work comes together, and where therest of this gets built up around it.
We've already retargeted some of the open PRs. If yours is still pointing at
main, it'd be a big help if you could switch it over yourself. Hit Edit nextto your pull request title and change the base branch to
develop.What that means in the meantime
We're going to be slower to merge. That's on purpose, and it isn't us losing
interest. PAIR runs background services on your machine, opens ports on your
network, and pairs with other machines you own. Getting something wrong in there
isn't a cosmetic bug. Until the public checks exist, careful review is the only
quality bar we've got, and I'd rather be slow than break a setup that's working
for you today.
Bug fixes are the exception. If something is broken for people already running
PAIR, that keeps moving, and it goes ahead of new functionality.
Leave your pull request open. We're not closing anything for going stale, and
we're not going to make you resubmit work. If your PR is quiet, that isn't a no.
Who you're talking to
Real fast, let me introduce the team you’ll see here on the repo:
That's where we're at. If you've got questions about any of it, ask them here. We're
opening discussions for exactly that purpose, we want to have a space to engage
with all of you about this project.
Thank you, genuinely, for the last week.
All reactions