J
Jeff Kluge
Guest
With the release of .NET 9 came a major leap for large .NET repositories: a new NuGet dependency graph resolver built to dramatically improve performance. If you’ve struggled with slow package restores in complex builds, this is the solution you’ve been waiting for. Faster restores mean less waiting, more productivity, and a smoother experience for developers working on large projects.
Internally at Microsoft, a large repository is migrating thousands of projects to .NET Core in order to reap the benefits of runtime performance improvements. This is no small task, with currently over 2,500 individual projects migrated and optimized for modern standards. The team contacted NuGet because their restores were taking over 30 minutes, which caused significant delays across hundreds of builds each day. These delays quickly added up, resulting in wasted time and frustration for developers.
To tackle this issue, a team worked tirelessly for months on optimizations, testing various solutions to speed up the restore process. These efforts managed to cut restore times down to 16 minutes, which was an improvement, but it still wasn’t enough. The time taken was still hindering productivity, and we knew there had to be a better way.
The old NuGet dependency resolution algorithm began as a temporary solution; proving that, as Milton Friedman famously said, “Nothing is as permanent as a temporary solution that works”. While it has served its purpose for a long time, it was not designed to handle the scale and complexity of large repositories. The original dependency resolver created a massive dependency graph with millions of nodes, representing every possible relationship between dependencies. This approach simply wasn’t scalable; it required vast amounts of memory and processing power, and as projects grew, so did the time and effort needed to resolve the graphs.
It was clear that a new approach was needed, so a dedicated team of engineers decided to start from scratch. Their goal was ambitious: create a simpler, more efficient resolver that would still produce the same results but in a fraction of the time. The new algorithm they developed uses a more streamlined approach, representing the graph as a flattened set where each node is created only once. This makes the in-memory dependency graph much smaller and easier to work with. Conflicts are resolved as the graph is being built, which avoids the need for the repetitive passes that the old dependency graph resolution algorithm required.
This new approach had dramatic results. The original dependency graph, which in our testing would create 1.6 million nodes for a complex project, was reduced to just 1,200 nodes. With fewer nodes to process, restore times dropped significantly; from 16 minutes down to just 2 minutes. This is a game-changer for developers working with large repositories, as it meant they could spend less time waiting and more time coding and building great products.
Changing a fundamental part of the build process like NuGet was not without its challenges. We understand that making big changes can be daunting, especially when it involves essential tools that developers rely on every day. However, we took this leap of faith—and it paid off. This success has paved the way for further innovation and improvements across the .NET ecosystem.
NuGet’s new dependency graph resolution algorithm is just the beginning. It’s the first step toward a performance-first approach that we want to apply across all of .NET. By focusing on performance and taking a fresh look at existing processes, we believe we can find new ways to make developers’ lives easier and more productive. We invite developers everywhere to join us in this journey; help us identify bottlenecks, suggest improvements, and work together to create the best possible development environment.
The new dependency graph resolution algorithm is included in .NET 9, and it’s on by default. That means if you upgrade to .NET 9, you’ll automatically get the benefits of faster restore times—no extra setup needed, no configuration changes required. Just upgrade, and you’ll see the difference immediately. If you experience any issues with it, please see our documentation on how to get support.
The post Dramatically faster package restores with .NET 9’s new NuGet resolver appeared first on .NET Blog.
Continue reading...
The Challenge
Internally at Microsoft, a large repository is migrating thousands of projects to .NET Core in order to reap the benefits of runtime performance improvements. This is no small task, with currently over 2,500 individual projects migrated and optimized for modern standards. The team contacted NuGet because their restores were taking over 30 minutes, which caused significant delays across hundreds of builds each day. These delays quickly added up, resulting in wasted time and frustration for developers.
To tackle this issue, a team worked tirelessly for months on optimizations, testing various solutions to speed up the restore process. These efforts managed to cut restore times down to 16 minutes, which was an improvement, but it still wasn’t enough. The time taken was still hindering productivity, and we knew there had to be a better way.
Reimagining Package Resolution
The old NuGet dependency resolution algorithm began as a temporary solution; proving that, as Milton Friedman famously said, “Nothing is as permanent as a temporary solution that works”. While it has served its purpose for a long time, it was not designed to handle the scale and complexity of large repositories. The original dependency resolver created a massive dependency graph with millions of nodes, representing every possible relationship between dependencies. This approach simply wasn’t scalable; it required vast amounts of memory and processing power, and as projects grew, so did the time and effort needed to resolve the graphs.
It was clear that a new approach was needed, so a dedicated team of engineers decided to start from scratch. Their goal was ambitious: create a simpler, more efficient resolver that would still produce the same results but in a fraction of the time. The new algorithm they developed uses a more streamlined approach, representing the graph as a flattened set where each node is created only once. This makes the in-memory dependency graph much smaller and easier to work with. Conflicts are resolved as the graph is being built, which avoids the need for the repetitive passes that the old dependency graph resolution algorithm required.
The Results
This new approach had dramatic results. The original dependency graph, which in our testing would create 1.6 million nodes for a complex project, was reduced to just 1,200 nodes. With fewer nodes to process, restore times dropped significantly; from 16 minutes down to just 2 minutes. This is a game-changer for developers working with large repositories, as it meant they could spend less time waiting and more time coding and building great products.
Changing a fundamental part of the build process like NuGet was not without its challenges. We understand that making big changes can be daunting, especially when it involves essential tools that developers rely on every day. However, we took this leap of faith—and it paid off. This success has paved the way for further innovation and improvements across the .NET ecosystem.
What’s Next?
NuGet’s new dependency graph resolution algorithm is just the beginning. It’s the first step toward a performance-first approach that we want to apply across all of .NET. By focusing on performance and taking a fresh look at existing processes, we believe we can find new ways to make developers’ lives easier and more productive. We invite developers everywhere to join us in this journey; help us identify bottlenecks, suggest improvements, and work together to create the best possible development environment.
The new dependency graph resolution algorithm is included in .NET 9, and it’s on by default. That means if you upgrade to .NET 9, you’ll automatically get the benefits of faster restore times—no extra setup needed, no configuration changes required. Just upgrade, and you’ll see the difference immediately. If you experience any issues with it, please see our documentation on how to get support.
The post Dramatically faster package restores with .NET 9’s new NuGet resolver appeared first on .NET Blog.
Continue reading...