Using Rust in Mercurial
Today, Mercurial is a Python application. It uses Python C extensions in various places to achieve better performance.
There are many advantages to being a Python application. But, there are significant disadvantages.
A major pain point of Python for Mercurial is startup overhead. It takes dozens of milliseconds to start a Python interpreter and load the Mercurial Python modules. If you have many extensions loaded, it could take over 100ms just to effectively get to a Mercurial command's main function. While the command itself may complete in mere milliseconds, Python overhead has already made hg seem non-instantaneous to end-users. A few years ago, we measured that CPython interpreter startup overhead amounted to 10-18% of the run time of Mercurial's test harness. The ability to skip Python completely for some hg commands would be a very compelling feature.
Python is also substantially slower than native code. While we embrace the hackability advantages of Python for large parts of Mercurial's functionality, there's a lot of lower-level, performance sensitive code that would benefit from not being implemented in Python. And the types of things we want to do (like using thread pools to handle CPU-intensive tasks like decompression and delta application) are not things that can be efficient in Python as long as the GIL is around.
As Mercurial's code base grows, the use of a dynamic programming language also hinders development velocity. There are tons of bugs that could be caught at compile time by languages that do such things.
When performance is an issue, Mercurial developers currently turn to C. But we treat C as a measure of last resort because it is just too brittle. It is too easy to introduce security vulnerabilities, memory leaks, etc. On top of vanilla C, the Python C API is somewhat complicated.
Furthermore, Mercurial needs to run on multiple platforms, including Windows. The nice things we want to do in native code are complicated to implement in C because cross-platform C is hard. The standard library is inadequate compared to modern languages. While modern versions of C++ are nice, we still support Python 2.7 and thus need to build with MSVC 2008 on Windows. It doesn't have any of the nice features that modern versions of C++ have.
For Mercurial, Rust is all around a better C. It is much safer, about the same speed, and has a usable standard library and modules system for easily pulling in 3rd party code.
https://www.mercurial-scm.org/wiki/OxidationPlan
discuss