Oxidizing your Python with some Rust

Python and its ecosystem are filled with a plethora of libraries and packages that make up some of Stack Overflow’s most popular tools & technologies [1]. The ecosystem is thoroughly embedded and the ease of use and learning curve is beginner-friendly. This is Python’s main strength: someone who is curious about the language can pick it up with little to no headache, especially with the vast number of resources to help kick-start their learning. However, with most things in life, the ease of use has its drawbacks.

Figure 1: Energy, Time, and Memory Consumption across programming languages
Figure 1: Energy, Time, and Memory Consumption across programming languages

Making Sense of Energy, Time, and Memory Performance

The figure above displays the energy, time, and memory performance across several programming languages. Python lags in energy, and time performance, but ranks slightly higher in the memory performance category[2]. Although Python is easy to use, its performance is not as strong as many other languages.

  • Reliability
  • Productivity

“Rust’s rich type system and ownership model guarantee memory-safety and thread-safety — enabling you to eliminate many classes of bugs at compile-time.”

An analysis of chromium, the core of Google Chrome & Chrome OS, found that 70% of their bugs are caused by memory safety issues [4]. Rust eliminates these classes of bugs at compile-time with the help of the Borrow Checker removing this class of bugs at compile time [5]. You can have confidence that a successful compilation will remain reliable and perform well.

Figure 2: Chromium High Severity Bug Classification

Getting Started — Building Extensions for Python

The pyo3 Rust crate (package) lets us build Python extensions that can allow us to build highly efficient, highly performant, and safe code in Rust. At the same time, Python users can utilize these tools without learning a whole new language.

New Package

First, we need to create a new Rust library using

cargo new oxidizer --lib

Implementation

Here is a trivial implementation of calculating the sum of primes from 0 to n. The key parts are the #[pyfunction] and #[pymodule] macros. These macros are what will be publicly visible to the python package. I will make use of using the primal crate to calculate the primality of a number. Both Rust and Python will be making use of the Miller-Rabin primality test [6].

from .oxidizer import sum_primes
maturin build --release
cd target/wheels && pip install oxidizer-{...}
maturin develop

Benchmarks

Below I’ve implemented a pytest micro-benchmark that compares the two implementations.

Figure 3: Visualization of Benchmark Results
Figure 3: Visualization of Benchmark Results

Appendix

Here are some Rust resources if this blog post hasn’t convinced you to get take a sip of the sweet, sweet Rust kool-aid yet.

  1. https://github.com/rust-unofficial/awesome-rust
  2. https://stackoverflow.blog/2020/01/20/what-is-rust-and-why-is-it-so-popular/

References

  1. https://insights.stackoverflow.com/survey/2020#technology-other-frameworks-libraries-and-tools
  2. https://greenlab.di.uminho.pt/wp-content/uploads/2017/10/sleFinal.pdf
  3. https://www.rust-lang.org/
  4. https://www.chromium.org/Home/chromium-security/memory-safety
  5. https://doc.rust-lang.org/1.8.0/book/references-and-borrowing.html
  6. https://en.wikipedia.org/wiki/Miller%E2%80%93Rabin_primality_test

Figures

  1. https://greenlab.di.uminho.pt/wp-content/uploads/2017/10/sleFinal.pdf
  2. https://www.chromium.org/Home/chromium-security/memory-safety

Data Engineering and DevOps / SRE Enthusiast

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store