Rust has a reputation of having good compiler error messages [citation needed].
I generally agree! However, blindly following the hints given by the compiler may sometimes hurt beginners who don’t fully understand the language. It doesn’t help that due to Rust’s excellent formatting of code suggestions [citation needed], the suggestions really seem Correct and Canonical.
Case Study
Consider this problem:
They likely don’t! But the pattern of “appending to an output” happens a lot, and that’s the main focus here.
Here’s how someone who is a beginner to Rust but is familiar with other programming languages might approach the problem:
fn (: , : &mut ) {
// ...
}
With the following thought process:
And here’s how they might write the function body:
fn (: , : &mut ) {
if .() {
= + ::("*");
}
}
With the following thought process:
And now the beginner is trapped:
--> src/main.rs:3:25
|
3 | result = result + String::from("*");
| ------ ^ ----------------- String
| | |
| | `+` cannot be used to concatenate a `&str` with a `String`
| &mut String
|
help: create an owned `String` on the left and add a borrow on the right
|
3 | result = result.to_owned() + &String::from("*");
| +++++++++++ +
For more information about this error, try `rustc --explain E0369`.
error: could not compile `testing` (bin "testing") due to previous error
Leading to:
fn (: , : &mut ) {
if .() {
= .() + &::("*");
}
}
Resulting in:
--> src/main.rs:3:18
|
1 | fn append_asterisk_if_ascii(target: String, result: &mut String) {
| ----------- expected due to this parameter type
2 | if target.is_ascii() {
3 | result = result.to_owned() + &String::from("*");
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `&mut String`, found `String`
|
help: consider dereferencing here to assign to the mutably borrowed value
|
3 | *result = result.to_owned() + &String::from("*");
| +
For more information about this error, try `rustc --explain E0308`.
error: could not compile `testing` (bin "testing") due to previous error
Leading to:
fn (: , : &mut ) {
if .() {
* = .() + &::("*");
}
}
And now the program compiles.
Space Analysis
Rustaceans People familiar with Rust might have been screaming for the past few paragraphs.
Let’s get the irrelevant (in this particular case study, but is very relevant in general and should be fixed) improvement out of the way:
fn (: , : &mut ) {
// ...
}
This function signature is overly specific.
Since the only thing we need target
for is the method .is_
, which does not mutate the String
, we can avoid taking ownership of the String
and use a &str
instead, which is an immutable string slice.
In a similar vein, result
should be &mut str
, since you can “provide” a &mut str
with types other than a String
, so enforcing the restriction that it must be a String
object is needlessly restrictive when all we are doing is appending a &str
.
Now to the meat and potatoes:
if target.is_ascii() {
*result = result.to_owned() + &::from("*");
}
This code is Not Good because of one reason: It makes plenty of unnecessary memory allocations. In fact, it makes 2 extra allocations per call, when in the ideal case it makes 0. The allocations are
-
result.to_
, which creates a clone ofowned() result
, which is aString
. -
String::from("*")
, which creates a clone of the&'static str
that is"*"
.
Note that the +
does not allocate a new string, but rather reuses the buffer of the LHS, which in this case is result.to_
.
Let’s find out!
We’ll use the heap profiling crate dhat-rs
.
Here’s the code:
#[]
static : dhat:: = dhat::;
fn (: &, : &mut ) {
if .() {
* = .() + &::("*");
}
}
fn () -> <(), <dyn std::error::>> {
let = ::(10);
let = dhat::::().().();
("ascii!", &mut );
let = dhat::::();
!(" Max blocks:\t{}", .);
!(" Max bytes:\t{}", .);
!("Total blocks:\t{}", .);
!(" Total bytes:\t{}", .);
(())
}
A few things are of note here:
-
We ensure the
result
string has sufficient capacity before the loop to avoid growing the string during the loop. Note that in this case, ensuring capacity does not change the memory used because the function replacesresult
each call. -
We create the heap profiler after creating the
result
string to avoid measuring the heap allocation during the creation ofresult
.
Here are the results:
Max blocks: 2
Max bytes: 9
Total blocks: 2
Total bytes: 9
From our analysis earlier we know why the maximum number of blocks is 2.
The breakdown for maximum number of bytes is rather complicated, but the TLDR is that the minimum heap allocation size when growing a String
is 8 bytes.
If you’re interested, here’s the stack trace:
alloc::raw_vec::finish_grow (core/src/result.rs:0:23)
alloc::raw_vec::RawVec<T,A>::grow_amortized (alloc/src/raw_vec.rs:404:19)
alloc::raw_vec::RawVec<T,A>::reserve::do_reserve_and_handle (alloc/src/raw_vec.rs:289:28)
alloc::raw_vec::RawVec<T,A>::reserve (alloc/src/raw_vec.rs:293:13)
alloc::vec::Vec<T,A>::reserve (src/vec/mod.rs:909:18)
alloc::vec::Vec<T,A>::append_elements (src/vec/mod.rs:1992:9)
<alloc::vec::Vec<T,A> as alloc::vec::spec_extend::SpecExtend<&T,core::slice::iter::Iter<T>>>::spec_extend (src/vec/spec_extend.rs:55:23)
alloc::vec::Vec<T,A>::extend_from_slice (src/vec/mod.rs:2438:9)
alloc::string::String::push_str (alloc/src/string.rs:903:9)
<alloc::string::String as core::ops::arith::Add<&str>>::add (alloc/src/string.rs:2264:14)
testing::append_asterisk_if_ascii (testing/src/main.rs:6:19)
testing::main (testing/src/main.rs:10:5)
with the relevant constant being MIN_
.
The 8 bytes, plus the 1 byte for String::from("*")
, makes 9 bytes.
Elementary, my dear duckson.
String::from
takes a different code path!
<alloc::alloc::Global as core::alloc::Allocator>::allocate (alloc/src/alloc.rs:241:9)
alloc::raw_vec::RawVec<T,A>::allocate_in (alloc/src/raw_vec.rs:184:45)
alloc::raw_vec::RawVec<T,A>::with_capacity_in (alloc/src/raw_vec.rs:130:9)
alloc::vec::Vec<T,A>::with_capacity_in (src/vec/mod.rs:670:20)
<T as alloc::slice::hack::ConvertVec>::to_vec (alloc/src/slice.rs:162:25)
alloc::slice::hack::to_vec (alloc/src/slice.rs:111:9)
alloc::slice::<impl [T]>::to_vec_in (alloc/src/slice.rs:441:9)
alloc::slice::<impl [T]>::to_vec (alloc/src/slice.rs:416:14)
alloc::slice::<impl alloc::borrow::ToOwned for [T]>::to_owned (alloc/src/slice.rs:823:14)
alloc::str::<impl alloc::borrow::ToOwned for str>::to_owned (alloc/src/str.rs:209:62)
<alloc::string::String as core::convert::From<&str>>::from (alloc/src/string.rs:2612:11)
testing::append_asterisk_if_ascii (testing/src/main.rs:6:40)
testing::main (testing/src/main.rs:10:5)
What this code path does exactly is outside my attention span pay grade.
Here it is:
fn (: &, : &mut ) {
if .() {
.('*');
}
}
Or this:
fn (: &, : &mut ) {
if .() {
* += "*";
}
}
Both do a whopping 0 extra allocations provided the result
still has enough capacity to fit the new content.
This is because the underlying buffer in result
is reused, instead of a new string being created to replace it.
We can see the effects more pronounced by doing more iterations of append_
:
let num_asterisks = 100_000;
let mut result = ::with_capacity(num_asterisks);
let _profiler = dhat::::builder().testing().build();
for _ in 0..num_asterisks {
append_asterisk_if_ascii("full ascii!", &mut result);
}
let stats = dhat::::get();
which still results in 0 allocations for the better version, but for the original…
Max blocks: 3
Max bytes: 399995
Total blocks: 299992
Total bytes: 14999949980
Time Analysis
Let’s use hyperfine to benchmark two versions of our program!
First, we’ll add the following to our Cargo.toml
:
[features]
slow = []
fast = []
Then, we can include our two versions of append_
:
#[(feature = "slow")]
fn append_asterisk_if_ascii(target: &, result: &mut ) {
if target.is_ascii() {
*result = result.to_string() + &::from("*");
}
}
#[(feature = "fast")]
fn append_asterisk_if_ascii(target: &, result: &mut ) {
if target.is_ascii() {
*result += "*";
}
}
We’ll keep the same number of iterations as before, and run a comparison of the two features:
Benchmark 1: cargo run --release --features fast
Time (mean ± σ): 40.4 ms ± 0.8 ms [User: 30.5 ms, System: 9.7 ms]
Range (min … max): 39.6 ms … 43.8 ms 67 runs
Benchmark 2: cargo run --release --features slow
Time (mean ± σ): 759.6 ms ± 2.2 ms [User: 257.7 ms, System: 496.8 ms]
Range (min … max): 755.8 ms … 762.3 ms 10 runs
Summary
cargo run --release --features fast ran
18.81 ± 0.36 times faster than cargo run --release --features slow
18.8 times faster. Cool!
Conclusion
To be clear, I am not saying you should disregard the hints or help messages given by the Rust compiler. However, you should not assume that the help provided is accurate or solves the underlying problem exactly. The diagnostics given by the compiler is usually narrowly focused, and local rather than global.
Unfortunately, this is a tough problem to solve. For people looking to learn Rust, there’s no way around taking the time to grok the reason for the language’s existence. Tools like Clippy help with writing idiomatic code, but it isn’t a panacea either. You just have to write code, possibly bad code, and keep telling yourself there must be a better way!