It's like reading "A Discipline of Programming", by Dijkstra. That morality play approach was needed back then, because nobody knew how to think about this stuff.
Most explanations of ownership in Rust are far too wordy. See [1]. The core concepts are mostly there, but hidden under all the examples.
- Each data object in Rust has exactly one owner.
- Ownership can be transferred in ways that preserve the one-owner rule.
- If you need multiple ownership, the real owner has to be a reference-counted cell.
Those cells can be cloned (duplicated.)
- If the owner goes away, so do the things it owns.
- You can borrow access to a data object using a reference.
- There's a big distinction between owning and referencing.
- References can be passed around and stored, but cannot outlive the object.
(That would be a "dangling pointer" error).
- This is strictly enforced at compile time by the borrow checker.
That explains the model. Once that's understood, all the details can be tied back to those rules.
That's not explaining ownership, that motivating it. Which is fine. The thing that's hard to explain and learn is how to read function signatures involving <'a, 'b>(...) -> &'a [&'b str] or whatever. And how to understand and fix the compiler errors in code calling such a function.
Ownership is easy, borrowing is easy, what makes the language super hard to learn is that functions must have signatures and uses that together prove that references don't outlive the object.
Also: it's better not store referenced object in a type unless it's really really needed as it makes the proof much much more complex.
That really doesn't explain the model because you have completely left out the distinction between exclusive/shared (or mutable/immutable) borrows. Rust made a large number of choices with respect to how it permits such borrows and those do not follow from this brief outline nor from intuition or common sense. For example, the no aliasing rule is motivated not by intuition or common sense but from a desire to optimize functions.
The most complicated aspect of the borrows comes about from the elision rules which will silently do the wrong thing and will work fantastically until they don't at which point the compiler error is pointing at a function complaining about a lifetime parameter of a parameter with the trait method implying that the parameter has to live too long but the real problem was a lifetime in the underlying struct or a previous broken lifetime bound. Those elision rules are again not-intuitive and don't fall out of your explanation axiomatically. They were decisions that were made to attempt to simplify the life of programmers.
And, after someone who doesn't know rust reads this neat and nice summary, they would still know nothing about rust. (Except "this language's compiler must have some black magic in it.")
Summarizing a set of concepts in a way that feels correct and complete to someone who understands them, is a much easier task than explaining them to someone who doesn't. If we put this in front of someone who's only worked with call-by-sharing languages, do you think they'll get it right away? I'm skeptical.
For me it really clicked when I realized ownership / lifetimes / references are just words used to talk about when things get dropped. Maybe because I have a background in C so I'm used to manual memory management. Rust basically just calls 'free' for you the moment something goes out of scope.
All the jargon definitely distracted me from grasping that simple core concept.
Rust also has the “single mutable reference” rule. If you have a mutable reference to a variable, you can be sure nobody else has one at the same time. (And the value itself won’t be mutated).
Mechanically, every variable can be in one of 3 modes:
1. Directly editable (x = 5)
2. Have a single mutable reference (let y = &mut x)
3. Have an arbitrary number of immutable references (let y = &x; let z = &x).
The compiler can always tell which mode any particular variable is in, so it can prove you aren’t violating this constraint.
If you think in terms of C, the “single mutable reference” rule is rust’s way to make sure it can slap noalias on every variable in your program.
This is something that would be great to see in rust IDEs. Wherever my cursor is, it’d be nice to color code all variables in scope based on what mode they’re in at that point in time.
"Rust basically just calls 'free' for you the moment something goes out of scope."
C++ does that too with RAII. Go ahead and use whatever STL containers you like, emplace objects onto them, and everything will be safely single-owned with you never having to manually new or delete any of it.
The difference is that C++'s guarantees in this regard derive from a) a bunch of implementation magic that exists to hide the fact that those supposedly stack-allocated containers are in fact allocating heap objects behind your back, and b) you cooperating with the restrictions given in the API docs, agreeing not to hold pointers to the member objects or do weird things with casting. You can use scoped_ptr/unique_ptr but the whole time you'll be painfully aware of how it's been bolted onto the language later and whenever you want you can call get() on it for the "raw" underlying pointer and use it to shoot yourself in the foot.
Rust formalizes this protection and puts it into the compiler so that you're prevented from doing it "wrong".
the tradeoff is that ~you have to guess where rust is doing the frees, and you might be wrong. in the end this would be strictly equivalent to an explicit instruction to free, with the compiler refusing to compile if the free location broke the rules.
Right. If you come to Rust from C++ and can write good C++ code, you see this as "oh, that's how to think about ownership". Because you have to have a mental model of ownership to get C/C++ code to work.
But if you come from Javascript or Python or Go, where all this is automated, it's very strange.
The list in the above comment isn’t a summary — it’s a precise definition. It can and must be carefully explained with lots of examples, contrasts with other languages, etc., but the precise definition itself must figure prominently, and examples and intuition should relate back to it transparently.
Practically, I think it suggests that learning the borrow checker should start with learning how memory works, rather than any concepts specific to Rust.
This explanation doesn't expose anything meaningful to my mind, as it doesn't define ownership and borrowing, both words being apparently rooted in an analogy with financial asset management.
I'm not acquainted with Rust, so I don't really know, but I wonder if the wording plays a role in the difficulty of concept acquisition here. Analogies are often double edged tools.
Maybe sticking to a more straight memory related vocabulary as an alternative presentation perspective might help?
The way I think about it is more or less in terms of how a C program would work: if you assume a heap allocated data structure, the owner is the piece of code that is responsible for freeing the allocation at the appropriate time. And a reference is just a pointer with some extra compile time metadata that lets the borrow checker prove that the reference doesn’t outlive the referent and that there’s no mutable aliasing.
If you've worked inside of CPython or other programs with manual reference counting, the idea of borrowing shows up there, where you receive a reference from another part of the program and then mess with the object without tweaking the reference count, "borrowing" an existing reference because any copies you've of the address will be short lived. The term shows up throughout CPython.
I find it strange that you relate borrowing and ownership to financial asset management.
From that angle, it indeed doesn’t seem to make sense.
I think, but might be completely wrong, that viewing these actions from their usual meaning is more helpful: you own a toy, it’s yours to do as tou please. You borrow a toy, it’s not yours, you can’t do whatever you want with it, so you can’t hold on to it if the owner doesn’t allow it, and you can’t modify it for the same reasons.
1. In real life I can borrow a toy from you and while I have that toy in my hands, the owner can exchange ownership with somebody else, while the object is borrowed by me. I.e. in real life the borrowing is orthogonal to ownership. In rust you can't do that.
2. Borrowing a toy is more akin to how mutable references work in rust. Immutable references allow multiple people to play with the same toy simultaneously, provided they don't change it.
The second bullet in the second section is overpromising badly. In fact there are many, many, many ways to write verifiably correct code that leaves no dangling pointers yet won't compile with rustc.
Frankly most of the complexity you're complaining about stems from attempts to specify exactly what magic the borrow checker can prove correct and which incantations it can't.
It took me a few tries to get comfortable with Rust—its ownership model, lifetimes, and pervasive use of enums and pattern matching were daunting at first. In my initial attempt, I felt overwhelmed very early on. The second time, I was too dogmatic, reading the book line by line from the very first chapter, and eventually lost patience. By then, however, I had come to understand that Rust would help me learn programming and software design on a deeper level. On my third try, I finally found success; I began rewriting my small programs and scripts using the rudimentary understanding I had gained from my previous encounters. I filled in the gaps as needed—learning idiomatic error handling, using types to express data, and harnessing pattern matching, among other techniques.
After all this ordeal, I can confidently say that learning Rust was one of the best decisions I’ve made in my programming career. Declaring types, structs, and enums beforehand, then writing functions to work with immutable data and pattern matching, has become the approach I apply even when coding in other languages.
I don't know how to read your comment other than "nothing hard is worth doing". Some things have benefits and drawbacks, is the existence of drawbacks always a non-starter for you?
I'm trying to phrase this as delicately as I can but I am really puzzled.
If someone wrote an article about how playing the harp is difficult, just stick with it... would you also say that playing the harp is a terrible hobby?
Maybe people need persuading to learn Rust not just because they think it's hard, but also because they think it's bad? Not everything hard is worth doing. Difficulty is just one of the factors to consider.
I started to learn Rust, but I was put off by the heavy restrictions the language imposes and the attitude that this is the only safe way. There's a lack of acknowledgement, at least in beginner materials, that by choosing to write safe Rust you're sacrificing many perfectly good patterns that the compiler can't understand in exchange for safety. Eventually I decided to stop because I didn't like that tradeoff (and I didn't need it for my job or anything)
Rust design decisions are pretty hard to understand sometimes, Mojo is another language with a borrow-checker but it is not nearly as hard to learn as Rust due to making a few decisions.
First is value semantics, in Rust people are told to always clone when learning, why isn't this semantics built into the language? It is what you have in most static languages - C, C++, Go, etc. This is the mental model many people come to Rust with.
Secondary, Mojo's lifetime does not tell the compiler when a value is safe to use but when it is safe to delete, in this way the lifetime is not scope based, references will extend the lifetime of the value they reference, but values will be destroyed immediately after their last use. In Mojo you'll never see "value does not live long enough".
Just these two design decisions defines away so many ergonomic issues.
> people are told to always clone when learning, why isn't this semantics built into the language?
Because cloning as opposed to copying is expensive and it generates a new instance of a type.
In C, you don't clone, you simply copy the struct or pointer, which will lead to a pointer to the same memory or a struct with members pointing to the same memory.
C++ on the other hand has a copy constructor, and you have to move explicitly, often generating unnecessary copies (in the sense of clone)
> Mojo's lifetime does not tell the compiler when a value is safe to use but when it is safe to delete,
What happens if you pass the variable mutably to a function?
> What happens if you pass the variable mutably to a function?
What happens in what manner? Mojo uses ASAP memory model, values will always be destroyed at the point of its last use. Mojo dataflow analysis will track this.
In terms of safety, Mojo will enforce `alias xor mutability` - like in Rust.
> C++ on the other hand has a copy constructor, and you have to move explicitly, often generating unnecessary copies (in the sense of clone)
Mojo also has copy and move constructors, but unlike in C++ these are not synthesised by default; the type creator has to either explicitly define the constructors or add a synthesiser. In Mojo, you can have types that are not copyable and not movable, these types can only be passed by reference. You can also have types that are copyable but not movable, or movable but not copyable.
The truth is that by the time you are a senior developer, you will have encountered the lessons that make rust worth learning but may not have truly understood all the implications.
Many people will think, I have a garbage collected language, rust has nothing to teach me. Even in garbage collected languages, people create immutable types because the possibility of shared references with mutability makes things incredibly chaotic that they look for immutability as a sort panacea. However, once you have immutable types you quickly realize that you also need ergonomic ways of modifying those objects, the methods you create to do so are often more cumbersome than what would be permitted for a mutable object. You wish there was some way to express, "There is a time where this object is mutable and then it becomes immutable." Enter the borrow checker.
Once you are borrow checking... why are you garbage collecting? Well, expressing those timelines of mutability and existence is a cost because you need to understand the timeline and most people would rather not spend that energy--maybe mutability or the poor ergonomics of immutable objects wasn't so bad. So, I garbage collect because I do not want to understand the lifetimes of my objects. Not understanding the lifetimes of objects is what makes shared mutability hard. Immutability eliminates that problem without requiring me to understand. Rust can teach this lesson to you so that you make an informed choice.
Of course, you can also just listen to me and learn the same lesson but there is value for many people to experience it.
Learning any programming language at all feels 10x as hard to beginners, so you might as well say programming is not worth learning period in this case. Anything new has a learning curve to it.
I think you can have a lot of debate on the design decisions on Rust, but I don't think the need for these articles tell you a lot about the language itself. I'd argue that Python needs articles like this more so than Rust does, but for entirely different reasons. In two decades of more and more programmers who aren't coming from an engineering background, I've yet to see anyone who used a Python generator or slots. Data Classes are less rare, but mainly in the form of pydantics "version". Which doesn't exactly matter for a lot of Python code... This is a world where 4chan can serve 4 million concurrent users an apache server running a 10k line PHP file neither of which have been updated since 2015... so you can be fine doing inefficient and entirely in-memory Python code 95% (or more) of the time.
That doesn't mean you should though. Imagine how much energy is being wasted globally on bad Python code... The difference is of course that anyone can write it, and not everyone can write Rust. I'm not personally a big fan of Rust, I'd chose Zig any day of the week... but then I'd also choose C over C++, and I frankly do when I optimise Python code that falls in those last 5%. From that perspective... of someone who really has to understand how Python works under the hood and when to do what, I'd argue that Rust is a much easier langauge to learn with a lot less "design smell". I suppose Python isn't the greatest example as even those of us who love it know that it's a horrible language. But I think it has quite clearly become the language of "everyone" and even more so in the age of LLM. Since our AI friends will not write optimised Python unless you specifically tell them to use things like generators and where to use them, and since you (not you personally) won't because you've never heard about a generator before, then our AI overlords won't actually help.
The article focuses on the learning curve rather than the problem Rust is solving, as an observation. Think you need both of those to draw a conclusion as to whether it’s worth doing.
> If a language needs an article like this, absolutely begging people to bite the bullet to learn it, maybe that's a language design smell.
The problem with articles like this is that they don't really get to the heart of the problem:
There are programs that Rust will simply not let you write.
Rust has good reasons for this. However, this is fundamentally different from practically every programming language that people have likely used before where you can write the most egregious glop and get it to compile and sometimes even kinda-sorta run. You, as a programmer, have to make peace with not being able to write certain types of programs, or Rust is not your huckleberry.
There _are_ more than two programming languages, though. I feel like most of the debates about Rust devolve into the same false choice between safety and ease.
Before Rust I was hearing the same argument from Haskell or Scala developers trying to justify their language of choice.
I know Rust is here to stay, but I think it’s mostly because it has a viable ecosystem and quality developer tools. Its popularity is _in spite of_ many of its language features that trade that extra 1% of safety for 90% extra learning curve.
> features that trade that extra 1% of safety for 90% extra learning curve.
I remember both MS and goog having talks about real-world safety issues in the range of 50% of cases were caused by things that safe rust doesn't allow (use after free, dangling pointers, double free, etc). The fact that even goog uses it, while also developing go (another great language with great practical applications) is telling imo.
I have taken the time to learn rust and you're absolutely right. It's a very complex, design-by-committee language. It has brilliant tooling, and is still much less complex than it's design-by-committee competitor C++, but it will never be easy to learn.
I find it relatively simple. Much simpler than C++ (obviously). For someone who can write C++ and has some experience wth OCaml/Haskell/F#, it's not a hard language.
Sure, C++ has a more complex spec, nobody can argue against that.
Complex is the wrong word. Baffling is a better word. Or counterintuitive, or cumbersome. If “easy enough for someone with experience in C++, OCaml, Haskell, and F#” were the same thing as “not hard” then I don’t think this debate would come up so frequently.
Of course, this is very subjective. For someone who only knows python or javascript at a superficial level, Rust may seem out of reach. But if you're ok with the most common programming paradigms, I don't find Rust baffling.
I mean, you can't expect to learn a new language in a few days, it'll always take a bit of work. My feeling is that people complaining of the language being hard aren't putting the effort.
My experience is that Rust is a relatively small language which doesn't introduce a lot of new concepts. The syntax is quite intuitive, and the compiler super helpful. The borrower checker was the only new thing for me. I'm not an expert at all, but my experience is that after spending 2 weeks full-time reading books and experimenting, I was able to work professionally with the language without feeling too much friction.
On the other hand, after spending much more time on C++, I don't feel really comfortable with the language.
What you call "baffling", I call "different". Being different doesn't mean it's "complex" or even "hard" (in isolation), but it can be baffling, in the same way that driving on the other side of the road for the first time can feel baffling (but doesn't mean it's "wrong").
Well, it does look like there is a will to mimic religious social structure in the community, be it as a satiric form of it. I mean, I guess they purposefully named their pancakes cargo, as in "cargo cult", didn't they? Rustacean, rustomicon, and the other few words I saw leak out of the community all seem to go in the same spirit. I'm almost surprised they didn't went with more fancy terms for these core concepts of ownership and borrowing. Perl was also full of religious stuff like blessing your object, though Larry was actually more in the "true devot" side of the line.
its not design by committee its design by Pull request
It doesn't have a central https://en.wikipedia.org/wiki/Benevolent_dictator_for_life like python used to
so people suggest and implement features as a group,
with code counting for a lot (although theoretical issues with safety/design also matter) as opposed to companies arguing for their pet features endlessly without much difference. Look at how long it takes C++ to get any new features.
My problem with rust is not the learning curve, but the absolute ugliness of the syntax. It's like Perl and C++ template metaprogramming had a child. I just can't stand it.
Python is my favourite, C is elegance in simplicity and Go is tolerable.
- it's very different from other languages. That's intentional but also an obstacle.
- it's a very complex language with a very terse syntax that looks like people are typing with their elbows and are hitting random keys. A single character can completely change the meaning of a thing. And it doesn't help that a lot of this syntax deeply nested.
- a lot of its features are hard to understand without deeper understanding of the theory behind them. This adds to the complexity. The type system and the borrowing mechanism are good examples. Unless you are a type system nerd a lot of that is just gobblygook to the average Python or Javascript user. This also makes it a very inappropriate language for people that don't have a master degree in computer science. Which these days is most programmers.
- it has widely used macros that obfuscate a lot of things that further adds to the complexity. If you don't know the macro definitions, it just becomes harder to understand what is going on. All languages with macros suffer from this to some degree.
I think LLMs can help a lot here these days. When I last tried to wrap my head around Rust that wasn't an option yet. I might have another go at it at some time. But it's not a priority for me currently. But llms have definitely lowered the barrier for me to try new stuff. I definitely see the value of a language like users. But it doesn't really solve a problem I have with the languages I do use (kotlin, python, typescript, etc.). I've used most popular languages at some point in my life. Rust is unique in how difficult it is to learn.
Borrow checker wouldn't get off my damn case - errors after errors - so I gave in. I allowed it to teach me - compile error by compile error - the proper way to do a threadsafe shared-memory ringbuffer. I was convinced I knew. I didn't. C and C++ lack ownership semantics so their compilers can't coach you.
Everyone should learn Rust. You never know what you'll discover about yourself.
It's an abstraction and convenience to avoid fiddling with registers and memory and that at the lowest level.
Everyone might enjoy their computation platform of their choice in their own way. No need to require one way nor another. You might feel all fired up about a particular high level language that you think abstracts and deploys in a way you think is right. Not everyone does.
You don't need a programming language to discover yourself. If you become fixated on a particular language or paradigm then there is a good chance you have lost sight of how to deal with what needs dealing with.
You are simply stroking your tools, instead of using them properly.
@gerdesj your tone was unnecessarily rude and mean. Part of your message makes a valid point but it is hampered by unnecessary insults. I hope the rest of your day improves from here.
I don’t specifically like Rust itself. And one doesn’t need a programming language to discover themselves.
My experience learning Rust has been that it imposes enough constraints to teach me important lessons about correctness. Lots of people can learn more about correctness!
I’ll concede- “everyone” was too strong; I erred on the side of overly provocative.
I think most of us enamoured with rust are c++ refugees glad the pain is lessened. The tooling including the compiler errors really are great though. I like the simplicity of c, but I would still pick rust for any new project just for the crates and knowing I'll never have to debug a segfault. I like pytorch and matlab fine for prototyping. Not much use for in-between languages like go or c# but I like the ergonomics of them just fine. I don't think it is at all weird for people coming from c++ or even c to like rust and prefer it over those other languages. We have already paid the cost of admission, and it comes with real benefits.
> You could see similar behavioural issues with C++ back in the days
I think that it's happened to some degree for almost every computer programming language for a whiles now - first was the C guys enamoured with their NOT Pascal/Fortran/ASM, then came the C++ guys, then Java, Perl, PHP, Python, Ruby, Javascript/Node, Go, and now Rust.
The vibe coding people seem to be the ones that are usurping Rust's fan boi noise at the moment - every other blog is telling people how great the tool is, or how terrible it is.
I know this feels like a positive vibe post and I don’t want to yuck anyone’s yum, but speaking for myself when someone tells me “everyone should” do anything, alarm bells sound off in my mind, especially when it comes to programming languages.
Python community famously learned the hard way that sometimes the programmer needs to know that there are multiple kinds of string.
Personally, I’ve been using to_owned instead. Some of the people looking at my code don’t write rust, and I figure it makes things a bit easier to understand.
I’m not sure why it’s counterintuitive that &str and String are different things. Do you also find it counterintuitive in C++ that std::string is different from const char* ? What about &[u8] and Vec<u8> ?
Strings are like time objects: most people and languages only ever deal with simplified versions of them that skip a lot of edge cases around how they work.
Unfortunately going from most languages to Rust forces you to speedrun this transition.
C++ is a horribly cmplicated language, sometimes I have to cast something to an int when it's already an integer. /s
I have a hard time understanding why people have such a hard time accepting that you need to convert between different text representations when it's perfectly accepted for numbers.
You may be able to draw one that way but it completely neglects the way people use the term ordinarily “a steep learning curve” is not an easy to learn thing.
In point of fact, I think the intended chart of the idiom is effort (y axis) to reach a given degree of mastery (x axis)
I don't think the idiom has in mind any particular curve. I think it's just another case of a misuse becoming idiomatic without any meaning beyond the phrase taken as a unit. E.g.
- another think coming -> another thing coming
- couldn't care less -> could care less
- the proof of the pudding is in the eating -> the proof is in the pudding
It's usually not useful to try to determine the meaning of the phrases on the right because they don't have any. What does it mean for proof to be in a pudding for example?
The idiom itself is fine, it's just a black box that compares learning something hard to climbing a mountain. But learning curves are real things that are still used daily so I just thought it was funny to talk as if a flat one was desirable.
Calling it inaccurate was too harsh; my definition only became common usage in 1970, and the original “time vs learning” is still used in academic circles.
Is there a concise document that explains major decisions behind Rust language design for those who know C++? Not a newbie tutorial, just straight to the point: why in-place mutability instead of other options, why encourage stack allocation, what problems with C++ does it solve and at what cost, etc.
Rust has better defaults for types than C++, largely because the C++ defaults came from C. Rust is more ergonomic in this regard. If you designed C++ today, it would likely adopt many of these defaults.
However, for high-performance systems software specifically, objects often have intrinsically ambiguous ownership and lifetimes that are only resolvable at runtime. Rust has a pretty rigid view of such things. In these cases C++ is much more ergonomic because objects with these properties are essentially outside the Rust model.
In my own mental model, Rust is what Java maybe should have been. It makes too many compromises for low-level systems code such that it has poor ergonomics for that use case.
Interestingly, CPU-bound high-performance systems are also incompatible with Rust’s model. Ownership for them is unambiguous, but Rust has another issue, doesn’t support multiple writeable references of the same memory accessed by multiple CPU cores in parallel.
A trivial example is multiplication of large square matrices. An implementation needs to leverage all available CPU cores, and a traditional way to do that you’ll find in many BLAS libraries – compute different tiles of the output matrix on different CPU cores. A tile is not a continuous slice of memory, it’s a rectangular segment of a dense 2D array. Storing different tiles of the same matrix in parallel is trivial in C++, very hard in Rust.
That's the tyranny of Gödel incompleteness (or maybe Rice's theorem, or even both): useful formal systems can be either sound or complete. Rust makes the choice of being sound, with the price of course being that some valid operations not being expressible in the language. C of course works the other way around; all valid programs can be expressed, but there's no (general) way to distinguish invalid programs from valid programs.
For your concrete example of subdividing matrixes, that seems like it should be fairly straightforward in Rust too, if you convert your mutable reference to the data into a pointer, wrap your pointer arithmetic shenanigans in an unsafe block and add a comment at the top saying more or less "this is safe because the different subprograms are always operating on disjoint subsets of the data, and therefore no mutable aliasing can occur"?
I don’t use C++ for most of my applications. I only use C++ to build DLLs which implement CPU-bound performance sensitive numeric stuff, and sometimes to consume C++ APIs and third-party libraries.
Most of my applications are written in C#.
C# provides memory safety guarantees very comparable to Rust, other safety guarantees are better (an example is compiler option to convert integer overflows into runtime exceptions), is a higher level language, great and feature-rich standard library, even large projects compile in a few seconds, usable async IO, good quality GUI frameworks… Replacing C# with Rust would not be a benefit.
Java should have been like Modula-3, Eiffel, Active Oberon, unfortunately it did not and has been catching up to rethink its design while preserving its ABI.
Thankfully C# has mostly catched up with those languages, as the other language I enjoy using.
After that, is the usual human factor on programming languages adoption.
> However, for high-performance systems software specifically, objects often have intrinsically ambiguous ownership
What is the evidence for this? Plenty of high-performance systems software (browsers, kernels, web servers, you name it) has been written in Rust.
Also Rust does support runtime borrow-checking with Rc<RefCell<_>>. It's just less ergonomic than references, but it works just fine.
Anyone that works on e.g. database kernels that do direct DMA (i.e. all the high-performance ones) experiences this. The silicon doesn’t care about your programming language’s ownership model and will violate it at will. You can’t fix it in the language, you have to accept the behavior of the silicon. Lifetimes are intrinsically ambiguous because objects have neither a consistent nor persistent memory address, a pretty standard property in databases, and a mandatory property of large databases. Yes, you can kind of work around it in idiomatic Rust but performance will not be anything like comparable if you do. You have to embrace the nature of the thing.
The near impossibility of building a competitive high-performance I/O scheduler in safe Rust is almost a trope at this point in serious performance-engineering circles.
To be clear, C++ is not exactly comfortable with this either but it acknowledges that these cases exist and provides tools to manage it. Rust, not so much.
New DB's like Tigerbeetle are written in Zig. Memory control was one of the prime reasons. Rust's custom allocators for the standard library have been a WIP for a decade now.
I think the major decisions behind Rust is being explicit and making the programmer make decisions. No NULLs, no Implicit conversions, no dangling pointers. Lifetimes, Optional, Results, each Match branch needs to exist, etc.
Side note: Stack allocation is faster to execute as there's a higher probability of it being cached.
I would say that RAII is very explicit: Resource Acquisition Is Initialization. When you initialize the struct representing the resource you are acquiring the resource. If you have a struct representing a resource you have the resource. Knowing this, you are also acquiring a call to drop when it goes out of scope. I would argue that the difference here isn't explicitness.
Instead, I would argue that rust is favoring a form of explicitness together with correctness. You have to clean up that resource. I have seen arguments that you should be allowed to leak resources, and I am sympathetic, but if we agree on explicitness as a goal then perhaps you might understand the perspective that a leak should be explicit and not implicit in a the lack of a call a some method. Since linear types are difficult to implement auto-drops are easier if you favor easily doing the correct thing. If you want to leak your resource, stash it in some leak list or unsafe erase it. That is the thing that should be explicit: the unusual choice, not all choices and not the usual choice alone.
But yeah, the drop being implicit in the explicit initialization does lead to developers ignoring it just like a leak being implicit if you forget to call a function often leads to unintentionally buggy programs. So when a function call ends they won't realize that a large number of objects are about to get dropped.
To answer your original question, the rationale is not in one concise location but is spread throughout the various RFCs that lead to the language features.
>> being explicit and making the programmer make decisions
>Why RAII then?
Their quote is probably better rephrased as _being explicit and making the programmer make decisions when the compiler's decision might impact safety_
Implicit conversion between primitives may impact the safety of your application. Implicit memory management and initialization is something the compiler can do safely and is central to Rust's safety story.
"Just write everything async" is not remotely a good solution to the problem. Not everything needs to be async (in fact most things don't), and it's much harder to reason about async code. The issue is very much not overblown.
Why is async code harder to reason about? I've been using it in C# and the entire point is that it lets you write callbacks in a way that appears nearly identical to synchronous code. If you dive into concurrency (which is a separate thing but can be utilized with async code, such as joining multiple futures at the same time), that parts hard whether you're doing it with async or with explicit threads.
One reason why async-await is trivial in .NET is garbage collector. C# rewrites async functions into a state machine, typically heap allocated. Garbage collector automagically manages lifetimes of method arguments and local variables. When awaiting async functions from other async functions, the runtime does that for multiple async frames at once but it’s fine with that, just a normal object graph. Another reason, the runtime support for all that stuff is integrated into the language, standard library, and most other parts of the ecosystem.
Rust is very different. Concurrency runtime is not part of the language, the standard library defined bare minimum, essentially just the APIs. The concurrency runtime is implemented by “Tokio” external library. Rust doesn’t have a GC; instead, it has a borrow checker who insists on exactly one owner of every object at all times, makes all memory allocations explicit, and exposed all these details to programmer in the type system.
These factors make async Rust even harder to use than normal Rust.
I don't know about C#, but at least in Rust, one reason is that normal (non-async) functions have the property that they will run until they return, they panic, or the program terminates. I.e. once you enter a function it will run to completion unless it runs "forever" or something unusual happens. This is not the case with async functions -- the code calling the async function can just drop the future it corresponds to, causing it to disappear into the ether and never be polled again.
That’s not a solution to the coloring problem any more than making everything red was in 2015 (ie, all the tradeoffs mentioned in the article [0] still apply).
It’s completely overblown. Almost every language with async has the same “problem”.
I’m not calling this the pinnacle of async design, but it’s extremely familiar and is pretty good now. I also prefer to write as much async as possible.
Async closures landed in stable recently and have been a nice QoL improvement, although I had gotten used to working around their absence well enough previously that they haven’t been revolutionary yet from the like “enabling new architectural patterns” perspective or anything like that.
I very rarely have to care about future pinning, mostly just to call the pin macro when working with streams sometimes.
While I'd like to have it, it doesn't stop me from writing a great deal of production code without those niceties.
When it came time for me to undo all the async-trait library hack stuff I wrote after the feature landed in stable, I realized I wasn't really held back by not having it.
This reads like a list of symptoms of what's wrong with the ergonomics of Rust. This is not to bash Rust. It has its uses. But you need to balance what you are sacrificing for what you are getting.
Bonus: do it with no heap allocation. This actually makes it easier because you basically don’t deal with lifetimes. You just have a state object that you pass to your input system, then your guest cpu system, then your renderer, and repeat.
My second (and final) rust project was a gameboy emulator that basically worked the same way.
But one of the best things about learning by writing an emulator is that there’s enough repetition you begin looking for abstractions and learn about macros and such, all out of self discovery and necessity.
I’ve found emulators to be a pretty poor first project for rust specifically for the reasons you alluded to: That you need to know to write it without heap allocation (or other hoop jumping so long as you avoid juggling lifetimes) when so much literature and example emulator code doesn’t do this is a recipe for a bad experience. Ask me how I know.
If you’re going to write an emulator in this style, why even use an imperative language when something like Haskell is designed for this sort of thing?
Compiler can figure that out, but the thing is compiler needs also to understand lifetimes at the site where this function is called. In general case compiler will not look into the code of a called function to see what it does, compiler relies on a function declaration.
That `longest` if defined without explicit lifetimes treated like a lifetime of a return value is the same as of the first argument. It is a rule "lifetime elision", which allows to not write lifetimes explicitly in most cases.
But `longest` can return a second reference also. With added lifetimes the header of the function says exactly that: the lifetime of a return value is a minimum of lifetimes of arguments. Not the lifetime of the first one.
You’re passing in two references and returning a reference.
The compiler knows the returned reference must be tied to one of the incoming references (since you cannot return a reference to something created within the function, and all inputs are references, the output must therefore be referencing the input). But the compiler can’t know which reference the result comes from unless you tell it.
Theoretically it could tell by introspecting the function body, but the compiler only works on signatures, so the annotation must be added to the function signature to let it determine the expected lifetime of the returned reference.
> Theoretically it could tell by introspecting the function body, but the compiler only works on signatures
Note that this is an intentional choice rather than a limitation, because if the compiler analyzed the function body to determine lifetimes of parameters and return values, then changing the body of a function could be a non-obvious breaking API change. If lifetimes are only dependent on the signature, then its explicit what promises you are or are not making to callers of a function about object lifetimes, and changing those promises must be done intentionally by changing the signature rather than implicitly.
To make a compiler automatically handle all of the cases like that, you will need to do an extensive static analysis, which would make compiling take forever.
> Use String and clone() and unwrap generously; you can always refactor later
At that point you might as well be writing Java or Go or whatever though. GC runtimes tend actually to be significantly faster for this kind of code, since they can avoid all those copies by sharing the underlying resource. By the same logic, you can always refactor the performance-critical stuff via your FFI of choice.
So long as you're aware that you're not optimizing, it's fine. Trying to build something useful as a new Rust dev while worrying about lifetimes is going to be quite challenging, unless your intention is to specifically learn about lifetimes and the borrow checker.
Yes the borrow checker is central to Rust, but there are other features to the language that people _also_ need to learn and explore to be productive. Some of these features may attract them to Rust (like pattern matching / traits / etc.)
"Significantly" and "this kind" are load bearing sentences here. In applications where predictable latency is desired, cloning is better than GC.
This is also the baby steps of learning the language. As a programmer gets better they will recognize when they are making superflous clones. Refactoring performance-critical stuff in FFI, however, is painful and wont get easier with time.
Furthermore, in real applications, this only really applies to Strings and vectors. In most of my applications most `clones` are of reference types - which is only marginally more expensive than memory sharing under a GC.
I went through this the first year that I did Advent of Code in rust, like okay I read in all the strings from the input file and now they're in a vector, so I'm going to iterate the vector and add references to those strings into this other structure, but of course they're still owned by the original vector, that's awkward. Oh wait I can iter_into and then I get owned objects and that ownership can be transferred to the other structure instead, but now I need them to also be keys in a map, do I use references for that too?
Cloning small objects is lightning fast, turns out in a lot of these cases it makes sense to just do the clone, especially when it's a first pass. The nice thing is that at least rust makes you explicitly clone() so you're aware when it's happening, vs other languages where it's easy to lose track of what is and isn't costing you memory. So you can see that it's happening, you can reason about it, and once the bones of the algorithm are in place, you can say "okay, yes, this is what should ultimately own this data, and here's the path it's going to take to get there, and these other usages will be references or clones.
It's really not, it's the way python works. Heap allocations are "fast" on modern CPUs that are too fast to measure for most stuff, but they're much (much) slower than the function call and code you're going to use to operate on whatever the thing it was you cloned.
Code that needs memory safety and can handle performance requirements like this has many options for source language, almost none of which require blog posts to "flatten the learning curve".
(And to repeat: it's much slower than a GC which doesn't have to make the clone at all. Writing Rust that is "Slower Than Java" is IMHO completely missing the point. Java is boring as dirt, but super easy!)
For people who don't get the reference, this might be referring to the notoriously gnarly task of implementing a doubly-linked lists in Rust [1]
It is doable, just not as easy as in other languages because a production-grade linked-list is unsafe because Rust's ownership model fundamentally conflicts with the doubly-linked structure. Each node in a doubly-linked list needs to point to both its next and previous nodes, but Rust's ownership rules don't easily allow for multiple owners of the same data or circular references.
You can implement one in safe Rust using Rc<RefCell<Node>> (reference counting with interior mutability), but that adds runtime overhead and isn't as performant. Or you can use raw pointers with unsafe code, which is what most production implementations do, including the standard library's LinkedList.
Rust still needs a way out of that mess. It's conceptually possible to have compile time checking for this. Think of RefCell/Weak and .upgrade() and .borrow() being checked at compile time.
I've discussed this with some of the Rust devs. The trouble is traits. You'd need to know if a trait function could borrow one of its parameters, or something referenced by one of its parameters. This requires analysis that can't be done until after generics have been expanded. Or a lot more attributes on trait parameters. This is a lot of heavy machinery to solve a minor problem.
In practice, it really doesn't. The difficulty of implementing doubly linked lists has not stopped people from productively writing millions of lines of Rust in the real world. Most programmers spend less than 0.1% of their time reimplementing linked data structures; rust is pretty useful for the other 99.9%.
Apologies since I have not taken the time to learn rust yet, but I've written a lot of modern C++. Is the ownership model kind of like std::unique_ptr and std::move, and `Rc<RefCell<Node>>` the same idea as `std::shared_ptr`? But less idiomatic? Or do I have the wrong idea?
Not really, because Rust enforces a "many readers or one writer" invariant on everything that has no C++ equivalent. That invariant is precisely what makes the doubly-linked list case hard (because every interior node in the list would be readable from two places, which means it can never be written to).
If that's a specific use case you need to handle, it's O(1) again if you have a pointer to both the node to be removed and the previous node.
Whether it's more efficient to carry a second pointer around when manipulating the list, or store a second pointer in every list node (aka double linked list) is up to your problem space.
So that you learn that loaning is for giving temporary shared^exclusive access within a statically-known scope, and not for storing data.
Trying to construct permanent data structures using non-owning references is a very common novice mistake in Rust. It's similar to how users coming from GC languages may expect pointers to local variables to stay valid forever, even after leaving the scope/function.
Just like in C you need to know when malloc is necessary, in Rust you need to know when self-contained/owning types are necessary.
The biggest thing I’ve run into where I really want self-referential types is for work that I want to perform once and then cache, while still needing access to the original data.
An example: parsing a cookie header to get cookie names and values.
In that case, I settled on storing indexes indicating the ranges of each key and value instead of string slices, but it’s obviously a bit more error prone and hard to read. Benchmarking showed this to be almost twice as fast as cloning the values out into owned strings, so it was worth it, given it is in a hot path.
I do wish it were easier though. I know there are ways around this with Pin, but it’s very confusing IMO, and still you have to work with pointers rather than just having a &str.
For me it was like Haskell. I spent an afternoon on it, my brain hurt too much, and I relegated it to the category of languages that are too complicated for what I need to do with a computer.
Languages I liked, I liked immediately. I didn’t need to climb a mountain first.
Unfortunately, yes. I still end up writing C++ instead of Rust for low-level system stuff. Since I also know Go - I usually prefer that when I need lean, middleware services. Learned Rust (somewhat) with great difficulty but still don't use it anywhere. Still haven't figured out how to design effectively using Rust and approaches suggested in that article like clone()/unwrap() stuff and refactor later just leave a bad taste in that mouth.
The thing is, once you internalized the concepts (ownership, borrowing, lifetimes), it's very hard to remember what made it difficult in the first place. It's "curse of knowledge" in some sense.
What's changed since 2015 is that we ironed out some of the wrinkles in the language (non-lexical lifetimes, async) but the fundamental mental model shift required to think in terms of ownership is still a hurdle that trips up newcomers.
100%. Newcomers still struggle a bit, especially if they've never used C/C++ before.
A good way to get people comfortable with the semantics of the language before the borrow checker is to encourage them to clone() strings and structs for a bit, even if the resulting code is not performant.
Once they dip their toes into threading and async, Arc<Lock<T>> is their friend, and interior mutability gives them some fun distractions while they absorb the more difficult concepts.
Do you mean `Arc<Mutex<T>>`? Yeah, I agree.
Wrote a blog post on that topic as well: https://corrode.dev/blog/prototyping/
The title is a bit of a misnomer, but it's about beginner-friendly escape hatches in the language. Perhaps it's useful to newcomers.
It's like reading "A Discipline of Programming", by Dijkstra. That morality play approach was needed back then, because nobody knew how to think about this stuff.
Most explanations of ownership in Rust are far too wordy. See [1]. The core concepts are mostly there, but hidden under all the examples.
That explains the model. Once that's understood, all the details can be tied back to those rules.[1] https://doc.rust-lang.org/book/ch04-01-what-is-ownership.htm...
That's not explaining ownership, that motivating it. Which is fine. The thing that's hard to explain and learn is how to read function signatures involving <'a, 'b>(...) -> &'a [&'b str] or whatever. And how to understand and fix the compiler errors in code calling such a function.
Is it a lot different from std::unique_ptr in C++?
I thought the Rust Book was too verbose but I liked Comprehensive Rust: https://google.github.io/comprehensive-rust/
I felt like I understood the stuff in the book based on cursory reading, but I haven't tried to actually use it.
>Is it a lot different from std::unique_ptr in C++?
Is knowing C++ a pre-requisite?
I think the most important lesson is this:
Ownership is easy, borrowing is easy, what makes the language super hard to learn is that functions must have signatures and uses that together prove that references don't outlive the object.
Also: it's better not store referenced object in a type unless it's really really needed as it makes the proof much much more complex.
That really doesn't explain the model because you have completely left out the distinction between exclusive/shared (or mutable/immutable) borrows. Rust made a large number of choices with respect to how it permits such borrows and those do not follow from this brief outline nor from intuition or common sense. For example, the no aliasing rule is motivated not by intuition or common sense but from a desire to optimize functions.
The most complicated aspect of the borrows comes about from the elision rules which will silently do the wrong thing and will work fantastically until they don't at which point the compiler error is pointing at a function complaining about a lifetime parameter of a parameter with the trait method implying that the parameter has to live too long but the real problem was a lifetime in the underlying struct or a previous broken lifetime bound. Those elision rules are again not-intuitive and don't fall out of your explanation axiomatically. They were decisions that were made to attempt to simplify the life of programmers.
And, after someone who doesn't know rust reads this neat and nice summary, they would still know nothing about rust. (Except "this language's compiler must have some black magic in it.")
Summarizing a set of concepts in a way that feels correct and complete to someone who understands them, is a much easier task than explaining them to someone who doesn't. If we put this in front of someone who's only worked with call-by-sharing languages, do you think they'll get it right away? I'm skeptical.
For me it really clicked when I realized ownership / lifetimes / references are just words used to talk about when things get dropped. Maybe because I have a background in C so I'm used to manual memory management. Rust basically just calls 'free' for you the moment something goes out of scope.
All the jargon definitely distracted me from grasping that simple core concept.
Almost all of it.
Rust also has the “single mutable reference” rule. If you have a mutable reference to a variable, you can be sure nobody else has one at the same time. (And the value itself won’t be mutated).
Mechanically, every variable can be in one of 3 modes:
1. Directly editable (x = 5)
2. Have a single mutable reference (let y = &mut x)
3. Have an arbitrary number of immutable references (let y = &x; let z = &x).
The compiler can always tell which mode any particular variable is in, so it can prove you aren’t violating this constraint.
If you think in terms of C, the “single mutable reference” rule is rust’s way to make sure it can slap noalias on every variable in your program.
This is something that would be great to see in rust IDEs. Wherever my cursor is, it’d be nice to color code all variables in scope based on what mode they’re in at that point in time.
"Rust basically just calls 'free' for you the moment something goes out of scope."
C++ does that too with RAII. Go ahead and use whatever STL containers you like, emplace objects onto them, and everything will be safely single-owned with you never having to manually new or delete any of it.
The difference is that C++'s guarantees in this regard derive from a) a bunch of implementation magic that exists to hide the fact that those supposedly stack-allocated containers are in fact allocating heap objects behind your back, and b) you cooperating with the restrictions given in the API docs, agreeing not to hold pointers to the member objects or do weird things with casting. You can use scoped_ptr/unique_ptr but the whole time you'll be painfully aware of how it's been bolted onto the language later and whenever you want you can call get() on it for the "raw" underlying pointer and use it to shoot yourself in the foot.
Rust formalizes this protection and puts it into the compiler so that you're prevented from doing it "wrong".
the tradeoff is that ~you have to guess where rust is doing the frees, and you might be wrong. in the end this would be strictly equivalent to an explicit instruction to free, with the compiler refusing to compile if the free location broke the rules.
It's really too bad rust went the RAII route.
Right. If you come to Rust from C++ and can write good C++ code, you see this as "oh, that's how to think about ownership". Because you have to have a mental model of ownership to get C/C++ code to work.
But if you come from Javascript or Python or Go, where all this is automated, it's very strange.
The list in the above comment isn’t a summary — it’s a precise definition. It can and must be carefully explained with lots of examples, contrasts with other languages, etc., but the precise definition itself must figure prominently, and examples and intuition should relate back to it transparently.
Practically, I think it suggests that learning the borrow checker should start with learning how memory works, rather than any concepts specific to Rust.
>the real owner has to be a reference-counted cell.
And what is that? Its easy to fall in the trap of making explanations that is very good (if you already understand).
This explanation doesn't expose anything meaningful to my mind, as it doesn't define ownership and borrowing, both words being apparently rooted in an analogy with financial asset management.
I'm not acquainted with Rust, so I don't really know, but I wonder if the wording plays a role in the difficulty of concept acquisition here. Analogies are often double edged tools.
Maybe sticking to a more straight memory related vocabulary as an alternative presentation perspective might help?
The way I think about it is more or less in terms of how a C program would work: if you assume a heap allocated data structure, the owner is the piece of code that is responsible for freeing the allocation at the appropriate time. And a reference is just a pointer with some extra compile time metadata that lets the borrow checker prove that the reference doesn’t outlive the referent and that there’s no mutable aliasing.
If you've worked inside of CPython or other programs with manual reference counting, the idea of borrowing shows up there, where you receive a reference from another part of the program and then mess with the object without tweaking the reference count, "borrowing" an existing reference because any copies you've of the address will be short lived. The term shows up throughout CPython.
I find it strange that you relate borrowing and ownership to financial asset management.
From that angle, it indeed doesn’t seem to make sense.
I think, but might be completely wrong, that viewing these actions from their usual meaning is more helpful: you own a toy, it’s yours to do as tou please. You borrow a toy, it’s not yours, you can’t do whatever you want with it, so you can’t hold on to it if the owner doesn’t allow it, and you can’t modify it for the same reasons.
Analogies often leak.
1. In real life I can borrow a toy from you and while I have that toy in my hands, the owner can exchange ownership with somebody else, while the object is borrowed by me. I.e. in real life the borrowing is orthogonal to ownership. In rust you can't do that.
2. Borrowing a toy is more akin to how mutable references work in rust. Immutable references allow multiple people to play with the same toy simultaneously, provided they don't change it.
Analogies are just analogies
The second bullet in the second section is overpromising badly. In fact there are many, many, many ways to write verifiably correct code that leaves no dangling pointers yet won't compile with rustc.
Frankly most of the complexity you're complaining about stems from attempts to specify exactly what magic the borrow checker can prove correct and which incantations it can't.
It took me a few tries to get comfortable with Rust—its ownership model, lifetimes, and pervasive use of enums and pattern matching were daunting at first. In my initial attempt, I felt overwhelmed very early on. The second time, I was too dogmatic, reading the book line by line from the very first chapter, and eventually lost patience. By then, however, I had come to understand that Rust would help me learn programming and software design on a deeper level. On my third try, I finally found success; I began rewriting my small programs and scripts using the rudimentary understanding I had gained from my previous encounters. I filled in the gaps as needed—learning idiomatic error handling, using types to express data, and harnessing pattern matching, among other techniques.
After all this ordeal, I can confidently say that learning Rust was one of the best decisions I’ve made in my programming career. Declaring types, structs, and enums beforehand, then writing functions to work with immutable data and pattern matching, has become the approach I apply even when coding in other languages.
If a language needs an article like this, absolutely begging people to bite the bullet to learn it, maybe that's a language design smell.
Disclaimer: I haven't taken the time to learn Rust so maybe don't take this too seriously..
I don't know how to read your comment other than "nothing hard is worth doing". Some things have benefits and drawbacks, is the existence of drawbacks always a non-starter for you?
I'm trying to phrase this as delicately as I can but I am really puzzled.
If someone wrote an article about how playing the harp is difficult, just stick with it... would you also say that playing the harp is a terrible hobby?
Maybe people need persuading to learn Rust not just because they think it's hard, but also because they think it's bad? Not everything hard is worth doing. Difficulty is just one of the factors to consider.
I started to learn Rust, but I was put off by the heavy restrictions the language imposes and the attitude that this is the only safe way. There's a lack of acknowledgement, at least in beginner materials, that by choosing to write safe Rust you're sacrificing many perfectly good patterns that the compiler can't understand in exchange for safety. Eventually I decided to stop because I didn't like that tradeoff (and I didn't need it for my job or anything)
Rust design decisions are pretty hard to understand sometimes, Mojo is another language with a borrow-checker but it is not nearly as hard to learn as Rust due to making a few decisions. First is value semantics, in Rust people are told to always clone when learning, why isn't this semantics built into the language? It is what you have in most static languages - C, C++, Go, etc. This is the mental model many people come to Rust with.
Secondary, Mojo's lifetime does not tell the compiler when a value is safe to use but when it is safe to delete, in this way the lifetime is not scope based, references will extend the lifetime of the value they reference, but values will be destroyed immediately after their last use. In Mojo you'll never see "value does not live long enough".
Just these two design decisions defines away so many ergonomic issues.
> people are told to always clone when learning, why isn't this semantics built into the language?
Because cloning as opposed to copying is expensive and it generates a new instance of a type. In C, you don't clone, you simply copy the struct or pointer, which will lead to a pointer to the same memory or a struct with members pointing to the same memory.
C++ on the other hand has a copy constructor, and you have to move explicitly, often generating unnecessary copies (in the sense of clone)
> Mojo's lifetime does not tell the compiler when a value is safe to use but when it is safe to delete,
What happens if you pass the variable mutably to a function?
> What happens if you pass the variable mutably to a function?
What happens in what manner? Mojo uses ASAP memory model, values will always be destroyed at the point of its last use. Mojo dataflow analysis will track this.
In terms of safety, Mojo will enforce `alias xor mutability` - like in Rust.
> C++ on the other hand has a copy constructor, and you have to move explicitly, often generating unnecessary copies (in the sense of clone)
Mojo also has copy and move constructors, but unlike in C++ these are not synthesised by default; the type creator has to either explicitly define the constructors or add a synthesiser. In Mojo, you can have types that are not copyable and not movable, these types can only be passed by reference. You can also have types that are copyable but not movable, or movable but not copyable.
The truth is that by the time you are a senior developer, you will have encountered the lessons that make rust worth learning but may not have truly understood all the implications.
Many people will think, I have a garbage collected language, rust has nothing to teach me. Even in garbage collected languages, people create immutable types because the possibility of shared references with mutability makes things incredibly chaotic that they look for immutability as a sort panacea. However, once you have immutable types you quickly realize that you also need ergonomic ways of modifying those objects, the methods you create to do so are often more cumbersome than what would be permitted for a mutable object. You wish there was some way to express, "There is a time where this object is mutable and then it becomes immutable." Enter the borrow checker.
Once you are borrow checking... why are you garbage collecting? Well, expressing those timelines of mutability and existence is a cost because you need to understand the timeline and most people would rather not spend that energy--maybe mutability or the poor ergonomics of immutable objects wasn't so bad. So, I garbage collect because I do not want to understand the lifetimes of my objects. Not understanding the lifetimes of objects is what makes shared mutability hard. Immutability eliminates that problem without requiring me to understand. Rust can teach this lesson to you so that you make an informed choice.
Of course, you can also just listen to me and learn the same lesson but there is value for many people to experience it.
Learning any programming language at all feels 10x as hard to beginners, so you might as well say programming is not worth learning period in this case. Anything new has a learning curve to it.
I think you can have a lot of debate on the design decisions on Rust, but I don't think the need for these articles tell you a lot about the language itself. I'd argue that Python needs articles like this more so than Rust does, but for entirely different reasons. In two decades of more and more programmers who aren't coming from an engineering background, I've yet to see anyone who used a Python generator or slots. Data Classes are less rare, but mainly in the form of pydantics "version". Which doesn't exactly matter for a lot of Python code... This is a world where 4chan can serve 4 million concurrent users an apache server running a 10k line PHP file neither of which have been updated since 2015... so you can be fine doing inefficient and entirely in-memory Python code 95% (or more) of the time.
That doesn't mean you should though. Imagine how much energy is being wasted globally on bad Python code... The difference is of course that anyone can write it, and not everyone can write Rust. I'm not personally a big fan of Rust, I'd chose Zig any day of the week... but then I'd also choose C over C++, and I frankly do when I optimise Python code that falls in those last 5%. From that perspective... of someone who really has to understand how Python works under the hood and when to do what, I'd argue that Rust is a much easier langauge to learn with a lot less "design smell". I suppose Python isn't the greatest example as even those of us who love it know that it's a horrible language. But I think it has quite clearly become the language of "everyone" and even more so in the age of LLM. Since our AI friends will not write optimised Python unless you specifically tell them to use things like generators and where to use them, and since you (not you personally) won't because you've never heard about a generator before, then our AI overlords won't actually help.
The article focuses on the learning curve rather than the problem Rust is solving, as an observation. Think you need both of those to draw a conclusion as to whether it’s worth doing.
I suspect an article like this says more about the author than the language.
Note I’m not being critical of the author here. I think it’s lovely to turn your passion into trying to help others learn.
> maybe that's a language design smell
why
> If a language needs an article like this, absolutely begging people to bite the bullet to learn it, maybe that's a language design smell.
The problem with articles like this is that they don't really get to the heart of the problem:
There are programs that Rust will simply not let you write.
Rust has good reasons for this. However, this is fundamentally different from practically every programming language that people have likely used before where you can write the most egregious glop and get it to compile and sometimes even kinda-sorta run. You, as a programmer, have to make peace with not being able to write certain types of programs, or Rust is not your huckleberry.
Maybe Rust is so complex, it is even more complex for an LLM to generate correct code (one-shot) without hallucinating non-existent functions.
Would rather have that than all the issues that JavaScript or any other weakly typed and dynamically typed language.
There _are_ more than two programming languages, though. I feel like most of the debates about Rust devolve into the same false choice between safety and ease.
Before Rust I was hearing the same argument from Haskell or Scala developers trying to justify their language of choice.
I know Rust is here to stay, but I think it’s mostly because it has a viable ecosystem and quality developer tools. Its popularity is _in spite of_ many of its language features that trade that extra 1% of safety for 90% extra learning curve.
> features that trade that extra 1% of safety for 90% extra learning curve.
I remember both MS and goog having talks about real-world safety issues in the range of 50% of cases were caused by things that safe rust doesn't allow (use after free, dangling pointers, double free, etc). The fact that even goog uses it, while also developing go (another great language with great practical applications) is telling imo.
I have taken the time to learn rust and you're absolutely right. It's a very complex, design-by-committee language. It has brilliant tooling, and is still much less complex than it's design-by-committee competitor C++, but it will never be easy to learn.
> It's a very complex
I find it relatively simple. Much simpler than C++ (obviously). For someone who can write C++ and has some experience wth OCaml/Haskell/F#, it's not a hard language.
Sure, C++ has a more complex spec, nobody can argue against that.
Complex is the wrong word. Baffling is a better word. Or counterintuitive, or cumbersome. If “easy enough for someone with experience in C++, OCaml, Haskell, and F#” were the same thing as “not hard” then I don’t think this debate would come up so frequently.
Of course, this is very subjective. For someone who only knows python or javascript at a superficial level, Rust may seem out of reach. But if you're ok with the most common programming paradigms, I don't find Rust baffling.
I mean, you can't expect to learn a new language in a few days, it'll always take a bit of work. My feeling is that people complaining of the language being hard aren't putting the effort.
My experience is that Rust is a relatively small language which doesn't introduce a lot of new concepts. The syntax is quite intuitive, and the compiler super helpful. The borrower checker was the only new thing for me. I'm not an expert at all, but my experience is that after spending 2 weeks full-time reading books and experimenting, I was able to work professionally with the language without feeling too much friction.
On the other hand, after spending much more time on C++, I don't feel really comfortable with the language.
What you call "baffling", I call "different". Being different doesn't mean it's "complex" or even "hard" (in isolation), but it can be baffling, in the same way that driving on the other side of the road for the first time can feel baffling (but doesn't mean it's "wrong").
There is a trade off. Rust gave us fast, and safe. It did not give us "easy to learn".
I think it is a very good example of why "design by committee" is good. The "Rust Committee" has done a fantastic job
Thank you
They say a camel is a horse designed by a committee (https://en.wiktionary.org/wiki/a_camel_is_a_horse_designed_b...)
Yes:
* Goes twice as far as a horse
* On half the food and a quarter the water of a horse
* Carries twice as much as a horse
Yes, I like design by committee. I have been on some very good, and some very bad committees, but there is nothing like the power of a good committee
Thank you Rust!
It's just a programming language, not a religion.
Well, it does look like there is a will to mimic religious social structure in the community, be it as a satiric form of it. I mean, I guess they purposefully named their pancakes cargo, as in "cargo cult", didn't they? Rustacean, rustomicon, and the other few words I saw leak out of the community all seem to go in the same spirit. I'm almost surprised they didn't went with more fancy terms for these core concepts of ownership and borrowing. Perl was also full of religious stuff like blessing your object, though Larry was actually more in the "true devot" side of the line.
its not design by committee its design by Pull request It doesn't have a central https://en.wikipedia.org/wiki/Benevolent_dictator_for_life like python used to so people suggest and implement features as a group, with code counting for a lot (although theoretical issues with safety/design also matter) as opposed to companies arguing for their pet features endlessly without much difference. Look at how long it takes C++ to get any new features.
> Look at how long it takes C++ to get any new features.
I’m not sure “it doesn’t have enough features” has ever been anyone’s complaint about C++.
My problem with rust is not the learning curve, but the absolute ugliness of the syntax. It's like Perl and C++ template metaprogramming had a child. I just can't stand it.
Python is my favourite, C is elegance in simplicity and Go is tolerable.
Rust has a few big hurdles for new users:
- it's very different from other languages. That's intentional but also an obstacle.
- it's a very complex language with a very terse syntax that looks like people are typing with their elbows and are hitting random keys. A single character can completely change the meaning of a thing. And it doesn't help that a lot of this syntax deeply nested.
- a lot of its features are hard to understand without deeper understanding of the theory behind them. This adds to the complexity. The type system and the borrowing mechanism are good examples. Unless you are a type system nerd a lot of that is just gobblygook to the average Python or Javascript user. This also makes it a very inappropriate language for people that don't have a master degree in computer science. Which these days is most programmers.
- it has widely used macros that obfuscate a lot of things that further adds to the complexity. If you don't know the macro definitions, it just becomes harder to understand what is going on. All languages with macros suffer from this to some degree.
I think LLMs can help a lot here these days. When I last tried to wrap my head around Rust that wasn't an option yet. I might have another go at it at some time. But it's not a priority for me currently. But llms have definitely lowered the barrier for me to try new stuff. I definitely see the value of a language like users. But it doesn't really solve a problem I have with the languages I do use (kotlin, python, typescript, etc.). I've used most popular languages at some point in my life. Rust is unique in how difficult it is to learn.
Rust is wonderful but humbling!
It has a built in coach: the borrow checker!
Borrow checker wouldn't get off my damn case - errors after errors - so I gave in. I allowed it to teach me - compile error by compile error - the proper way to do a threadsafe shared-memory ringbuffer. I was convinced I knew. I didn't. C and C++ lack ownership semantics so their compilers can't coach you.
Everyone should learn Rust. You never know what you'll discover about yourself.
"Rust is wonderful but humbling!"
It's an abstraction and convenience to avoid fiddling with registers and memory and that at the lowest level.
Everyone might enjoy their computation platform of their choice in their own way. No need to require one way nor another. You might feel all fired up about a particular high level language that you think abstracts and deploys in a way you think is right. Not everyone does.
You don't need a programming language to discover yourself. If you become fixated on a particular language or paradigm then there is a good chance you have lost sight of how to deal with what needs dealing with.
You are simply stroking your tools, instead of using them properly.
@gerdesj your tone was unnecessarily rude and mean. Part of your message makes a valid point but it is hampered by unnecessary insults. I hope the rest of your day improves from here.
I don’t specifically like Rust itself. And one doesn’t need a programming language to discover themselves.
My experience learning Rust has been that it imposes enough constraints to teach me important lessons about correctness. Lots of people can learn more about correctness!
I’ll concede- “everyone” was too strong; I erred on the side of overly provocative.
Wow who pissed in your coffee? he likes rust ok?
And he's telling other people they should like it as well, because he has seen the light.
My gut feeling says that there's a fair bit of Stockholm Syndrome involved in the attachments people form with Rust.
You could see similar behavioral issues with C++ back in the days, but Rust takes it to another level.
I think most of us enamoured with rust are c++ refugees glad the pain is lessened. The tooling including the compiler errors really are great though. I like the simplicity of c, but I would still pick rust for any new project just for the crates and knowing I'll never have to debug a segfault. I like pytorch and matlab fine for prototyping. Not much use for in-between languages like go or c# but I like the ergonomics of them just fine. I don't think it is at all weird for people coming from c++ or even c to like rust and prefer it over those other languages. We have already paid the cost of admission, and it comes with real benefits.
> You could see similar behavioural issues with C++ back in the days
I think that it's happened to some degree for almost every computer programming language for a whiles now - first was the C guys enamoured with their NOT Pascal/Fortran/ASM, then came the C++ guys, then Java, Perl, PHP, Python, Ruby, Javascript/Node, Go, and now Rust.
The vibe coding people seem to be the ones that are usurping Rust's fan boi noise at the moment - every other blog is telling people how great the tool is, or how terrible it is.
> Everyone should learn Rust.
I know this feels like a positive vibe post and I don’t want to yuck anyone’s yum, but speaking for myself when someone tells me “everyone should” do anything, alarm bells sound off in my mind, especially when it comes to programming languages.
Got recommended learning paths? I tend to prefer follow along adventures via video.
Check out Jon Gjengset.
https://www.youtube.com/@jonhoo
>For instance, why do you have to call to_string() on a thing that’s already a string?
It's so hard for me to take Rust seriously when I have to find out answers to unintuitive question like this
Python community famously learned the hard way that sometimes the programmer needs to know that there are multiple kinds of string.
Personally, I’ve been using to_owned instead. Some of the people looking at my code don’t write rust, and I figure it makes things a bit easier to understand.
I’m not sure why it’s counterintuitive that &str and String are different things. Do you also find it counterintuitive in C++ that std::string is different from const char* ? What about &[u8] and Vec<u8> ?
Better analogy is std::string_view vs std::string
Just because a language is not high level enough to have a unique concept of "string" type doesn't mean you shouldn't take it seriously.
Strings are like time objects: most people and languages only ever deal with simplified versions of them that skip a lot of edge cases around how they work.
Unfortunately going from most languages to Rust forces you to speedrun this transition.
The question is worded weird for fun. One is a string slice (like char*) and one is String or &String, which is closer to an object.
C++ is a horribly cmplicated language, sometimes I have to cast something to an int when it's already an integer. /s
I have a hard time understanding why people have such a hard time accepting that you need to convert between different text representations when it's perfectly accepted for numbers.
A learning curve measures time on the x axis and progress on the y axis.
A flat learning curve means you never learn anything :-\
You may be able to draw one that way but it completely neglects the way people use the term ordinarily “a steep learning curve” is not an easy to learn thing.
In point of fact, I think the intended chart of the idiom is effort (y axis) to reach a given degree of mastery (x axis)
I don't think the idiom has in mind any particular curve. I think it's just another case of a misuse becoming idiomatic without any meaning beyond the phrase taken as a unit. E.g.
- another think coming -> another thing coming
- couldn't care less -> could care less
- the proof of the pudding is in the eating -> the proof is in the pudding
It's usually not useful to try to determine the meaning of the phrases on the right because they don't have any. What does it mean for proof to be in a pudding for example?
The idiom itself is fine, it's just a black box that compares learning something hard to climbing a mountain. But learning curves are real things that are still used daily so I just thought it was funny to talk as if a flat one was desirable.
https://en.m.wikipedia.org/wiki/Learning_curve
It should be called "learning hill" instead.
People (colloquially) use phrases like "steep learning curve" because they imagine learning curve is something you climb up, a.k.a. a hill.
"Flattening the derivative of Rust's learning curve" really doesn't roll off the tongue though
Yeah that's true. But it would be on brand for a post that emphasizes the importance of accuracy and attention to detail.
A steep line still has a flat derivative
This is incorrect. A learning curve measures expertise on the x axis and effort on the y axis. Hence the saying "steep learning curve".
Calling it inaccurate was too harsh; my definition only became common usage in 1970, and the original “time vs learning” is still used in academic circles.
https://en.wikipedia.org/wiki/Learning_curve
It is unclear how this comment was meant; in any case, it is appreciated. As stated in the link:
“The common English usage aligns with a metaphorical interpretation of the learning curve as a hill to climb.”
Followed by a graph plotting x “experience” against y “learning.”
That’s interesting. I always intuitively assumed x-axis was progress and y-axis was cumulative effort.
What we want is an "effort/difficulty curve" that measures how difficult something typically is over time from introduction to proficiency
It also could mean you don't need to learn beyond a certain point.
Is there a concise document that explains major decisions behind Rust language design for those who know C++? Not a newbie tutorial, just straight to the point: why in-place mutability instead of other options, why encourage stack allocation, what problems with C++ does it solve and at what cost, etc.
Rust has better defaults for types than C++, largely because the C++ defaults came from C. Rust is more ergonomic in this regard. If you designed C++ today, it would likely adopt many of these defaults.
However, for high-performance systems software specifically, objects often have intrinsically ambiguous ownership and lifetimes that are only resolvable at runtime. Rust has a pretty rigid view of such things. In these cases C++ is much more ergonomic because objects with these properties are essentially outside the Rust model.
In my own mental model, Rust is what Java maybe should have been. It makes too many compromises for low-level systems code such that it has poor ergonomics for that use case.
Interestingly, CPU-bound high-performance systems are also incompatible with Rust’s model. Ownership for them is unambiguous, but Rust has another issue, doesn’t support multiple writeable references of the same memory accessed by multiple CPU cores in parallel.
A trivial example is multiplication of large square matrices. An implementation needs to leverage all available CPU cores, and a traditional way to do that you’ll find in many BLAS libraries – compute different tiles of the output matrix on different CPU cores. A tile is not a continuous slice of memory, it’s a rectangular segment of a dense 2D array. Storing different tiles of the same matrix in parallel is trivial in C++, very hard in Rust.
That's the tyranny of Gödel incompleteness (or maybe Rice's theorem, or even both): useful formal systems can be either sound or complete. Rust makes the choice of being sound, with the price of course being that some valid operations not being expressible in the language. C of course works the other way around; all valid programs can be expressed, but there's no (general) way to distinguish invalid programs from valid programs.
For your concrete example of subdividing matrixes, that seems like it should be fairly straightforward in Rust too, if you convert your mutable reference to the data into a pointer, wrap your pointer arithmetic shenanigans in an unsafe block and add a comment at the top saying more or less "this is safe because the different subprograms are always operating on disjoint subsets of the data, and therefore no mutable aliasing can occur"?
Hard in safe rust. you can just use unsafe in that one area and still benefit in most of your application from safe rust.
I don’t use C++ for most of my applications. I only use C++ to build DLLs which implement CPU-bound performance sensitive numeric stuff, and sometimes to consume C++ APIs and third-party libraries.
Most of my applications are written in C#.
C# provides memory safety guarantees very comparable to Rust, other safety guarantees are better (an example is compiler option to convert integer overflows into runtime exceptions), is a higher level language, great and feature-rich standard library, even large projects compile in a few seconds, usable async IO, good quality GUI frameworks… Replacing C# with Rust would not be a benefit.
Java should have been like Modula-3, Eiffel, Active Oberon, unfortunately it did not and has been catching up to rethink its design while preserving its ABI.
Thankfully C# has mostly catched up with those languages, as the other language I enjoy using.
After that, is the usual human factor on programming languages adoption.
> However, for high-performance systems software specifically, objects often have intrinsically ambiguous ownership
What is the evidence for this? Plenty of high-performance systems software (browsers, kernels, web servers, you name it) has been written in Rust. Also Rust does support runtime borrow-checking with Rc<RefCell<_>>. It's just less ergonomic than references, but it works just fine.
Anyone that works on e.g. database kernels that do direct DMA (i.e. all the high-performance ones) experiences this. The silicon doesn’t care about your programming language’s ownership model and will violate it at will. You can’t fix it in the language, you have to accept the behavior of the silicon. Lifetimes are intrinsically ambiguous because objects have neither a consistent nor persistent memory address, a pretty standard property in databases, and a mandatory property of large databases. Yes, you can kind of work around it in idiomatic Rust but performance will not be anything like comparable if you do. You have to embrace the nature of the thing.
The near impossibility of building a competitive high-performance I/O scheduler in safe Rust is almost a trope at this point in serious performance-engineering circles.
To be clear, C++ is not exactly comfortable with this either but it acknowledges that these cases exist and provides tools to manage it. Rust, not so much.
New DB's like Tigerbeetle are written in Zig. Memory control was one of the prime reasons. Rust's custom allocators for the standard library have been a WIP for a decade now.
I think the major decisions behind Rust is being explicit and making the programmer make decisions. No NULLs, no Implicit conversions, no dangling pointers. Lifetimes, Optional, Results, each Match branch needs to exist, etc.
Side note: Stack allocation is faster to execute as there's a higher probability of it being cached.
Here is a free book for a C++ to Rust explanation. https://vnduongthanhtung.gitbooks.io/migrate-from-c-to-rust/...
> being explicit and making the programmer make decisions
Why RAII then?
> C++ to Rust explanation
I've seen this one. It is very newbie oriented, filled with trivial examples and doesn't even have Rust refs to C++ smart pointers comparison table.
I would say that RAII is very explicit: Resource Acquisition Is Initialization. When you initialize the struct representing the resource you are acquiring the resource. If you have a struct representing a resource you have the resource. Knowing this, you are also acquiring a call to drop when it goes out of scope. I would argue that the difference here isn't explicitness.
Instead, I would argue that rust is favoring a form of explicitness together with correctness. You have to clean up that resource. I have seen arguments that you should be allowed to leak resources, and I am sympathetic, but if we agree on explicitness as a goal then perhaps you might understand the perspective that a leak should be explicit and not implicit in a the lack of a call a some method. Since linear types are difficult to implement auto-drops are easier if you favor easily doing the correct thing. If you want to leak your resource, stash it in some leak list or unsafe erase it. That is the thing that should be explicit: the unusual choice, not all choices and not the usual choice alone.
But yeah, the drop being implicit in the explicit initialization does lead to developers ignoring it just like a leak being implicit if you forget to call a function often leads to unintentionally buggy programs. So when a function call ends they won't realize that a large number of objects are about to get dropped.
To answer your original question, the rationale is not in one concise location but is spread throughout the various RFCs that lead to the language features.
>> being explicit and making the programmer make decisions
>Why RAII then?
Their quote is probably better rephrased as _being explicit and making the programmer make decisions when the compiler's decision might impact safety_
Implicit conversion between primitives may impact the safety of your application. Implicit memory management and initialization is something the compiler can do safely and is central to Rust's safety story.
This might not be exactly what you're looking for, but I really like "References are like jumps": https://without.boats/blog/references-are-like-jumps/
The whole learning curve seems overblown to me. You don’t need to grok every part of the language to start using it and being productive.
I thought it was quite manageable at beginner level…though I haven’t dived into async which I gather is a whole different level of pain
Async and the "function color" "problem" fall away if your entire app is in an async runtime.
Almost 90% of the Rust I write these days is async. I avoid non-async / blocking libraries where possible.
I think this whole issue is overblown.
"Just write everything async" is not remotely a good solution to the problem. Not everything needs to be async (in fact most things don't), and it's much harder to reason about async code. The issue is very much not overblown.
Why is async code harder to reason about? I've been using it in C# and the entire point is that it lets you write callbacks in a way that appears nearly identical to synchronous code. If you dive into concurrency (which is a separate thing but can be utilized with async code, such as joining multiple futures at the same time), that parts hard whether you're doing it with async or with explicit threads.
> I've been using it in C#
One reason why async-await is trivial in .NET is garbage collector. C# rewrites async functions into a state machine, typically heap allocated. Garbage collector automagically manages lifetimes of method arguments and local variables. When awaiting async functions from other async functions, the runtime does that for multiple async frames at once but it’s fine with that, just a normal object graph. Another reason, the runtime support for all that stuff is integrated into the language, standard library, and most other parts of the ecosystem.
Rust is very different. Concurrency runtime is not part of the language, the standard library defined bare minimum, essentially just the APIs. The concurrency runtime is implemented by “Tokio” external library. Rust doesn’t have a GC; instead, it has a borrow checker who insists on exactly one owner of every object at all times, makes all memory allocations explicit, and exposed all these details to programmer in the type system.
These factors make async Rust even harder to use than normal Rust.
> Why is async code harder to reason about?
I don't know about C#, but at least in Rust, one reason is that normal (non-async) functions have the property that they will run until they return, they panic, or the program terminates. I.e. once you enter a function it will run to completion unless it runs "forever" or something unusual happens. This is not the case with async functions -- the code calling the async function can just drop the future it corresponds to, causing it to disappear into the ether and never be polled again.
That’s not a solution to the coloring problem any more than making everything red was in 2015 (ie, all the tradeoffs mentioned in the article [0] still apply).
[0]: https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
It’s completely overblown. Almost every language with async has the same “problem”.
I’m not calling this the pinnacle of async design, but it’s extremely familiar and is pretty good now. I also prefer to write as much async as possible.
The "function color is a problem" people invented a construct that amplifies the seriousness. It's not really a big deal.
Async's issue is not coloring.
How are async closures / closure types, especially WRT future pinning?
Async closures landed in stable recently and have been a nice QoL improvement, although I had gotten used to working around their absence well enough previously that they haven’t been revolutionary yet from the like “enabling new architectural patterns” perspective or anything like that.
I very rarely have to care about future pinning, mostly just to call the pin macro when working with streams sometimes.
While I'd like to have it, it doesn't stop me from writing a great deal of production code without those niceties.
When it came time for me to undo all the async-trait library hack stuff I wrote after the feature landed in stable, I realized I wasn't really held back by not having it.
Good article, thoughtful and well-written and (idioms aside) much of it applicable to learning other languages.
This reads like a list of symptoms of what's wrong with the ergonomics of Rust. This is not to bash Rust. It has its uses. But you need to balance what you are sacrificing for what you are getting.
What is it that make that rust is said to have steep learning curve compared to other programing languages (in the same category)?
It doesn’t. The only language I know of in the same category as Rust is C++, which is much harder to learn and use.
Write a CHIP8 emulator!
Bonus: do it with no heap allocation. This actually makes it easier because you basically don’t deal with lifetimes. You just have a state object that you pass to your input system, then your guest cpu system, then your renderer, and repeat.
And I mean… look just how incredibly well a match expression works for opcode handling: https://github.com/ablakey/chip8/blob/15ce094a1d9de314862abb...
My second (and final) rust project was a gameboy emulator that basically worked the same way.
But one of the best things about learning by writing an emulator is that there’s enough repetition you begin looking for abstractions and learn about macros and such, all out of self discovery and necessity.
I’ve found emulators to be a pretty poor first project for rust specifically for the reasons you alluded to: That you need to know to write it without heap allocation (or other hoop jumping so long as you avoid juggling lifetimes) when so much literature and example emulator code doesn’t do this is a recipe for a bad experience. Ask me how I know.
If you’re going to write an emulator in this style, why even use an imperative language when something like Haskell is designed for this sort of thing?
Compared to other programming languages, Rust's compiler and linters go a long way to implement best practices at build time.
Regarding the first example, the longest() function, why couldn't the compiler figure it out itself? What is the design flaw?
Compiler can figure that out, but the thing is compiler needs also to understand lifetimes at the site where this function is called. In general case compiler will not look into the code of a called function to see what it does, compiler relies on a function declaration.
That `longest` if defined without explicit lifetimes treated like a lifetime of a return value is the same as of the first argument. It is a rule "lifetime elision", which allows to not write lifetimes explicitly in most cases.
But `longest` can return a second reference also. With added lifetimes the header of the function says exactly that: the lifetime of a return value is a minimum of lifetimes of arguments. Not the lifetime of the first one.
You’re passing in two references and returning a reference.
The compiler knows the returned reference must be tied to one of the incoming references (since you cannot return a reference to something created within the function, and all inputs are references, the output must therefore be referencing the input). But the compiler can’t know which reference the result comes from unless you tell it.
Theoretically it could tell by introspecting the function body, but the compiler only works on signatures, so the annotation must be added to the function signature to let it determine the expected lifetime of the returned reference.
> Theoretically it could tell by introspecting the function body, but the compiler only works on signatures
Note that this is an intentional choice rather than a limitation, because if the compiler analyzed the function body to determine lifetimes of parameters and return values, then changing the body of a function could be a non-obvious breaking API change. If lifetimes are only dependent on the signature, then its explicit what promises you are or are not making to callers of a function about object lifetimes, and changing those promises must be done intentionally by changing the signature rather than implicitly.
> changing the body of a function could be a non-obvious breaking API change
This. Many trival changes breaks API. This is not ideal for library developers.
You can argue it is broken already, but this is forcing the breakage onto every api caller, not just some broken caller.
It's a design choice.
To make a compiler automatically handle all of the cases like that, you will need to do an extensive static analysis, which would make compiling take forever.
Would be nice if an IDE can autofix it.
Maybe autofix as we type, or autofix when it save the document / advance to next line.
> Use String and clone() and unwrap generously; you can always refactor later
At that point you might as well be writing Java or Go or whatever though. GC runtimes tend actually to be significantly faster for this kind of code, since they can avoid all those copies by sharing the underlying resource. By the same logic, you can always refactor the performance-critical stuff via your FFI of choice.
So long as you're aware that you're not optimizing, it's fine. Trying to build something useful as a new Rust dev while worrying about lifetimes is going to be quite challenging, unless your intention is to specifically learn about lifetimes and the borrow checker.
Yes the borrow checker is central to Rust, but there are other features to the language that people _also_ need to learn and explore to be productive. Some of these features may attract them to Rust (like pattern matching / traits / etc.)
>significantly faster for this kind of code
"Significantly" and "this kind" are load bearing sentences here. In applications where predictable latency is desired, cloning is better than GC.
This is also the baby steps of learning the language. As a programmer gets better they will recognize when they are making superflous clones. Refactoring performance-critical stuff in FFI, however, is painful and wont get easier with time.
Furthermore, in real applications, this only really applies to Strings and vectors. In most of my applications most `clones` are of reference types - which is only marginally more expensive than memory sharing under a GC.
> At that point you might as well be writing Java or Go or whatever though.
And miss Option/Result, enums, exhaustive pattern matching and the rest other dozens of features that make your life manageable
I'm not using Rust for the borrow checker.
I went through this the first year that I did Advent of Code in rust, like okay I read in all the strings from the input file and now they're in a vector, so I'm going to iterate the vector and add references to those strings into this other structure, but of course they're still owned by the original vector, that's awkward. Oh wait I can iter_into and then I get owned objects and that ownership can be transferred to the other structure instead, but now I need them to also be keys in a map, do I use references for that too?
Cloning small objects is lightning fast, turns out in a lot of these cases it makes sense to just do the clone, especially when it's a first pass. The nice thing is that at least rust makes you explicitly clone() so you're aware when it's happening, vs other languages where it's easy to lose track of what is and isn't costing you memory. So you can see that it's happening, you can reason about it, and once the bones of the algorithm are in place, you can say "okay, yes, this is what should ultimately own this data, and here's the path it's going to take to get there, and these other usages will be references or clones.
> Cloning small objects is lightning fast
It's really not, it's the way python works. Heap allocations are "fast" on modern CPUs that are too fast to measure for most stuff, but they're much (much) slower than the function call and code you're going to use to operate on whatever the thing it was you cloned.
Code that needs memory safety and can handle performance requirements like this has many options for source language, almost none of which require blog posts to "flatten the learning curve".
(And to repeat: it's much slower than a GC which doesn't have to make the clone at all. Writing Rust that is "Slower Than Java" is IMHO completely missing the point. Java is boring as dirt, but super easy!)
Not to mention that even though you can always refactor later, will you really? It’s much easier not to.
In my experience, hobbyist Rust projects end up using unwrap and panic all over the place, and it’s a giant mess that nobody will ever refactor.
Surrender! to compile
Weather the ferocious storm
You will find, true bliss
> Treat the borrow checker as a co-author, not an adversary
Why would I pair-program with someone who doesn’t understand doubly-linked lists?
For people who don't get the reference, this might be referring to the notoriously gnarly task of implementing a doubly-linked lists in Rust [1]
It is doable, just not as easy as in other languages because a production-grade linked-list is unsafe because Rust's ownership model fundamentally conflicts with the doubly-linked structure. Each node in a doubly-linked list needs to point to both its next and previous nodes, but Rust's ownership rules don't easily allow for multiple owners of the same data or circular references.
You can implement one in safe Rust using Rc<RefCell<Node>> (reference counting with interior mutability), but that adds runtime overhead and isn't as performant. Or you can use raw pointers with unsafe code, which is what most production implementations do, including the standard library's LinkedList.
https://rust-unofficial.github.io/too-many-lists/
Rust still needs a way out of that mess. It's conceptually possible to have compile time checking for this. Think of RefCell/Weak and .upgrade() and .borrow() being checked at compile time.
I've discussed this with some of the Rust devs. The trouble is traits. You'd need to know if a trait function could borrow one of its parameters, or something referenced by one of its parameters. This requires analysis that can't be done until after generics have been expanded. Or a lot more attributes on trait parameters. This is a lot of heavy machinery to solve a minor problem.
> Rust still needs a way out of that mess.
It has one: use raw pointers and unsafe. People are way too afraid of unsafe, it's there specifically to be used when needed.
> Rust still needs a way out of that mess.
In practice, it really doesn't. The difficulty of implementing doubly linked lists has not stopped people from productively writing millions of lines of Rust in the real world. Most programmers spend less than 0.1% of their time reimplementing linked data structures; rust is pretty useful for the other 99.9%.
Doubly linked lists are rare, but backlinks to the owner are often needed. It's the same problem, mostly.
Apologies since I have not taken the time to learn rust yet, but I've written a lot of modern C++. Is the ownership model kind of like std::unique_ptr and std::move, and `Rc<RefCell<Node>>` the same idea as `std::shared_ptr`? But less idiomatic? Or do I have the wrong idea?
Not really, because Rust enforces a "many readers or one writer" invariant on everything that has no C++ equivalent. That invariant is precisely what makes the doubly-linked list case hard (because every interior node in the list would be readable from two places, which means it can never be written to).
I am working on a code base, that among its many glories and poo balls every list is a doubly linked list.
Stop!
If you are using a doubly linked list you (probably) do not have to, or want to.
There is almost no case where you need to traverse a list in both directions (do you want a tree?)
A doubly linked list wastes memory with the back links that you do not need.
A singly linked list is trivial to reason about: There is this node and the rest. A doubly linked list more than doubles that cognitive load.
Think! Spend time carefully reasoning about the data structures you are using. You will not need that complicated, wasteful, doubly linked list
> There is almost no case where you need to traverse a list in both directions
But you might need to remove a given element that you have a pointer to in O(1), which a singly linked list will not do
If that's a specific use case you need to handle, it's O(1) again if you have a pointer to both the node to be removed and the previous node.
Whether it's more efficient to carry a second pointer around when manipulating the list, or store a second pointer in every list node (aka double linked list) is up to your problem space.
Or whether an O(n) removal is acceptable.
Getting the pointer to that element means randomly hopping around the heap to traverse the list though.
Linked lists are perfect for inserting/deleting nodes, as long as you never need to traverse the list or access any specific node.
So that you learn that loaning is for giving temporary shared^exclusive access within a statically-known scope, and not for storing data.
Trying to construct permanent data structures using non-owning references is a very common novice mistake in Rust. It's similar to how users coming from GC languages may expect pointers to local variables to stay valid forever, even after leaving the scope/function.
Just like in C you need to know when malloc is necessary, in Rust you need to know when self-contained/owning types are necessary.
The biggest thing I’ve run into where I really want self-referential types is for work that I want to perform once and then cache, while still needing access to the original data.
An example: parsing a cookie header to get cookie names and values.
In that case, I settled on storing indexes indicating the ranges of each key and value instead of string slices, but it’s obviously a bit more error prone and hard to read. Benchmarking showed this to be almost twice as fast as cloning the values out into owned strings, so it was worth it, given it is in a hot path.
I do wish it were easier though. I know there are ways around this with Pin, but it’s very confusing IMO, and still you have to work with pointers rather than just having a &str.
I'd rather pair program with someone wary of double-linked lists, but is really hot on understanding ownership than the other way around.
Because most of my code is not doubly-linked lists!
Because you care about productivity and safety more than l33t h4x0r hazing rituals?
Does anyone still have trouble learning Rust? I thought that was kind of a 2015 thing
For me it was like Haskell. I spent an afternoon on it, my brain hurt too much, and I relegated it to the category of languages that are too complicated for what I need to do with a computer.
Languages I liked, I liked immediately. I didn’t need to climb a mountain first.
Unfortunately, yes. I still end up writing C++ instead of Rust for low-level system stuff. Since I also know Go - I usually prefer that when I need lean, middleware services. Learned Rust (somewhat) with great difficulty but still don't use it anywhere. Still haven't figured out how to design effectively using Rust and approaches suggested in that article like clone()/unwrap() stuff and refactor later just leave a bad taste in that mouth.
Coming from C++, I don’t find it hard to learn. I do find it annoying, don’t love the syntax and absolutely hate cargo.
To each his own, I guess….
The thing is, once you internalized the concepts (ownership, borrowing, lifetimes), it's very hard to remember what made it difficult in the first place. It's "curse of knowledge" in some sense.
What's changed since 2015 is that we ironed out some of the wrinkles in the language (non-lexical lifetimes, async) but the fundamental mental model shift required to think in terms of ownership is still a hurdle that trips up newcomers.
100%. Newcomers still struggle a bit, especially if they've never used C/C++ before.
A good way to get people comfortable with the semantics of the language before the borrow checker is to encourage them to clone() strings and structs for a bit, even if the resulting code is not performant.
Once they dip their toes into threading and async, Arc<Lock<T>> is their friend, and interior mutability gives them some fun distractions while they absorb the more difficult concepts.
Do you mean `Arc<Mutex<T>>`? Yeah, I agree. Wrote a blog post on that topic as well: https://corrode.dev/blog/prototyping/ The title is a bit of a misnomer, but it's about beginner-friendly escape hatches in the language. Perhaps it's useful to newcomers.
Any lock, but that's generally the best choice.
Great post! It's got a ton of advice for being productive, and it should be especially useful for beginners.
Yeah I struggled with it.