All right. Welcome, everybody.
My name is Arthur O'Dwyer.
I'm the chair of the Back to Basics track here at CppCon 2019.
And number one, before we get started with this talk, I just wanted to remind everyone...
...that there is a site where you can give feedback. On, I believe, dascandy.org. You've all got an email about giving feedback after talks?
So please do go there — especially if you're participating in the Back to Basics track.
Please do leave feedback for the speakers. Please do rate the talks, you know.
Tell us what we can do better next year, for whoever comes to the Back to Basics track next year.
So: please rate the talks.
With that out of the way... also... My name is Arthur O'Dwyer, and this is a talk about smart pointers.
Let's get started!
In the next hour I'm going to tell you a little bit about the various kinds of smart pointers in C++.
`unique_ptr`, `shared_ptr`... I'm going to tell you how to make them...
We're going to talk a little bit about `weak_ptr` and `enable_shared_from_this`.
Those are tending into the less "basic" area of smart pointers in C++...
...but I did want to cover them, because I don't think you can say that you came away from CppCon "knowing" smart pointers...
...unless you knew a little bit about `weak_ptr`, `enable_shared_from_this`, that kind of thing.
And at the end, I hope we'll have time for questions.
Number one, here is the list of smart pointer types in C++.
And I see that I haven't updated this slide in a little while... But the list also hasn't changed since C++14.
Except for the very first one on the list, which is `auto_ptr`.
`auto_ptr` was a thing from the bad old days of the 1990s.
It was deprecated when move semantics came in, and it is now gone. And I am not going to talk about it again!
Let us just never mention the name `auto_ptr` again.
`unique_ptr` is the C++11 replacement for `auto_ptr`. We're going to talk a lot about that on the next few slides.
`shared_ptr` is a reference-counted pointer. And `weak_ptr` is for something called "weak references."
We're going to talk about all of these, but I wanted to get all the names up there,
so that you have that basic list of names as we go forward.
I also wanted to put up— we're not going to talk a whole lot about the SYNTAX of smart pointers in this talk...
I'm not really going to tell you what your code is going to look like after you put smart pointers in it...
...because it's going to look very similar to how it looks today with raw pointers.
Right? I can make a variable of a smart pointer type just like I can make a variable of native pointer type. I can say `int *p`, I can say `shared_ptr p`.
I have to say what it points to.
The syntax for creating heap-allocated objects looks a little bit different. I don't use `operator new` explicitly.
We're going to talk a little bit about that about halfway through this talk.
But `shared_ptr`s are assignable just like regular pointers are...
they are copy-constructible...
you can assign `nullptr`. You can compare against `nullptr`. They implicitly convert to `bool`.
They do all the things that ordinary, native, core-language pointers do.
This is all through the magic of operator overloading. And as I said, we're not going to talk much more about what they look like.
I want to tell you more about what they do and how they work.
And I'm going to do this through Chapter 4 of Scott Meyers' "Effective Modern C++." [ https://amzn.to/2ogk33z ]
This is a great book. It came out many years ago now. it has been getting some updates.
So I highly recommend— basically anything Scott Meyers has ever done, but especially the book "Effective Modern C++" [ https://amzn.to/2ogk33z ]
which is all about how to update your code base to take advantage of what was, at least at the time, "modern C++."
We're talking C++11 and C++14 features.
And Chapter 4 in that book is all about smart pointers.
...and in particular, it has some items...
And Item #18 says, "Use `unique_ptr` for exclusive-ownership resource management."
What does "exclusive-ownership resource management" mean?
Well, if you saw my talk on "RAII and the Rule of Zero," given earlier this week—
—And if you're watching on YouTube, go see that talk!— [ https://youtu.be/7Qgd9B1KuMQ ]
What we're talking about here is managing a resource — namely, a heap allocation — that we would otherwise have to manage manually.
When we are done with a heap allocation, we need to call `delete` on that pointer.
And the destructor of `unique_ptr` is the thing that's going to call `delete` for us.
So here's how we might work with a raw pointer.
I've drawn a little diagram here. I love this kind of diagram.
On the left-hand side here, I have my stack. This is my program stack with all my local variables, my temporary variables.
And then over on the right side, we have my heap. That's where my dynamic-lifetime objects live — the things I get from `new` and `delete`.
And in the bad old world of raw pointers, I would write `T *ptr;`...
so that makes a variable of type pointer, and it would go over here on the left-hand side, on the stack.
And it is initialized to point to the address of what I call the "controlled object," of type `T`, over here on the heap.
I heap-allocate an object with `new`, and then later I have to remember to `delete` the pointer.
It's my responsibility to write `delete ptr;` or else I get a memory leak
But, I don't have to manage that responsibility myself.
I can push that responsibility— delegate it onto a well-tested piece of library code, such as `unique_ptr`.
So here's `unique_ptr`. It's a class template; again, templated on what it points to.
And it has, inside itself, just one data member. It's a private data member, so you're not allowed to access it directly, as a user of `unique_ptr`.
But it just holds a raw pointer inside itself. And it has a destructor that will call `delete` on that pointer.
If you've been to my Rule of Zero talk, you know that it also has a copy— er, sorry, it DOESN'T have a copy constructor!
It has a move constructor and a move assignment operator. I'm not showing those here.
The important thing is that when that variable is destroyed, it calls `delete` on the pointer for you.
This makes your code more safe. It makes it more resilient against exceptions getting thrown.
`unique_ptr` is move-only!
Right? I almost tripped up and said it had a copy constructor.
A raw pointer is copyable. I can have a raw pointer to `T` here...
I've removed one layer of boxes. So we're back in the realm of asterisks and stars.
If I copy this pointer... You have this pointer, and then I make a copy of it...
They both hold the same memory address. That means they both point to the same object.
Which one of them has ownership of that object? Which one of them has the responsibility for cleaning up?
Well, it's unclear.
`unique_ptr` is move-only. And its move constructor actually nulls out the source pointer.
So, that reifies, or materializes, the ownership and the responsibility into the code.
When you have a `unique_ptr` object, and I attempt to copy it...
Number one, I can't copy it. I can't copy the responsibility for cleaning up. That is a unique ownership, a unique responsibility.
But I can `std::move` it from one place to the other. And when I do, the source object gets nulled out automatically as part of the side effects of the move constructor or the move assignment operator.
This preserves the unique ownership.
There is only one pointer, one `unique_ptr` in this program that points at that controlled object.
There's only one way to get to it. And there's only one of these `unique_ptr` objects that has the responsibility for cleaning it up.
This is why we're going to use these smart pointers...
both to help us guard against bugs — right? to make sure that we don't leak memory, that someone cleans it up...
but also to make the ownership clearer.
`unique_ptr` is specifically for representing exclusive ownership and exclusive responsibility for cleaning up.
That ownership can travel around the program, but it's always unique.
`unique_ptr` also has a specialization for array types. If I have a `unique_ptr` to an array of `int`, or a `unique_ptr` to an array of `Widget`, that's perfectly fine
I could get a pointer from the array form of `new`...
and then encapsulate it, transfer ownership, into a `unique_ptr` or a `unique_ptr`.
And at the end of its lifetime, it will call the array form of `delete`.
Remember, there are two different forms of `delete` and two different forms of `new`; and you have to make sure they match up?
Well, `unique_ptr` has a specialization that can take care of that for you
In fact, `unique_ptr` actually has one more template parameter — one more template type parameter — that I didn't even show on the previous slides! But it was there.
It was defaulted. Just like when I have a `std::vector`, there's actually an allocator also associated. An `allocator` parameter that you don't have to write, because it's defaulted.
`unique_ptr` also has one of those. It defaults to the `default_delete` for that particular `T`...
but you can put it in explicitly.
If you put in your own "Deleter" type, then you can set its value in the constructor,
and in the destructor, it will invoke the call operator — the `operator()` — of your deleter.
We're going to see an example of that, in fact.
So— right. `unique_ptr` is always a template of two parameters... If you provide no second parameter, it's defaulted to `default_delete`...
Here's the implementation of `default_delete`. It has a call operator, and all that does is to call `delete`.
There's also a `default_delete` for arrays of things, and it calls the array form of `delete`.
So this is all happening for you behind the scenes as a result of this defaulted template parameter.
Normally, we don't write that.
But we can do some neat things with a custom deleter on `unique_ptr`.
For example if I had a little tag type here...
A struct with no particular members other than the call operator. And this call operator takes a `FILE*`.
And it doesn't call delete on the file star, right? That's not what we do.
It calls `fclose` on it.
So now I've got a type that represents the operation of closing a `FILE`.
So if I were to call `fopen` to get a `FILE*`... then I have a responsibility to call `fclose` on that when I'm done with it.
I can encapsulate that responsibility into a `unique_ptr`.
So here I have a `unique_ptr` to a `FILE`, and the deleter is a `FileCloser`.
So when this `unique_ptr` goes out of scope, it will call the `operator()` of the `FileCloser` and call `fclose` on the file for me.
So in this way I can, again, get more exception safety and get more clarity about whose responsibility it is.
If you hold a non-null `unique_ptr` of this type, then it's your responsibility to call `fclose`. And in fact the `unique_ptr` will do that for you.
If you have a raw `FILE*`, it must not be your responsibility to call `fclose`.
In fact, I work on a codebase, and it uses a lot of OpenSSL.
How many people in this room, like, use OpenSSL, as your job?
[A SURPRISING NUMBER OF HANDS] Yeah.
I highly, highly recommend, then—
OpenSSL is a C API, and it has a lot of these pairs of things— it's a very nice object-oriented API, but it's all C.
And so it has the equivalent of constructors and destructors. You have to remember to destroy things when you're done using them.
So I highly recommend a pattern something like this.
I make a custom deleter type. This one happens to delete certificate signing requests.
You pass it a raw pointer...
...it calls `X509_REQ_free`, which is the function that OpenSSL uses to destroy a certificate signing request.
And then I have this business logic type. This is the actual C++ type that I'm going to use to represent a signing request.
And it has a constructor that takes a `unique_ptr`.
So you're passing it a pointer to one of these underlying OpenSSL types, that's managed through these C functions...
But once that gets into this class `MyCSR`, it's going to have a data member of type `unique_ptr` with an appropriate deleter, so that it gets cleaned up properly.
So this constructor takes a `unique_ptr` by value. I would say that this is a "sink"...
...a `unique_ptr` sink.
You have to have a `unique_ptr` before you can call this constructor.
So you have to have accepted your responsibility.
You can't just pass any old raw pointer into this business logic constructor and say, "Here, you manage it."
I have to demonstrate that I have accepted responsibility for this `X509_REQ` object.
And the way I do that is, I explicitly wrap it up into a `unique_ptr` before passing it off to the business logic class.
I say, "Yes, I really own this. Here is my responsibility."
I've reified my responsibility into the code, in the form of this `unique_ptr`...
...and now I am passing it off to you, and now it is your responsibility.
So `unique_ptr` here is forming a sort of glue layer between the low-level OpenSSL C code...
...and the high-level business object, which is following the Rule of Zero (as all good business objects should).
Some rules of thumb for using all kinds of smart pointers, not just—
Oh, we have a question. We'll take a question.
[AUDIENCE] So, I'm a little confused. Why wouldn't you pass it by reference? If you're taking ownership, when you pass it in, shouldn't you pass it by reference, so that you can take away the ownership from the caller?
Ah. Excellent question. And in fact that's covered on my next slide, as the number one bullet!
My rule of thumb is "pass smart pointers by value."
If you saw Chandler's talk a couple days ago called "There Are No Zero-Cost Abstractions," [ https://youtu.be/rHIkrotSwcc ]
you will know that this abstraction is not quite zero-cost.
But it has a benefit. And the benefit is: simplicity, and lack of bugs.
So I highly recommend: Treat your smart pointer types just like you would treat raw pointer types.
You saw in terms of memory footprint a `unique_ptr` is just a very small — you know, 8-byte — value.
It is true that, being non-trivial, it will get passed on the stack.
This is fine. This is fine. Please do not panic, if you were in Chandler's talk.
So treat smart pointer types just like raw pointer types. Pass by value, return by value.
In order for the caller to call my function, and pass me a `unique_ptr` by value...
`unique_ptr` is not copyable. They're not passing me a copy of a `unique_ptr` they still hold.
They have moved it into me. They have transferred me ownership of that `unique_ptr`.
And hopefully that answers that question.
Right? By passing by copy, I am not saying that there is a copy MADE of the `unique_ptr`. It is still unique. They HAD to move it.
I don't have to worry about how they moved it. I don't go taking rvalue references or anything like that. Certainly—
Just like with a native pointer. If you have a native pointer and you're passing it to a function, you don't pass it by reference; you pass by value.
Passing by reference merely adds an extra indirection on top of whatever you would already have.
So similarly, I don't pass smart pointers by reference.
So, a function taking a `unique_ptr` by value shows transfer of ownership.
And it shows exactly WHAT responsibility is being transferred. Because, remember, the responsibility is encoded in that deleter type.
If you don't see the deleter type, it's `default_delete`. So the responsibility being transferred is the responsibility for calling `operator delete`.
On the other hand, if you see that the `unique_ptr` is parametrized by `X509_REQ_Deleter`,
that means the responsibility being transferred is whatever that does, which is that OpenSSL call.
Usually the responsibility is simply to call `delete`.
Smart pointers are frequently used as I just used them, as a glue layer between the low level and the high level...
...or as an implementation detail so that you can write a class following the Rule of Zero.
Delegate all that manual management to the smart pointer, so that you don't have to write explicit copy constructors and move constructors and so on.
To bake a smart pointer type into your interface — into what your actual users are doing — MAY be a code smell.
Not necessarily saying that it IS...
But I would be careful about writing an entire codebase based around passing around `shared_ptr`s or passing around `unique_ptr`s.
I would try to hide those in business logic classes such as my `MyCSR` class, that you saw earlier.
Scott Meyers also talks in this chapter about `std::shared_ptr`, and says to use that for "shared-ownership resource management."
But this slogan does not really tell you anything that the name of the type didn't already tell you.
Yes, it is shared. What do we mean, "shared"?
Well, it expresses shared ownership. And what I mean by "shared ownership" is, "reference counting."
I like to explain reference counting by saying, it's as if—
The lights are on in this room. We're all in this room.
We're eventually all going to leave this room. And when we do, we would like the lights to be off.
So a good algorithm to implement that — a protocol to implement that, if we had to — would be to say, "The last person out of the room should turn out the lights."
However, this is a very big room. And as you're leaving, you may not necessarily know if you are the last person out of the room.
So what we're going to do, in order to keep track of how many people are currently in this room,
participating in the ownership of that light switch, is...
We're going to keep a jar of tokens by the door.
When you enter, you put a token in the jar. When you leave, you take a token out of the jar.
It doesn't matter which token. Right? They're all the same.
But if you take the last token out of the jar, such that the jar is empty after you do so...
...then you must be the last person out of the room.
I hope this makes sense to everybody.
As long as everyone involved is cooperating. We can't have anyone coming in, and not putting in a token...
...because if they do that we might end up leaving someone in the room when the lights go out. And that would be bad.
That would be a dangling pointer bug.
But we need a place to put that jar of tokens.
That jar of tokens, in our example, is going to be an `atomic`.
Some languages, and some codebases, implement ref-counting by putting that jar right here in the room — in the `Widget`.
So every `Widget` has a `usecount_`.
And maybe it has a couple of members such that I can retain a little bit of ownership in this `Widget` by incrementing the use count;
I can release my ownership of the `Widget`, decrement the use count;
and if it reaches zero then I delete the object.
This would be perfectly reasonable.
But not every C++ class is orchestrated in this way.
[QUESTIONER TAPS THE MICROPHONE] I don't think this is on.
[ARTHUR] Oh, there it is. [QUESTIONER] There it goes. Thank you.
[QUESTIONER] So the intention of using the `atomic` right here is to make it thread-safe, is that correct?
[ARTHUR] Yes. The intention of the `atomic` is to make it thread-safe.
[QUESTIONER] And so is that to say that the reference-counting solution is NOT thread-safe?
[ARTHUR] This IS a reference-counting solution. It is not quite yet the one used by `shared_ptr`.
But if I wanted multiple— [QUESTIONER] But as opposed to not using an `atomic`.
If I didn't use an `atomic` here, then it would be fine as long as no two people were trying to get their token out of the jar at the same time.
What `atomic`— [QUESTIONER] So it's not thread-safe, then. Without the atomic.
[ARTHUR] Well, we haven't said what "it" is. if "it" is what I'm showing, then it's thread-safe, because it uses `atomic`.
On the previous slide, I didn't show anything but a jar of tokens. That is not thread-safe, in the sense that the opening of the jar is maybe not big enough for two people to get their hands in at once.
So we need to do something to make sure that each person's hand going in "synchronizes-with" everyone else's hand going in.
And the way we express the "synchronizes-with" relationship is with an `atomic`.
But we're not going to do this anyway. We're going to use `shared_ptr`.
Because the problem with this is that your own class, whatever it is, probably doesn't have a `usecount_` member.
And certainly an `int` doesn't have a `usecount_` member, right?
In C++ we want our user-defined types to behave just like the primitive types.
`std::string` doesn't have a `usecount_` member. This is not built in to types in C++. We want things to be fast.
So we're going to need somewhere else to store this reference count.
Oh yeah. Right. "There's a problem: we don't contain refcounts." I forgot I wrote all these slides. All right.
So what about a `shared_ptr` to an `int`?
Where would we store the reference count if we had a `shared_ptr` to an `int`?
Well, we're going to have to allocate somewhere else to put it
So, let's suppose my controlled object in this case is a pointer that I got from `new int`.
It's a heap-allocated `int`, or any other kind of object.
And I'm going to transfer ownership of that `int` to a `shared_ptr`.
Now, the `shared_ptr` over here... Notice that its memory footprint looks bigger.
It is not as small as `unique_ptr`. It's actually twice as big.
Because it has not only a pointer to the controlled object (so that I can dereference it);
it also needs that reference count.
And so it's going to allocate some space on the heap to store that reference count.
That space that it's allocating on the heap is associated with the controlled object. We call it the "control block" associated with that object.
We don't get one control block per `shared_ptr`. We get one control block per controlled object.
And in that control block, we have a reference count. That's exactly the reference count I just described; and yes, it would be atomic.
We have a weak reference count, which I'll get to.
We might even have a custom deleter, if there was some other responsibility than just calling `delete` on the controlled object.
I could actually put the custom deleter right in there.
And we also have another pointer to the controlled object, to tell the deleter WHICH object it is that it wants to delete. We'll get back to that.
So my `shared_ptr` stores the reference count in a separate control block. And when I copy the `shared_ptr`—
When I copy the `shared_ptr`, I get a copy of the pointer to `T`, and I get a copy of the pointer to the control block...
So far, that's just copying the bits.
But there's one more thing I need to do, besides copying the bits of the actual `shared_ptr`; and that is...
...to follow the pointer to the control block and increment the reference count.
The copy constructor for `shared_ptr` also does this.
And the move constructor does the appropriate thing, and the move assignment operator, and so on.
They manage this reference count for us.
When I make a copy, I increment the reference count. When I destroy a copy, I decrement the reference count.
And when the reference count goes to zero, I destroy the controlled object.
So, why does the control block need a pointer to the controlled object...
...when the `shared_ptr` itself already has a pointer to `T` that points directly at the controlled object?
When the ref count goes to zero, why do I need to follow the pointer in the control block...
...as opposed to the pointer that is already stored in the `shared_ptr`?
Well, to explain this, we need to do a little bit of a detour and talk about C++ physical class layout.
Here, I have two classes `Fruit` and `Vegetable`. They each have one integer member...
...and their memory footprint looks like what I have over here. `Fruit` looks like that... `Vegetable` looks like that...
They're both probably 4 bytes on your average desktop system.
Now, if I have an `Apple`, an `Apple` IS-A `Fruit`.
And what that means is that if I have a pointer to an `Apple`, I also have a pointer to a `Fruit`.
Therefore, if I have a pointer to the first byte of this `Apple`, I must also have a `Fruit` object there.
I do. It is the `Fruit` base class subobject.
So here's the `Fruit` part of the `Apple`, and then any other, non-`Fruit` parts of the `Apple` come afterwards.
Similarly, I can have multiple inheritance.
A `Tomato` is both a `Fruit` and a `Vegetable` (for the sake of this example)...
...and so it has all the parts of a `Fruit`, all the parts of a `Vegetable`, and whatever tomatoey parts it has.
It has both [Fruit and Vegetable parts] and they can't both be at offset zero.
When I have multiple inheritance in C++, one of those base classes is going to have to be a little bit offset from the other.
So what this means is that if I have a `shared_ptr` and a `shared_ptr`, and they both point to the same `Tomato` object...
...one of them is going to have to point to the `Vegetable` subobject of that `Tomato` object. It's going to be a little bit offset.
And so it's not going to point directly at the controlled object that needs deleting.
So that's why the control block not only stores what action do I need to perform—
my `default_delete` — what action do I need to perform — but also,
what data do I need to do that action on? What pointer am I actually going to call `delete` on?
And that's a `Tomato` pointer.
So the `Tomato*` is kept in the control block,
and the `shared_ptr`s might be shared_ptr`s to base classes. They might be `shared_ptr`. They might even be more exotic things.
Right, so, `shared_ptr` (unlike `unique_ptr`) places a layer of indirection between the physical heap-allocated object and this notion of "ownership."
Your `unique_ptr` directly controls who calls delete on that object.
Your `shared_ptr`, not so much. Your `shared_ptr` is really participating in ownership of the CONTROL BLOCK.
The control block itself is the arbiter who decides what it means to delete the controlled object.
That is set up during construction of the first `shared_ptr`. During construction of the control block.
It says, "When everyone's out of this room, here is how you turn out the lights."
And none of us, none of the people in the room, can change what it means to turn out the lights. That is fixed.
So that pointer, that `ptr to X` over here that I've drawn with a little squiggly arrow — that doesn't play any role in ownership.
That is merely pointing at the controlled object, or at some part of it,
or even at something else entirely!
I am totally not going to go over this example because I don't think we quite have time...
...but this would be showing a `shared_ptr` that is— uh—
When we construct the `shared_ptr`, the controlled object is a `vector`.
The `vector` has ownership of an array of ints.
And then we use what's called the aliasing constructor,
and make a `shared_ptr` that points at just the the second element of that vector.
We can then drop the original `shared_ptr`. We no longer have a `shared_ptr>` anywhere in this system.
Nobody can actually access that vector anymore, except for...
...when that last `shared_ptr` to the control block goes away, it will destroy the vector, which will destroy those ints.
So this is called the aliasing constructor of `shared_ptr`.
And this is totally not something that should be in a Back to Basics talk!
Scott Meyers also says, "Prefer `make_unique` and `make_shared` to direct use of `new`."
This is good advice. You should do this.
This is because one of our goals in modern C++ is to avoid raw `new` and `delete`.
Why do we want to avoid them?
Well, an even more primitive rule that's been around since 1998 is, "Every `new` must have a matching `delete`, and vice versa."
Every time I write `new` I should see where I'm calling `delete`.
So the second rule of memory management, the one you've already learned, is "Use smart pointers."
When you use a smart pointer, that responsibility for calling `delete` goes into the destructor of the smart pointer type.
Your code, your business logic code, does not have calls to `delete` in it anymore.
Those calls have been delegated to some destructor.
So, if my code doesn't have calls to `delete` in it, should my code have calls to `new` in it?
Right? This code over here, on the left-hand side, this is bad code...
This is bad C++98 code. It calls `new` and it fails to call `delete`. This is a memory leak.
This code over here [on the right] does not leak memory.
But it still calls `new` and it doesn't call `delete`.
I mean, the responsibility for the `delete` has been taken care of by the destructor of the `shared_ptr`, so it doesn't leak...
But it looks just as bad!
I don't want to have to code review and figure out whether you've got code like on the left or code like on the right.
If I don't see `delete`, I better not see `new`.
So here's what we're going to do.
We're going to hide that call to `new`.
We hid the call to `delete` behind a destructor. We're going to hide the call to `new` in a factory function.
And the standard library provides factory functions.
It provides `std::make_shared`...
That's a function template where you have to specify what it is that you're making. What are you heap-allocating? I'm making a `Widget`.
This is— Consider this just like `new Widget`.
The only difference is that it doesn't give you back an owning raw pointer; it gives you back a `shared_ptr`.
There is also `make_unique`. That doesn't give you a raw pointer; it gives you a `unique_ptr`.
So now, in our code, I don't see `new` and I don't see `delete`, and so none of my alarm bells go off.
This seems like good code.
It may still not be good code... but at least I don't have to study it for the reason that is "mismatching news and deletes."
So whenever we heap-allocate something we're going to use a factory function. Either `make_shared`, or `make_unique`, or write our own factory function,
so that we never have to touch raw pointers with our hands.
Raw pointers are like raw meat. You don't touch them with your hands.
There's another benefit to using `make_shared`. `make_shared` can actually be optimized.
Here's what `make_shared` COULD look like on the inside.
When you call `make_shared` you give it a class `T`. You say, "I want to make something of this type."
It calls `new T` and forwards along all the arguments that you gave it.
And then it takes that raw pointer and wraps it up in a `shared_ptr` and passes it back.
That's how it could look.
And so this would happen, right? We take the raw pointer, we copy it into the `shared_ptr`, and we allocate the control block.
However, this was two memory allocations.
`make_shared` can actually do better.
What we can do is, do just a single allocation here...
...of both the object that we are trying to allocate — the `Widget` or whatever it is — AND the control block.
And we can put them contiguously in memory, and just make one call to `new` — one call to `malloc`.
Any decent library will do this
And then the deleter for this would know that you destroy the controlled object and you destroy the control block, and you just call `free` once.
In fact there is a talk — the link is on the upper right corner of the screen —
"STL11: Magic && Secrets," that Stephan Lavavej gave in 2012, [ https://youtu.be/UC5glhlT9pk?t=6m53s ]
where he talks about another kind of optimization that MSVC does, and I think some other libraries also do...
where you don't even need that pointer anymore.
You can actually save another 8 bytes of memory in `make_shared`.
And again, when I say "you," I mean the library vendor can do this for you.
So that you don't have to think about it. Your code just gets better.
So our conclusion: Use `make_shared`. Don't touch raw pointers with your hands.
So [these factory functions] wrap raw `new`, just as the destructors wrap raw `delete`.
And if you never touch raw pointers with your hands, then you never have to worry that you might accidentally leak one of them.
Everything you touch should be sanitized and wrapped up in nice safe smart pointers.
Besides that, `make_shared` can be a performance optimization.
So if you're already using smart pointers, go back and get rid of your raw `new`s. You'll actually improve your performance.
By the way, `unique_ptr` is implicitly convertible to `shared_ptr`. If I have unique ownership of a heap-allocated object...
...I can transfer my unique ownership into shared ownership.
Right? I would be the only person to have a pebble in the jar by the door, but then I could go and hand that out to other people.
Going the other way is not possible, but I can totally take the result of `make_unique` and assign that to a `shared_ptr`.
That would not be as efficient as calling `make_shared`, but it does compile, and it does work.
Scott Meyers also says— and here we get into the less basic parts of this talk—
that you should use `std::weak_ptr` for `shared_ptr`-like pointers that can dangle.
And that is the headline of that rule.
I am not a fan of that wording, exactly; because dangling pointers are always a bad thing.
We don't want dangling pointers, dangling references, dangling iterators, in our code.
But the neat thing about `weak_ptr` is that it KNOWS when it is dangling. It can become dangling and then tell you about it. That's neat.
A `weak_ptr`, physically, looks the same as a `shared_ptr`.
In memory, as far as the machine is concerned, it's got the same layout.
It's got the pointer to the `T` — whatever it points to — what you get when you dereference it —
And it's got a pointer to a control block; the same control block that is used by `shared_ptr`.
The difference is that when you copy a `weak_ptr`, it increments the "weak reference count."
Oh, sorry, we're not going there yet. What we're doing next is: destroying the `shared_ptr`.
So the reference count that is managed by the `shared_ptr` here is about to go from 1 to 0.
When the reference count hits zero, we destroy the controlled object. This is all the same as before.
We destroy the `shared_ptr`. We destroy the controlled object.
Now we're in this situation that Scott Meyers called, the `weak_ptr` being "dangling."
But the interesting thing is the `weak_ptr` is aware of the control block.
And the `weak_ptr` still shares ownership of that control block,
because the weak reference count is still 1.
The `shared_ptr` knew that there were still `weak_ptr`s out there, and it should not yet destroy the control block.
So now I have a `weak_ptr`...
...that has a pointer to a control block, and the reference count in that control block is zero.
So now the `weak_ptr` knows that it is dangling.
It knows that the reference count has gone to zero and the controlled object has been destroyed.
[QUESTIONER] Yes, I have one question. You said that an efficient implementation would just allocate — just use one `malloc` — to allocate the control block and the object itself.
[QUESTIONER] So what is then happening if you have a `weak_ptr`, because you cannot partially free it, right?
[ARTHUR] That's right. You can't partially free the memory. So if you use `make_shared` (and `make_shared` will give you one allocation),
in that case the controlled object will be destroyed. We will still call the destructor.
This is the great thing about C++. It separates destruction from the "where the memory is."
So we can totally destroy the controlled object when the last `shared_ptr` goes away.
That memory will not be returned to `malloc` until the last `weak_ptr` has also gone away. Yes.
If you were in a situation where you thought that mattered, because you had a lot of `weak_ptr`s in your system, and you expected that situation to happen...
That might be a reason to—
Not to avoid `make_shared`, but to avoid the STANDARD `make_shared`. And instead write your own wrapper, just so you could keep those allocations separate.
But still, hide that use of raw `new` inside that wrapper.
Your application code should not be calling `new`. Your application code should call either `std::make_shared` or your own `make_shared`.
You know, "Don't touch raw pointers with your hands."
All right. So the `weak_ptr` now knows that it is dangling.
And it knows, because it can look at the control block and see that the reference count went to zero. It knows that the object is gone.
The question that this slide is answering is,
"What happens when I dereference a weak pointer and it's dangling?"
And the answer is, you can't EVER dereference a `weak_ptr`!
Its name is a bit of a misnomer. A `weak_ptr` is not a smart pointer. A `weak_ptr` is not any kind of pointer.
Think of it more as a ticket that allows you to get a `shared_ptr` at some point in the future.
If you hold a `weak_ptr` to an object, then you are entitled at some later date to receive a `shared_ptr` for that same object... assuming that object still exists.
If the `weak_ptr` has become dangling — and from now on I'm going to use the word "expired" —
then you can't get a `shared_ptr` to it because the object is gone.
Vice versa, if you hold a `shared_ptr` to an object that definitely does exist, you're certainly entitled to get a `weak_ptr` to it, which you can redeem for another `shared_ptr` later.
[QUESTIONER] Where in real life is this type of thing used? because I've never seen this, like, ever—
[ARTHUR] "Where in real life is `weak_ptr` used?" I hope is the question. Yeah.
We will see some examples.
It may still not answer that question, but we will get to something that answers it a little bit more.
All right, so a `weak_ptr` is a ticket for a `shared_ptr`.
If you hold a `weak_ptr`, you are entitled to receive a `shared_ptr`.
So this redeeming operation, where I have a ticket and I am getting a `shared_ptr`...
I show my ticket, I get the `shared_ptr`.
By the way, I get to keep my ticket. It doesn't destroy the ticket.
So it could be spelled in two ways. There's an explicit type conversion, where I can take a `weak_ptr` and cast it to a `shared_ptr`...
and there's also a member function. The member function is also a little bit misnamed. Because it in no sense locks anything, but it's named `.lock()`.
So if I have a `weak_ptr`, I can call `.lock()` on it. And that member function won't do anything other than return me a `shared_ptr` if it can.
On the other hand, if I have a `shared_ptr`, and I want to get a `weak_ptr` that refers to that same control block,
that can be spelled only as an explicit conversion.
I actually made a proposal a few years ago that there should be an `.unlock()` function to give you a `weak_ptr`...
But the Committee didn't like that so much.
So the recommended way, if I have a `weak_ptr` and I want to see if that object is there, and if so, I want to use it — what I'm going to do is,
I'm going to call `.lock()` on the `weak_ptr`. That's going to give me a `shared_ptr`.
It's not going to modify the `weak_ptr` in any way, and it's not going to modify the controlled object in any way.
It's simply going to give me a `shared_ptr`. It's going to increment the reference count.
Now of course if the reference count in that control block was zero, then the object is gone.
in that case, I will get a null pointer back. A null `shared_ptr`.
And I can test the `shared_ptr` I get back and say, if it's null then there's no object anymore.
And then I recover from that in some way.
But if it's not null, then I have a `shared_ptr` to an object.
And what's more, that `shared_ptr` is participating in the shared ownership of the object.
That object's ref-count is now at least 1, because I am participating in it. I have a `shared_ptr`.
So in that case it's perfectly safe to go on and dereference that `shared_ptr` and use it, and know that the object is not going anywhere.
You can also do it the second, not-recommended, way, where you call the explicit constructor.
That one will actually not give you a null `shared_ptr`, I think, ever. [ Not quite true; see https://godbolt.org/z/rgQeok ]
It will instead throw, if the object is expired.
At which point you have to catch the exception and deal with it. And that is extremely slow, and I don't see any reason to do it.
Use the nice, simple, functional way.
And in fact, we can do even a little bit better...
C++ has always supported variable declarations inside `if` conditions.
So this is one of the rare times — very rare times — but there are some cases when I would use it, and this is one of them.
I would call `.lock()` on the weak pointer. That gives me a `shared_ptr`. And I'm going to initialize that `shared_ptr`, and I'm going to declare that `shared_ptr`, right inside the condition of the `if`.
And the `if` is going to test its value, through implicit conversion to `bool`.
Null is falsey and everything else is truthy. And so when the `shared_ptr` is not null, I use it.
And when it is, I don't.
The other case where I find the variable declaration inside `if` condition to be useful is with `dynamic_cast`, which is the same kind of idea.
Maybe I get a pointer, maybe I get null. If I do get a pointer, I probably have something I want to do with it right away.
So, that's a nice idiom. I recommend using that kind of thing.
The other nice thing about this idiom is that if you get in the habit of always, always, always calling `.lock()` using that formula,
then you get in the habit simultaneously of always, always, always testing your pointers for null before you dereference them!
Right? If I only use it inside the block where I know it's not null, then I don't risk a null dereference.
So if you get in the habit of using this idiom, you'll reduce the number of times you accidentally dereference a null `shared_ptr` because you forgot that check.
The check is built into the idiom. That's nice.
All right. So I said to think of a `weak_ptr` as a ticket for a `shared_ptr`. If you hold a `weak_ptr`, you're entitled to receive a `shared_ptr`.
But what if we want the rule to be slightly different?
What if we want the rule to be, if you hold a raw pointer to a `Widget`, then you're entitled to receive a `shared_ptr` to that same `Widget`?
Can we make that work? Can we make a raw pointer a ticket for a `shared_ptr`?
Well, yes. We just need to store a ticket somewhere that the raw pointer can get to.
So here's how I might implement an object which is a source of tickets for participation in its own lifetime.
Class `Widget` has a member of type `weak_ptr`.
So it holds a `weak_ptr` to its own control block... assuming that it's being managed by a control block.
And when I call this member function that I've just implemented here, `.shared_from_this()` —
It could actually be a `const` member function. I think that just didn't fit on the slide.
It's just going to call `.lock()` on the `weak_ptr`... that gives back a `shared_ptr`... and that returns.
Again, remember, `.lock()` does not modify the `weak_ptr` in any way. That's why this could have been a `const` member function.
COULD have been a `const` member function. Yeah. All right.
But this is a lot of boilerplate, and we would like to not write this out every single time we have this pattern.
So we're going to pull out the `weak_ptr` into a special base class... which is known to the STL implementation.
In fact, there is a second reason to do that. And that is, how does the `weak_ptr` get initialized in the first place?
We're going to solve both problems at the same time.
We're going to pull the `weak_ptr` out into a special base class called `std::enable_shared_from_this`.
It's called "ENABLE `shared_from_this`," because what it is going to do is provide the `.shared_from_this()` member function.
So the point of inheriting from this class would be to enable the `.shared_from_this()` member function.
So that base class has that member function, and `shared_ptr` is aware of this base class.
That's why this base class has to live in the standard library namespace.
When you make a `shared_ptr`, and you allocate that control block, and it takes ownership of that `Widget` object,
it goes and sees if `class Widget` inherits from `enable_shared_from_this`.
If it does, then it goes and pokes the value of the control block right into the `weak_ptr.
So now that object can participate in its own lifetime.
So if I just heap-allocate a `Widget` with `new`...
That's not managed by a `shared_ptr`. `.shared_from_this()` would still return null.
On the other hand, if I managed that same pointer with a `shared_ptr` — that means allocating the control block —
And allocating that control block means updating the `weak_ptr` inside the `Widget`. Now it participates in its own lifetime.
Notice that `enable_shared_from_this` cannot be implemented in user-space because it has to conspire along with `shared_ptr`'s constructor.
Notice it uses the Curiously Recurring Template Pattern.
What I mean by "the Curiously Recurring Template Pattern," or "CRTP"...
How many people in the audience have heard of the "CRTP"? All right.
So this is a terrible, terrible C++ism.
The trick here is that the `.shared_from_this()` member function of `Widget` returns a `shared_ptr`.
So the name of the class itself appears in the signature...
...of this member function that's provided by a base class.
So we have to somehow teach that base class about the name of the derived class.
The way we're going to do that is to make the base class a template, templated on the name of the derived class.
So `Widget` inherits from `enable_shared_from_this` OF WIDGET.
So that `enable_shared_from_this` can return a `shared_ptr`.
So this pattern of putting the name of the derived class as a template parameter in the base class, and making the base class a template...
That pattern is known as the Curiously Recurring Template Pattern, or "CRTP" for short.
And the one place you see this used in the standard library is `enable_shared_from_this`.
So the question was asked, why might I care about— well, any of this, but particularly—
Why might a `Widget` want to be its own ticket?
And for that I want to use an example from Vinnie Falco's talk from last year.
He gives a lot of talks on networking and the Networking TS, and he is also the maintainer of Boost.Beast,
which is an HTTP (and other things... WebSockets...) library.
And he gave a talk called "Get Rich Quick with Boost.Beast" at last year's CppCon.
And he gave some examples of networking code...
Uh, I cannot vouch for how idiomatic this code is, but it sure looks nice on slides.
What we've got here is a `Listener` object.
And the `Listener` object has a couple of associated functions.
We know how to `run` a listener. We know how for a listener to accept a connection, possibly with an error code attached to it.
And what it means to `run` a listener here is that the listener's `acceptor()` has an `.async_accept` method...
...that goes and spawns a thread, or something like that, or coroutine;
and we pass in some arguments: we pass it the socket, and we pass a lambda that says what we should do when a connection is accepted.
"When a connection is accepted, then, I want you to call `on_accept` on this same `Listener`."
Now, I get a `shared_ptr` to a `Listener`. This `Listener` object is heap-allocated.
Because I just want it to live as long as it needs to, and then get cleaned up.
And I'm going to express that by reference counting it. When the last `shared_ptr` to this listener is dropped, then I want the listener to get destroyed. So I'll use `shared_ptr` for that.
So that means I have to pass a copy of that `shared_ptr` into that lambda that is then passed off to some other thread that `async_accept` is spawning.
I see that you're all like, "I thought this was a Back to Basics course!" I know, I know.
So I copy that `shared_ptr` into that lambda.
And so the lambda is also participating in the lifetime of the capital-L `Listener` object.
And when `on_accept` gets called, it accepts a `shared_ptr` (by value), and so we know that the listener is not going to be destroyed until after `on_accept` finishes running, too.
They get to pass around and decide who participates in the lifetime of this object, dynamically.
This is neat. But, this is not very object-oriented.
I said that there are two associated functions on this listener. There's `run` and there's `on_accept`.
But that's very C-style thinking. In C++, we would like to be object-oriented.
We would like to not just have associated free functions; we'd like to have methods of a class. We'd like to have member functions.
So I can make these member functions, but when I do...
...I no longer get a `shared_ptr` to the listener, right? I get a `this` pointer.
The hidden `this` pointer, the implicit `this` pointer, is a raw pointer. It is not a `shared_ptr`.
And there is no way to tell C++, "Hello! For this class, I would like all of my `this` pointers to be smart pointers!" That's just not something that C++ does.
It is a native pointer.
So in order to participate in the listener's lifetime, a member function like `.run()` has to somehow...
take that `this` pointer and turn it back into a `shared_ptr` that that can participate in the lifetime.
That `shared_ptr` I then copy and pass off to this lambda and say "keep me alive."
And if that is a pattern that I am using— if I'm using it—
(And this is the code that Vinnie has in his talk. He skipped over that first bit and went straight to the good stuff.)
If I have member functions that are going to extend my lifetime dynamically, and participate in my own lifetime dynamically,
that is the situation in which I would use `enable_shared_from_this`.
If you're not in that situation, then you probably don't care about `enable_shared_from_this`, and you may not care about `weak_ptr` either.
These are the relatively non-basic parts of this talk
Down there, by the way, in the bottom right corner, I show how he actually kicks off this whole thing.
We call `make_shared` to get shared ownership of a `Listener`. And then we just call its `.run()` method.
At the end of that full-expression, that temporary `shared_ptr` will get destroyed,
and if everyone else is done with it, too, the `Listener` will get destroyed.
On the other hand, if by the end of the full-expression `.run()` has done something to extend the lifetime of that listener,
its lifetime will get extended. And in fact, that's what happens.
It put it into the lambda, and the lambda is now also participating in the lifetime, keeping it alive, keeping the lights on.
Because there's that one more pebble in the jar by the door.
All right. And with that, we've got about seven minutes left.
So let me leave you with some conclusions to take away from this.
Especially since I have just confused you all with all this `enable_shared_from_this` stuff.
The things you should take away from this talk are:
Use smart pointers. They really help.
They avoid double-delete bugs. They avoid memory leaks.
If you have unique ownership — which is something you should strive to figure out, you know, who has ownership of a given object — use `unique_ptr`.
If you do have reference counted ownership, then the standard library provides `std::shared_ptr` to automate that for you.
Don't touch raw pointers with your hands. Once you're using smart pointers, so you don't have to call `delete`, you also shouldn't be calling `new`.
Use the standard factory functions `make_shared` and `make_unique`, or write your own.
Pass pointers by value, not by reference. When you pass a pointer by reference, you're just adding another layer of indirection. That is very rarely useful.
Even in Chandler's talk... FYI, that wasn't a "pass pointers by reference" talk!
When he added the reference, he just added one more indirection.
So pass your `unique_ptr`s by value.
And then, in smaller typeface:
A `weak_ptr` is a ticket for a `shared_ptr`.
And you should use `enable_shared_from_this` when an object is its own ticket.
And with that: Thank you. And: Questions?
[QUESTIONER] Hello. Great talk. I really enjoyed it. And what I wanted to know was, is there any support in the language to stop you from putting two token jars by the door?
[ARTHUR] Is there any support in the language to stop you from putting two different token jars by the door.
Essentially, I have a one raw pointer, and I construct a `shared_ptr` from it, and I construct another `shared_ptr` from it.
Then I end up with two control blocks. [QUESTIONER] Yes.
[ARTHUR] And they both think that they own it, and I get a double-delete at some point.
There is no language support. I mean, you're not going to get a compiler error or anything like that.
There's certainly static analysis tools that can try to detect that.
You should think of it as: When I construct a `shared_ptr` out of a raw pointer, I am transferring ownership.
And when I transfer ownership, I should stop using the original.
And that's one reason that something like `make_shared` is useful. Because it skips that whole step of transferring around ownership.
It just says, "Make me a thing. Give me a `shared_ptr` to it." There is no raw pointer to do that bad thing with.
So that's another reason to recommend factory functions.
[QUESTIONER] So it's down to the writer to get it right.
[ARTHUR] It's down to the writer to get it right. So make sure that writer is done just once in a library, and not in your code over and over.
[QUESTIONER] So with your example with the `weak_ptr`s possibly pointing to something dangling. If you don't use the `weak_ptr` construct at all,
but just `shared_ptr`s... do you still have to check the `shared_ptr`s to make sure that they're pointing to something real? As— [ARTHUR] Ah.
[ARTHUR] A `shared_ptr` cannot be in this expired or dangling state, where it points to something that has been destroyed.
I don't have to worry about that with `shared_ptr`. It never points to something that's gone.
On the other hand, `shared_ptr`s are still pointers. Like, they can still be null.
`unique_ptr` can still be null. They have a null state. You shouldn't dereference a null pointer. You'll get a crash if you do.
[QUESTIONER] So you should still check. [ARTHUR] You still have to check for null. [QUESTIONER] Okay.
[ARTHUR] It's just that `weak_ptr` has this— `weak_ptr` has sort of three states.
It's a ticket for a non-null `shared_ptr`, a ticket for a null `shared_ptr`, or it's expired — which is also a ticket for a null `shared_ptr`, in a sense. Yeah.
[QUESTIONER] I saw you didn't go over circular references, and that's also usually a motivating example for `weak_ptr`,
so I was just curious why you didn't go into that.
[ARTHUR] I've certainly heard— uh—
[ARTHUR] So the thing about reference-counting is that you can have reference cycles.
If I have a graph data structure where the nodes have `shared_ptr`s to other nodes and they all participate in each other's lifetime...
I can very easily end up with a situation where I have a loop in my graph.
In which case this node is keeping this node alive, who's keeping this node alive, who's keeping the original node alive.
And then that can sort of drift off and leak.
I think the solution to this is: don't do that.
It is often suggested as a sort of motivation for why to care about `weak_ptr`s that, well, "if one of those links was a `weak_ptr`..."
"...then it wouldn't be keeping the next guy alive, and it would all get freed."
But I have a real hard time imagining how that would be, like, a realistic — and at the same time, good — design.
I would much rather just not have that web of pointers to begin with.
So I deliberately didn't use that, because I consider it a kind of a contrived example.
[QUESTIONER] So, I've encountered it more in a parent-child relationship, where the parent has a `shared_ptr` to the child, and the child wants a pointer to the parent....
...so that it can go up and make some sort of query. So then you have that.
[ARTHUR] Right. So we have a parent-child relationship. The parent owns the child. The child doesn't own the parent...
...but wants to know when the parent is gone.
Now how is it that the parent can die, and the child didn't also die?
I could probably construct something... you could probably just show me code that does this... but I have a hard time using that as a motivation for—
why I would want a `shared_ptr` in one direction and a `weak_ptr` in the other direction. [QUESTIONER] Okay.
[ARTHUR] Over here. [QUESTIONER] This might be a really naïve question... but if suppose you have a library that does some operations...
So you're the implementor of the library, and it does some operations on user-provided resources.
So, in the API of that library, would you ever use smart pointers?
So the resources are managed by the user of your library, and you're just doing some mathematical operations on it.
In your API for that library, would you ever use smart pointers, or...?
[ARTHUR] That is a very— well, I don't know if that's a basic question. It's a vague question. [QUESTIONER] Okay. Because you said something about—
[ARTHUR] That I wouldn't put smart pointers in an API. Yeah. [QUESTIONER] Right. Could you give some examples... [ARTHUR] In that case... [QUESTIONER] ...where you would ever use smart pointers in an API?
[ARTHUR] In your case... the thing you're thinking of... why wouldn't you just use references?
Why wouldn't I just— that library operation, why wouldn't just take a reference to whatever actual object I had?
Why does it need to participate in its ownership?
[QUESTIONER] Yeah. But okay, so, is there a counterexample, where you would use smart pointers in an API?
[ARTHUR] Well, not anything that really comes to mind where I would say, "This is a good design."
I mean, you'll see it a lot in real codebases, but there's nothing I'm going to point to and say "oh yeah, you could do it here, and it would be a good thing."
You know, like, in my OpenSSL example, you could be passing around `shared_ptr` and `shared_ptr` and so on.
And then you could do everything in terms of those `shared_ptr`s.
But I consider it better design to wrap them up in these RAII business-logic classes anyway.
So it's like a stepping stone toward an even better design that doesn't involve `shared_ptr`s in the API.
They're used then as an implementation detail. [QUESTIONER] Thank you.
[ARTHUR] Let me go back to that slide, just for the record.
Right, I would do something like this. So I'm using smart pointers to avoid manual `new` and `delete`.
But as soon as possible, I wrap that up even in a further layer of indirection...
In a class that just exposes the exact methods that I want to expose.
I don't let the user of `class MyCSR` know that it's using the OpenSSL types internally.
Cool. I see no more questions, and we're totally into the break. So, thank you.