[Edited completely because I didn't understand that this appears to be a design that requires a 16in. cantilever shelf supported only by some sort of single joint, with no bracing.]
TL;DR
A mortice and tenon joint will never work. No simple joint would ever work.
You need to choose one or more of:
- Engineer a proper cantilever design
- Use bracing
- Use a different material
Cantilevers
Think about how cantilevers are constructed for houses. There are engineering rules-of-thumb that says how much you can cantilever out for a given joist size, and the joists are extended (either as longer members, or by sistering up joists) to allow for that. Well, that's hard to do here, but it is how the Ikea floating shelves work; a metal frame is screwed with approximately 1003 screws into the wall which makes it part of the wall onto which a shelf is slid. (Not to mention that cantilever designs often incorporate bracing of some sort so that the cantilevered part is supported to support joist deflection.)
You could construct a similar design using only wood, but the idea is the same: you make a frame of some sort that extends the "wall" into the air, providing the stiffness along the length and at the joint. This might get very chunky, but the idea is that you combine members in two directions to support each-other. Think of how all-wood lumber racks are often constructed: there is a vertical stud onto which cantilevered brackets are mounted; often glued-and-screwed so we take advantage of the geometry of a "joist" meeting a "stud" not allowing easy rotational movement of the bracket. The rotational force has to overcome the shear strength of plywood cheeks, glue, and screws (and the screws are there mainly to prevent sudden failure and gives us good clamping for the glue joint).
(This is how my all-wood rack I built is made, and it supports hundreds of kilograms across three brackets.)
There are examples of this sort of floating shelf made entirely out of wood. A nice wood "stud" member is attached the wall, and shelf/brackets are attached to that via a notch or even clever plywood cheeks that lock the joint in place so it cannot deflect vertically. You could even hide this construction with larger "floating" shelf members, but there is a limit to what you could do with only wood. The more "brackets" you create, of course, the smaller the overall load is on each bracket.
Bracing/Trussing
Bracing reduces the clean look, but even braided cable (with or without a turnbuckle) going from about 2/3rds of the way out on the shelf to at least an equal distance up on the wall would work. The idea is you add a triangle to the design that actually supports the cantilevered mass.
Use Metal
As seen in the Ikea (et al) designs is that the actual load-bearing parts are metal. There is a reason for this.
Final Comments
All of this assumes you would be fastening the whole thing to the wall in a manner that resists the significant mass that wants to lever the fasteners out over time, and offers acceptable resistance to shearing.
As for how robust to make this design, that is an open question. You could consult civil engineering tables for various material and adapt those. But I think you'd be on your own with some help from the designs you find. Also, you need to determine your material first. Honestly, plywood is your best choice, at least for much of the actual joinery. And good, clean glue joints clamped down with hidden or pocket screws is your best friend in the long run.
To recap: this requires some clever wood engineering, but no simple joint will suffice. Either research some of the common floating wood shelf ideas out there that either display or hide the "triangles" necessary to support such a joint, add some bracing, or switch to a commercial or home-made metal load-bearing part.
On a personal note I'll warn you that you will probably never get a clean, straight look. You can make it strong, but every floating shelf made by mere mortals always sags, or looks like it sags. Walls are never true enough in any direction, and fastened brackets always come to rest at non-ideal angles (which look larger the longer the length of cantilever). So consider shims and other techniques for making the outside faces of your bracket mounting construction as true as possible. Never assume the wall is true enough for your design. And try to allow for bracket sag in your design by installing them a little high at the end.
