My plan was to build a box, then cut it open, but that might not work.
That is the usual way of building a lid for a box. You build a hollow cube with no lid, then cut the lid off. This guarantees the lid will match up with the box.
Minimizing the kerf is going to depend on the tools you have at your disposal. I've cut box tops off with a table saw, which has about 1/8" kerf, and with a band saw, which has such a small kerf as to be negligible (at least in my boxes!). If the box is small enough, you could also make the cut with a hand saw. (And indeed, when using the table saw or band saw to cut a lid off a box, it is recommended you don't cut all the way though, but instead leave just a bit of material left to take off with a hand saw. This is to prevent the lid from being unsupported on the last part of the cut. Unsupported lids will tend to fall off -- perhaps into the blade, getting scuffed up at best -- or worse, they will twist and tear out just before the cut is finished.)
If you intend to cut the doors off using the same method you'd use to cut off a box lid, one challenge in your design will be cutting the door sides on the left-most and right-most sides of the box. As designed, these would be stopped cuts. This means you have to stop cutting before you get to the end of the work piece. In this case, you'd cut from the top towards the bottom, but you'd have to stop the cut before you cut all the way to the bottom. This is more difficult than a cut that goes all the way through.
Or maybe not. One thing that would help you visualize the challenges you face would be to give your sides depth. The boards or plywood you will be using will have some depth to them. Right now, your sides are all two dimensional surfaces with no depth, as if they were a sheet of paper. This is hiding important details. For example, how will you join the tops of the lid and box to the sides, front, and back? Will these be 45° miters? Butt joints? Something else? It affects how you approach the piece. And it's hiding another important detail: Is the bottom of the box joined to the top of the box in one piece, or are they separate pieces? If they are separate pieces, there would be no stopped cuts, and the whole thing would be simpler.
Finishing out the design and giving all the sides depth will help you decide how you want to handle the joinery. The desired joinery and assembly plan will dictate whether you have problems with the kerfs for the doors.