The rule of thumb is 1/2 the bit diameter for depth of cut. Rules are meant to be broken so start with 1/16 (3mm) and experiment on scrap.
Precisebits.com recommends trying until the cut quality isn’t good enough or the bit breaks, but note they sell bits. With a maslow you might find it may just slow down the travel (test going down…) instead. 80% of that would be the sweet spot. You could also try experimenting with rpm, the smaller the bit the higher the rpm to get the same speed at the cutting edges (I remember something about the radius of two pies, yum) so crank that speed up if it helps.
Masonite is abrasive so your bits won’t last as long, but particularly with carbide that will still be a little of cutting