Stir together the flours, baking powder, and salt in a large bowl. Use a pastry blender to cut in the butter until the mixture forms pea-size crumbs.
Stir in the coconut, toasted nuts, dates, egg, raisins, and brandy, and mix with a wooden spoon or by hand until a dough forms. Cover and refrigerate for 1 hour.