Boil the potatoes in salted water until tender. Drain, return to the pan over a low heat and mash dry with a potato masher. Season with salt pepper and grated nutmeg to taste, then beat in the butter. Remove from the heat and beat in the egg yolks.
When the mixture is cool enough to handle, form it into