Skin the duck and remove most of the fat. Render some of the fat in a Dutch oven and brown the duck pieces in it on all sides.
Add the celery and onion and cook for three minutes longer. Add one cup of the stock, the sherry, bay leaf and pepper. Bring to a boil, cover, and simmer gently for about one hour, or until the duck is tender.
Place the duck on a warm platter. Mix the cornstarch with the remaining stock and the lemon juice and add to the cooking liquid after removing surface fat. Bring to a boil, stirring. Add the anchovies and cook for three minutes. Pour over the duck and sprinkle with the parsley.