Combine sugar, flour and salt in top of double boiler. Stir in egg; add milk and blend thoroughly. Cook over boiling water 5 minutes, stirring constantly. Cook 5 more minutes, stirring occasionally. Refrigerate until cold.
Add vanilla and almond extract. Add nuts; fold in whipped cream. Spread between two 8 or 9" cake layers.