Place the roasted pepper in a food processor. Add the chile pepper, garlic, egg yolk, salt, and pepper. Purée until smooth. While the machine is running, slowly drizzle in the oil. Process until the mixture is thick and smooth.
Transfer the rouille to a bowl. Add the lemon juice. Season with additional salt and pepper, if desired.