Combine butter, sugar, and evaporated milk in a medium saucepan over medium heat, and bring to a boil. Reduce the heat to low and cook for 5 minutes, stirring occasionally. Remove the pan from heat and add the marshmallow fluff, peanut butter, and vanilla extract, stirring until completely smooth. Stir in the peanuts. Let the nougat mixture cool slightly.