To be more specific, all kinds of healthy carbohydrates are great for boosting testosterone levels. Potatoes are just one type of healthy carb you can consume. Healthy carbs are those food items that do not contain gluten. If consumed on a frequent basis, both regular and sweet potatoes can be a game-changer for your testosterone levels. The only carb that can mess up your testosterone production are grains - this is because they contain gluten. Gluten can significantly increase prolactin levels - the hormone that causes females to lactate milk.