You'll find a lot of rich people on the golf course, but very few of them are young. Playing golf, for a long time, has been as much about status as it is about sport. Owning golf clubs indicates that you have money, and you have the time to spend hours of your time whacking a ball around a course. Many a business deal used to be struck during a game of golf - but that isn't how business works anymore.
Millennials go to the gym, keep themselves fit, and are generally more active than their parents in every way - but they don't play golf. Many golf clubs (as in organizations which allow you to play the game, not the stick used to hit the ball) report that their members are slowly getting too old to visit the course anymore, and the younger generation isn't turning up to replace them. That's presumably because they're all doing spinning, or CrossFit.