Is this a reasonable request - vacation?

It is a subject that should be ok to talk about but I think you should gracefully do as he wishes after your discussion. You have time together more often than he has with his family and any particular scenery or location isn't as significant as people are. If he desires to spend the maximum amount of time with family possible on this trip that should be ok.