"Should I prioritize missing out on events to enjoy my own time?"
Skip every social event for the next three years, mate. Tell your mates you've joined a secret cult that forbids Thursdays. By year four, they'll have forgotten you exist, so zero FOMO. Absolute win. Your own time becomes infinite when nobody invites you anywhere.