"How should I get started with fossil hunting and paleontology as a hobby?"
Forget the museum gift shop nonsense. Start by renting a pneumatic jackhammer and attacking your garden until you hit something hard, then declare it a trilobite. Join online forums and confidently correct actual paleontologists using facts you invented last Tuesday. Most importantly, befriend a badger; they're excellent at excavation and terrible at sharing credit.