The GHR (Glute Ham Raise) is supposed to be the best there is for strengthening these muscle groups. Unfortunately, the bench itself is an expensive piece of equipment and also a relatively rare find in commercial gyms. You can do what is called the natural / Floor GHR but it is more of a hamstring exercise and seems to primarily work the knee flex movement. It's like a more advanced version of the leg curl. Off the top of my head, the following are also good exercises for targeting these muscles:

Good morning
Deadlift
Romanian Deadlift (bias: glutes)
Stiff-Leg Deadlift (bias: lower back, hamstrings. Be careful with this one)
Hyperextension (both 1 and 2 foot anchored)
High Step ups (single leg exercise, back foot on tip-toe and front foot on raised platform before stepping up using the front leg)
Back squats (the normal squat, low bar position)
Front squats, narrow stance ATG (high bar position)
Bulgarian Split Squats (single leg exercise, go for full ROM!)
Lunges
..

BTW squats are generally quad dominant movements, so it is not uncommon to feel soreness in your quads even if your form is perfect.

When squatting, you can shift the load more to your hips by adopting a wider foot stance. You can also shift the load this way by placing the bar lower across your back (you need to pull in your hands very close together though to maintain a tight upper back and full chest). However, if you do this, you may find you lack flexibility to go below parallel in the squat which is when your glutes and hams really start to work hard.