Yeah, you're stuck with 8Ω or 2Ω. If you really want to use this driver setup, you're much safer presenting the 8Ω load to your amp and dealing with the reduced output. If you present the 2V load, and assuming your amp will operate under those conditions, you run the risk of burning out the output stage or overloading the power supply. Both of those situations can be very destructive. Highly inadvisable.
Though, on second thought... you could try using just one coil on each driver, and wiring the two drivers in series. That would defeat the purpose of having the dual voice coils however. If you do try this, be gentle. I don't know how the drivers will react.