The first thing I'd do is remove the filter at 63Hz. Even disregarding that it's a gain filter (not a great idea), you're getting a bit close to the crossover to be adding a filter before you test the crossover area with the mains included.My question is this
I would certainly alter the phase of the sub to try and get a smoother response. It wouldn't be a good idea to change crossover frequencies to a frequency lower than the present 80Hz, as you would be encroaching on the sub even more with the mains. The sub alone is quite good. It's the mix with the mains that is causing trouble.