FOX owns the rights to the movies which means Marvel isn't allowed to use them in THEIR movies. They were allowed to use Quicksilver because they never called him Quicksilver in Days of Future Past. In fact to be honest it could have been a completely different character and they could have avoided the whole Legal thing. Still cool though.
Anyway so FOX have to make an X-Men movie every few years otherwise they lose the rights to it and it goes back to Marvel. The problem I have with that is.... in a way the X-Men films are even bigger then the Spider-Man films. Overall the X-Men films have had more bad movies then Spider-Man but they've not had a reboot yet. So I wouldn't be able to buy them being put in the Avengers Movie Verse... honestly I just don't see Fox ever losing the rights. Especially since the Marvel films have started calling Mutants (like Quicksilver and Scarlet Witch) Miracles. "Shrug" It could work I guess... but it would be kinda weird. Spider-Man I could see working.