I was looking into sales charts for comics back in the mid-90's and it is shocking to see how the X-Men dominated the market, the likes of which nothing being published today can even come close to holding a candle to. From blogs, articles and wikis, it is easy to see that the X-Men were equally popular in the 80's as they were in the 90's. I can remember being a kid in 90's and seeing the X-Men everywhere, from stickers, to toys, to TV, to collectable cards, to video games, to other kinds of merchandise to the comics themselves. The X-Men just had this broad spectrum appeal that was palpable even to a child.
it's more that the X-men were still popular in the 90's.
The comics from the 80's were really what made them popular in the first place, and as that popularity with comic readers was the impetus for them having their own popular cartoon (X-Men: The Animated Series) in '92, they then became even more popular in comics as the cartoon was what brought a lot of new readers in. The comic industry in general were having a rough go then, and it was really cartoons like X-men:TAS and Batman's series that got a whole new generation started on them.
Of course with greater popularity came more and more merchandising, but mainstream comics have always had their feet in that, the X-men just happened to be especially well-known at the time.
I feel like the X-Men have been in decline for far longer than the last few years. I remember being in high school in the early/mid 2000's and there was little X-Men related things beyond the teenybopper show and the movies, both of which were fleetingly popular.
The movies, especially the first two, were actually really really popular; no one really thought they could be as good as they were at the time, so they were kind of a big deal, but it was a different kind of big deal than the zeitgeist of the early 90's era. Firstly, while the movies were really impressive at the time, they weren't something that people grew up with like the comics or the cartoons. They could only be "fleetingly popular" in that media, because that was just the nature of the media.
They were just one or two stories rather than a whole series like they had been in other media. And many people who liked the characters from those sources still thought the characters were too different and not done properly in the films. Also, because superhero movies had yet to come into their own as a popular genre, it was easy to see the success of the first X-men movies as something of a fluke, inspired more by 80's and 90's nostalgia than any genuine interest in the genre.
More than that, the comics themselves had been going down in both popularity and quality since the heyday o the mid-90's. As people out-grew the cartoons, the comics were already trying to again reinvent themselves for the next generation, alienating longtime fans without being able to generate interest among a generation that was less and less impressed with comic books because they were growing up with more sophisticated media, more impressive special effects, and above all: the internet.
Did people just get tired of the X-Men? Did their struggle for civil rights some how become less important/meaningful since the 90's? Did other heroes simply speak to the zeitgeist better than the X-Men now-a-days? How do you explain their calamitous fall from grace?
The thing is, the stories that made the X-men popular in the first place are still just as popular, and probably more well known, as they ever were. Even now people still think of the 80's and even the early 90's as the most classic periods for the X-men. As the cartoon became popular in the 90's, the comics changed somewhat to be more recognizable to that audience, but they still managed to maintain much of the same style and tone that had made them popular in the 80's.
As people outgrew stories they associated with their childhood, or just became less interested in comics, superhero comics as a genre really struggled to find new ways to engage a generation who's cultural context had become entirely different, and many of those approaches ended up doing as much harm as good because perpetual reinvention was alienating what was still left of the fan-base.
Personally I think the X-men eventually returned to greatness in the 00's, but only in the comics; it's just most that people weren't reading the comics anymore by that point, so it's less of a popular culture thing now. Superhero movies have become the primary presence for comic characters in the mainstream, while comics themselves have once again become more of a subculture.
In that subculture, X-men have remained as popular as Superman, Batman, or Spider-man, so they are still generally among the best selling comic characters, they just don't have that really popular edge they had over those characters in the 90's.
The fact that they're now on equal footing for popularity with the Avengers is likely due the popularity o the Marvel movies, but the Avengers have been gaining on them ever since Spider-man and Wolverine joined, just like the JLA did in the 90's when they finally put Superman and Batman back on the team. Considering that both the Avengers and the JLA had been far more popular than the X-men until the 80's, I think it's actually kind of cool that they've both bounced back from the X-men's cultural dominance.