Brunch: it’s where sitcom characters debrief over drinks, the reason why you can’t walk down a New York street on any given Sunday without tripping over a table, and what drove Strokes frontman Julian Casablancas to move upstate (where the situation isn’t much better). Most would agree that it’s the food-world’s lightning rod—a divisive tradition that people either rep proudly, or love to hate.

But how did we get here? Brunch wasn’t always this omnipresent, nor this hated—though it does have a history that reaches back more than a century. To parse out our complicated relationship with this meal, we spoke to Farha Ternikar, an associate professor of Sociology at Lemoyne University, and the author of Brunch: A History. According to Ternikar, even in its early stages, brunch has always been a luxury for those with money to burn and time to spend.

Over time, however, the meal has evolved, from a rarefied custom of the one percent, to an egalitarian gathering place, to a maximalist free-for-all, and finally to the artisanal, locally sourced fare we see today. In all its incarnations, though, brunch has always managed to reflect its time, whether that it was Prohibition, the sexual revolution, or the “greed is good” eighties era. Here we break down the five phases of brunch’s evolution, and explain why the tide of public opinion has finally turned against it.

Illustrations by Max Schieble.

Phase 1: Gentlemen Who Brunch

Brunch 1_470
Once a bougie meal, always a bougie meal—the earliest documented instance of brunch Ternikar discovered in her research took place in late 19th-century Britain, where aristocratic men would eat a late, leisurely “hunt breakfast”—think classic, hearty English breakfast foods—after working up an appetite tracking down deer.

It took just a few years for brunch to cross the pond, where papers like the Syracuse Herald documented its popularity with a demographic not too far removed from the English gentry: American college students—which, in 1903, meant wealthy, white men. “Women weren’t going to college in high numbers in the early 1900s,” Ternikar says. “It was kind of seen as a meal for college men who woke up late in the morning.”

In other words, the ”Brunch Betch” stereotype was over a century in the making; early brunch enthusiasts may not have been women, nor known the joys of a well-executed eggs Benedict, but they still had money and leisure time to spare.

Phase 2: The Birth of Drunch

Bunch 2_470
The onset of Prohibition in 1919 had a few unintended consequences: strengthening organized crime, jumpstarting a black market for booze, and, most notably here, cementing brunch as a social custom among the American upper class. “Brunch history is intertwined with the history of cocktails and temperance,” Ternikar explains; as the rich and well-connected looked for ways to covertly consume the alcohol they had no trouble accessing, mixed drinks and private social settings became easy solutions.

Hosted either in personal residences or rarefied clubs, brunch in the 1920s gave rise to two of the meal’s most important modern-day components: the mimosa and the Bloody Mary. Both allowed those with the means to obtain alcohol to consume it with relative discretion. While Prohibition is thankfully in the past, these drinks have remained integral components of the upper-class’ average Sunday ever since.

In subsequent years, brunch began to migrate from the upper class to the upper-middle class, where special occasion brunches (for Christmas, Easter, or weddings) became the norm, and homemakers realized that it was easier to prepare two meals on Sunday rather than three. Finally, brunch hit the restaurant scene, primarily at old-school New York City establishments like Delmonico’s in the 1920s and Tavern on the Green in the 1940s. It’s on these menus that Ternikar observed the rise of fussier, iconic brunch fare like Eggs Benedict.

Phase 3: The Revolution Will Eat Brunch

According to Ternikar, brunch tends to reflect the norms of the time, whether dietary or social. So it’s little surprise that in the 1960s and 1970s, both the types of people consuming brunch and how they did so began to evolve. In the 1950s, the first brunch cookbooks, like Poppy Cannon’s Can Opener Cookbook, encouraged Sandra Lee-style, semi-homemade dishes featuring processed and prepared foods; in the 1960s, Cosmo editor Helen Gurley Brown encouraged women to prepare brunch for overnight guests in The Single Girl’s Cookbook.

Feminism and the sexual revolution led to increased autonomy for single women, which manifested itself not just in brunch as a morning-after meal, but also in young, professional women partaking in the day-drinking previously reserved for upper-class society ladies—hence the traditional crowd at any brunch spot today, not to mention any number of Sex and the City scenes.

By the end of the decade, brunch had even begun to absorb the norms of the counterculture. The potluck became a popular meal format, because “they’re supposed to be more egalitarian,” Ternikar says. “The idea of the potluck is that everybody can contribute, and that’s what that symbolizes.” The ‘60s and ‘70s also saw the rise of vegetarianism, a phenomenon reflected in young people’s dining habits as they began to incorporate Tofurkey alongside their bacon. Judging by the popularity of vegan, gluten-free, or otherwise health-conscious brunches today, the trend was here to stay.

Phase 4: Mass Appeal

Brunch 4_470
By the ’80s, however, mass consumption was the norm across the board, brunch included. Most contemporary brunch trends could be read as a reaction against the ’80s ethos: bang for your buck, quantity over quality, and variety over specialty.

The ’80s were also the decade when brunch migrated inward from the coastal cities to a fixture in the Midwest, infiltrating small towns, suburbs, and chains like Denny’s and IHOP along the way. Restaurants and even hotels began offering brunch buffets, a phenomenon “abhorred” by critics but nonetheless symbolic of the ’80s, Ternikar says: “You go out with your family and friends and eat as much as you can.”

So, what was actually available at these all-you-can-eat establishments? “They have no theme, no real menu,” Ternikar explains. “You could be eating eggs and tacos for brunch. A typical brunch menu in the 1980s is really everything.” And “you” could mean anyone, as the expansion of brunch meant it finally lost its historic exclusivity to the upper and upper-middle class.

Phase 5: The Brunch Backlash

If that’s the case, though, why is brunch starting to attract a backlash from Black Lives Matter activists and New York Times opinion writers alike?

The answer, Ternikar says, lies in our stereotypes of brunch, rather than what brunch actually is. Anyone reading a food site in 2016 can picture the “white people having brunch” Julian Casablancas famously complained about a few years ago, complete with kale scrambles and excessively garnished Bloody Marys. Even though far more diverse groups of people now consume brunch, the meal still “symbolizes this kind of gentrification the same way as when a Starbucks or a Target come into your neighborhood… It’s not necessarily about hating brunch; it’s about hating those indicators of changes,” says Ternikar.

And since the 1920s, when brunch really was limited to rich people, brunch has migrated from private social settings into full public view (see: your Instagram feed, Saturdays and Sundays from 11 to 4). Combine that with social anxiety over gentrification in the major cities where brunch has always been most popular, and you have the perfect storm of stigmatization.