See more of the story

Read an old novel, or watch old movies, and you might be struck (as I am) by the dearth of restaurant meals. Only rich characters dine out frequently. The everyman might get a quick breakfast at a drugstore or a coffee shop, but a real sit-down dinner? For regular people, that was for special occasions, not a regular occurrence. Most of the food that people ate — and the overwhelming majority of dollars they spent on food — was home preparation, not restaurant service.

Times certainly have changed. The average American spends slightly more on bars and restaurants than they do on home cooking. Since restaurant meals cost more, this still means that most people get most of their food from the refrigerator. Nonetheless, the average American eats out about five times a week. Compared with any time in history, this is stunning. And it leads to a stunning question: Why aren't restaurants doing better financially?

A recent article in QSR magazine reported: "[R]estaurants have now posted four consecutive quarters of declining year-over-year sales. The last time the industry experienced a year with all negative quarters was 2009, when the economy was suffering the effects of the great recession." Traffic has declined and check growth is nearly stagnant, which adds up to bad news for the restaurant industry.

This may seem hard to believe for folks in cities like Washington, where we seem to be in a golden age of dining out. The renaissance in my city has been particularly noticeable, because when I moved to D.C. 10 years ago, the city was oversupplied with college bar food and steakhouses where lobbyists could take congressmen, and undersupplied with every other kind of restaurant. But it's been happening in other cities too — virtually everywhere you go in the U.S., the variety and quality of restaurants has only increased.

Which could partly explain the industry's financial problem. Sure, demand for restaurant meals has risen. But when demand rises, often supply also rises as well.

In recent decades, food and beverages have become the sort of thing that upper-middle-class folks aspire to make as well as to eat. For most of its history, food preparation was unglamorous: something done by millions of uninspired housewives every night. Fifty years ago, its professional counterpart was a fundamentally working-class occupation because restaurant work, however well done, is hard physical labor.

But as mass-produced food options became better and cheaper, cooking became optional — and then it became a craft, rather than a tedious necessity. Thanks to immigration and labor-saving devices, it became one of the few things that middle-class people still do with their hands. It acquired artisanal glamour and become the sort of thing that upper-middle-class people might aspire to do, mostly in service to the increasingly varied and sophisticated tastes of other upper-middle-class people.

When an industry becomes glamorous, we expect that more people will pour into that industry than it can actually support, putting downward pressure on earnings. And yet that's not quite what we see. As Derek Thompson of the Atlantic points out, 2016 brought a net decline in the number of independent restaurants in major cities like New York, San Francisco and DC; what growth there was came from fast-food chains. So while supply may be part of the story, to understand what's happening to restaurants, we also need to look at demand.

One place to look is at your local shopping mall. Malls have been, for decades, incredible drivers of restaurant sales; they have a captive audience, hungry from walking around. Now malls seem to be dying. Most of what they sell can now be conveniently ordered online, and much of the rest of the mall can't survive without the foot traffic driven by retail stores. It doesn't make sense to maintain a gigantic indoor space so people can comfortably walk between the movie theater, the Apple store and the Auntie Anne's pretzels.

Meanwhile, the alternatives to eating out have gotten better and better. No, you can't get a meal from Le Cirque in your supermarket's freezer case, but you can get an increasing variety of frozen meals that are a decent substitute for a stop at TGI Friday's. Or you can amble over to the sushi counter, the salad bar or the cook-at-home pizza counter and get something that is tastier, better for you, and cheaper than you could have gotten by stopping at some mediocre chain. This supermarket revolution predates the great recession, but the financial squeeze only accelerated it, as strapped middle-class consumers sought alternatives to eating out. They may have found, once the recession ended, that they were quite satisfied with their workarounds.

That's before we even take into account the increasing convenience of online takeout ordering, and services like Blue Apron that enable busy professionals to enjoy a high-quality home-cooked meal without the time-consuming shopping and prep. So even as consumers become more interested in the quality and convenience that a restaurant provides, restaurants face more competition to actually give it to them.

And in the major cities where we find the biggest concentration of folks with disposable incomes and food interest, restaurants face another problem: rising costs. New York restaurateurs have complained for years about nosebleed rent hikes, a complaint that is echoed wherever gentrification threatens to alter the underlying economics of beloved eateries. More places are also passing major increases in the minimum wage, a heavy burden for restaurants, because despite the glamour, the back of the house is still fundamentally a working-class place. Rising costs and increased competition are a recipe for disaster in a business where even successful places generally enjoy only decent profit margins.

That's not to say that the restaurant industry is doomed. There's still plenty of demand for things that are hard to make well at home, from fried chicken to snail porridge. But it does suggest that the industry may be due for a shakeout. And that when we do dine out, in the future, it may once again be a special occasion.

Megan McArdle is a Bloomberg View columnist. She wrote for the Daily Beast, Newsweek, the Atlantic and the Economist and founded the blog Asymmetrical Information. She is the author of "The Up Side of Down: Why Failing Well Is the Key to Success."