Chad Rubin began selling vacuum cleaner parts on Amazon.com Inc. in 2008 and turned it into a multi-million-dollar business. But in recent years, Rubin has found it increasingly difficult to compete on the cluttered site, where he has been forced to buy advertising that cuts into his profit. Last year, he decided to shift some of his business to an up-and-coming website where it’s easier and cheaper to stand out: Walmart.com.

“Amazon sells 25,000 different kinds of toilet paper holders,” says Rubin, who is actually undercounting a product with more than 70,000 search results on the site. “How do you win and differentiate in that environment?” In the last year, thousands of merchants like Rubin have answered that question by signing on with Amazon’s biggest rival.

Wal-Mart fulfillment center in Bethlehem, Pennsylvania.

The notion of Amazon merchants selling through Wal-Mart Stores Inc. would have been risible just a few years ago. But much has changed. Wal-Mart’s struggling e-commerce operations have shown signs of life since the company acquired startup Jet.com last year and put its co-founder, Marc Lore, in charge. Meanwhile, among merchants, Amazon has earned a reputation as a demanding and cranky landlord, kicking sellers off its marketplace when customers complain and reaching deeper into their pockets with steeper fees and new advertising products that merchants say are now required to get a sale.

For merchants, Wal-Mart today is what Amazon was a decade ago: a vast frontier with room to grow. Amazon, meanwhile, is reaping the rewards of its popularity. It attracts 180 million unique web visitors each month, about double Wal-Mart’s traffic, and can charge merchants accordingly to get a piece of the action.

The overcrowding on Amazon.com is of particular concern to merchants generating almost all of their sales there. Last year, Robert Roque began putting inventory from his health and beauty business on Walmart.com and Jet. About half his sales are now on Amazon—down from 90 percent—due to the quick surge he’s seeing on Wal-Mart. “We had to diversify,” he says. “We could not sit back and depend on Amazon and have all our eggs in one basket.”

Wal-Mart has a long and contentious history…