WASHINGTON—Specialty stores and department stores cut jobs as the holiday shopping season began in November, the Labor Department’s monthly employment report showed Friday.

Apparel and specialty stores cut a seasonally adjusted 17,600 jobs from payrolls last month to employ 1.34 million, while department stores shed 5,100 jobs to employ 1.3 million. General merchandisers, a category that includes department stores and discounters, boosted payroll slightly by 800 to employ 3.2 million.

In the overall economy, employers added 178,000 jobs and the unemployment rate fell to 4.6 percent from 4.9 percent in October.

Apparel employers added 400 seasonally adjusted jobs to employ 131,200, while employment at mills making apparel fabric and yarn fell 200 to 112,000 last month. Textile product mills added 100 jobs to employ 114,600.

Scott Hoyt, senior director of consumer economics at Moody’s Analytics, said the specialty store employment decline was surprising, noting that many retailers announced their seasonal hiring plans would be comparable with last year.

He said additional hiring for many retailers is in their e-commerce divisions, which is measured in the retail warehouse and distribution center segment of the government’s report.

Hoyt noted that employment at department stores is down 4,600 year over year, while employment at apparel and accessories stores is down 29,600, Hoyt said. Department stores had been doing “relatively well” in terms of employment, posting gains for the past five consecutive months until November, he said.

The outlook for employment into the first quarter is bright for the overall job market, but “less bright” for retail, he noted, adding that “they’ve got pressure that is continuing to rise in the online share of sales in addition to pricing pressure.”

Jack Kleinhenz, chief economist at the National Retail Federation, said, “The market continues on its variable path toward full employment and today’s report points to a rate hike by the Federal Reserve. We continue to expect a positive holiday season for retailers and it appears that retailers should have the employees they need in place to meet the expected solid increase in holiday sales.”

Nariman Behravesh, chief economist at IHS Markit, said November was a “solid month for jobs growth in the U.S. and consistent with an economy with underlying growth rate in the 2 to 2.5 percent range.”

“With the recent improvements in productivity growth, this solid but modest growth range implies that employment gains are set to slow further over the next couple of years,” Behravesh said.

On the downside, the participation in the labor force edged down slightly last month, which contributed to the decline in the unemployment rate. Wage growth also fell 0.1 percent, after posting gains of 0.4 percent in October and 0.3 percent in September.

“While the data on both the labor force and wage growth are disappointing, there is a strong seasonal factor in both and both have been quite volatile in the past six months,” Behravesh said. “It is important to keep in mind that, despite the declines the past two months, the U.S. labor force is 2 million people larger than it was a year ago.”

In addition wage growth was 2.5 percent higher year over year in November, above the average of 2.3 percent in 2015, he said.

“There is nothing in today’s employment report that is likely to deter the Fed from raising interest rates later this month,” he added.