NEW YORK — J. Crew Group’s women’s business is still struggling – and the retailer doesn’t expect things to get any better for the rest of the year.

And that’s dragging down the value of the brand itself. For the second time within six months, the retailer has written down the value of the Crew brand, this time by another $533.4 million in noncash impairment charges. The move follows a similar writedown, of $684 million, in the third quarter of last year.

But Millard “Mickey” Drexler, the brand’s chairman and chief executive officer, insisted the company is moving rapidly to fix things. Addressing what is ailing the women’s business, he pointed to sweaters and knits as the culprits on Thursday, just after the company reported a net loss of $462.4 million for the three months ended May 2. This compares with a loss of $30.1 million in the first quarter of last year.

Drexler’s message was that overall, the women’s business is not broken.

“Frankly, there are many things that we do well and the lion’s share of our women’s issue is isolated to knits and sweaters,” Drexler said during a conference call. “Aside from those two, we’re quite pleased with the balance of our women’s assortments.”

He called those two categories “outsized” and said that going forward, “we’ll be tightening our assortments and investments and mix and our sweaters will reflect more classics.” It is believed that sweaters and knits represent 25 to 30 percent of J. Crew’s business.

Drexler also acknowledged that “we may have put too much emphasis in marketing our Collection product,” J. Crew’s most expensive range, to the degree that it “overshadowed the rest of the assortment.” However, it is believed that Collection represents only a low-single-digit percent of the overall business and that the company did not increase inventory on Collection, only the marketing. That created the perception that the company had raised prices. Officials said prices were not increased.

Total revenues slipped 1.7 percent to $581.8 million from $592 million, while comparable company sales fell 8 percent on top of a 2 percent decrease in the year-ago quarter. Sales at J. Crew decreased 5.2 percent to $508.7 million, but at Madewell jumped 32.6 percent to $61.9 million.

“We are, needless to say, clearly disappointed with the results and the related impairment charge….It is our job, as always, to focus on what we can control in the business,” Drexler said. “We recognize there was work to be done and we’re on it. I’ve been through this before. It’s never fun. It’s hard work. But we’re all committed to getting it right.”

With the assortment, “anything and everything is up for debate,” except for J. Crew’s “iconic core classics,” which encompasses ballet flats, cashmere, the Regent blazers, washed shirts and the “toothpick” denim. Customers, officials indicate, are gravitating toward Crew’s more refined and put-together pieces to create outfits, including suiting pieces and woven shirts, and matching classic cardigans with pants, or cashmere T-shirts with pencil skirts, which are areas where the company will be investing in more.

Drexler said pants have Crew’s better fits and styles, and tops are less boxy and more tailored.

To generate more traffic, the company is “leveraging mobile capabilities, personal stylists and channel marketing,” Drexler said.

In addition, the company is “committed to growing men’s and Crewcuts.” A Crewcuts shop opened in Harrods in “a new partnership,” Drexler said.

Madewell, he said, “continues to be a strong performer” and is on track to add 20 stores this year. Also, 21 J. Crew factory outlets will open. Men’s wear has also been performing well.

“The women’s fixes, in my mind, are not that complicated, and that’s notwithstanding an overall trend in the women’s challenges of the world,” Drexler said. “But we just feel that we’re going to do this, do it right, fix the goods, and frankly, right now, we have adjusted all our investments for the third quarter and holiday.

“We’re making a lot of the right changes. You will see the iconic classics coming back in full strength.”

The company is also working to reduce expenses, which will involve organizational changes that were not specified.

“I was traveling yesterday just in preparation to get my head set right for this call. I shopped all the competition, and I didn’t exactly see that many stores where I’d run in and buy a lot of women’s clothes,” he said.

The company said declines at its J. Crew concept were driven primarily by the performance of women’s apparel and accessories, which it said it expects to continue “at least through fiscal 2015.”

Given the current and expected future operating results, the company determined that the carrying value of the J. Crew reporting unit exceeded its fair value and recorded an estimated noncash goodwill impairment charge of $341 million. In addition, the company also took a noncash impairment charge of $190 million to write down the intangible asset related to the J. Crew trade name.

There were no impairment charges required in connection with its Madewell concept.

Eighteen months ago, there was strong speculation that J. Crew’s owners, Leonard Green & Partners and TPG, would go for an initial public offering of the retailer some time in 2014, although Drexler always insisted that nothing was imminent. Given the retailer’s performance, and the tough overall retail scene, it’s unlikely an IPO is in the cards for some time.