“The best way I can help now is if I step aside,” he said, while continuing to deny the claims. The resignation will take effect in 14 days.
Lt Governor Kathy Hochul will become the first woman to lead New York state.
Mr Cuomo had been facing pressure to resign from fellow Democrats, including President Joe Biden.
Just a year ago he was basking in adulation as millions of Americans tuned in daily to his no-nonsense televised briefings on the coronavirus pandemic.
Mr Cuomo is the third New York governor in a row to leave office under a cloud of scandal.
Why did Cuomo resign?
The independent investigation by the New York Attorney General’s office found that Mr Cuomo, 63, sexually harassed 11 women, including state employees.
Women alleged that he made sexual comments, inappropriately touched or groped them, and kissed them without consent.
The report led many prominent Democrats to turn against Mr Cuomo, including Speaker of the House of Representatives Nancy Pelosi, Senate leader Chuck Schumer and New York’s two US senators.
His fellow New York Democrats had begun plans to impeach him. He is still facing criminal investigations relating to the harassment claims.
As he announced his resignation on Tuesday, Mr Cuomo continued to deny allegations of sexual harassment but said he wanted to “deeply, deeply” apologise to any women who may have been offended by his actions.
“In my mind I’ve never crossed the line with anyone. but I didn’t realise the extent to which the line has been redrawn,” he said.
He added that his instinct was “to fight through this controversy, because I believe it is politically motivated”.
But he said he was resigning because the “current trajectory” of the scandal would generate months of distractions and “cost taxpayers millions of dollars”.
Mr Cuomo said the allegations had harmed his relationship with his daughters.
“I have sat on the couch with them, hearing the ugly accusations for weeks. I have seen the look in their eyes, and the expression on their faces. It hurt,” he said.
He said he told them he “never did and would never intentionally disrespect a woman”.
Following Mr Cuomo’s announcement, New York City Mayor Bill de Blasio, a long-time critic of the governor, said “it was past time” for him to resign.
“I respect the governor’s decision,” Mr Biden told reporters at the White House. “I respect the decision he made.”
White House Press Secretary Jen Psaki earlier said the president had not spoken to his old friend since the report was published and did not have advance notice of his resignation.
Mr Cuomo’s brother, CNN primetime anchor Chris Cuomo, has also faced calls to resign after it emerged he had been advising the governor ahead of his Tuesday announcement.
The younger Cuomo in May had apologised to viewers for strategizing with his brother about how to handle the initial misconduct claims. He recused himself from covering the matter on air, and has not commented on the official report. He has since gone on holiday.
A separate federal inquiry is also looking into whether the governor’s office deliberately undercounted the number of Covid deaths in New York care homes.
Who is Cuomo’s replacement?
Kathy Hochul will become the first female governor of the fourth most populous US state.
The 62-year-old is a centrist Democrat from the Buffalo area of New York state. She joined the governor’s team in 2014, and has held the largely ceremonial role of lieutenant governor.
She was not part of Mr Cuomo’s inner circle, but he described her on Tuesday as “smart and competent”.
Ms Hochul had joined the chorus of politicians denouncing the New York governor last week, and said his resignation on Tuesday was the “right thing to do”.
“As someone who has served at all levels of government and is next in the line of succession, I am prepared to lead as New York State’s 57th Governor,” she said in a statement.
It’s unclear whether she will seek a full term in the 2022 election.