UNHCR, the United Nations refugee agency, said the fire was linked in part to poor living conditions and a prevailing sense of uncertainty among many. A police official in Athens said two riot police squads were being deployed to the island.
"Last night's fires... symbolize the shortcomings of the European response to the refugee crisis," said Panos Navrozidis, the International Rescue Committee's Greece director.
It was not clear what caused the blaze, but Greek media said clashes had erupted following a rumor that hundreds of people would be deported.
A police official on Lesbos said nearly 60 percent of the Moria camp was destroyed in the fire.
Tensions have boiled over at overcrowded camps on Greece's islands as the slow processing of asylum requests adds to frustration over difficult living conditions there.
On Lesbos, tents and containers used for housing and social provision had burned down. Nearly everyone,including unaccompanied children, had evacuated the site, the police official said on condition of anonymity. Many later returned.
Roland Schoenbauer, a spokesman in Greece for the UNHCR, said people were "sick of waiting" in the camps.
"They don't know when their asylum claims will be processed, some people feel they don't have enough information," he said.