Mr Wickremesinghe has been appointed acting president by President Gotabaya Rajapaksa, who has fled the country.
But the decision to leave him in charge triggered further protests demanding that the prime minister must also go.
Sri Lanka has been suffering from its worst economic crisis in decades.
Many blame the Rajapaksa administration for the crisis and see Mr Wickremesinghe, who became prime minister in May, as part of the problem.
On Wednesday, for the second time in less than a week, protesters broke into a highly secure state building. This time it was the prime minister's office.
Echoing earlier scenes over the weekend of the occupied president's official residence, people in the prime minister's office lounged on plush sofas snapping photos, while others stood on chairs and desks waving the Sri Lankan flag.
In a television address, Mr Wickremesinghe called on protesters to leave his occupied office and other state buildings and co-operate with authorities.
"We can't tear up our constitution. We can't allow fascists to take over. We must end this fascist threat to democracy," he said.
But asked whether the prime minister's statement was an indication that the army might take control, a human rights lawyer in Colombo, Bhavani Fonseka, told the BBC's World at One that Sri Lanka "doesn't have a history where military has played an active role in politics or government - unlike some of the other countries in the neighbourhood".
"We have had a very robust democracy and it's been elected representatives in that role. But we are also in an unprecedented situation, so anything is possible," she added.
Outside the president's office, the BBC's Tessa Wong said armed soldiers stood by impassively watching the protesters celebrate inside the office.
And demonstrators ignored the prime minister's calls for the office to be emptied.
"Our goal is for Gota to go home. And Ranil and other cabinet members to go home," one protester at the prime minister's office, Nixon Chandranathan, told the BBC. "We need truthful and honest leaders to build up Sri Lanka now."
"We feel proud," Satish Bee, a businessman who came to explore the compound after it was stormed told AFP. "There's no proper governance in this country. It has never been good... The youngsters, they don't want to continue like this."
The continued unrest came as the news arrived that President Rajapaksa had fled to the Maldives.
The president went into hiding and pledged to resign after his official residence was stormed on Saturday.
The leader, who has enjoyed immunity from prosecution as president, is believed to have wanted to flee abroad before stepping down to avoid the possibility of arrest by the new administration.
He is, however, yet to submit a formal letter of resignation.
Sri Lanka: The basics
- Sri Lanka is an island nation off southern India: It won independence from British rule in 1948. Three ethnic groups - Sinhalese, Tamil and Muslim - make up 99% of the country's 22 million population.
- One family of brothers has dominated for years: Mahinda Rajapaksa became a hero among the majority Sinhalese in 2009 when his government defeated Tamil separatist rebels after years of bitter and bloody civil war. His brother Gotabaya, who was defence secretary at the time, is the current president but says he is standing down.
- Presidential powers: The president is the head of state, government and the military in Sri Lanka but does share a lot of executive responsibilities with the prime minister, who heads up the ruling party in parliament.
- Now an economic crisis has led to fury on the streets: Soaring inflation has meant some foods, medication and fuel are in short supply, there are rolling blackouts and ordinary people have taken to the streets in anger with many blaming the Rajapaksa family and their government for the situation.
The president's departure threatens a potential power vacuum in Sri Lanka, which needs a functioning government to help start digging it out of financial ruin.
Politicians from other parties have been talking about forming a new unity government but there is no sign they are near agreement yet. It's also not clear if the public would accept what they come up with.
Picking a new leader will also have its difficulties.
In a press statement on Wednesday Mr Wickremesinghe's team said he had asked the speaker of parliament to nominate a new prime minister "who is acceptable to both the government and opposition".
Earlier on Monday, the main opposition leader Sajith Premadasa told the BBC he would be tilting for the presidency. But he - like Mr Wickremesinghe - lacks public support. There is also deep public suspicion of politicians in general.
The protest movement which has brought Sri Lanka to the brink of change also does not have an obvious contender for the country's leadership.