Attendees had been told to arrive early to allow time for coronavirus precautions such as hand-washing and temperature checks, AFP reports.
The fast-tracked ceremony comes after his predecessor Pierre Nkurunziza died suddenly last week.
Ndayishimiye is a former rebel leader, like Mr Nkurunziza.
He was backed by his predecessor and was declared the winner of May’s presidential election, which the opposition said was rigged.
Nkurunziza died aged 55 on 8 June after suffering a cardiac arrest, the government said. There are unconfirmed reports that he was suffering from Covid-19.
After 15 years in power, Nkurunziza was due to step down in August.
According to Burundi’s constitution, if the head of state dies in office then the president of the national assembly, currently Pascal Nyabenda, should succeed him.
But after a ruling by Burundi’s Constitutional Court, Ndayishimiye’s inauguration in the capital, Gitega, was brought forward.
The new president is taking over a country that is diplomatically isolated and on bad terms with donors.
His predecessor was accused of presiding over a government that carried out grave human rights violations and a crackdown on the opposition, journalists and activists. The country is also dealing with the coronavirus, which was downplayed by his predecessor.
Following the failed coup attempt in 2015, thousands of Burundians have fled the country and into refugee camps in neighbouring countries.
