Major General Godefroid Niyombareh said a salvation committee had been set up, but his level of support is unclear.
Thousands protesting against the president in the capital, Bujumbura, are now reported to be celebrating.
President Nkurunziza is currently in Tanzania. The presidency’s Twitter site said the coup attempt had failed.
Mr Nkurunziza has been meeting other East African leaders to discuss the crisis. His aide dismissed the coup claims as “a joke”.
In a series of Twitter postings, the Burundi presidency at first said the situation was under control and that there was no coup, then that the attempted coup had failed.
The BBC’s Maud Jullien in Bujumbura says there is gunfire as protesters escorted by the army head to the city centre.
Soldiers loyal to the president have been shooting to protect the offices of the state broadcaster. There are some reports that forces supporting the coup are now trying to enter the building and are meeting resistance.
Our correspondent says at least three people were killed in the capital’s Kabondo district but the circumstances of their deaths remain unclear.
One eyewitness in Bujumbura told BBC Southern Africa correspondent Karen Allen that thousands of people were moving towards the Place de la Independence, with police watching from the sidelines.
Some soldiers were giving the “thumbs up” sign to protesters, the eyewitness said.
In some outlying districts, protesters were being held up at checkpoints and prevented from moving into the city by security forces, but the mood was said to be largely celebratory.
Gen Niyombareh, in a statement read to reporters in a military base, said he did not recognise the leadership because the president’s bid for a third term violated the constitution.
In a radio broadcast on a private station, Gen Niyombareh said: “The masses have decided to take into their own hands the destiny of the nation to remedy this unconstitutional environment into which Burundi has been plunged.
“The masses vigorously and tenaciously reject President Nkurunziza’s third-term mandate… President Pierre Nkurunziza has been relieved of his duties. The government is overthrown.”
Gen Niyombareh’s “national salvation committee” comprises at least five other army and police generals.
He said in the radio broadcast that the mission of the body was the “restoration of national unity… and the resumption of the electoral process in a peaceful and fair environment”.
The unrest began on 26 April and has led to the deaths of more than 20 people.
Tens of thousands of Burundians have fled to neighbouring states in recent weeks.
President Nkurunziza has rejected calls to postpone next month’s election.
The 51-year-old former rebel leader argues that he is entitled to run for a third term because he was first appointed to the role by parliament in 2005.
The constitution states a president should govern only for two terms, but earlier this month a court upheld Mr Nkurunziza’s interpretation.