Kabanshi becomes the first casualty with many more expected to go in a scandal that has forced some donors to suspend aid to Zambia.
The UK has frozen aid funding to Zambia over the $4.3m (£3.3m) under the Social Cash Transfer Scheme.
The move follows allegations of corruption within President Edgar Lungu’s administration.
The UK takes a “zero-tolerance approach to fraud and corruption”, the UK’s Department for International Development (DfID) said in a statement.
Ireland, Finland and Sweden have also suspended aid.
President Lungu announced that he had given Secretary to the Cabinet one week to give him concrete answers but within 24 hours the axe has fallen on Kabanshi.