Can't remove public folder mailbox after completing hybrid migration

I migrated public folders to Exchange Online. So far everything worked.
When doing the post-Tasks and cleaning up the Exchange 2013 server I ran into an issue.

I tried to delete the public folder mailbox and recieved the following error:

"No active public folder mailboxes were found. This happens when no public folder mailboxes are provisioned or they are provisioned in 'HoldForMigration' mode. If you're not currently performing a migration, create a public folder mailbox."

How can I delete the the public folder mailbox?

Some values from the get-organozationConfig (onPremise) which may be helpful

PublicFoldersLockedForMigration              : True
PublicFolderMigrationComplete                : True
PublicFolderMailboxesLockedForNewConnections : True
PublicFolderMailboxesMigrationComplete       : True
PublicFoldersEnabled                         : Remote
PublicComputersDetectionEnabled              : False
RootPublicFolderMailbox                      : 34a050b3-2f50-4bd3-9807-680fd2d0ded7

Thanks