Okta recently released a new feature that allow delete the user from their cloud tenant. Below screenshot showing the okta delete button.
1. Login to the okta admin console and under the Directory Menu select People. From there select the user link to edit. Now you will see the Delete button.
2. Once you click the okta Delete button, a confirmation windows will be displayed Click Delete and now your user will be deleted.
The delete button will only appears if the user was previously deactivated.