AWS provides method to delete FHIR data stores using SDK or CLI.
AWS documentation details the use of the DeleteFHIRDatastore function. This allows users to programmatically remove entire FHIR data stores within the AWS HealthLake environment.
AWS HealthLakeFHIR