Modules are smart contracts that have direct access to your Safe. To remove access from this module permanently, you can remove the module from your Safe. For this, first navigate to Settings > Advanced.
Initiate the module removal process by hovering over the Module you would like to remove and click the remove icon. You will then be asked to confirm the module removal. This will trigger a transaction that will have to be signer with your signer wallet and potential other signers of the Safe.