If a website should ever refuse to allow you to close your account with them (or just make it difficult to do so), you can still make sure that any info they collected when you registered can’t hurt you if hackers should ever breach that website. Instead of closing the account, just update your account with incorrect info. For instance, if the account asked for birth date and birthplace, or mother’s maiden name, or other potentially sensitive info, just change those items so that a hacker breach won’t net them any useful info. If it's a site you're paying for automatically with a credit card, nose around and see if you can change payment terms from "automatic" to "manual."
This site offers tips for closing accounts with 20+ online services: