As developers, we often work with sensitive information such as API keys, passwords, and other secrets. It’s essential to keep this information secure and out of the hands of unauthorized users.

However, accidents can happen, and sometimes we forgot to add a file containing secrets to the .gitignore and it is accidentally pushed to a Git repository. Even if the secrets are later removed from the repository, they can still be accessed through the commit history which is a potential breach of security.

It’s important to take steps to remove secrets from a Git repository as soon as possible after they are accidentally pushed to ensure your secret information remains safe without affecting other files and commits.

Remove secrets using the filter-branch command

If you’ve accidentally pushed secrets to a Git repository, don’t worry – there are several ways to remove them entirely from the repository. One way is to use the filter-branch command, which is a built-in Git command that allows you to rewrite the commit history of a repository.

Step 1: Use the filter-branch command

Use the filter-branch command to remove a file from the commit history of a repository:

git filter-branch --force --index-filter \
  "git rm --cached --ignore-unmatch <PATH-TO-YOUR-FILE-WITH-SENSITIVE-DATA>" \
  --prune-empty --tag-name-filter cat -- --all

Run this command to rewrite the commit history of the repository, removing any references to the given file. It’s important to note that this command can be destructive, so make sure you understand what it does before using it.

• git filter-branch: Used to rewrite the Git revision history.

• --force: Overrides some safety checks.

• --index-filter: Specifies a filter that modifies the index, or staging area, of each commit. In this case, the filter is "git rm --cached --ignore-unmatch <PATH-TO-YOUR-FILE-WITH-SENSITIVE-DATA>".

• "git rm --cached --ignore-unmatch <PATH-TO-YOUR-FILE-WITH-SENSITIVE-DATA>": Removes the specified file from the index, but not from the working tree. The --ignore-unmatch option prevents the command from failing if the file is not present in a particular commit.

• --prune-empty: Removes any empty commits that may be created as a result of removing the file.

• --tag-name-filter cat: Specifies how to handle tags. In this case, it specifies that tags should be rewritten to point to the same commit as before.

• -- --all: Specifies that all branches and tags should be rewritten.

In addition to the filter-branch command, there are other tools you can use to remove secrets from a Git repository, such as BFG Repo-Cleaner and git-filter-repo. You can read more about these tools in the GitHub documentation.

Step 2: Add secrets into .gitignore

After filtering your repository, don't forget to add the files which you've removed using filter-branch containing the secret into your .gitignore file to prevent it from accidentally push to the repository.

Step 3: Push your local changes to a remote repository

Once you are happy with the state of your repository, force-push your local changes to overwrite your remote repository, as well as all the branches you've pushed up:

git push origin --force

Step 4: Push against your Git tags

In order to remove the sensitive file from your tagged releases, you'll also need to force-push against your Git tags:

git push origin --force --tags

Conclusion

In conclusion, accidentally pushing secrets to a Git repository can be a security risk, but there are several ways to remove them entirely from the repository. Using the filter-branch command or other tools such as BFG Repo-Cleaner and git-filter-repo, you can rewrite the commit history of the repository and remove any references to the secrets.