I've felt it. You've felt it.
More often than not, in my case at least, this feeling of guilt morphs into feelings of shame and self-loathing. I've been lucky in that I have been able to hold these feelings in check and not let them spiral into depression. But others are not so lucky. Guilt triggers a strong rejection and devaluation of the self.
This slide from guilt into shame and self-loathing is natural. But more and more I've been trying to redirect my guilt into another emotion.
When I do something wrong and feel guilty I'm trying to spend less time on how I'm such a screw up than on how, in light of my own failures, I should be more forgiving of the failures of others. If I'm such a screw up I should extend sympathy, empathy and compassion to others when they make mistakes. How can I judge others when I make similar or even worse mistakes?
My guilt is a trigger to extend grace to others.
This movement from guilt to compassion for the failings of others isn't as intuitive as moving into self-censor, but the connection isn't too far-fetched and becomes easier when practiced. But you do have to practice this new train of thought. You have to practice seeing your guilt as a sign of your membership in the "democracy of sinners," to use the description of G.K. Chesterton.
Guilt doesn't instinctively trigger compassion, but it can if you work on it. In light of your own failures you shift away from shame to say, If I'm making mistakes should I not extend grace for your mistakes as well?