What affect on our applications will changing the Heroku API Key have?

Our organization has a number of Rails applications (websites) deployed to Heroku. A former devleoper has left the organization, and as good practice we want to change the Heroku API key associated with our account to prevent any modifications to the apps via the Heroku CLI.

I know that the Heroku API Key is used for Heroku CLI access (it gets cached in ~/.heroku/credentials), but not certain what else it is used for. Specifically, do 3rd-party add-ons in the Heroku platform (e.g. New Relic, Hoptoad/Airbrake, Sendgrid, etc) use this, and therefore require reconfiguring if the API Key is changed? Heroku throws up a fairly generic (and non-informative) error message when you click the "regenerate" button to change it.

Because the term "API Key" is so generic, want to be clear that this is the single API Key associated with each Heroku account accessible via "My Account" link. Image (and warning message) below.

Asked Heroku Support. This is what I got back:

"you can safely change your API key at any time, as we don't give it to any add-on providers. That alert is meant to remind you that if you added your API key to any application or service (ie for auto scaling, manually provision workers, etc) it will stop working until you provide it a new key."

I requested that they update the interface/documentation to make this more clear.

Also remove him from being a collaborator on all your projects so he can't push to them via git.

Out of curiousity (i'd never seen reset key in the admin) I tried it. When I then tried to use the CLI against one of my apps I was asked to reauthenticate - but i can't now get back in - doh! The same username/password works via the site. I'll ping support and report back,

UPDATE: So it appears my problem is entirely due to the Heroku Accounts (https://github.com/ddollar/heroku-accounts) plugin that I'm using which stores a copy of the key in the ~/.heroku/accounts/ file. Support got me to remove the folder and it all works now - just something to be aware of if you reset your API key.