unbind C-m from RET

In my emacs config file I have created a minor mode so that I can re-bind keys without having to rebind them separately for each major mode.

In doing so, I have re-mapped C-m to kill-ring-save. However, in default emacs C-m is the same as RET. Therefore when I have the following in my .emacs :

 (define-key my-minor-mode-map (kbd "C-m") 'kill-ring-save)

When I press the return key, kill-ring-save is executed

How should I go about fixing my config file so that I don't run into these problems?

I am also open to taking a different approach to creating a key binding that works in all major modes.

Edit: I am running in graphical mode

This won't work in a non-graphical mode emacs. When run in a terminal, return and C-m are not distinguishable.

If you're not running terminal mode emacs, just rebind <return> and C-m separately.

For example:

(cond (window-system  ; ensure not running in a terminal
       (local-set-key (kbd "<return>") 'newline)
       (local-set-key (kbd "C-m") 'kill-ring-save)))