How to change the html code inserted on "Add Media"?

There's an "Add Media" button just above TinyMCE editor when you are editing the page content. That link inserts a link to that file into page content.

How do I change the html code that is inserted based on file type/mime type?

You need to use the filter media_send_to_editor.

add_filter('media_send_to_editor', 'so_6884350_send_to_editor', 10, 3 );

function so_6884350_send_to_editor( $html, $send_id, $attachment )
{
    /* Manipulate $html result */
    return $html;
}

$html is the code that's going to be inserted in your post. Something like <a href='http://example.com/wp-content/uploads/2012/11/README.rtf'>README</a>

$send_id is the ID of the attachment being inserted. Use it to get information about the attachment, i.e., get_post_mime_type($send_id);.

$attachment is an array with the following structure:

array(
    ['menu_order'] =>
    ['post_title'] => 'README'
    ['post_excerpt'] =>
    ['post_content'] =>
    ['url'] => 'http://example.com/wp-content/uploads/2012/11/README.rtf'
)