Make my own video conversion program? Viable?

How are video conversion programs made? Do they seriously understand every single file format of every single type they implement or is it using direct show filters or something? Basically i am interested in making my own but if i actually have to research every single file format to convert then i think ill pass O_O

So whats the go, direct show filters? libraries? are the free? Like say i want to convert mp4 to avi or mkv and what not; where shoul开发者_如何学JAVAd i start my learnings >:D

Start here, ffmpeg is open source and the defacto standard as a free codec library.