VOIP functionality (real time voice streaming across max of 5 users) over Silverlight 4.0?

As SL 4.0 has got video and Mic support...

How feasible it is to provide VOIP functionality (real time voice streaming across max of 5 users) over Silverlight 4.0, for a web based application?

What all are the related challenges?

A part of the solution has recently been provided in the form of CSpeex, a port of an old Java port of an old version of Speex (c. 2003). I've played around with it, and it definitely works (e.g., 10x or better compression). However, you'll still need to stream those packets up to a server somewhere, and there don't appear to be any Silverlight implementations of RTP/RTSP/SIP or RTMP, so at this point you'd have to roll your own network stream container support. Also, unless you want to make all your users wear headphones, you're going to want acoustic echo cancellation, and that's another ballgame altogether. There are some very good open source implementations (e.g., speexdsp), but they're all straight C, and hard to port. So you're on your own there as well. There are likely going to be some open source answers to all this over the next few months/years, but the pieces aren't in place yet.

A lot depends on how the new SL4 mic/video support is implemented. You would need to include code that is capable of

1. Encoding the voice into a byte stream
2. Send the byte stream to the other party/parties
3. Receiving/decoding the byte stream into an audio signal

I believe SL4 has some COM support planned, so if there are any open source (or paid) unmanaged C++ VOIP or audio-encoding libraries out there, you might be able to hook into them.

There are multiple libraries that one can use. For example: conaito, ABTO llc, IndependentSoft SIP, that will work with Silverlight 4.0. Conaito and ABTO use java scripting to link into their web client, whilst IndependentSoft have a managed assembly.

Conaito and ABTO are good but their libraries are a bit outdated due to them being ocx's and using COM dll's which may break your deployment.

IndedenpentSoft is easier to deploy because it's managed code, but requires quite a lot more work to implement as it basically provides a transport layer and thats it, you have to specify the codec's and trnasport protocol.

The main issues you need to consider are your firewall requirements. My suggestion would be to look for a managed assembly thats uses STUN, TURN and especially ICE.