Projects

From Mumble Wiki
Revision as of 01:45, 20 February 2010 by Slicer (talk | contribs)
Jump to: navigation, search

These are medium-sized Mumble projects that we've though about, and which would be really beneficial to have, but which we so far haven't even scratched the surface of. We'd really like help with these, but be aware that they non-trivial to implement.

Shared surface overlay

Skills: Low level Windows graphics, Windows Kernel.

Mumble's overlay currently renders the overlay in the Mumble process with the CPU in a shared memory segment, then signals the in-game injected overlay to update its full screen texture. This is resource-intensive. On OSX, you can share surfaces, and on Linux you can share contexts, which allows you to render-to-texture the overlay, and then have the in-game overlay use that rendered texture without any memory transfer. On Windows, in Vista/Win7, you have DXGI Shared Surfaces.

The focus of this project is twofold. First, get DXGI Shared Surfaces working. Ideally we'd like to render with OpenGL in the Mumble process, and have the resulting surface be used both in OpenGL and Direct3D applications.

Second, have a peek at moving the overlay into the driver layer, either as part of the userspace graphics driver or in the kernel. Right now the overlay is rendered by patching the running processes, which works most of the time, but also causes problems with some anti-cheat utilities and does not deal with faulty applications very well.

HRTF

Skills: Signal Processing, Qt GUI

A lot of work has been done on HRTF research, and we'd like to use it in Mumble. Positional audio already works well, but nowhere near as well as it could with proper HRTF applied. The focus here will be on headphone based HRTF, as there is no way to guarnatee speaker placement for users. This project consists of three separate tasks.

First, find a way to apply HRTFs to the current audio engine with minimal latency increase.

Second, find a way to smoothly interpolate the HRTFs as audio sources and listeners move around.

Third, create a simple wizard to guide the user to choosing the correct HRTF for his/her ears.

Automatic network diagnostics

Skills: Low level IP, Qt Networking, Qt GUI

Mumble uses QoS by marking IP packets with Type of Service, which (in theory) gives it higher presedence than other traffic. Quite a few ISPs these days offer VoIP telephony to their customers, meaning they also give them home routers that support QoS. The problem is that quite a lot of routers do not support QoS, and some have broken support for it, either crashing when receiving packets with ToS bits set, dropping the packets or mangling the header without recomputing the checksum. Some servers being hosted behind buggy firewalls or incomplete VM implementations also suffer from this problem.

We'd like to add some automated testing to both the server and client. By trying to send and receive packets with and without QoS from a "known good" server, the client and server can individually find out what their QoS support is like. Both the method of querying and the "known good" server need to be implemented, and has to scale to several hundred thousand simultaneous users.

Jitter Buffer for Home Networks

Skills: Math, algorithm design, Patience

In an ideal world, every packet sent on a network reaches it target, and does so in deterministic time. In reality, streams of packets are frequently delayed, clustered together, have packets dropped or the order of packets reversed and so on. The classic solution to this, used by streamed non-interactive media, is to buffer data for long enough that any late packets arrive. For a low-latency solution, you need to adaptively keep your buffer as short as possible (we prefer less than 20 ms) yet maintain a low probability that packets arrive late or not at all. Mumble currently uses the SpeexDSP jitter buffer, but when switching to 10 ms packets, it isn't quite able to adapt quickly enough.

This project should focus on designing a jitter algorithm that can quickly adapt to changing conditions, and that can also support discontinous transmission of audio. It should also support save/load of state, so that the next session can use the previous session as "initial guesses".