Difference between revisions of "Projects"

From Mumble Wiki
Jump to: navigation, search
Line 5: Line 5:
 
= Shared surface overlay =
 
= Shared surface overlay =
  
Skills: Low level Windows graphics, Windows Kernel.
+
Skills: C++, Low level Windows graphics, Windows Kernel.
  
 
Mumble's overlay currently renders the overlay in the Mumble process with the CPU in a shared memory segment, then signals the in-game injected overlay to update its full screen texture. This is resource-intensive. On OSX, you can share surfaces, and on Linux you can share contexts, which allows you to render-to-texture the overlay, and then have the in-game overlay use that rendered texture without any memory transfer. On Windows, in Vista/Win7, you have DXGI Shared Surfaces.
 
Mumble's overlay currently renders the overlay in the Mumble process with the CPU in a shared memory segment, then signals the in-game injected overlay to update its full screen texture. This is resource-intensive. On OSX, you can share surfaces, and on Linux you can share contexts, which allows you to render-to-texture the overlay, and then have the in-game overlay use that rendered texture without any memory transfer. On Windows, in Vista/Win7, you have DXGI Shared Surfaces.
Line 15: Line 15:
 
= HRTF =
 
= HRTF =
  
Skills: Signal Processing, Qt GUI
+
Skills: C++, Signal Processing, some Qt GUI
  
 
A lot of work has been done on HRTF research, and we'd like to use it in Mumble. Positional audio already works well, but nowhere near as well as it could with proper HRTF applied. The focus here will be on headphone based HRTF, as there is no way to guarnatee speaker placement for users. This project consists of three separate tasks.
 
A lot of work has been done on HRTF research, and we'd like to use it in Mumble. Positional audio already works well, but nowhere near as well as it could with proper HRTF applied. The focus here will be on headphone based HRTF, as there is no way to guarnatee speaker placement for users. This project consists of three separate tasks.
Line 27: Line 27:
 
= Automatic network diagnostics =
 
= Automatic network diagnostics =
  
Skills: Low level IP, Qt Networking, Qt GUI
+
Skills: C++, Low level IP, Qt Networking, some Qt GUI
  
 
Mumble uses QoS by marking IP packets with Type of Service, which (in theory) gives it higher presedence than other traffic. Quite a few ISPs these days offer VoIP telephony to their customers, meaning they also give them home routers that support QoS. The problem is that quite a lot of routers do not support QoS, and some have broken support for it, either crashing when receiving packets with ToS bits set, dropping the packets or mangling the header without recomputing the checksum. Some servers being hosted behind buggy firewalls or incomplete VM implementations also suffer from this problem.
 
Mumble uses QoS by marking IP packets with Type of Service, which (in theory) gives it higher presedence than other traffic. Quite a few ISPs these days offer VoIP telephony to their customers, meaning they also give them home routers that support QoS. The problem is that quite a lot of routers do not support QoS, and some have broken support for it, either crashing when receiving packets with ToS bits set, dropping the packets or mangling the header without recomputing the checksum. Some servers being hosted behind buggy firewalls or incomplete VM implementations also suffer from this problem.
Line 35: Line 35:
 
= Jitter Buffer for Home Networks =
 
= Jitter Buffer for Home Networks =
  
Skills: Math, algorithm design, Patience
+
Skills: C/C++, Math and Algorithm design, Patience
  
 
In an ideal world, every packet sent on a network reaches it target, and does so in deterministic time. In reality, streams of packets are frequently delayed, clustered together, have packets dropped or the order of packets reversed and so on. The classic solution to this, used by streamed non-interactive media, is to buffer data for long enough that any late packets arrive. For a low-latency solution, you need to adaptively keep your buffer as short as possible (we prefer less than 20 ms) yet maintain a low probability that packets arrive late or not at all. Mumble currently uses the SpeexDSP jitter buffer, but when switching to 10 ms packets, it isn't quite able to adapt quickly enough.
 
In an ideal world, every packet sent on a network reaches it target, and does so in deterministic time. In reality, streams of packets are frequently delayed, clustered together, have packets dropped or the order of packets reversed and so on. The classic solution to this, used by streamed non-interactive media, is to buffer data for long enough that any late packets arrive. For a low-latency solution, you need to adaptively keep your buffer as short as possible (we prefer less than 20 ms) yet maintain a low probability that packets arrive late or not at all. Mumble currently uses the SpeexDSP jitter buffer, but when switching to 10 ms packets, it isn't quite able to adapt quickly enough.
  
 
This project should focus on designing a jitter algorithm that can quickly adapt to changing conditions, and that can also support discontinous transmission of audio. It should also support save/load of state, so that the next session can use the previous session as "initial guesses".
 
This project should focus on designing a jitter algorithm that can quickly adapt to changing conditions, and that can also support discontinous transmission of audio. It should also support save/load of state, so that the next session can use the previous session as "initial guesses".
 +
 +
= Waiting Games =
 +
 +
Skills: C++, Qt GraphicsView
 +
 +
Every once in a while, your game server goes down for a few minutes (or hours), or you're waiting for that last person to show up for your raid, or you're just waiting for an unusually slow map load. In short, you're bored. For these occations, it would be beneficial to have a waiting game that can be played in Mumble.
 +
 +
This project would include implementing one or more small games to be played either in the Mumble overlay or in a dedicated window (base it on QGraphicsView and Mumble will handle the rest), and should include both client and server side modifications necesarry. It is recommended to start with an existing game concept (Tetris, XBlast); anything that can potentially be done by several people and have a bit of a competetive edge to it.
 +
 +
 +
 +
 +
Skills: C++

Revision as of 16:04, 27 February 2010

These are medium-sized Mumble projects that we've though about, and which would be really beneficial to have, but which we so far haven't even scratched the surface of. We'd really like help with these, but be aware that they non-trivial to implement.

Shared surface overlay

Skills: C++, Low level Windows graphics, Windows Kernel.

Mumble's overlay currently renders the overlay in the Mumble process with the CPU in a shared memory segment, then signals the in-game injected overlay to update its full screen texture. This is resource-intensive. On OSX, you can share surfaces, and on Linux you can share contexts, which allows you to render-to-texture the overlay, and then have the in-game overlay use that rendered texture without any memory transfer. On Windows, in Vista/Win7, you have DXGI Shared Surfaces.

The focus of this project is twofold. First, get DXGI Shared Surfaces working. Ideally we'd like to render with OpenGL in the Mumble process, and have the resulting surface be used both in OpenGL and Direct3D applications.

Second, have a peek at moving the overlay into the driver layer, either as part of the userspace graphics driver or in the kernel. Right now the overlay is rendered by patching the running processes, which works most of the time, but also causes problems with some anti-cheat utilities and does not deal with faulty applications very well.

HRTF

Skills: C++, Signal Processing, some Qt GUI

A lot of work has been done on HRTF research, and we'd like to use it in Mumble. Positional audio already works well, but nowhere near as well as it could with proper HRTF applied. The focus here will be on headphone based HRTF, as there is no way to guarnatee speaker placement for users. This project consists of three separate tasks.

First, find a way to apply HRTFs to the current audio engine with minimal latency increase.

Second, find a way to smoothly interpolate the HRTFs as audio sources and listeners move around.

Third, create a simple wizard to guide the user to choosing the correct HRTF for his/her ears.

Automatic network diagnostics

Skills: C++, Low level IP, Qt Networking, some Qt GUI

Mumble uses QoS by marking IP packets with Type of Service, which (in theory) gives it higher presedence than other traffic. Quite a few ISPs these days offer VoIP telephony to their customers, meaning they also give them home routers that support QoS. The problem is that quite a lot of routers do not support QoS, and some have broken support for it, either crashing when receiving packets with ToS bits set, dropping the packets or mangling the header without recomputing the checksum. Some servers being hosted behind buggy firewalls or incomplete VM implementations also suffer from this problem.

We'd like to add some automated testing to both the server and client. By trying to send and receive packets with and without QoS from a "known good" server, the client and server can individually find out what their QoS support is like. Both the method of querying and the "known good" server need to be implemented, and has to scale to several hundred thousand simultaneous users.

Jitter Buffer for Home Networks

Skills: C/C++, Math and Algorithm design, Patience

In an ideal world, every packet sent on a network reaches it target, and does so in deterministic time. In reality, streams of packets are frequently delayed, clustered together, have packets dropped or the order of packets reversed and so on. The classic solution to this, used by streamed non-interactive media, is to buffer data for long enough that any late packets arrive. For a low-latency solution, you need to adaptively keep your buffer as short as possible (we prefer less than 20 ms) yet maintain a low probability that packets arrive late or not at all. Mumble currently uses the SpeexDSP jitter buffer, but when switching to 10 ms packets, it isn't quite able to adapt quickly enough.

This project should focus on designing a jitter algorithm that can quickly adapt to changing conditions, and that can also support discontinous transmission of audio. It should also support save/load of state, so that the next session can use the previous session as "initial guesses".

Waiting Games

Skills: C++, Qt GraphicsView

Every once in a while, your game server goes down for a few minutes (or hours), or you're waiting for that last person to show up for your raid, or you're just waiting for an unusually slow map load. In short, you're bored. For these occations, it would be beneficial to have a waiting game that can be played in Mumble.

This project would include implementing one or more small games to be played either in the Mumble overlay or in a dedicated window (base it on QGraphicsView and Mumble will handle the rest), and should include both client and server side modifications necesarry. It is recommended to start with an existing game concept (Tetris, XBlast); anything that can potentially be done by several people and have a bit of a competetive edge to it.



Skills: C++