Visualizing music

Not long ago I came across this article: http://www.perl.com/pub/2011/01/visualizing-music-with-sdl-and-perl.html

Now, I'm no stranger to perl, I have used it before, mainly for socket programming and http requests, I knew it had various bindings for various libraries, but still this solution seemed elegant.

This gave me the idea to recreate this old xmms spectrum analyzer OpenGL plugin with Ogre3d.

I had two weeks vacation between semester 2 and 3 so I had time to study a bit of OpenGL to help me draw things in my (optional) Computational Geometry class.

So I read about the fast fourier transform and how to extract the frequencies out of the stream. I decided to use FFTW because it's fast and I came up with a solution to normalize the values (not sure if it's the best, but it works, if you have tips on how to do this more correctly I'm very eager to hear them).

I came up with this code:

https://github.com/balazsbela/OgreVisualizer



It works, it plays ogg files and visualizes them in 3d pretty well. The framerate is pretty good too.

There is one problem though, it crashes after a while,seemingly randomly, sometimes it
plays through the whole song without problems, sometimes it crashes at the half of the song.

I posted questions on Ogre3d forum, StackExchange GameDev section, Stackoverflow, but I have yet to read a good response.

Here's the main class: https://github.com/balazsbela/OgreVisualizer/blob/master/src/VisualizerApplication.cpp

Here's the backtrace:

balazsbela@darknet:~/workspace/OgreVisualizer/Release$ gdb OgreVisualizer core
GNU gdb (GDB) 7.2-debian
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i486-linux-gnu".
For bug reporting instructions, please see:
...
Reading symbols from /home/balazsbela/workspace/OgreVisualizer/Release/OgreVisualizer...done.
[New Thread 17705]
[New Thread 17702]
[New Thread 17703]
[New Thread 17700]
Reading symbols from /usr/lib/libv4l/v4l1compat.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libv4l/v4l1compat.so
Reading symbols from /usr/local/lib/libOgreMain.so.1.7.1...done.
Loaded symbols for /usr/local/lib/libOgreMain.so.1.7.1
Reading symbols from /usr/lib/libfftw3.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libfftw3.so.3
Reading symbols from /usr/lib/libSDL_sound-1.0.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libSDL_sound-1.0.so.1
Reading symbols from /usr/lib/libSDL-1.2.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libSDL-1.2.so.0
Reading symbols from /usr/lib/libSDL_mixer-1.2.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libSDL_mixer-1.2.so.0
Reading symbols from /usr/lib/libOIS-1.2.0.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libOIS-1.2.0.so
Reading symbols from /usr/lib/libstdc++.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libstdc++.so.6
Reading symbols from /lib/i686/cmov/libm.so.6...Reading symbols from /usr/lib/debug/lib/i686/cmov/libm-2.11.2.so...done.
done.
Loaded symbols for /lib/i686/cmov/libm.so.6
Reading symbols from /lib/libgcc_s.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libgcc_s.so.1
Reading symbols from /lib/i686/cmov/libc.so.6...Reading symbols from /usr/lib/debug/lib/i686/cmov/libc-2.11.2.so...done.
done.
Loaded symbols for /lib/i686/cmov/libc.so.6
Reading symbols from /lib/i686/cmov/libpthread.so.0...Reading symbols from /usr/lib/debug/lib/i686/cmov/libpthread-2.11.2.so...done.
done.
Loaded symbols for /lib/i686/cmov/libpthread.so.0
Reading symbols from /usr/local/lib/libv4l1.so.0...done.
Loaded symbols for /usr/local/lib/libv4l1.so.0
Reading symbols from /usr/lib/libfreetype.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libfreetype.so.6
Reading symbols from /usr/lib/libSM.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libSM.so.6
Reading symbols from /usr/lib/libICE.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libICE.so.6
Reading symbols from /usr/lib/libX11.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libX11.so.6
Reading symbols from /usr/lib/libXext.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libXext.so.6
Reading symbols from /usr/lib/libXt.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libXt.so.6
Reading symbols from /usr/lib/libXaw.so.7...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libXaw.so.7
Reading symbols from /lib/i686/cmov/libdl.so.2...Reading symbols from /usr/lib/debug/lib/i686/cmov/libdl-2.11.2.so...done.
done.
Loaded symbols for /lib/i686/cmov/libdl.so.2
Reading symbols from /usr/lib/libboost_thread.so.1.42.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libboost_thread.so.1.42.0
Reading symbols from /usr/lib/libboost_date_time.so.1.42.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libboost_date_time.so.1.42.0
Reading symbols from /usr/lib/libfreeimage.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libfreeimage.so.3
Reading symbols from /usr/lib/libzzip-0.so.13...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libzzip-0.so.13
Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libz.so.1
Reading symbols from /usr/lib/libsmpeg-0.4.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libsmpeg-0.4.so.0
Reading symbols from /usr/lib/libmikmod.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libmikmod.so.2
Reading symbols from /usr/lib/libvorbis.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libvorbis.so.0
Reading symbols from /usr/lib/libvorbisfile.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libvorbisfile.so.3
Reading symbols from /usr/lib/libFLAC.so.8...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libFLAC.so.8
Reading symbols from /usr/lib/libogg.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libogg.so.0
Reading symbols from /usr/lib/sse2/libspeex.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/sse2/libspeex.so.1
Reading symbols from /usr/lib/libasound.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libasound.so.2
Reading symbols from /lib/i686/cmov/librt.so.1...Reading symbols from /usr/lib/debug/lib/i686/cmov/librt-2.11.2.so...done.
done.
Loaded symbols for /lib/i686/cmov/librt.so.1
Reading symbols from /usr/lib/libdirectfb-1.2.so.9...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libdirectfb-1.2.so.9
Reading symbols from /usr/lib/libfusion-1.2.so.9...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libfusion-1.2.so.9
Reading symbols from /usr/lib/libdirect-1.2.so.9...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libdirect-1.2.so.9
Reading symbols from /usr/lib/libvga.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libvga.so.1
Reading symbols from /lib/ld-linux.so.2...Reading symbols from /usr/lib/debug/lib/ld-2.11.2.so...done.
done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /usr/local/lib/libv4l2.so.0...done.
Loaded symbols for /usr/local/lib/libv4l2.so.0
Reading symbols from /lib/libuuid.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libuuid.so.1
Reading symbols from /usr/lib/libxcb.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libxcb.so.1
Reading symbols from /usr/lib/libXmu.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libXmu.so.6
Reading symbols from /usr/lib/libXpm.so.4...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libXpm.so.4
Reading symbols from /usr/lib/libjpeg.so.62...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libjpeg.so.62
Reading symbols from /usr/lib/libmng.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libmng.so.1
Reading symbols from /usr/lib/libopenjpeg.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libopenjpeg.so.2
Reading symbols from /lib/libpng12.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib/libpng12.so.0
Reading symbols from /usr/lib/libIlmImf.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libIlmImf.so.6
Reading symbols from /usr/lib/libImath.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libImath.so.6
Reading symbols from /usr/lib/libHalf.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libHalf.so.6
Reading symbols from /usr/lib/libIex.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libIex.so.6
Reading symbols from /usr/lib/libIlmThread.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libIlmThread.so.6
Reading symbols from /lib/libx86.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libx86.so.1
Reading symbols from /usr/local/lib/libv4lconvert.so.0...done.
Loaded symbols for /usr/local/lib/libv4lconvert.so.0
Reading symbols from /usr/lib/libXau.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libXau.so.6
Reading symbols from /usr/lib/libXdmcp.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libXdmcp.so.6
Reading symbols from /usr/lib/liblcms.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/liblcms.so.1
Reading symbols from /usr/local/lib/OGRE/RenderSystem_GL.so...done.
Loaded symbols for /usr/local/lib/OGRE/RenderSystem_GL.so
Reading symbols from /usr/lib/libGLU.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libGLU.so.1
Reading symbols from /usr/lib/libGL.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libGL.so.1
Reading symbols from /usr/lib/libXrandr.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libXrandr.so.2
Reading symbols from /usr/lib/libGLcore.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libGLcore.so.1
Reading symbols from /usr/lib/tls/libnvidia-tls.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/tls/libnvidia-tls.so.1
Reading symbols from /usr/lib/libXrender.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libXrender.so.1
Reading symbols from /usr/lib/libXcursor.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libXcursor.so.1
Reading symbols from /usr/lib/libXfixes.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libXfixes.so.3
Reading symbols from /lib/i686/cmov/libnss_compat.so.2...Reading symbols from /usr/lib/debug/lib/i686/cmov/libnss_compat-2.11.2.so...done.
done.
Loaded symbols for /lib/i686/cmov/libnss_compat.so.2
Reading symbols from /lib/i686/cmov/libnsl.so.1...Reading symbols from /usr/lib/debug/lib/i686/cmov/libnsl-2.11.2.so...done.
done.
Loaded symbols for /lib/i686/cmov/libnsl.so.1
Reading symbols from /lib/i686/cmov/libnss_nis.so.2...Reading symbols from /usr/lib/debug/lib/i686/cmov/libnss_nis-2.11.2.so...done.
done.
Loaded symbols for /lib/i686/cmov/libnss_nis.so.2
Reading symbols from /lib/i686/cmov/libnss_files.so.2...Reading symbols from /usr/lib/debug/lib/i686/cmov/libnss_files-2.11.2.so...done.
done.
Loaded symbols for /lib/i686/cmov/libnss_files.so.2
Reading symbols from /usr/lib/alsa-lib/libasound_module_rate_speexrate.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/alsa-lib/libasound_module_rate_speexrate.so
Reading symbols from /usr/lib/sse2/libspeexdsp.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/sse2/libspeexdsp.so.1
Core was generated by `./OgreVisualizer'.
Program terminated with signal 11, Segmentation fault.
#0 0xb6dc563d in std::_Rb_tree_insert_and_rebalance(bool, std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::_Rb_tree_node_base&) ()
from /usr/lib/libstdc++.so.6
(gdb) bt
#0 0xb6dc563d in std::_Rb_tree_insert_and_rebalance(bool, std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::_Rb_tree_node_base&) ()
from /usr/lib/libstdc++.so.6
#1 0xb73bb3c2 in std::_Rb_tree, std::less, Ogre::STLAllocator > >::_M_insert_(std::_Rb_tree_node_base const*, std::_Rb_tree_node_base const*, Ogre::Node* const&) ()
from /usr/local/lib/libOgreMain.so.1.7.1
#2 0xb73b5a52 in _M_insert_unique (this=0xb6157ea0, child=0xb616aff8, forceParentUpdate=false) at /usr/include/c++/4.4/bits/stl_tree.h:1182
#3 insert (this=0xb6157ea0, child=0xb616aff8, forceParentUpdate=false) at /usr/include/c++/4.4/bits/stl_set.h:411
#4 Ogre::Node::requestUpdate (this=0xb6157ea0, child=0xb616aff8, forceParentUpdate=false)
at /home/balazsbela/Downloads/ogre_src_v1-7-1/OgreMain/src/OgreNode.cpp:805
#5 0xb73b6a40 in Ogre::Node::needUpdate (this=0xb616aff8, forceParentUpdate=92)
at /home/balazsbela/Downloads/ogre_src_v1-7-1/OgreMain/src/OgreNode.cpp:789
#6 0xb73b5038 in Ogre::Node::setScale (this=0x1825c, scale=...) at /home/balazsbela/Downloads/ogre_src_v1-7-1/OgreMain/src/OgreNode.cpp:638
#7 0x0805d306 in VisualizerApplication::adjustNodes (this=0x9cd4808) at ../src/VisualizerApplication.cpp:236
#8 0xb6e867f0 in ?? () from /usr/lib/libSDL_mixer-1.2.so.0
#9 0xb6e8719a in ?? () from /usr/lib/libSDL_mixer-1.2.so.0
#10 0xb6ed9b0d in ?? () from /usr/lib/libSDL-1.2.so.0
#11 0xb6ee185e in ?? () from /usr/lib/libSDL-1.2.so.0
#12 0xb6f2e0bd in ?? () from /usr/lib/libSDL-1.2.so.0
#13 0xb6bc7955 in start_thread (arg=0xb198ab70) at pthread_create.c:300
#14 0xb6ca6e7e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:130
(gdb)

I've had this bug for weeks, it's very frustrating.
It seems like Ogre keeps an std::set with the Nodes which need updating, this std::set is a Red Black Tree.

It instantly crashes once the node gets into the set at one point.
I used the sdl semaphores to make sure that the callback function does not call visualize multiple times, before the previous call finished.

I don't know what causes this crash. Maybe the std::set fills up from the frequent updating ? Maybe I should limit the framerate ?

I tried using a NodeListener to only update a node if the previously queued update operation finished, but since it crashes when it's placed in the update queue, not when it gets updated, it didn't help.


Relevant methods for the crash, snippets from Ogre3d 1.7.1 source code:

void Node::setScale(const Vector3& scale)
{
assert(!scale.isNaN() && "Invalid vector supplied as parameter");
mScale = scale;
needUpdate();
}

void Node::requestUpdate(Node* child, bool forceParentUpdate)
{
// If we're already going to update everything this doesn't matter
if (mNeedChildUpdate)
{
return;
}

mChildrenToUpdate.insert(child);
// Request selective update of me, if we didn't do it before
if (mParent && (!mParentNotified || forceParentUpdate))
{
mParent->requestUpdate(this, forceParentUpdate);
mParentNotified = true ;
}

}

typedef set::type ChildUpdateSet;

/// List of children which need updating, used if self is not out of date but children are
mutable ChildUpdateSet mChildrenToUpdate;

STL Set:

// insert/erase
/**
* @brief Attempts to insert an element into the %set.
* @param x Element to be inserted.
* @return A pair, of which the first element is an iterator that points
* to the possibly inserted element, and the second is a bool
* that is true if the element was actually inserted.
*
* This function attempts to insert an element into the %set. A %set
* relies on unique keys and thus an element is only inserted if it is
* not already present in the %set.
*
* Insertion requires logarithmic time.
*/
std::pair
insert(const value_type& __x)
{
std::pair __p =
_M_t._M_insert_unique(__x);
return std::pair(__p.first, __p.second);
}

STL Tree:

template
pair::iterator, bool>
_Rb_tree<_key,>::
_M_insert_unique(const _Val& __v)
{
_Link_type __x = _M_begin();
_Link_type __y = _M_end();
bool __comp = true;
while (__x != 0)
{
__y = __x;
__comp = _M_impl._M_key_compare(_KeyOfValue()(__v), _S_key(__x));
__x = __comp ? _S_left(__x) : _S_right(__x);
}
iterator __j = iterator(__y);
if (__comp)
{
if (__j == begin())
return pair(_M_insert_(__x, __y, __v), true);
else
--__j;
}
if (_M_impl._M_key_compare(_S_key(__j._M_node), _KeyOfValue()(__v)))
return pair(_M_insert_(__x, __y, __v), true);
return pair(__j, false);
}

template
typename _Rb_tree<_key,>::iterator
_Rb_tree<_key,>::
_M_insert_equal(const _Val& __v)
{
_Link_type __x = _M_begin();
_Link_type __y = _M_end();
while (__x != 0)
{
__y = __x;
__x = _M_impl._M_key_compare(_KeyOfValue()(__v), _S_key(__x)) ?
_S_left(__x) : _S_right(__x);
}
return _M_insert_(__x, __y, __v);
}

So yeah, this bug is a stubborn one.
If you would like to build the source code I will provide the instructions to do so, I also compiled it under windows, I have a build if you need it. If anyone has any idea how to fix the crash, I would be willing to donate beer :)).

For now, here's a screenshot:



Oh yeah, and you have full control of the camera, like in OgrePathfinder.