19 May '13 - 10:28 *
Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
 
   Home   Help Search Login Register  
Pages: [1] 2  All
  Reply  |  Print  
Author Topic: Using BASS.NET Visualisations in WPF  (Read 3758 times)
Evan
Posts: 8


« on: 18 Sep '10 - 11:40 »
Reply with quoteQuote

I upgraded my app from Windows Forms to WPF. One of the high points of the app was the spectrum supplied with BASS.NET, and I definitely wanted to take that along with me. I've written a small guide for other people who want to do the same.

In case you haven't seen the visualisations, take a look at Simple.cs in the samples – they're great. Essentially, the functions draw bitmaps of the music during the timer tick event, and putting the bitmaps on controls works like an animation.

WPF introduced a new way of handling bitmaps. Windows Forms uses items from the System.Drawing namespace. WPF uses System.Windows.Media.Imaging, a different Bitmap object, plus a different way of initialising it.

The strategy was to translate the bitmap in memory. Then there were two threading issues to solve. First, WPF threading is much less forgiving in general. With Windows Forms I had a PictureBox, and changing the image "just worked", even though I was calling the UI from a music-playing thread. WPF complains. Therefore you need to call the UI with a delegate function, using the Dispatcher from the UI thread.

There's one more threading complication. Because of the new bitmap style, you can't send bitmaps across threads. A WPF bitmap is initialised by setting the .Source member, and initialisation happens when it's appropriate. WPF still complains, (probably) because the Stream assigned to the .Source member is on the music-playing thread. The workaround is to convert the bitmap to a byte array, send the array across the threads, and then convert it to a bitmap after it reaches the UI thread. It's a little roundabout, but it does the job.

Here's how it works.

   // In the class definition, before the constructor:
        public delegate void DrawMeterDelegate(Byte [] buffer);


        /// <summary>
        /// Slightly altered Drawing function (mSpectrumBackground and mUiDispatcher set elsewhere)
        /// </summary>
        /// <param name="width">Bitmap width</param>
        /// <param name="height">Bitmap height</param>
        private void DrawSpectrum(int width, int height)
        {
            System.Drawing.Bitmap bitmap;
            Byte [] buffer;

            try
            {
                bitmap =
                    mPlayerVisuals.CreateSpectrumLinePeak(mPlayerStream, width, height,
                    System.Drawing.Color.LightBlue, System.Drawing.Color.SlateBlue, System.Drawing.Color.Cyan,
                    mSpectrumBackground, 2, 1, 2, 10, false, false, false);
            }
            catch
            {
                return;
            }
            if (bitmap != null)
            {
                // Convert bitmap to byte[] array
                buffer = DrawingToBuffer(bitmap);
                // Update control with UI dispatcher we passed ourselves elsewhere
                mUiDispatcher.Invoke(new DrawMeterDelegate(DrawMeter), buffer);
            }
        }


        /// <summary>
        /// Converts a System.Drawing.Bitmap to a byte [] array
        /// </summary>
        /// <param name="drawingBitmap">Bitmap to convert</param>
        /// <returns>Byte [] array</returns>
        private byte [] DrawingToBuffer(System.Drawing.Bitmap drawingBitmap)
        {
            MemoryStream ms;
            Byte[] buffer;

            ms = new MemoryStream();
            drawingBitmap.Save(ms, System.Drawing.Imaging.ImageFormat.Bmp);
            buffer = new byte[ms.Length];
            buffer = ms.ToArray();
            ms.Dispose();
            return buffer;
        }


        /// <summary>
        /// On the UI thread, translates a bitmap, and sets the Image control named "Meter" (reference passed elsewhere)
        /// </summary>
        /// <param name="buffer">System.Drawing.Bitmap as byte[] array</param>
        private void DrawMeter(byte [] buffer)
        {
            BitmapImage bitmapImage;
            MemoryStream ms;

            ms = new MemoryStream(buffer);

            bitmapImage = new System.Windows.Media.Imaging.BitmapImage();
            ms.Seek(0, SeekOrigin.Begin);

            // Create the bitmap in risk-averse fashion
            bitmapImage.BeginInit();
            bitmapImage.CreateOptions = BitmapCreateOptions.IgnoreColorProfile;
            bitmapImage.CacheOption = BitmapCacheOption.OnLoad;
            bitmapImage.StreamSource = ms;
            bitmapImage.EndInit();
            // Get rid of memory stream
            ms.Dispose();

            // Assign new bitmap to Image control
            Meter.Source = bitmapImage;
        }

Logged
ken
Posts: 630


« Reply #1 on: 18 Sep '10 - 12:28 »
Reply with quoteQuote

I use this in WPF to convert image (use it for WaveForm)

private BitmapSource BitmapSourceFromImage(System.Drawing.Image img)
{

    if (img != null)
    {

          System.IO.MemoryStream memStream = new System.IO.MemoryStream();

          //save the image to memStream as a png
           img.Save(memStream, System.Drawing.Imaging.ImageFormat.Png);

           //gets a decoder from this stream
          System.Windows.Media.Imaging.PngBitmapDecoder decoder = new System.Windows.Media.Imaging.PngBitmapDecoder(memStream, System.Windows.Media.Imaging.BitmapCreateOptions.PreservePixelFormat, System.Windows.Media.Imaging.BitmapCacheOption.Default);

                return decoder.Frames[0];
            }
            else
                return null;
        }

And for Thread issues in WPF (updating GUI) I use this code

Dispatcher.Invoke(DispatcherPriority.Normal, (Action)delegate()
{
     //Do your stuff with controls in GUI here, like...
     myImage.Source =  "some code..."
});

Logged
KenS
Guest
« Reply #2 on: 21 Sep '10 - 21:18 »
Reply with quoteQuote

Thank you.
Logged
BaseHead
Posts: 91


« Reply #3 on: 28 Nov '10 - 12:20 »
Reply with quoteQuote

Evan!

I'm going thru your example now!
so happy I found this since I'm WPF also...... Cheesy
Awesome stuff!

I have the code popped and most all things sorted out, but need to know what these 4 variables or controls are first before I can make my test app build and it's best not to guess.....hehehe

mPlayerStream =  I assume the BASS channel stream that is playing??
mPlayerVisuals
mSpectrumBackground  = I assume an Image control?
mUiDispatcher 

Thx so much for this!

Steve

« Last Edit: 28 Nov '10 - 12:27 by BaseHead » Logged
BaseHead
Posts: 91


« Reply #4 on: 29 Nov '10 - 00:07 »
Reply with quoteQuote

Ken,

I'm interested in your Waveform converting code also, since this in the next step after getting the Spectrum working for me.

First off I'm having a bit of trouble with the test examples that are in Simple.cs that I'm moving over to a test WPF app now.
how did you get around this line
WF2 = new WaveForm("test.mp3",new WAVEFORMPROC(MyWaveFormCallback), this);

i'm getting this error in my WPF project.
The best overloaded method match for 'Un4seen.Bass.Misc.WaveForm.WaveForm(string, Un4seen.Bass.Misc.WAVEFORMPROC, System.Windows.Forms.Control)'


If you can post anymore of your code that shows the full flow of how you pulled off waveform drawing under WPF this would be great!!
I'm still trying to get my head around C# so any help would be appreciated!   Cool

thx!

Steve
Logged
radio42
Posts: 4012


« Reply #5 on: 29 Nov '10 - 07:51 »
Reply with quoteQuote

Seems, that your 'this' member is not a 'System.Windows.Forms.Control'?
So you might try one of the other WaveForm constructor overloads or set this parameter to 'null'.
Its used to invoke the NOTIFYPROC async on the UI thread the control was created on.
If not set the NOTIFYPROC might be called from the WaveForm background thread.
Logged
ken
Posts: 630


« Reply #6 on: 29 Nov '10 - 13:48 »
Reply with quoteQuote

I'm interested in your Waveform converting code also, since this in the next step after getting the Spectrum working for me.

Here is a snippet from my WPF code. Hope that helps you.


private void DoWaveForm()
{
  WF = new WaveForm(FileName, new WAVEFORMPROC(WFcallback), null);
  WF.FrameResolution = 0.01f;
  WF.CallbackFrequency = 2000;
  WF.DrawWaveForm = WaveForm.WAVEFORMDRAWTYPE.HalfMono;
  WF.RenderStart(true, BASSFlag.BASS_DEFAULT);
}


private void WFcallback(int framesDone, int framesTotal, TimeSpan elapsedTime, bool finished)
{
    if (finished)
    {

       //You need Dispatcher.Invoke if you run in separate thread.
       Dispatcher.Invoke(DispatcherPriority.Normal, (Action)delegate()
       {
            //Do your stuff with controls in GUI here, like...
            wfImage.Source = BitmapSourceFromImage(WF.CreateBitmap((int)wfImage.Width, (int)wfImage.Height, -1, -1, true));
        });

     }
}

Logged
BaseHead
Posts: 91


« Reply #7 on: 29 Nov '10 - 23:19 »
Reply with quoteQuote

radio (Bernd),
Thanks for that tip!
I set it to null and seems to get past that problem for now.

Ken!
you are a God for sharing this!
It's drawing like a champ now!

Just need to get the spectrum meters going and I will have all the pieces in place and roughly working on the visual side of things.

radio, if you have time, maybe include a WPF example of Simple.cs in the future since I'm sure this will come up more in the future and all 4 drawing things to be concerned about would be in that one example.
I was on my way to creating one as a good exercise, but got excited seeing bits work in my main app that I kind of stopped progress on it......hehehehe
It's crazy how many differences there are between C#/WPF code examples compared to just C# and it's such a battle in the Google trying to find the right code examples....... Cool

Thanks Again guys!

Steve
Logged
BaseHead
Posts: 91


« Reply #8 on: 30 Nov '10 - 04:23 »
Reply with quoteQuote

Ken,
are you able to get Markers to draw using this method?
mine aren't showing up so far.

thx again!

Steve

Logged
BaseHead
Posts: 91


« Reply #9 on: 30 Nov '10 - 04:44 »
Reply with quoteQuote

Scrap that last question Ken!

Got them to work!...... Cool

Steve
Logged
aybe
Posts: 129


« Reply #10 on: 30 Nov '10 - 17:02 »
Reply with quoteQuote

There are numerous paths to achieve this in WPF :

Mine,

- Create a BitmapData from the Bitmap (Forms),
from here you could copy the bytes to a WriteableBitmap but this will never give you 60 fps as it's double-buffered blabla etc ...
but this class provides a simple usage for writing to a bitmap

- Imaging.CreateBitmapSourceFromMemorySection by far the fastest since
it will copy the data straight to the GPU, but it's a little more complex since
you'll need MapViewOfFile function from Win32 plus CopyMemory or DMACopyMemory which is in BASS.
here the Image.Source should be InteropBitmap (the type returned from above method) GetAsFrozen() method,
this will suppress the tearing you might experience. (Huge optimization)
Also, there's a bug in .NET which leaks memory, this gets fixed by doing GC.Collect on each frame, for instance, in the event below

- To have the smoothest scrolling possible and to avoid any Dispatcher.BeginInvoke stuff, use CompositionTarget.Rendering but be careful,
it can be overkill.
http://msdn.microsoft.com/en-us/library/system.windows.media.compositiontarget.rendering.aspx

I use all these three, first one to pull the image, second to make it for WPF and the fastest possible, third for polishing ...

Ideally you should render the whole bitmap but it can be big in RAM,

The main issue I had initially was speed for big bitmaps, and also that Image from WPF
doesn't accept +32000 pixels or wide or almost, depending height, even though it accepts them,
scrolling is unpractical;
And for 1920*256 or more, and at 60fps;
there's quite some output that is being transferred : 1920*256*4 ~= 2Mb * 60 fps = 120Mb/s
it is absolutely vital to optimize your app to handle such burden without going CPU crazy ...
this solution seems to be the one  Grin
« Last Edit: 30 Nov '10 - 17:17 by aybe » Logged
aybe
Posts: 129


« Reply #11 on: 30 Nov '10 - 17:05 »
Reply with quoteQuote

I use this in WPF to convert image (use it for WaveForm)

And for Thread issues in WPF (updating GUI) I use this code

Dispatcher.Invoke(DispatcherPriority.Normal, (Action)delegate()
{
     //Do your stuff with controls in GUI here, like...
     myImage.Source =  "some code..."
});


Hello,

The Invoke is a little counter-intuitive for drawing since WPF is not an immediate rendering mode compared to GDI which is.
Hence it is blocking and sometimes it doesn't even work, depending how many times you call it  Cheesy
« Last Edit: 30 Nov '10 - 17:19 by aybe » Logged
BaseHead
Posts: 91


« Reply #12 on: 2 Dec '10 - 12:42 »
Reply with quoteQuote

Ken,

Are you using the zoom functions?
If so, did you figure out a way to zoom without having to re-render and draw the waveform every time with you method?

Aybe,
Thanks for that info also!  Good to know!

Steve
Logged
ken
Posts: 630


« Reply #13 on: 2 Dec '10 - 12:56 »
Reply with quoteQuote

Ken,

Are you using the zoom functions?
If so, did you figure out a way to zoom without having to re-render and draw the waveform every time with you method?


I use the Zoom function like the C# example for Bass.NET. I aslo have used a WPF's ScrollViewer for shooth scrolling of waveform (I make the WF image much wider than the screen, the scoll it inside a ScrollViewer.

/Ken
Logged
aybe
Posts: 129


« Reply #14 on: 2 Dec '10 - 20:42 »
Reply with quoteQuote

Ken,

Are you using the zoom functions?
If so, did you figure out a way to zoom without having to re-render and draw the waveform every time with you method?

Aybe,
Thanks for that info also!  Good to know!

Steve


By using ratios, much like any sound edition related software, and writing bytes directly to memory,
you won't have to recreate that image every time.
I have started working on a sound editor but before I just spent some time spying Wavelab,
they do create a stripe of the sound that goes down to 1:256.

Below that ratio, they start to create the bitmap for what is visible.
Above that ratio is really simple, for 1:512 I draw one pixel, step one, etc ...
Extremely easy to implement and yet very effective.  Wink

If you use a ScrollViewer you should consider something that updates only up to N times per second,
when you move the scroll by hand, it can triggers really a lot of events per second,
this will significantly lower the burden on your CPU as well ...
Logged
BaseHead
Posts: 91


« Reply #15 on: 3 Dec '10 - 09:50 »
Reply with quoteQuote

Ken,
I just needed to use this line of yours and I'm all good with the zoom not re-drawing........Cool

 imgWaveform.Source = BitmapSourceFromImage(WF2.CreateBitmap((int)imgWaveform.Width, (int)imgWaveform.Height, _zoomStart, _zoomEnd, true));

sweet!
need to figure out how change the waveform gain without re-drawing is next on my list........hehehe


Aybe,
Thx for those tips as always!!
You got some really good ideas!


What are you guys making? 
I'm making an audio sound effects search database program that displays the wave of the current selected sound with playback/zoom of that selected sound.
Not many WPF apps out there still so would be cool to see what you are doing also if released already.

cya!

Steve



Logged
BaseHead
Posts: 91


« Reply #16 on: 3 Dec '10 - 10:20 »
Reply with quoteQuote

Also, for the future if any WPF newbies are looking at this thread I'd like to contribute how I'm drawing the playhead in WPF, since it's a bit different then the Simple.cs example for winforms.
Hope this helps somebody out...... Grin

//Trigger this from a timer

 Line playHead = new Line();  //Creates playhead line
        long pos = 0;
        long len = 0;
 private void DrawPlayHead()
        {

            double bpp = 0;
            bpp = len / (double)this.grdWaveform.Width;  // bytes per pixel              
            int x = (int)Math.Round(pos / bpp);  // position (x) where to draw the line

            try
            {
                playHead.Stroke = System.Windows.Media.Brushes.Yellow;
                playHead.X1 = x;
                playHead.X2 = x;
                playHead.Y1 = grdWaveform.Height;  //Height

                playHead.VerticalAlignment = VerticalAlignment.Center;
                playHead.StrokeThickness = 2;
                grdWaveform.Children.Add(playHead);
            }
            catch
            {

            }
            finally
            {
                //Add cleanuup code here
            }


I'm gonna update this once I get the zoom code working!

Steve
« Last Edit: 4 Dec '10 - 09:50 by BaseHead » Logged
aybe
Posts: 129


« Reply #17 on: 3 Dec '10 - 17:21 »
Reply with quoteQuote

Ken,
I just needed to use this line of yours and I'm all good with the zoom not re-drawing........Cool

 imgWaveform.Source = BitmapSourceFromImage(WF2.CreateBitmap((int)imgWaveform.Width, (int)imgWaveform.Height, _zoomStart, _zoomEnd, true));

sweet!
need to figure out how change the waveform gain without re-drawing is next on my list........hehehe


Aybe,
Thx for those tips as always!!
You got some really good ideas!


What are you guys making? 
I'm making an audio sound effects search database program that displays the wave of the current selected sound with playback/zoom of that selected sound.
Not many WPF apps out there still so would be cool to see what you are doing also if released already.

cya!

Steve





Well good tips, I'd rather say they are necessary as there are many other things that will need CPU cycles such as VSTs etc ...

I am working on a DJ app, with timecode etc ... but what makes it special is the online service for cataloging and a lot more.
Being an old FinalScratch customer and felt stolen mainly because they did support it 1 year for something that was ~1500€,
I decided starting my own complete solution which so far, works MUCH better than everything that's out there, especially regarding precision and responsiveness.
Now I'm eager to nail it down with killer online features.

Otherwise my second project would be an audio framework powered by BASS but this went a little more complicate than expected,
so I'm going to finish the DJ app first then focus on the framework. Also going to target Windows/Linux as BASS is available on both platforms.

All this stuff will one day end on the web as open source, hopefully ...

Regarding WPF, the common myth that it's slower at graphics is just wrong, actually things can even be faster that Forms whose GDI is great but outdated now.

About your example, don't you think there's going to be a problem since you add your markers to your Grid ??

I think Radio42 solution is the best, since it just draws the line over the bitmap,
yours involves changing the visual tree / logical tree which then involves measuring again etc ... I'm pretty sure your way will crash sooner or later ....
« Last Edit: 3 Dec '10 - 17:26 by aybe » Logged
BaseHead
Posts: 91


« Reply #18 on: 4 Dec '10 - 09:53 »
Reply with quoteQuote

Hey!
I'm not using the BASS markers for the playhead.
I left out this line Line playHead = new Line();
updated above.
I'm drawing a vector graphic line (i assume) over the grid.

I did try using those BASS markers in early tests as a playhead also, but they don't like being updated that much and it did crash like mad then.......Cool

Rock solid now tho!

cya!


s>
Logged
aybe
Posts: 129


« Reply #19 on: 4 Dec '10 - 17:21 »
Reply with quoteQuote

You're not drawing over the Grid, actually you are adding an UIElement to your Window, which is radically different.

what I'd do :
  • Radio42 solution
  • Create a 3 columns Grid, middle column has 1 pixel width and hosts a Line or a Rectangle,
    then you draw and offset it so what you hear is under that line,
    you could also set a GridSplitter which would allow you to set the position as you wish

But honestly, n°1 is the best and that's what I'm doing, the others involves layout & co and you might have to play
with UseLayoutRounding and SnapToDevicePixels to get an exactly one pixel wide line ....

How did you made it crash ?

WaveForm markers are great for static markers, not for a moving marker ... because you have to draw again & again !
Logged
Pages: [1] 2  All
  Reply  |  Print  
 
Jump to:  

Powered by SMF 1.1.18 | SMF © 2013, Simple Machines