Cad Cam Development



Unmanaged C++ OpenGL Drawing and C# (WinForms/WPF) interoperability

I’m programming 3C milling simulator. I’ll write a post about some miscellaneous information about simulator later but today I’m going to show you how to do unmanaged C++ drawing and combine it with windows forms or WPF. I’m performing like that while completing the work I’m doing with 3C simulator. I have rendering in umanaged OpenGL that’s performing fast and well and nicely UI written in WPF that allows me to control the process of simulation. There are a lot of programming tasks like this one. It’s not first time I had to provide nice GUI and provide quick, fast, and memory efficient algorithms in an unmanaged environment. It would be hard to generate big enough Z-Buffer for ITO path generation method in C#. The first drawback would be the speed and the second one the managed environment. I’m allocating big amount of memory and don’t want to have garbage collector running on my huge Z-Buffer structure so that my paths milling generations will fail. There are other tasks like digitall sculpturing programs, CUDA simulations which requires speed, sophisticated graphics performance or usage of special technology like CUDA and highly usable UI that would allow for controlling the process that is going in behind. So it’s important to know how to do it. It’s a bit difficult and requires some technology knowledge, but is possible to do and really worth doing!

The following solutions to the problem has been presented:

  1. By the image creation and transferring it to managed application.
  2. By the on-HDC graphic context operations

You can find a general discussion and a short comparison of both solutions at the end of the post.

1st solution: By the image creation and transferring it to managed application

In windows forms you can handle drawing for numerous ways.  You can get HDC and draw either with GDI or GDI+, you can get HBitmap and so on. Our task is to perform fast reliable and efficient drawing in unmanaged OpenGL and then transform the result to the managed application to have it shown.  The first idea is create function that will generate an image for you and than create a table out of byte data and display it on a form. That’s relatively simple if you know how BMP images are generated, because it has some headings and initial informations. None of the both technologies (WPF + WinForms) will display images in raw format like it’s done on OpenGL Texturing or OpenGL Pixel Buffer Object that is the byte array with following triples inside :

byte * arr = new byte[img_size*3]
arr[index] = red;
arr[index+1] = green;
arr[index+2] = blue;

So you need to provide a valid image to WinForms or WPF. This would reqiure some knowledge like what the structure of bmp file looks like, or how the byte arrays are interpreted by windows forms (the order of colour, and what’s with transparency). I want you to make aware what order of bytes are you providing to your windows controls, is this:

struct Color
{
byte alfaTransparency;
byte red;
byte green;
byte blue;
};

or

struct Color
{
byte red;
byte green;
byte blue;
byte alfaTransparency;
};

The above structs will be differently interpreted while bitmap creation and you should choose proper memory struct layout to get it working. Normally it’s the second memory layout, with alfa transparency at the end. The following two code snippets displays how generate manually an image in C# language.


public Bitmap CopyDataToBitmap(byte[] data)
 {
 //Here create the Bitmap to the know height, width and format
 Bitmap bmp = new Bitmap( this.Width, this.Height, PixelFormat.Format24bppRgb );

 //Create a BitmapData and Lock all pixels to be written
 BitmapData bmpData = bmp.LockBits(
 new Rectangle( 0, 0, bmp.Width, bmp.Height ),
 ImageLockMode.WriteOnly, bmp.PixelFormat );

 //Copy the data from the byte array into BitmapData.Scan0
 Marshal.Copy( data, 0, bmpData.Scan0, data.Length );
 //Unlock the pixels
 bmp.UnlockBits( bmpData );
 //Return the bitmap
 return bmp;
 }

Code snippet above is pasted from here and displays the fastest pixel copying to provide bitmap content.


private void SimulatorControlGL_Paint(object sender, PaintEventArgs e)
 {
 if (LicenseManager.UsageMode == LicenseUsageMode.Designtime || this.DesignMode)
 return;
 byte[] bmpData = new byte[this.Width * this.Height * 3];
 int index = 0;
 for (int i = 0; i < this.Height; i++)
 {
 for (int j = 0; j < this.Width; ++j)
 {
 bmpData[index++] = 0;//b
 bmpData[index++] = 0;//g
 bmpData[index++] = 255;//r
 }
 }
 //FillImage( bmpData, this.Width, this.Height );
 // Bitmap bmp = new Bitmap( this.Width, this.Height, e.Graphics );

 Bitmap b1 = CopyDataToBitmap( bmpData );
 b1.Save( "myBmp.bmp" );
 Graphics g = e.Graphics;
 g.DrawImage( b1, 0, 0 );
 g.Flush();
 b1.Dispose();
}

The code snipper above illustrates the process of generation custom bitmap. Firstly, we fill the byte[] array with values (note the memory byte order for different colours) and than we fill our bitmap with data we’ve computed and render it on a Graphics device. This code works. Note that we heaven’t done a bitmap structure from scratch. The BMP headings have been created by the contructor of Managed GDI+ Bitmap class. Suppose we want to our colour table be computed in an unmanaged environment. The simple one solution is to create a dll file and a proper C# interop handler for this. Look at the code below how to do it. The first code snippet presents the unmanaged method for filling the byte* array in C++ unmanaged dll. The second one presensts a simple C# interop method invoker. If you require further information about C# interop check it on internet. There is a MSDN site here


extern "C" __declspec(dllexport) void FillImage(byte * img,int width,int height)
{
 int index = 0;
 UINT size = width*height*3;
 for(int i=0;i {
 for(int j=0;j {
 img[index++] = 0;//b
 img[index++] = 255;//g
 img[index++] = 0;//r
 }
 }
}


[DllImport( "3CSimulatorLib.dll", CharSet = CharSet.Auto, SetLastError = true, ExactSpelling = true )]
public static extern void FillImage([MarshalAs( UnmanagedType.LPArray )]byte[] b, int width, int height);

Instead of lines 6-15 of code snippet write DLL method invocation to draw your image in an unmanaged environment or just uncomment the 16th line in our test example. Sum up what has been told, implement those 4 code snippets from above and you’re done. You can render an image in an unmanaged environment and display it to a C# GUI. There some problems on performing it like the memory alignment or byte array (what’s the memory layout of following colours).  It would be necessary in some cases to manually generate bitmap in C++, so if you don’t know how to do it I’ve listed out some important resources below:

  1. Wikipedia description of bmp file format
  2. Wikipedia link to a sample BMP C++ class
  3. BMP format description
  4. Another BMP format description
  5. Creating and writing to a bitmap

So, we’re almost done with the first solution to it. To sum up: firstly, you create an unmanaged dll with some dllexport function that will handle the image filling. Secondly, you either create an managed Bitmap object and filling it in DLL or create a whole Bitmap object in a DLL. Note that the process of a bitmap creation should be done carefully because either Windows Forms or WPF won’t display invalid image. At last, you display the result to the screen.

2nd Solution:  By the on-HDC graphic context operations

WPF Airspace and WPF and Windows Forms technology differences

First of all note, that there is a striking difference between screens pixel handling in Windows Forms and WPF. In Windows Forms there exists any manager of pixel that is, you can get access to your pixel from anywhere and draw on everywhere. In WPF there is an airspace which means that pixel belongs only to one region. You can say that in WPF every pixel has its owner and the owner would be only one (monogamy)… and to comparison there were no owners of pixels in Windows Forms. Read the following articles if you don’t know the idea of airspace and wpf has changed:

  1. Technology Regions Overview
  2. WPF Interoperation: ”Airspace” and Window Regions Overview

Windows forms pixel screen management

WPF pixel arrangment

As we can see on left picture there is intersection of the blue, orange and green area. In Windows Forms every technology could have a simultaneous access to the screen buffer. On the right picture we can see how  screen buffer is shared within WPF technologies. Each pixel belongs to one drawing technology. We can easily setup a sharing region with unmanaged directX but there will be some problems with OpenGL.  That knowledge is substantial to our task because we want have our pixels rendered in other than WPF or GDI environment. That means, if we want to have OpenGL displayed on a WPF window, we must take Airspaces into account because it won’t render by disobeying airspace rules. We didn’t have the airspace problem in the previous solution. We must delegate the drawing procedure to other than WPF drawing environment. That means we have to change the owner of our pixels. I hope you got the point. Now how to do it. There are a few examples in the internet I played with:

WPF airspace example 1

WPF airspace example 1 - How to host win32 and share an airspace

How to do airspace and unmanaged DirectX drawing on WPF

How to do airspace and unmanaged DirectX drawing on WPF

There are two working examples that would be helpful once you want to do correct airspace management and delegate drawing for other technology like unmanaged directX. Just click on the photo above to go there. If you want to see the another one go here . if you download the example from DirectX Airspace sharing you’ll notice that it’s not so dificcult to use difficulties. It seems that Microsoft have provided support for use of only their technology once again. I couldn’t get OpenGL working in Airspace domain. I use windows forms control and host it on the wpf window. Works fine and smooth and is good wrapper technology for use OpenGL in managed WPF. Read the following paragraphs to know how to do HDC – context based OpenGL rendering.

How to setup OpenGL?

You have to use WGL library. Note that I’m rendering in windows DLL and all the technicall knowledge about DLL address spaces, variables and so on is required to write a well-working GL rendering library. You don”t have the standard glutInit, glut double buffergin and so on functions. Instead you need to provide appropriate  functionality with wgl. In order to get your unmanaged rendering done you need to fulfill following points:

  1. Choose pixel format
  2. The device you’re using(screen) is working on some settings like color byte ordering(r,g,b), double buffering, how many bytes are used per pixel and so on etc. OpenGL can be used either for rendering to device (this case) or rendering to bitmap. All the informations you set up here. You need to fulfill the managed object PIXELFORMATDESCRIPTOR carefully so that rendering to openGL could be used successfully. This and following stage is indispensable in order to get OpenGL rendered for the managed environment. The code snippet below presents my code for PixelFormatDescriptor just when you won’t know my code settings:

     [StructLayout( LayoutKind.Sequential )]
        public struct PIXELFORMATDESCRIPTOR
        {
            public void Init()
            {
                nSize = (ushort)Marshal.SizeOf( typeof( PIXELFORMATDESCRIPTOR ) );
                nVersion = 1;
                dwFlags = PFD_FLAGS.PFD_DRAW_TO_WINDOW | PFD_FLAGS.PFD_SUPPORT_OPENGL | PFD_FLAGS.PFD_DOUBLEBUFFER | PFD_FLAGS.PFD_SUPPORT_COMPOSITION;
                iPixelType = PFD_PIXEL_TYPE.PFD_TYPE_RGBA;
                cColorBits = 24;
                cRedBits = cRedShift = cGreenBits = cGreenShift = cBlueBits = cBlueShift = 0;
                cAlphaBits = cAlphaShift = 0;
                cAccumBits = cAccumRedBits = cAccumGreenBits = cAccumBlueBits = cAccumAlphaBits = 0;
                cDepthBits = 32;
                cStencilBits = cAuxBuffers = 0;
                iLayerType = PFD_LAYER_TYPES.PFD_MAIN_PLANE;
                bReserved = 0;
                dwLayerMask = dwVisibleMask = dwDamageMask = 0;
            }
            ushort nSize;
            ushort nVersion;
            PFD_FLAGS dwFlags;
            PFD_PIXEL_TYPE iPixelType;
            byte cColorBits;
            byte cRedBits;
            byte cRedShift;
            byte cGreenBits;
            byte cGreenShift;
            byte cBlueBits;
            byte cBlueShift;
            byte cAlphaBits;
            byte cAlphaShift;
            byte cAccumBits;
            byte cAccumRedBits;
            byte cAccumGreenBits;
            byte cAccumBlueBits;
            byte cAccumAlphaBits;
            byte cDepthBits;
            byte cStencilBits;
            byte cAuxBuffers;
            PFD_LAYER_TYPES iLayerType;
            byte bReserved;
            uint dwLayerMask;
            uint dwVisibleMask;
            uint dwDamageMask;
        }
    
  3. Create gl context
  4. You have to create a GL context. This would be the context the GL performs on. The following function performs task from the previous step and current one:

    public void Func_Init(IntPtr hwnd)
            {
                this._hdc.ManangedObject = GetDC( hwnd );
                var pixelFormatDescriptor = new PIXELFORMATDESCRIPTOR();
                pixelFormatDescriptor.Init();
    
                var pixelFormat = ChoosePixelFormat( _hdc.ManangedObject, ref pixelFormatDescriptor );
                if (!SetPixelFormat( _hdc.ManangedObject, pixelFormat, ref pixelFormatDescriptor ))
                    throw new Win32Exception( Marshal.GetLastWin32Error() );
                if ((this._hglrc.ManangedObject = wglCreateContext( _hdc.ManangedObject )) == IntPtr.Zero)
                    throw new Win32Exception( Marshal.GetLastWin32Error() );
            }
    
  5. Initialize gl rendering context to be able to render

How to Render?
Firstly, you have to know how to draw with OpenGL on win32 windows. A very helpful article that describes the basics of it can be read here. There are some articles on how to do GL rendering in win32 in google so search if you don’t know it. The steps you’ve to perform to render on WinForms are very similar. You create a proper pixel format, than GL context and then rendering as it was displayed in points above. You can’t neither setup pixelformat in unmanaged dll nor create GL context in unmanaged environment, because it won’t work. If you created properly GL context and pixelformat and you get black screen than you have to try to invoke GL function:

 [DllImport( "opengl32.dll", EntryPoint = "glGetString", CharSet = CharSet.Auto, SetLastError = true, ExactSpelling = true )]
        static extern IntPtr _glGetString(StringName name);
        public static string glGetString(StringName name)
        {
            return Marshal.PtrToStringAnsi( _glGetString( name ) );
        }

Firstly, I got black screen. After I invoked the glGetString function opengl started to be visible. So call it to get GL working on WinForms. I call it in constructor, even before context creation and setting pixel format. This is kind of magic line that makes it work.
How to avoid flickering?
You need to provide double buffering so that no flickering could be visible. This is achieved through proper PixelFormaDescription structure initialization. This is achieved using SwapBuffer function.

      [DllImport( "gdi32.dll", CharSet = CharSet.Auto, SetLastError = true, ExactSpelling = true )]
        public static extern bool SwapBuffers(IntPtr hdc);

This function is to invoke after each rendering and Windows forms screen refresh. But it’s not everything. Not that every refresh action sends a WM_ERASEBKGND message to the window. This is a culprit of flickering despite swapping buffers( because it is being invoked after we have rendered image and updated a HDC context). To get rid of this you have to either provide proper window style WS_CLIPCHILDREN style or just filter out this message from a WndProc. I use the second one. Overriding WndProc is easy and done in two steps:

 [System.Security.Permissions.PermissionSet( System.Security.Permissions.SecurityAction.Demand, Name = "FullTrust" )]
        protected override void WndProc(ref Message m)
        {
            if (m.Msg == 0x14)//filter out the WM_ERASEBACKGROUND
            {
                return;
            }
            base.WndProc( ref m );
        }

And this code in the class constructor:

  this.SetStyle( ControlStyles.EnableNotifyMessage, true );

That’s everything about flickering. There is quite helpful conversation on stackoverflow here

Managed environment miscellanities

You have to pin the GL and HDC context in an managed environment. What is pinned object in C#? A pinned object is one that has a set location in memory. Read this series of articles if you’re lacking knowledge about memory management in .NET. Pinning is essential step because you’re working on address spaces that cannot be moved by the garbage collector. So if you don’t pin those objects your GL rendering over managed environment will be UNSTABLE! Garbage collector should keep your unmanged DLL address space unchanged and you need to display it over screen buffer always in the same place. That’s the reason addresses of contexts you create should be unchanged. This is very important to know. I use PinnedObject structure from this article. That’s the code:

 public class PinnedObject where T:struct
    {
        protected T managedObject;
        protected GCHandle handle;
        protected IntPtr ptr;
        public bool disposed { get; protected set; }

        public T ManangedObject
        {
            get
            {
                return (T)handle.Target;
            }
            set
            {
                Marshal.StructureToPtr(value, ptr, false);
            }
        }

        public IntPtr Pointer
        {
            get { return ptr; }
        }

        public PinnedObject()
        {
            handle = GCHandle.Alloc(managedObject, GCHandleType.Pinned);
            ptr = handle.AddrOfPinnedObject();
        }

        ~PinnedObject()
        {
            Dispose();
        }

        public void Dispose()
        {
            if (!disposed)
            {
                handle.Free();
                ptr = IntPtr.Zero;
                if (managedObject is IDisposable)
                    (managedObject as IDisposable).Dispose();
                disposed = true;
            }
        }
    }

General considerations – PROS AND CONS
The best cross platform rendering is done using DirectX. There are almost no problems. Airspaces are supported, managing with Windows Forms is also less demanding because to render DirectX on windows forms the only thing you have to provide is HWND of the control to render on. No fuzz with GL rendering context and pixel format.
The first solution with bitmap is versatile but requires custom and step by step creation of bitmap. It spares you some troubles with airspaces and all this stuff like, once again, pixel format and GL rendering context.
The second solution is reliable but requires a few steps to perform. You need to know Marshalling pretty good and some advanced C++ like dll and DLL address spaces. But provides good results and I’m using this one. If you do it once, you can create yourself a managed library that will handle those setting up of a pixel format and creating GL context and you’ll only connect to dll what will be convenient.
Both of the presented solution generate nice results and are very fast. So you get what you want: advanced, fast C++ rendering over stunning GUI in WPF or Windows Forms.
Sourcecodea and executable: I will update a post here and provide sourcode if I finish my 3C milling simulator

Advertisements

Trackbacks & Pingbacks

Comments

  1. * Jeff says:

    Hello,

    Nice blog, any progress on your 3C milling simulator? Thanks.

    | Reply Posted 6 years, 2 months ago


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: