.NET and Multimedia - Introduction of managed DirectX Andreas Lennartz University of Applied Sciences, Fulda firstname.lastname@example.org Abstract any CLR compliant language can optionally be used) (?). DirectX is a multimedia API that DirectX allow it to make non-hardware- provides a standard interface to speciﬁc function calls. For this purpose, interact with graphics and sound DirectX is accountible for the execution on cards, input devices and more. Di- the hardware. If the function is supported rectX abstracts code from the spe- from the hardware, the hardware acceleration ciﬁc hardware and translates it to makes the API functions work fast. If not, Di- a common set of instructions into rectX has software emulation that takes over. hardware speciﬁc commands. The It will be slower, but in spite of that the actual implementation of DirectX same low-level API can be used. Developer contains an integration in managed can focus on the application development it- Code, which combines the advan- self. The layer of DirectX that takes care tages of both world, and is called of hardware-supported features is called the managed DirectX. hardware acceleration layer (HAL), and the software emulation layer that kicks in if the1 Introduction hardware doesn’t support a feature is called the hardware emulation layer (HEL).Currently there are two vendors oﬀering APIs When writing applications with managedfor developing 3D-Applications available. On DirectX or unmanaged DirectX, managed Di-one hand Microsoft provides its DirectX-API rectX has the prejudice that it is slower thanand on the other hand there is the open source unmanaged. But this is only true in certainalternative OpenGL. circumstances. So far a lot of games were OpenGL is an environment for developing created with managed code or using a mixportable, interactive 2D and 3D graphics ap- of managed and unmanaged code. Writingplications. Introduced in 1992, OpenGL is in managed code makes the developers moreone of the industry’s ﬁrst choice for 2D and productive and thus write more code in less3D graphics APIs (?). Because of its stan- time and produces safer code.darisation it is platform independent. WithOpenGL, rapid application development with 2 Managed Codea broad set of rendering, texture mapping,special eﬀects and other powerful visualiza- Managed code is code that has its executiontion functions can be archieved. managed by the .NET Framework Common DirectX ﬁrst appeared in 1995 and was then Language Runtime (CLR). It refers to a con-called the ”GameSDK”. Originally it was tar- tract of cooperation between natively execut-geted at developers using C and C++. With ing code and the runtime. In this contract itthe release of the ﬁrst managed version (9.0) is speciﬁed that at any point of execution, theof the API in December 2002 it is possible to runtime may stop an executing CPU. Then ituse C# or VB.NET with DirectX (Actually may retrieve information speciﬁc to the cur-
rent CPU instruction address. Information tections in place around memory managementthat must be queryable generally belongs to and port I/O and so forth, but the systemruntime state, such as register or stack mem- doesn’t actually know what the application isory contents. doing. Therefore, it can’t make any guaran- The necessary information is encoded in an tees about what happens when the applica-Intermediate Language (IL) and associated tion runs.”’ see (?)metadata. The symbolic information that de-scribes all of the entry points and the con- 3 DirectX componentsstructs is exposed in the IL (e.g., methods, DirectX 9.0 for Managed Code is made up ofproperties) and their characteristics. With the following major components (The depre-the Common Language Infrastructure (CLI) cated parts of DirectX will not be included inStandard, a description is given how the in- the further discussion):formation has to be encoded. Programminglanguages that target the runtime will build • Direct3D Graphics provides a single APIthe correct encoding. Therefore, the devel- that can be used for 3D graphics pro-oper only has to know that any of the lan- gramming.guages that target the runtime produce man- • DirectInput provides support for a vari-aged code. This code contains encoded ﬁles ety of input devices, including full sup-that contain IL and metadata. There are a lot port for force-feedback technology.of languages to choose from, because there areover 20 diﬀerent languages provided by third • DirectSound provides support for playingparties in addition to the standard languages and capturing prerecorded digital sam-from Microsoft. (C#, J#, VB .Net, Jscript ples..Net, and C++). • AudioVideoPlayback allows for playback Before execution of the code, the Virtual and simple control of audio and videomachine compiles the IL into native exe- media.cutable code. While the compilation happensby the managed execution environment, the 3.1 Direct 3D Graphicsmanaged execution environment can make Direct3D Graphics is for device independentguarantees about what the code is going to access to the 3D-graphic hardware and accel-do. The enviroment can insert traps and ap- eration. If no hardware acceleration is sup-propriate garbage collection hooks, exception ported, the software renderer will emulate thehandling, type safety, array bounds and index hardware. (?)checking, etc. E.g, such a compiler makes sure Complicated 3D-ﬁgures and objects areto lay out stack frames and everything just usually modeled with a 3D-software and thenright so that the garbage collector can run saved to ﬁle. The .x ﬁle format is an ex-in the background on a separate thread, con- ample for such a ﬁle. (?),(?) Microsoft Di-stantly walking the active call stack, ﬁnding rect3D can create objects from these ﬁles us-all the roots, chasing down all the live ob- ing meshes. A mesh is an abstract data con-jects. In addition because the IL has a notion tainer with the data for a complex model. Itof type safety the execution engine will main- contains resources such as textures and mate-tain the guarantee of type safety eliminating rials, and attributes such as position data anda whole class of programming mistakes that adjacency data. Meshes itselves are some-often lead to security holes. what complicated, but accessing and using ”‘Contrast this to the unmanaged world: them with Direct3D makes it easier. In theUnmanaged executable ﬁles are basically a following an example is shown how to load,binary image, x86 code, loaded into mem- render, and unload a mesh. With this exam-ory. The program counter gets put there and ple an overview will be given over the func-that’s the last the OS knows. There are pro- tionality of Direct3D.
First of all, the mesh object has to be ini- Furthermore in the application deﬁned On-tialized. Also we need a device for the out- ResetDevice method it is possible to initializeput. There are two device types in DirectX: an ExtendedMaterial object as an array thata hardware device and a reference device. is used to capture the mesh ﬁle data to a Ma-The hardware device is the primary device terial structure. In the method the device canwhich supports hardware-accelerated raster- be adjusted, for e.g. turning on the z-buﬀerization as well as hardware vertex processing. and white ambient lighting. This function isFor this purpose the display adapter of the called when the device is initialized and incomputer has to support Direct3D. The hard- some diﬀerent cases reseted.ware device then implements all or part of the Now the mesh object has to be loaded.transformations, lighting, and rasterization in As shown in the following code fragmenthardware. The application itself does not ac- as a part of the OnResetDevice method acess the hardware device directly - it calls Di- mesh is loaded that represents a texture-rect3D functions and methods which access mapped 3D tiger from a tiger.x ﬁle. In thisthe hardware through a hardware abstraction Mesh.FromFile method call, the SystemMem-layer (HAL). If the hardware supports Di- ory constant of the MeshFlags enumerationrect3D, these functions and methods are exe- indicates that the mesh is to be loaded intocuted with best performance of the hardware. system RAM not typically accessible by theDirect3D supports additional a device called device.reference device. This reference device emu- mesh = Mesh.FromFile("tiger.x", MeshFlags.lates all Direct3D functions and methods via SystemMemory, device, out extendedMaterials);software without any hardware acceleration. After the mesh is loaded, a meshMateri-Furthermore, if Direct3D functionalities are als Material object extendedMaterials has itsused which are not implemented in the hard- members ﬁlled with the Material structuresware, the reference device will then emulate of the materials object that came from thethe corresponding part of the hardware de- mesh ﬁle. These materials could now be setvice. inside the OnResetDevice method to a speciﬁc To create a new Direct3D device, the fol- ambient color or could be set to a MeshTex-lowing command has to be executed, e.g. in ture object loaded from a texture of a ﬁle. Aa Windows Form Class: texture object ﬁle could be loaded with thedevice = new Device(Manager.Adapters.Default TextureLoader.FromFile method. For exam-.Adapter, DeviceType.Hardware,this,CreateFlags. ple, it is possible to set for all extendedMa-SoftwareVertexProcessing,newPresentParameters()) terials of a mesh the ambient material color. This could take place by an iteration over the For creating a new device these parameters length of the materials from the mesh. Af-are needed: terwards the texture had to be loaded. The 1. A graphics adapter to connect to the out- following code fragment (with i as the itera- put device, e.g. to connect one monitor tion counter) shows this: to the graphics card. meshMaterials[i].Ambient = meshMaterials[i]. Diffuse; 2. A parameter which decides whether a meshTextures[i] = TextureLoader.FromFile hardware of software rendering is done. (device, extendedMaterials[i]. TextureFilename) 3. The class (e.g. the windows form) which Finally, after the mesh has been loaded and connects Direct3D with the application. initialized, the next step is to render it. For this purpose we have to call the private Ren- 4. Some ﬂags for setting the behavior of the der method. This method is called every time rendering pipeline. an object needs to be rendered. The function 5. Parameters for the presentation behav- is accountable for making the Mesh object ior. looking as desired. Inside the render method,
the rendering pipeline can be adjusted. To does not depend on the existence of particu-render the mesh itself, it is divided into sub- lar device objects (e.g. speciﬁc buttons). Ac-sets, one for each material that was loaded. A tion mapping simpliﬁes the input loop andloop needs to be run for rendering each ma- therefore reduces the need for custom gameterial subset. This loop can for example be drivers and custom conﬁguration user inter-used to make the following operations on the faces in games. (?)object: To capture the device input using Microsoft DirectInput, it is necessary to know from • Setting the material property which device the input has to be captured. (And, of course, if it is currently attached to • Draw the material subset with the the user’s computer). To do this, there is a mesh.DrawSubset() method enumeration of DeviceInstance objects which can be looped. This Enumeration serves the With the commands device.EndScene() following purposes:and device.Present() the end of the render-ing pipeline is indicated and the object will • Reports what DirectInput devices areget visible. available. Now, it is realized to load, render and • Supplies a globally unique identiﬁerpresent a mesh with a Direct3D device. If the (GUID) for each DirectInput device.graphic device supports Direct3D, the ben-eﬁts of hardware acceleration would be used. • Possibility to create a DeviceInstance ob-For a deeper look at this example and its code ject for each DirectInput device as it isthere are more information in (?) enumerated. With this the type of de- vice and its capabilities can be checked.3.2 Direct Input An example code fragment for this loopWith Microsoft DirectInput it is possible to would beestablish a direct communication between the foreach(DeviceInstance di in Manager.Devices)hardware drivers of an input device. Nor- string name = di.InstanceName;mally the communication would be provided Capturing the device input itself with Di-over a Windows message relay supported by rectInput can be done in three parts. Firstthe Microsoft Win32 API. With DirectIn- a device object that represents the input de-put the application retrieves the data directly vice needs to be created. After the creation offrom the device, even when the application is the device object the conﬁguration has to beinactive. Furthermore, with DirectInput all done. After the conﬁguration the device statekind of input devices with DirectInput drivers can be checked within the application. Thereare supported, as well as functionalities for will be callback functions registered which areForce Feedback. With Force Feedback it is called after the corresponding action from theallowed to send messages and properties to a user (e.g. pressing a button.)user’s DirectInput device to relay feedback in An example for the conﬁguration of a joy-the form of physical force. stick device could be: DirectInput implements the technology of foreach(DeviceObjectInstance doi inaction mapping. Action mapping applica- joystick.Objects)tions can retrieve input data without knowing joystick.Properties.SetRange(what kind of device is being used to generate ParameterHow.ById, doi.ObjectId, new InputRange(-5000,5000));it. DirectInput does not provide any specialsupport for keyboard devices or mouse navi- Furthermore, in the callback function Up-gation. Action mapping provides a possibil- dateJoystick() which is registered for joystickity to establish a connection between input events, the state of the buttons could be re-actions and input devices. This connection quested with this command:
byte buttons = joystick.CurrentJoystickState. by a dot and the operation name. For exam-GetButtons; ple sound.Play() starts playing the sound or As seen in the examples, DirectInput pro- sound.Format.* gives back additional soundvides an easy way to access an input device. information like the number of channels orThe beneﬁt of DirectInput devices is the di- samples per second.rect communication between hardware and When a secondary buﬀer is created withapplication, as long as the devices have corre- enabled ﬂag for the control3D property, Di-sponding hardware drivers. rectSound oﬀers the possibility to set an al- gorithm to simulate 3D spatial location of3.3 Direct Sound a sound. By default, no head-related trans-Microsoft DirectSound provides a system to fer function (HRTF) processing is performed,capture sounds from input devices and play and the location of the sound relative to thesounds through various playback devices. listener is indicated by panning and volumeThe input devices are realized in several sec- only. To optimize the 3D eﬀects for the user’sondary software or hardware sound buﬀers. system DirectSound also uses the speaker con-Software buﬀers keep their data always in ﬁguration. The SpeakerConﬁg property fromsystem memory and are mixed by the CPU. the sound device allows users to set up theHardware buﬀers can keep their data either sound system to the given speaker system byin system memory or, if the application re- switching the ﬂag to true, for example mono,quests it and resources are available, in on- stereo, 5.1 or 7.1 surround sound.board memory and are mixed by the sound One of the most important features of Di-card processor. Each of them contains a static rectSound is the possibility to apply real-timesingle sound or a dynamic stream of audio. audio eﬀects to the sounds in playback. AllWhen sounds in secondary buﬀers are played, the eﬀects are applied in hardware (that is,DirectSound mixes them in the so called pri- implemented by the sound card driver) when-mary buﬀer and sends them to the output ever possible, otherwise emulated via soft-device. Only the available processing time ware. The latter causes an important droplimits the number of secondary buﬀers that in the performance. There are 9 eﬀects exist-DirectSound can mix. The primary buﬀer is ing: Chorus, Compressor, Distortion, Echo,always a hardware buﬀer on the soundcard, Flangler, Gargle, Interactive3DLevelReverb,which means there is only one available and ParamEqualizer and WavesReverb. Each ofits memory size is limited. So the primary them has its own diﬀerent settings ((?)). Tobuﬀer holds the audio that the listener will implement an eﬀect on a buﬀer ﬁrst the eﬀecthear. Unlike the primary buﬀer there ex- ﬂag in the buﬀer description has to be set toists several settings for the secondary sound true and then the SecondaryBuﬀer.SetEﬀectsbuﬀer that can be changed. When creating method has to be used. This method takesa new secondary buﬀer the most important an array of EﬀectDescription structures thatcommand line is: describe the eﬀects. An eﬀect can only besound = new SecondaryBuffer (filePath, applied if the sound is not played. An exam-bufferDescription, SoundDevice) ple and a great summary of the Direct Sound As shown each secondary buﬀer has a buﬀer functionalities gives The Ultimate Manageddescription which contains one or more con- DirectSound 9 Tutorial 1 . It creates a smalltrol properties (like pan, volume, frequency, application which demonstrates all Direct-eﬀects, 3D) from the BuﬀerCaps structure Sound elements that have been described be-(?). To allow the control of these properties fore. A big disadvantage of Microsoft Direct-the corresponding ﬂags just have to be en- Sound is that it plays only wave and Pulseabled. According to those settings diﬀerent Code Modulation (PCM) ﬁles. Waveform au-operations can be executed by simply typing 1 http://www.codeproject.com/cs/media/Directin the name of the secondary buﬀer followed Sound9p1.asp
dio data consists of digital samples of the graphs have to be combined. For further in-sound at a ﬁxed sampling rate, whereas PCM formation on that topic see (?).the representation of an analog signal by a se-quence of numbers means. Managed Direct- 4 DirectX versus OpenGLSound does neither support compressed WAV OpenGL is an open standard, so everyone canformats nor audio formats with more than two use it for free. The fact that it is a standardchannels. Therefore AudioVideoPlayback is for graphics cards makes it running on everythe better choice for playback other formats operating system (as long as the OS consid-like mp3 for example. ers a graphic user interface). Compared to that, DirectX is an API developed by Mi-3.4 Audio Video Playback crosoft and consists of several subsystems likeThe AudioVideoPlayback application pro- shown in the previous chapters. So only thegramming interface (API) provides like the DirectX graphic component can be comparedolder DirectX version the playback and simple with OpenGL. Considering the DirectX soundcontrol of audio and video ﬁles. But Microsoft part, the analogue would be OpenAL, whichhas highly simpliﬁed the usage. All function- is an open audio standard. As already men-alities for the playback are packed into a sin- tioned, DirectX is a Microsoft product, so itgle dll-ﬁle. Thus the integration of a video can only be found on Microsoft operating sys-into an application became much easier, be- tems where it is preferred to OpenGL. Never-cause the user only needs to refer to the Mi- theless Microsoft cannot ignore OpenGL, be-crosoft.DirectX.AudioVideoPlayback names- cause it is an industrial standard. So thepace. The command: graphics card manufaturers support their cus-Video ourVideo = new Video("C:Example.avi") tomers with the latest OpenGL driver ver- sions and makes OpenGL usable under Win- creates an instance of the video class dows.and provides the possibility to control Considering OpenGL and DirectX boththe playback. For instance the method are quite similar and oﬀer equal functionali-ourVideo.Stop() stops the playback and ties. Anyway there still exist some diﬀerencesourVideo.Play() plays it. The size of the play- which are illustrated in table 1 on the follow-back window can be set by ourVideo.Size = ing page.new Size(480, 320). If the video ﬁle containsaudio, the Video.Audio property returns an 5 ConclusionAudio object that can be used to change thevolume or the stereo balance for the audio, for With Managed DirectX there exists a struc-example Video.Audio.Volume = 100. tured and easy to use low-level API. While in C++ the developer took great pain to sim-Audio ourAudio = new Audio("C:AudioFile.mp3") plify and integrate unmangaged C++ classes, The Audio object is similar to the Video with the .NET style and the design of names-object, but it supports only properties that paces, classes and properties, programmingrelate to audio, such as Volume and Balance. Managed DirectX is very comfortable forThat means the Audio class is primarily de- the developer. Compared to OpenGL, Di-signed for very simple playback scenarios, or rectX has some very important advantagesfor use with the Video class. Whenever a for game developers. With Managed DirectX,greater control over the audio playback is the developers also write more productive andneeded, it is better to use Microsoft Direct- therefore more code in less time and produceSound to play audio ﬁles. safer code. The managed DirectX AudioVideoPlay- After a short introduction in Managed Di-back API doesn’t support video converting. rectSound, DirectInput, Direct3D and Au-The only way to do this is using the old dioVideoPlayback APIs the reader should forunmanaged C++ code, where several ﬁlter- himself recognize the obviously advantages of
Features OpenGL DirectX object oriented no yes operating system many only all Windows versions useable drivers available for high end graphics cards nearly all graphics cards driver quality mostly bad often better than OpenGL driver usage university, research and development, CAD game industry documentations, tutorials, samples many less than for OpenGL version release every 5 years every 15 months trademark of Silicon Graphics Inc. Microsoft Table 1: The list compares the key features between OpenGL and DirectXmanaged DirectX. With regard to the comingWindows Vista where DirectX is an elemen-tal component, DirectX will play a more im-portant role even for application designers inevery part of future application development.