Check these out next
GDC Europe 2014: Unreal Engine 4 for Programmers - Lessons Learned & Things to Come
Apr. 19, 2015•0 likes•12,525 views
32 likes
Be the first to like this
Show More
views
Total views
0
On Slideshare
0
From embeds
0
Number of embeds
0
Download to read offline
Report
Software
A high-level overview of Unreal Engine 4, its game framework, the Slate user interface library, Unreal Motion Graphics, and Editor and Engine extensibility. Presented at GDC Europe in Cologne, Germany. Also includes bonus slides on concurrency and parallelism features, general tips for programmers and Epic's build and automation infrastructure.
Gerke Max PreussnerFollow
Video Game Developer at Epic GamesRecommended
East Coast DevCon 2014: The Slate UI Framework - Architecture & ToolsGerke Max Preussner
East Coast DevCon 2014: Concurrency & Parallelism in UE4 - Tips for programmi...Gerke Max Preussner
FMX 2017: Extending Unreal Engine 4 with Plug-ins (Master Class)Gerke Max Preussner
Scalability for All: Unreal Engine* 4 with Intel Intel® Software
Forts and Fights Scaling Performance on Unreal Engine*Intel® Software
Plug-ins & Third-Party SDKs in UE4Gerke Max Preussner
GDC Europe 2014: Unreal Engine 4 for Programmers - Lessons Learned & Things to Come
- Unreal Engine 4 for Programmers Lessons Learned & Things To Come Gerke Max Preussner max.preussner@epicgames.com
- Then… • “We ain’t gonna need it” • Fixed, rigid feature set • First Person Shooter centric • Programmer centric • Closed source • Monolithic, aged code base • Secret development schedule • PC & console focused • Millions of $ studio licenses Now… • “You might need it” • Plug-ins and customizable tools • Game type agnostic • Artist & game designer friendly • Public source • Modular code base, largely rewritten • Public feature voting on trello.com • Seamless multi-platform development • $19/month subscription Unreal Engine in 2014
- Unreal Engine in 2014
- Engine Overview
- The Big Picture Directories Configuration Modules Projects Root Directory • /Engine – All code, content & configuration for the Engine • /MyProject – All files for the game project ‘MyProject’ • /Templates – Templates for creating new projects Inside the /Engine and Project Directories • /Binaries – Executables & DLLs for the Engine • /Build – Files needed for building the Engine • /Config – Configuration files • /Content – Shared Engine content • /DerivedDataCache – Cached content data files (Engine only) • /Intermediate – Temporary build products (Engine only) • /Plugins – Shared and project specific plug-ins • /Saved – Autosaves, local configs, screenshots, etc. • /Source – Source code for all the things!
- The Big Picture Directories Configuration Modules Projects INI Files • Hold class default properties • Will be loaded into CDOs on startup • Organized in a hierarchy • Higher INIs override lower ones • Organized in sections • Key-value pairs within sections • Important ones exposed in Editor UI • Low-level access with FConfig
- The Big Picture Directories Configuration Modules Projects INI Files • Hold class default properties • Will be loaded into CDOs on startup • Organized in a hierarchy • Higher INIs override lower ones • Organized in sections • Key-value pairs within sections • Important ones exposed in Editor UI • Low-level access with FConfig Class Constructor BaseXXX.ini DefaultXXX.ini XXX.ini
- Editor & Project Settings
- The Big Picture Directories Configuration Modules Projects Modularity Promotes • Reusability • Extensibility • Maintainability • Decoupling • Efficiency Monolithic builds are still possible though!
- The Big Picture Directories Configuration Modules Projects Module Types • Developer – Used by Editor & Programs, not Games • Editor – Used by Unreal Editor only • Runtime – Used by Editor, Games & Programs • ThirdParty – External code from other companies • Plugins – Extensions for Editor, Games, or both • Programs – Standalone applications & tools Module Dependency Rules • Runtime modules must not have dependencies to Editor or Developer modules • Plug-in modules must not have dependencies to other plug-ins
- The Big Picture Directories Configuration Modules Projects Module Type UnrealEd App Game Runtime √ √ √ ThirdParty √ √ √ Plugins √ √ √ Developer √ √ X Editor √ X X Module usage across different types of applications
- The Big Picture Directories Configuration Modules Projects Important Modules for Beginners • Core – Fundamental core types & functions • CoreUObject – Implements the UObject sub-system • Engine – Game classes & engine framework • OnlineSubsystem – Online & social networking features • Slate – Widget library & high-level UI features
- The Big Picture Directories Configuration Modules Projects Interesting Modules for Advanced Programmers • DesktopPlatform – Useful APIs for Windows, Mac & Linux • DetailCustomizations – Editor’s Details panel customizations • Launch – Main loop classes & functions • Messaging – Message passing sub-system • Sockets – Network socket implementations • Settings – Editor & Project Settings API • SlateCore – Fundamental UI functionality • TargetPlatform – Platform abstraction layer • UMG – Unreal Motion Graphics implementation • UnrealEd – Unreal Editor main frame & features
- The Big Picture Directories Configuration Modules Projects Interesting Modules for Cool Features • Analytics – Collects usage statistics from Editor & games • AssetRegistry – Database for assets in Unreal Editor • GameLiveStreaming – Twitch Streaming • HeadMountedDisplay – HMD Support API (Oculus, etc.) • JsonUtilities & XmlParser – Handle Json & XML files • SourceControl – API for custom source control providers
- The Big Picture Directories Configuration Modules Projects Your Game Projects can… • Use Blueprints, C++ Code or both • Contain any number of modules & plug-ins • Be moved around and shared with others Project Templates to Get You Started • Blank (with or without sample content) • First Person Shooter • Side scroller, top-down & puzzle games • Flying & driving games • They all come in Blueprint and C++ flavors • More to come, and make your own!
- Project Templates & Samples
- Project Packaging & Deployment
- Game Framework
- Game Framework History UE1 and UE2 • Designed for First Person Shooters (FPS) • UnrealScript game scripting language UE3 • Kismet Visual Scripting added • More modular game classes • But still very FPS centric UE4 • UnrealScript replaced with Blueprints • Game genre agnostic • Lots of sample projects!
- UnrealScript vs. C++ vs. Blueprints UnrealScript was: • An object-oriented scripting language • Similar in syntax to C, C++, Java, but also somewhat different • Compiled to virtual machine byte code • Adding interesting features, such as States, Timers, Delegates Blueprints are: • A visual scripting system that is artist and designer friendly • Using the same virtual machine as UnrealScript • Almost as powerful as UnrealScript, and in many ways better C++ has: • Always been part of UE game programming • Tight bi-directional integrations with the virtual machine • Been greatly improved in UE4 to replace UnrealScript for coders C++ Blueprints VM
- So What Is The Game Framework? Overview Why Use It? Set Of Foundation Classes • Provide basic structure and functions of your game • You derive from the classes that you need • Fill in the details for your particular game What’s Missing? • Game genre specific implementations • No concept of health and death • No built-in classes for weapons, inventory, etc.
- So What Is The Game Framework? Overview Why Use It? Reduced Learning Curve • Hides the low-level details of game engines • Benefit from 20+ years of game developing experience • Many samples and tutorials for solving common tasks High Productivity • Focus on what really matters to you: making your game • Solves many tricky problems in games (movement, input, etc.) • Greatly reduces boilerplate code needed for all games Scalability • Smoothly grow your game & team from prototype to AAA
- Fundamental Concepts Actors Components Pawn Character Controller HUD GameMode What is an Actor? • Entity in a game level • Usually contains one or more Actor Components • Supports network replication for multiplayer games Things to know about Actors • Don’t have Location, Rotation (stored in root component) • Created with SpawnActor() method • Must be destroyed explicitly with Destroy() method • Will not be garbage collected during the game
- Fundamental Concepts Actors Components Pawn Character Controller HUD GameMode What is an ActorComponent? • Reusable functionality that can be added to an Actor • Contain the most interesting functionality & events Example Components: • Scene Component – Adds transforms and attachments • Primitive Component – Adds collision and rendering • UAudioComponent, UArrowComponent, UInputComponent, ULightComponent, UMeshComponent, UParticleSystemComponent and many more!
- Fundamental Concepts Actors Components Pawn Character Controller HUD GameMode PrimiviteComponent Event Examples • Hit – Called when bumping into a wall • Begin/EndOverlap – Walk into or out of a trigger • Begin/EndCursorOver • Clicked/Released • InputTouchBegin/End • Begin/EndTouchOver Also accessible in Blueprints!
- Fundamental Concepts Actors Components Pawn Character Controller HUD GameMode What is a Pawn? • An agent in the world • Optionally possessed by a Controller • Usually handles movement and input Things to know about Pawns • Good place to implement health • No movement or input code by default
- Fundamental Concepts Actors Components Pawn Character Controller HUD GameMode What is a Character? • Special Pawn that can walk • Comes with useful Components Things to know about Controllers • Handles collision • Client-side movement prediction • Much improvement from UE3
- Fundamental Concepts Actors Components Pawn Character Controller HUD GameMode What is a Controller? • A brain that can possess a Pawn • PlayerController: Represents a human player • AIController: Computes AI behavior for Pawns Things to know about Controllers • Possess one Pawn at a time • Can persist after possessed Pawn dies
- Fundamental Concepts Actors Components Pawn Character Controller HUD GameMode PlayerController • Interface for players to agents • Handles touches, clicks, keyboard • Showing/hiding mouse cursor • Good place for non-agent code • Menus, voice chat, etc. • Many other useful options
- Fundamental Concepts Actors Components Pawn Character Controller HUD GameMode What is a HUD? • Head-Up Display • Responsible for in-game UI Things to know about Controllers • Immediate-mode drawing API • No tools for building HUDs • UMG will provide all the things!
- Fundamental Concepts Actors Components Pawn Character Controller HUD GameMode What is a GameMode? • Implements the game rules • Configures default classes for Pawn, Controller, HUD, etc. • Can be accessed from anywhere (GetGameMode()) Things to know about GameModes • Only exists on the server and single player instances! • GameState is used to replicate game state to clients • Default game mode can be set in Project Settings • Per-map overrides in World Settings
- Other Important Concepts Input Collision Replication Axes & Actions • Can bind to mouse, keyboard, touch, etc. • Defined in Project Settings Input Processing Order 1. PlayerController 2. Level Blueprint 3. Possessed Pawn
- Other Important Concepts Input Collision Replication Various Collision Query Functions • Line traces (ray casts) • Geometry sweeps • Overlap tests Simple Collision • Box, sphere, capsule, convex • Authoring tools in Editor • Used for movement, physics, etc. Complex collision • Actual graphics triangles • Used for weapons, kinematics, etc.
- Other Important Concepts Input Collision Replication Transfer game state between Server and Clients • Remote Procedure Calls (RPC) • Data transfer (variables, structs, dynamic arrays) • Editor supports multiplayer PIE testing
- Game Framework Flowchart
- Slate UI Framework
- UE1, UE2 and UE3
- Slate Design & Principles Overview Features Concepts Tools Architecture • Written entirely in C++ • Platform agnostic (works on mobile and consoles, too!) • SlateCore module provides low-level functionality • Slate module contains library of common UI widgets • Does not require Engine or Editor modules Current Use Cases • Unreal Editor • Standalone desktop applications • Mobile applications • In-game UI
- Slate Design & Principles Overview Features Concepts Tools Styling • Customize the visual appearance of your UI • Images (PNGs and Materials), Fonts, Paddings, etc. • Customizable user-driven layouts Input Handling • Keyboard, mouse, joysticks, touch • Key bindings support Render Agnostic • Supports both Engine renderer and standalone renderers Large Widget Library • Layout primitives, text boxes, buttons, images, menus, dialogs, message boxes, navigation, notifications, dock tabs, list views, sliders, spinners, etc.
- Slate Design & Principles Overview Features Concepts Tools Declarative Syntax • Set of macros for declaring widget attributes • Avoids layers of indirection Composition • Compose entire widget hierarchies in a few lines of code • Uses fluent syntax for ease of use • Preferred over widget inheritance • Any child slot can contain any other widget type • Makes it very easy to rearrange UIs in code
- // Example custom button (some details omitted) class STextButton : public SCompoundWidget { public: SLATE_BEGIN_ARGS(SMyButton ) { } // The label to display on the button. SLATE_ATTRIBUTE(FText, Text) // Called when the button is clicked. SLATE_EVENT(FOnClicked, OnClicked) SLATE_END_ARGS() // Construct this button void Construct( const FArguments& InArgs ); }; // Button implementation (some details omitted) void STextButton::Construct ( const FArguments& InArgs ) { ChildSlot [ SNew(SButton) .OnClicked(InArgs._OnClicked) [ SNew(STextBlock) .Font(FMyStyle::GetFontStyle(“TextButtonFont")) .Text(InArgs._Text) .ToolTipText(LOCTEXT(“TextButtonToolTip", “Click Me!")) ]; ]; }
- Slate Design & Principles Overview Features Concepts Tools Widget Inspector • Visually debug and analyze your UI • Can jump directly to widget code in Visual Studio or XCode UDK Remote • iOS app for simulating touch devices on your PC • Remote server is a plug-in (enabled by default) • Primarily used for game development
- Demo
- Demo
- Going A Little Deeper State Updates Widget Roles Anatomy Attributes Polling instead of Invalidation • Avoids duplicate state data • Exception: Non-trivial data models (use caches instead) • Performance depends on number of visible widgets
- Going A Little Deeper State Updates Widget Roles Anatomy Attributes Fundamental Widget Types • SCompoundWidget – Can have nested child widgets • SLeafWidget – Does not contain child widgets • SPanel – Base class for layout panels Special Widgets • SWidget – Root base class for all widgets (do not inherit!) • SNullWidget – Empty default widget User Widgets • More efficient in terms of compile time
- Going A Little Deeper State Updates Widget Roles Anatomy Attributes Common Interfaces • Arguments – Widget parameters that do not change • Attributes – Parameters that are polled • Event handlers – Usually named ‘OnSomeEvent’ Common Internals • ComputeDesiredSize() - Calculates widget’s desired size • ArrangeChildren() - Arranges children within allotted area • OnPaint() – Draws the widget
- Going A Little Deeper State Updates Widget Roles Anatomy Attributes Common Attributes • Enabled state, Visibility, Hit testability • Tooltip Widget, Tooltip Text, Cursor Style • Horizontal & Vertical Alignment , Padding Attributes Can Be: • Constants, i.e. IsEnabled(false) • Delegate bindings, i.e. IsEnabled(this, &SMyWidget::HandleIsEnabled)
- Current In-Game UI Features HUD Canvas VP Widgets Game Menus FCanvas • Low-level C++ API for drawing directly to the screen • Has been part of Unreal Engine for many years • All functions are in FCanvas class • DrawText(), DrawTexture(), DrawTile(), etc. • Use AHUD.Canvas to access the canvas object HHitProxy • Provides basic interaction support for FCanvas • Create one hit proxy per interactive object • Hit proxy ID is sent to GPU for per-pixel hit tests
- Current In-Game UI Features HUD Canvas VP Widgets Game Menus UGameViewportClient • Allows usage of Slate widgets inside game view port • Use all features of Slate (except SWindow) • Add/RemoveViewportWidgetContent() Things to keep in mind • All added widgets will be layered on top of each other (SOverlay) • Widgets should use TWeakObjPtr for UObject references
- Current In-Game UI Features HUD Canvas VP Widgets Game Menus The Hard Way • Use FCanvas to draw your own menus • Not recommended The Custom Way • Use HUD Widgets to create any menu layout The Lazy Way • Use GameMenuBuilder for paged menus • FGameMenuPage - Single menu page • FGameMenuItem - An option in a menu page • Can be customized and styled • Mostly used for settings screens
- Unreal Motion Graphics (UMG)
- Unreal Motion Graphics Overview Scripting Upcoming One UI Solution To Rule Them All • Built on top of Slate • Adds real-time animation and transformation to widgets • Integrated with Blueprints • WYSIWYG Editor for artists and designers • No programming required (unless you want to) • Not officially released yet, but already in the code base
- Unreal Motion Graphics Overview Scripting Upcoming Adding Behavior to your UI • Designers should not have to write code! • Blueprints allow scripting of UI • You can still use C++ as well, but probably won’t
- Unreal Motion Graphics Overview Scripting Upcoming Currently working on: • Materials! • Style assets • 2D Transforms • Special effects • In-world 3D UI • Workflow polish
- Arbitrary 2D Transforms
- • Attachable UI Actor Components • Injected into render stream • Can intersect with world geometry • Render to texture (optional) • 3D Transformations In-World 3D User Interfaces
- Extensibility
- Why Do We Want Extensibility? Custom Requirements • Features that are too specific to be included in UE4 • Features that UE4 does not provide out of the box Third Party Technologies • Features owned and maintained by other providers • Scaleform, SpeedTree, CoherentUI, etc. Flexibility & Maintainability • More modular code base • Easier prototyping of new features
- How To Extend The Engine General Games Editor Plug-ins UE3: Engine Code Changes • Only accessible to licensees • Required deep understanding of code base • Merging Engine updates was tedious UE4: Extensibility APIs • Modules, plug-ins, C++ interfaces • Native code accessible to everyone • Also supports non-programmers
- How To Extend The Engine General Games Editor Plug-ins Blueprint Construction Scripts • Blueprints as macros to create & configure game objects • Activated when an object is created in Editor or game • Check out our excellent tutorials on YouTube!
- How To Extend The Engine General Games Editor Plug-ins Details View Customization • Change the appearance of your types in the Details panel • Customize per class, or per property • Inject, modify, replace, or remove property entries Menu Extenders • Inject your own options into the Editor’s main menus Tab Manager • Register your own UI tabs • Allows for adding entirely new tools and features
- Default Appearance Detail Customizations
- How To Extend The Engine General Games Editor Plug-ins Blutilities • Blueprints for the Editor! • No C++ programming required • Can fire off events that effect the Editor • Currently experimental, but already working • Scheduled for 4.5 (may change)
- Procedural Textures, L-Systems, Flipbooks
- How To Extend The Engine General Games Editor Plug-ins Overview • Extend the Engine, the Editor, or both • Are completely self contained • Can be enabled and disabled per project • Everything should be a plug-in! Examples • BlankPlugin, UObjectPlugin – Use these to start coding • Perforce & Subversion support for the Editor • Oculus Rift support • Movie players, Twitch Live Streaming, Slate Remote • And many more!
- How To Extend The Engine General Games Editor Plug-ins All Plug-ins • Will be loaded automatically on startup (if enabled) • Most not be dependencies of the Engine or other plug-ins Plug-ins with Code • Have their own ‘Source’, ‘Binaries’, ‘Intermediates’ folders • Can have one or more code modules • May declare new UObject and UStruct types • Seldom have Public header files in modules Plug-ins with Content • Have their own ‘Content’ folder • Configuration files (INIs) not supported yet
- How To Extend The Engine General Games Editor Plug-ins Descriptor Files (.uplugin) { "FileVersion" : 3, "FriendlyName" : "Paper2D", "Version" : 1, "VersionName" : "1.0", "CreatedBy" : "Epic Games, Inc.", "CreatedByURL" : "http://epicgames.com", "EngineVersion" : "4.2.0", "Description" : "Paper2D.", "Category" : "2D.Helpers", "EnabledByDefault" : true, "Modules" : [ // module definitions omitted ], "CanContainContent" : true }
- Vertex Snapping, Ocean Surface, Rollercoaster More on the Wiki!
- Questions? Documentation, Tutorials and Help at: • AnswerHub: • Engine Documentation: • Official Forums: • Community Wiki: • YouTube Videos: • Community IRC: Unreal Engine 4 Roadmap • lmgtfy.com/?q=Unreal+engine+Trello+ http://answers.unrealengine.com http://docs.unrealengine.com http://forums.unrealengine.com http://wiki.unrealengine.com http://www.youtube.com/user/UnrealDevelopmentKit #unrealengine on FreeNode
- Concurrency & Parallelism
- Synchronization Primitives Atomics Locking Signaling Waiting FPlatformAtomics • InterlockedAdd • InterlockedCompareExchange (-Pointer) • InterlockedDecrement (-Increment) • InterlockedExchange (-Pointer) 64- and 128-bit overloads on supported platforms
- Synchronization Primitives Atomics Locking Signaling Waiting // Example class FThreadSafeCounter { public: int32 Add( int32 Amount ) { return FPlatformAtomics::InterlockedAdd(&Counter, Amount); } private: volatile int32 Counter; };
- Synchronization Primitives Atomics Locking Signaling Waiting Critical Sections • FCriticalSection implements synchronization object • FScopeLock for scope level locking using a critical section • Fast if the lock is not activated Spin Locks • FSpinLock can be locked and unlocked • Sleeps or spins in a loop until unlocked • Default sleep time is 0.1 seconds
- Synchronization Primitives Atomics Locking Signaling Waiting Semaphores • Like mutex with signaling mechanism • Only implemented for Windows and hardly used • API will probably change • Use FEvent instead
- Synchronization Primitives Atomics Locking Signaling Waiting FEvent • Blocks a thread until triggered or timed out • Frequently used to wake up worker threads FScopedEvent • Wraps an FEvent that blocks on scope exit // Example for scoped events { FScopedEvent Event; DoWorkOnAnotherThread(Event.Get()); // stalls here until other thread triggers Event }
- High Level Constructs Containers Helpers General Thread-safety • Most containers (TArray, TMap, etc.) are not thread-safe • Use synchronization primitives in your own code where needed TLockFreePointerList • Lock free, stack based and ABA resistant • Used by Task Graph system TQueue • Uses a linked list under the hood • Lock and contention free for SPSC • Lock free for MPSC TDisruptor (currently not part of UE4) • Lock free MPMC queue using a ring buffer
- High Level Constructs Containers Helpers FThreadSafeCounter FThreadSingleton • Singleton that creates an instance per thread FMemStack • Fast, temporary per-thread memory allocation TLockFreeClassAllocator, TLockFreeFixedSizeAllocator • Another fast allocator for instances of T FThreadIdleStats • Measures how often a thread is idle
- Parallelization Threads Task Graph Processes Messaging FRunnable • Platform agnostic interface • Implement Init(), Run(), Stop() and Exit() in your sub-class • Launch with FRunnableThread::Create() • FSingleThreadRunnable when multi-threading is disabled FQueuedThreadPool • Carried over from UE3 and still works the same way • Global general purpose thread pool in GThreadPool • Not lock free
- Parallelization Threads Task Graph Processes Messaging Game Thread • All game code, Blueprints and UI • UObjects are not thread-safe! Render Thread • Proxy objects for Materials, Primitives, etc. Stats Thread • Engine performance counters
- Parallelization Threads Task Graph Processes Messaging Task Based Multi-Threading • Small units of work are pushed to available worker threads • Tasks can have dependencies to each other • Task Graph will figure out order of execution Used by an increasing number of systems • Animation evaluation • Message dispatch and serialization in Messaging system • Object reachability analysis in garbage collector • Render commands in Rendering sub-system • Various tasks in Physics sub-system • Defer execution to a particular thread
- Parallelization Threads Task Graph Processes Messaging FPlatformProcess • CreateProc() executes an external program • LaunchURL() launches the default program for a URL • IsProcRunning() checks whether a process is still running • Plus many other utilities for process management FMonitoredProcess • Convenience class for launching and monitoring processes • Event delegates for cancellation, completion and output
- Parallelization Threads Task Graph Processes Messaging Unreal Message Bus (UMB) • Zero configuration intra- and inter-process communication • Request-Reply and Publish-Subscribe patterns supported • Messages are simple UStructs Transport Plug-ins • Seamlessly connect processes across machines • Only implemented for UDP right now (prototype)
- Upcoming Features Critical sections & events • Better debugging and profiling support Task Graph • Improvements and optimizations UObjects • Thread-safe construction and destruction Parallel Rendering • Implemented inside renderers, not on RHI API level Messaging • UDP v2 (“Hammer”), BLOB attachments, more robust, breakpoints • Named Pipes and other transport plug-ins More lock-free containers
- Appendix A: Programming Tips
- Getting Started Tools: • Windows: Visual Studio, UnrealVS, Visual Assist X (recommended) • MacOS: XCode For everything else see: https://docs.unrealengine.com/latest/INT/Programming/QuickStart/
- Common Blockers Compiling Acronyms Entry Point Compiling is handled through UBT • UBT – Unreal Build Tool • Solution/Projects in Visual Studio and Xcode are a lie!
- Common Blockers Compiling Acronyms Entry Point Acronym Soup (and Code Names, too) • UBT – Unreal Build Tool • UHT – Unreal Header Tool • UAT – Unreal Automation Tool • UFE – Unreal Frontend • BP – Blueprint • CDO – Class Default Object • INI – Text Based Configuration File • Cooking – Optimizing game content • Lightmass, Persona, Cascade, Swarm and other tools • etc. pp.
- Common Blockers Compiling Acronyms Entry Point I Want To Understand the Engine - Where Is the Main Loop? • LaunchEngineLoop.cpp • It’s really complicated (and everybody hates it) • Please don’t bother with this – start with our tutorials!
- Unrealisms Type Names UObjects Basic Types Strings Macros We Use Prefixes for All Types • U – UObject derrived class, i.e. UTexture • A – AActor derrived class, i.e. AGameMode • F – All other classes and structs, i.e. FName, FVector • T – Template, i.e. TArray, TMap, TQueue • I – Interface class, i.e. ITransaction • E – Enumeration type, i.e. ESelectionMode • b – Boolean value, i.e. bEnabled Everything in Unreal is Pascal Case (Upper Camel Case) • Function names and function parameters, too • Even local and loop variables!
- Unrealisms Type Names UObjects Basic Types Strings Macros UObjects Work Around Limitations in C++ • Run-time reflection of class properties and functions • Serialization from/to disk and over the network • Garbage collection • Meta data • Also: Blueprint integration Decorate regular C++ Classes with Magic Macros • UCLASS – for class types • USTRUCT – for struct types • UFUNCTION – for class and struct member functions • UPROPERTY – for class and struct variables
- Unrealisms Type Names UObjects Basic Types Strings Macros // Example (not actual UE4 code – omitting some more advanced details) USTRUCT() struct FVector2D { UPROPERTY() float X; UPROPERTY() float Y; UFUNCTION () float GetLength() const; };
- Unrealisms Type Names UObjects Basic Types Strings Macros Fundamental Types • We don’t use C++ integer types (char, short, int, long, etc.) • Custom typedef’s for ints & strings in GenericPlatform.h (int32, uint32, uint64, TCHAR, ANSICHAR etc.) • Numeric type traits in NumericLimits.h Common Structures • FBox, FColor, FGuid, FVariant, FVector, TBigInt, TRange • And many more in Core module
- Unrealisms Type Names UObjects Basic Types Strings Macros Containers • TArray, TSparseArray – Dynamic arrays • TLinkedList, TDoubleLinkedList • TMap – Key-value hash table • TQueue – Lock free FIFO • TSet – Unordered set (without duplicates) • And many more in Core module Delegates • Unicast and multicast delegates • Also thread-safe variants
- Unrealisms Type Names UObjects Basic Types Strings Macros Smart Pointers • TSharedPtr, TSharedRef – for regular C++ objects • TWeakPtr – for regular C++ objects • TWeakObjPtr – for UObjects • TAutoPtr, TScopedPtr • TUniquePtr • Similar to boost:: & std:: implementations • Also thread-safe variants
- Unrealisms Type Names UObjects Basic Types Strings Macros String Types • FString – Regular string • FText – Localized string, used heavily in Slate UI • FName – String hash, used heavily in UObjects String Literals • TEXT() – Creates a regular(!) string, i.e. TEXT(“Hello”); • LOCTEXT() – Creates a localized string, i.e. LOCTEXT(“Namespace”, “Name”, “Hello”); • NSLOCTEXT() – LOCTEXT with scoped namespace, i.e. NSLOCTEXT(“Name”, “Hello”);
- Unrealisms Type Names UObjects Basic Types Strings Macros FNames are case-insensitive! • Can’t rename ‘MisSpelled’ to ‘Misspelled’ in the Editor • Can’t name a property ‘Blast’ if one ‘bLast’ exists
- Unrealisms Type Names UObjects Basic Types Strings Macros Logging • UE_LOG, also GLog->Logf() Assertions • check(), checkSlow(), ensure() Localization • LOCTEXT_NAMESPACE, LOCTEXT, etc. Slate (UI Framework) • SLATE_BEGIN_ARGS, SLATE_ATTRIBUTE, etc.
- Best Practices Guidelines Principles Coding Guidelines • Posted on http://docs.unrealengine.com • There are some inconsistencies in the code base • If in doubt, follow existing style in current code file Naming Conventions • Choose descriptive names that are as short as possible • Also for local and loop variables! • Avoid your own acronyms
- Best Practices Guidelines Principles General Principles • KISS, YAGNI • Composition vs. inheritance • Avoid tight coupling of code and modules • Many trivial instead of few complicated components Design Patterns • SOLID (especially S, O, L and I; DI is not elegant in C++) • Hollywood Principle (especially for Slate & game code) • GOF, EIP Methodologies • DDD, TDD (we support unit tests), AOP
- Appendix B: Build Automation
- Build Tools UHT UBT UAT Unreal Header Tool (UHT) • Written in C++ • Parses all C++ headers containing UClasses • Generates glue code for all Unreal classes & functions • Generated files stored in Intermediates directory
- Build Tools UHT UBT UAT Unreal Build Tool (UBT) • Written in C# (may convert to C++ in the future) • Scans solution directory for modules and plug-ins • Determines all modules that need to be rebuilt • Invokes UHT to parse C++ headers • Creates compiler & linker options from .Build.cs & .Target.cs • Executes platform specific compilers (VisualStudio, LLVM) Other UBT Features • Project file generation (GenerateProjectFiles.bat) • Remote Compilation (iOS, MacOS)
- Build Rules
- Dependency Graph
- Build Tools UHT UBT UAT Unreal Automation Tool (UAT) • Written in C# (may convert to C++ in the future) • Automates repetitive tasks through Automation Scripts • Build, cook, package, deploy and launch projects • Invokes UBT for compilation Other UAT Scripts • Analyze and fix up game content files • Code surgery when updating to new Engine versions • Distributed compilation (XGE) & build system integration • Generate code documentation • Automated Testing of code and content • And many others – you can add your own scripts!
- UBT, UHT, UAT Projects
- Build Automation SCC CIS Promotion Testing Source Code Control (SCC) • We use Perforce, but you don’t have to • Not just code, but also content and everything else • Also contains compiled binaries from build system (!) • Used to version many other things (Markting, QA, etc.) GitHub Integration • Check-ins are pushed to GitHub Master in near real-time • Script for converting GitHub pull requests to P4 CLs
- Build Automation SCC CIS Promotion Testing Continuous Integration System (CIS) • Verifies all check-ins of code and content • Grand Unified Build Process (GUBP) • Thousands of build tasks a day Backend Software • Used custom build server for several years • Experimented with Jenkins, but not scalable enough • Now using Electric Commander • Complex workflows with hundreds of jobs and sub-tasks Build Hardware • Virtualized farm of Windows and MacOS build servers
- GUBP
- Build Automation SCC CIS Promotion Testing Build Promotion • Selected successful CIS builds are tested by QA • Builds that pass QA will be promoted to stable • Selected stable builds become release candidates • Approved stable builds are released to public
- Build Automation SCC CIS Promotion Testing UnrealSync • Tool for artists to fetch the latest promoted build • Aware of P4 branches and projects • Notifies user when new promoted build is available
- Build Automation SCC CIS Promotion Testing Automated Testing • Simple and complex tests • Unit tests for C++, content verification • Parallel testing on multiple platforms & devices • Can also run from command line (as a Commandlet) Simple Tests • Single atomic test that can pass or fail • Unit tests for C++, feature tests for content • Examples: Play a map in PIE, verify text wrapping in Slate, etc. Complex Tests • Run same test code on a number of inputs • Examples: Load all maps in the Editor, compile all Blueprints, etc.
- // Example for a simple automation test that runs in a game IMPLEMENT_SIMPLE_AUTOMATION_TEST(FSetResTest, "Windows.SetResolution", ATF_Game) bool FSetResTest::RunTest( const FString& Parameters ) { FString MapName = TEXT("AutomationTest"); FEngineAutomationTestUtilities::LoadMap(MapName); int32 ResX = GSystemSettings.ResX; int32 ResY = GSystemSettings.ResY; FString RestoreResolutionString = FString::Printf(TEXT("setres %dx%d"), ResX, ResY); ADD_LATENT_AUTOMATION_COMMAND(FEngineWaitLatentCommand(2.0f)); ADD_LATENT_AUTOMATION_COMMAND(FExecStringLatentCommand(TEXT("setres 640x480"))); ADD_LATENT_AUTOMATION_COMMAND(FEngineWaitLatentCommand(2.0f)); ADD_LATENT_AUTOMATION_COMMAND(FExecStringLatentCommand(RestoreResolutionString)); return true; } Simple Automation Test
- Automation Test Demo
- Questions? Documentation, Tutorials and Help at: • AnswerHub: • Engine Documentation: • Official Forums: • Community Wiki: • YouTube Videos: • Community IRC: Unreal Engine 4 Roadmap • lmgtfy.com/?q=Unreal+engine+Trello+ http://answers.unrealengine.com http://docs.unrealengine.com http://forums.unrealengine.com http://wiki.unrealengine.com http://www.youtube.com/user/UnrealDevelopmentKit #unrealengine on FreeNode
Editor's Notes
- *** Prepared and presented by Gerke Max Preussner for GDC Europe 2014, August 11-13th]
- *** Prepared and presented by Gerke Max Preussner for GDC Europe 2014, August 11-13th]
- The very first things you will encounter after downloading Unreal Engine 4 are the various directories on your hard drive. Most of the files are located within the Engine directory, which contains everything needed to make and run games. Over time, you will add your own directories, one for each project you are creating.
- After looking at a few sample projects and perhaps creating your own first project, you may wonder how the Engine knows what to load and what to do. The answer to this are configuration files – we call them INIs. Many systems, such as the Engine, the Editor and games have their own INI file stack.
- There is not just one INI file, but multiple ones that are stacked hierarchically.
- In previous versions of Unreal Engine, users had to modify INI files quite frequently. This was tedious and error prone. In Unreal Engine 4 we are making a big push to expose all the important settings directly in the Editor. We hope that, in the future, users will no longer have to edit INI files directly.
- In previous versions of Unreal Engine all code was split into a small number of DLLs. Each DLL had a large number of responsibilities, which lead to a lot of very tightly coupled code that was difficult to maintain and extend. For Unreal Engine 4 we have rewritten large parts of the code base from scratch and reorganized it into hundreds of modules, each of which has one particular purpose. Modules can be compiled into individual DLLs or into static libraries to be linked into one large monolithic executable. The important point is that the code base is now much more organized and easier to use.
- We split our modules into various categories. Modules that are used by the Engine can be found in the Runtime directory. Modules that are used to build the Editor can be found in the Editor directory. We also have some standalone programs, which share their modules with the Editor. Those modules are located in the Developer directory. One important convention to remember is that Runtime modules must not have any dependencies to Developer or Editor modules.
- Like I said, there are now hundreds of modules in the code base, but you only need to know a few of them. When you first begin programming with Unreal Engine, you will frequently use features from Core, CoreUObject, Engine and Slate. Nearly all other modules in the code base are built on top of one or more of these.
- Once you are more familiar with programming games and tools for Unreal Engine, you may also be interested in other more advanced modules. Most of the remaining modules in the code base implement features for Unreal Editor. You can use them as a learning resource for implementing your own plug-ins and applications.
- There are also several modules that will make it easier for you to add cool new features to the Editor or your game. We recently added the GameLiveStreaming module, which allows your users to stream their game play to Twitch. There is also a module for 3D Head Mounted Display support – the Oculus plug-in uses it to add Oculus support to the Engine. There are also various utility modules, such as Json and XML helpers.
- All these modules are used in the Engine, the Editor and our tools. However, you will actually spend most of your time in your own projects. Projects are really easy to create in Unreal Engine 4. We provide a number of both Blueprint and C++ project samples for various types of games. More templates will be added over time, and you can even add your own.
- Here you can see the project template browser and the sample project browser.
- When your project is ready for testing, you can cook, package and deploy it. Cooking is the process of optimizing your game’s content for the desired target platforms. Packaging will arrange all content files and binaries in a way that is best for the target device, usually a .pak file. Deployment is the process of getting your packaged files onto the device. Cooking and packaging each time is very time consuming. We recommend that you use the Quick Launch feature instead, which utilizes Cook-On-The-Fly.
- Previous versions of Unreal Engine were primarily designed for first person shooter games, because that’s the kind of games Epic used to make. Unreal Engine 4 is much more game agnostic, and it ships with a ton of sample games for different genres. Game scripting was part of the engine from the very beginning. After spending several years of learning Unreal Script, you may be excited to hear that we got rid of it. Unreal Script has been replaced with Blueprints.
- Unreal Script was a completely separate programming language, although it was similar to C++ and Java. Blueprints are almost as powerful as UnrealScript. C++ has been augmented with more macros to give it more of the abilities that Unreal Script used to have.
- Unreal Engine comes with the so called Game Framework that makes it easy to create new games. It is a set of classes that provide all the basic functionality needed for most games. Since the Engine is now more game agnostic, it no longer contains concepts for health and weapons. You have to create those yourself for your particular game, or use one of the project templates as a starting point.
- Why should you use the Game Framework? Because it takes care of many of the low-level things that every game needs. Epic Games has been making games for over 20 years, and all that experience flows back into the framework. Let us worry about the technicalities – you focus on making cool games.
- There are a number of concepts in the Game Framework, but some are particularly important. The most important one is Actor, because every interactive entity in your game will be an Actor. Actors have built-in support for network replication, and they can be heavily customized with Actor Components.
- Actor Components are little reusable pieces of functionality that you can attach to Actors. They really implement the most interesting parts of what an Actor actually does and what it looks like in your game. Scene and Primitive Components are the most frequently used Actor Components. They provide things like appearance, collision and 3D transformations.
- Actor Components can generate events that are accessible in both Blueprints and C++. For example, when an Actor touches a wall or some other actor, you can react to it by monitoring the Hit event. Many Actor Components define their own events.
- The second most important concept is the Pawn. A Pawn is an entity in your game that can be controlled by a Player or some artificial intelligence. In earlier versions of Unreal Engine, Pawns were mostly used for humanoid creatures. In UE4 they can be anything, including space ships, submarines or eight legged robots.
- A Character is a special Pawn class that contains walking logic. It comes with useful components that make it easier to create humanoid game entities. Characters have client-side movement prediction code that allows for smooth multiplayer experiences. They have been improved greatly from Unreal Engine 3.
- Pawns do not control themselves. They are controlled by so called Controllers, which you can think of as the brain of a Pawn. Human players use a special controller, called the Player Controller. AI pawns can be controlled with AI Controllers. A Pawn may die in your game and disappear, but the Controller can stick around. For example, when a player respawns, we usually attach the same PlayerController to a new Pawn.
- PlayerControllers also process input, which is forwarded to the currently possessed Pawn. We also PlayerControllers to manage in-game menus, head-up displays and voice chat. It is a good place to put anything that has to do with human players.
- The HUD is responsible for drawing in-game UI, such as health bars and ammo counters. It uses a code-only API including functions like DrawTexture() and DrawText(). Unreal Motion Graphics will soon provide awesome tools for creating animated in-game UIs.
- Finally, the Game Mode contains your game’s state machine and game rules. It also determines which Pawn, Controller and HUD classes are used by your game. The Game Mode only exists on the server, because the server is the authority over what happens in the game. Clients learn about the game state through another class, called the GameState. The Game State forwards relevant details about the game to all clients.
- There are some other important concepts you should know about. The input system has support for a variety of input types, such as buttons, keys, analog controllers and mice. All input is processed in a certain order. This may affect the implementation of your game.
- The Engine provides a variety of functions for collision testing. We distinguish between simple and complex collsion. Simple collision is very cheap and uses 3D primitives for overlap checking. Complex collision is much more accurate, because it uses individual triangles, but it is also more expensive.
- Finally, Replication is the Engine’s mechanism to ensure that all players in a multiplayer game share the same experience. It is responsible for transferring game relevant data and events from the client to the server, and the server to all clients. If you wish to learn more about Replication, make sure to check out our next talk in Room XYZ.
- And here is a flow chart for the game framework classes. The only class we have not discussed is the PlayerCameraManager. It is part of the PlayerController and controls the movement of the player’s camera.
- In previous versions of Unreal Engine we used different third party UI frameworks, such as MFC, wxWidgets, Windows Forms and WPF. All these user interfaces were complicated, cluttered and not very customizable. In Unreal Engine 4 we have drastically changed how we approach user interfaces. We want our programs to be more intuitive, customizable, extensible and modern.
- The result of this effort is our own UI framework called Slate. Here you can see a screenshot of the latest Unreal Editor, which is entirely done in Slate.
- Slate is written entirely in C++ and runs on all platforms supported by Unreal Engine 4. We used it for the Unreal Editor, standalone desktop applications and tools, as well mobile apps. It can also be used for in-game UI, but I will talk about that in the second part of this presentation. Slate used to be just one C++ module, but we recently split it into two, and we may split up further in the near future.
- The appearance of widgets can be styled with a C++ based style system. We are currently working on making it better as part of UMG. The core of Slate takes care of all user input and translates it into events that your application can consume. Slate can render its UI with the Engine and without. Of course, Slate also comes with a large library of common UI widgets.
- We have a special application called Slate Viewer. It is a great way to learn about Slate. It shows the look and feel of all the different built-in widget types. It also demonstrates the built-in tab docking framework and the many different ways to create layouts.
- The two main concepts in Slate programming are Declarative Syntax and Composition. Declarative syntax is our way for creating UI widgets in C++ code. It is very compact and easy to read. Other UI frameworks heavily rely on widget inheritance, but Slate uses composition instead.
- Here is a simple contrived example of these two concepts. On the left we declare a new button widget using Slate’s declarative syntax. The first macro declares an attribute for the button text, and the second macro declares an event delegate for when the button is clicked. There are several other macros available for declaring other types of widget attributes. On the right we see the button’s construction code. Notice that STextButton inherits from SCompoundWidget, one of the fundamental widget types, and we use composition to populate the button’s content. In this case we are simply wrapping an SButton widget that contains an STextBlock as its content, and we forward any widget arguments that may have been passed in.
- We also have a couple tools to help you develop your Slate based user interfaces. Slate comes with a cool visual debugging tool called Widget Inspector – I will demo it shortly. There is also an app for iOS that allows you to simulate touch input while developing your game or application on a desktop PC.
- You can get the UDK Remote app for iOS on the Apple App Store. We will soon rewrite this app in Slate, so it also works on Android.
- [Widget Reflector Demo]
- With a little bit of practice you will be able to visualize in your mind what Slate code will look like when rendered. On the top left we see the Color Picker widget. Below is the Widget Reflector with focus on the Color Wheel widget inside the color picker. Notice how all three representations show the same hierarchical relationships (blue, red, green boxes).
- For updating its UI, Slate uses Polling instead of Invalidation. This avoid duplication of UI state data. It also means you have to be careful about the amount of work you are doing inside callback delegates. You can see in example for the color wheel that all properties are bound to delegates. The non-event delegates will be executed every tick.
- Although we mostly use composition for creating new widgets, SlateCore defines three truly fundamental widget types that are meant to be inherited. Most of your custom widgets will inherit from SCompoundWidget, or SLeafWidget if they have no children. Child slots must always contain a valid widget. SNullWidget is used to fill empty slots. User widgets were recently added and are still somewhat experimental.
- Widgets share a common anatomy. A widget’s appearance is usually controlled with Arguments and Attributes. You can also use setter and getter functions if you prefer. All widgets contain functions to compute their desired size, arrange their child widgets (if any) and paint themselves.
- While our previous presentation mainly focused on building UI for the Editor and standalone applications, this talk will be about game UI. The most common type of in-game UI is the Head-Up Display. The Engine currently provides two mechanisms to generate HUDs: Canvas and HUD Widgets.
- A better way for creating in-game UI is to use viewport widgets. The game’s viewport client has an API that allows for adding and removing regular Slate widgets. Since all UI is usualy displayed inside the game’s window, the use of SWindow is generally discouraged (unless you really want to open external windows from your game).
- And here is what the virtual joystick looks like in game. Notice the two circles at the bottom left and right, which can be used to control player movement on a mobile device.
- Another important type of game UI are in-game menus for settings or game controls. You can use either of the two techniques just mentioned, or you can leverage the new GameMenuBuilder to get up and running quickly. Most of our own in-game Uis are currently built with viewport widgets.
- So far we have talked about in-game UI that is entirely written in C++ and is, for the most part, rather static. We are currently building a new tool set on top of Slate that will add a much more artist centric workflow and provide cool features, such as UI animations and transformations. This tool set is called Unreal Motion Graphics. Let’s take a sneak peek at what we have in store for you…
- This is a screenshot of an early version of UMG.
- In the center you see the Preview Area. This is where you build and rearrange your user interface. You can select and modify various parts of your UI.
- The panel in the lower left shows the structural view of the UI you are creating. The currently selected widget is shown here as well.
- On the right is the Details panel. It shows the properties of the selected widget and allows you to modify them.
- The panel at the bottom is called the Sequencer. It lets you animate all properties of a widget over time. You can move the scrubber to preview how the animation behaves.
- Here it is in motion.
- In the upper left you can see the Widget Library. UMG ships with a large number of reusable widget types that you can drag and drop into your Preview Area. Of course, you can also create your own widgets and add them to the library.
- Here it is in motion.
- You can preview your user interface within your game at any time in the Level Editor. Simply hit the Play In Editor button, and your game and its user interface will be simulated in the Editor’s view port.
- I have shown you how to build a user interface with UMG, but how do we add some actual behavior to it? Unreal Engine uses a visual scripting system called Blueprints for a lot of scripting tasks. UMG is tightly integrated with Blueprints as well and gives designers full control over their UI’s behavior. It allows you to build interactive experiences without writing a single line of C++, C# or JavaScript.
- Here is an example of a simple Blueprint graph that is bound to the Click event of a button.
- We still have a lot of work to do to polish the workflow and make it fun. One thing we are working on right now is support for Engine Materials in your UI. On the right you can see a Material we built for a circular progress bar…
- … and this is what the result looks like.
- UMG will also support arbitrary 2D transformation, such as rotation, translation, scale and shear. You will also be able to arrange and layer your UI in 3D inside your game world. That will bring us one step closer to the IronMan and Minority report style interfaces.
- So, why do we want extensibility in the first place? Unreal Engine 4 is an extremely powerful toolset for game developers that ships with many features out of the box. However, many developers have special requirements that may require custom solutions. We also want to allow other companies to integrate their own specialized tools into our Engine.
- In Unreal Engine 3 and earlier, licensees had to change the Engine code directly in their own code branch. Of course, that brought all kinds of problems with it, and UDK users could not make changes at all. In Unreal Engine 4 we have a number of extensibility APIs. They are still pretty new, but already work and are continuously being improved.
- When it comes to games, Blueprint Construction Scripts are a great way for procedurally creating and configuring game objects. This is quite a large topic, and if you joined our Blueprint related talks today, you already know how they work and what they can do. Otherwise, check out our tutorials on YouTube!
- Here you can see a screenshot of a spline based road editing tool that was built entirely using Blueprint Construction Scripts by one of our technical artists.
- The Editor provides various APIs to add your own features and modify existing ones. The most frequently used ones are Details View Customization, Menu Extenders and the Tab Manager. Unfortunately, I don’t have the time today to go into programming details. Please take a look at the various examples in our code base!
- Let’s take a closer look at Detail Customization, which is used heavily in Unreal Editor. The Details Panel will generate a default appearance for all properties, but often those are not user friendly. With Detail Customization you can completely override how each property will be shown to the user.
- The Editor can also be extended with Blueprints, which do not require any C++ programming. With the so called Blutilities you can add entire new features to the Editor. Our technical artists have already built some great tools with it, and there will be more to come.
- Here are some examples of tools that were built entirely using Editor Blueprints.
- Plug-ins are the most powerful way to extend the Engine. They are completely self contained packages of code and/or content that can be enabled on a per project basis. The Engine ships with a number of plug-ins, including some templates that you can use as a starting point. Our Editor uses more and more plug-ins to implement its features, and ideally nearly everything would be a plug-in at some point, but we’re not quite there yet.
- Plugins can be located inside the Engine directory or inside your game project’s directory. Engine plug-ins will be shared between the Editor and all your projects. Project plug-ins only apply to the particular project. The Plugin Manager in the Editor allows users to see which plug-ins are currently installed.
- Plug-ins can have code or content or both. They will be loaded automatically by the Engine, if enabled. Content plug-ins are still work in progress and require some special care. Please visit our online documentation to learn more about the various plug-ins types.
- All plug-ins require a descriptor file to be present in their root directory. This file contains meta data that is used by the Engine and the Editor for managing and loading the plug-in. The example shown here shows the descriptor file of the Paper2D plug-in, which consists of multiple modules. Each module can be configured independently, but I have omitted the details to keep things readable.
- Paper2D is actually a completely new Editor tool that has been implemented entirely as a plug-in! It includes complex features, such as 2D spline & collision editing, custom view ports, and flipbook animation.
- We are also starting to see a number of plug-ins developed by the community. Check out the Community Wiki for examples and tutorials.
- Make sure to check out our extensive materials on the internet, all of which are available for free – even if you don’t have a subscription yet. Any questions?
- Games and applications that perform concurrent tasks on one or multiple CPUs or CPU cores are generally hard to write, proof correct, debug and maintain. Not only are there many concepts and mechanisms you have to learn, but also a wide range of very subtle issues that are often dependent on the operating system and the processor it is running on. Unreal Engine provides a range of features, helpers and tools to develop for multi-core computers and mobile devices more easily.
- At the lowest level, the Engine provides a number of synchronization primitives used to synchronize memory access across different threads. Commonly used primitives are atomic operations, mechanisms for locking and signaling threads, as well as for waiting on events from other threads. Atomic operations allow the CPU to read and write a memory location in the same indivisible bus operation. Although this seems to be a rather trivial feature, it is the foundation of many higher level multi-threading mechanisms. The main advantage of atomics is that they are very quick compared to locks, and they avoid the common problems of mutual exclusion and deadlocks. Unreal Engine exposes various atomic operations, such as incrementing and compare-and-exchange in a platform agnostic API. Most atomic operations operate on 32-bit values, but we also provide APIs for 64- and 128-bit values, if the particular platform supports them.
- Here is an example from our code base that uses atomics to implement a thread-safe counter. A normal increment instruction for integers may involve multiple bus operations that read the current value, increment it and write it back to memory. During these multiple steps it is possible – in principle and in practice – that the operating system suspends our thread, and another thread that also wishes to increment the counter modifies our value, causing that increment to be lost when our thread resumes. The use of ‘interlocked add’ in this example ensures that the increment operation cannot be interrupted. Because the counter value can change in asynchronous ways that the compiler cannot foresee, we use the volatile keyword to prevent optimizations of code that accesses the value.
- The main locking mechanisms used in Unreal Engine are Critical Sections and Spin Locks. Critical Sections provide mutually exclusive access to a single resource, usually the internals of a thread-safe class. In combination with Scope Locks they ensure that a given block of code can be executed by only one thread at a time. Compared to atomic operations, they can be a lot slower when the lock is accessed by more than one thread. Spin Locks also provide exclusive access, but instead of using the mutex facilities provided by the OS they sleep in a loop until the lock is removed. The sleep time is configurable (0.1 seconds by default), and for very small durations they may actually just do a no-op spin loop under the hood.
- Semaphores are similar to mutual exclusion locks, but they also contain a signaling mechanism. There is currently an implementation in FPlatformProcess, but it is not quite what one would expect, only implemented for Windows and only used in Lightmass right now. A better choice for signaling between threads would be the FEvent class…
- … which also uses a mutex under the hood. We frequently use events in worker threads to notify them about newly added work items or that we wish to shut them down. If you just want to send a one-shot signal from another thread, you can pass an FScopedEvent into it, and your current thread will stall when the event exits its scope until it is triggered.
- Most container types and classes in the Engine, including TArray and TMap are not thread-safe. We usually just manually add atomics and mutual exclusion locks in our code on an as-needed basis. Of course, locks are not always desirable, so we are also starting to implement basic lock-free containers that are thread-safe. A very useful lock-free container is TQueue, which is lock- and contention-free for SPSC and lock-free for MPSC. It is used in many places that transfer data from one thread to another. (We also implemented a lock-free MPMC queue using the Disruptor pattern at some point – it is not part of UE4 right now, but if you need it, send me an email!)
- We also provide some high level helper objects that are built on top of the synchronization primitives, some of which are shown here. The thread-safe counter we have already seen in the code example earlier. Thread-safe singletons are special singletons that create one instance of a class per thread instead of per process. Memory stacks allow for ultra-fast per-thread memory allocations. We also have a way to measure thread idle times that is always enabled, even in Shipping builds.
- Traditionally, the main work horse of multi-threaded programming are operating system threads. We expose those in the form of the FRunnable base class that you can inherit from. If your thread object inherits from FSingleThreadRunnable, it can also be used when multi-threading is disabled or unavailable. The thread pool from Unreal Engine 3 is still available as well. You can create your own thread pools or just use the global one in GThreadPool.
- Unreal Engine 4 uses threads for a number of things, but the most important ones are GameThread, RenderThread and StatsThread. All code that interfaces with UObjects must run on the GameThread, because UObjects are not thread-safe. The RenderThread allows us to perform rendering related tasks in parallel with the game logic. We recently also added the StatsThread that collects and processes a number of perforce counters for profiling.
- Threads are very heavyweight. Task Based Multi-Threading allows for more fine grained, lightweight use of multiple cores. A task is a small, parallelizable unit of work that can be scheduled to available worker threads that are managed by our Task Graph system. As the name suggests, tasks can also have dependencies on each other, so that certain tasks are not executed until other tasks completed successfully. The Task Graph is used by an increasing number of systems in the Engine, and we encourage you to use it, too.
- The Profiling Tools in the Editor come with a handy feature for visually debugging threads. It shows the cycle counter based statistics, as well as Task Graph usage.
- Unreal Engine 4 also has platform agnostic APIs for creating and monitoring external processes. You can start programs, launch web URLs or open files in other applications. If you need fine control over creating and monitoring external processes, you can use the FMonitoredProcess class.
- Messaging is another important architectural pattern for multi-threaded and distributed applications. Unreal Message Bus implements a message passing infrastructure that can be used to send commands, events or data to other threads, processes and even computers. Our main goal was to unify all the different network communication technologies we used for our tools in UE3, and to allow users to create their own distributed applications and tools without having to deal with low level technicalities, such as network protocols and how exactly messages get from A to B. Messages are simple Unreal structures (that currently require a tiny bit of boilerplate, but we will fix that soon). They can be sent between devices, even across the internet - if desired – and all our Unreal Frontend tools heavily rely on it. Messaging is a great way to get information out of your code classes in a loosely coupled way, and future use cases may include achievement systems, event processing for AI and real-time remote editing on game consoles and mobile phones.
- Debugging the flow of messages tends to be quite cumbersome in code, so the Messaging system comes with a visual debugging tool as well. It is not quite finished yet, but already functional.
- We are continuously working on improving and extending Unreal Engine 4. That UObjects are not thread-safe is sometimes a limitation, and we are currently working on allowing their construction and destruction on non-Game threads. We are also working on parallelizing the Renderer. This will be implemented not in the RHI API, but in the renderers themselves and utilize the Task Graph. The Messaging System will become very important for the new Editor tools we have scheduled for next year, so we will make it more powerful and easier to use as well. Much of our thread-safe code is currently lock based, and we have gotten good results with it, but we are also planning to add more generic lock-free containers.
- By the way, we have really great documentation on our website for getting you started, so I won’t go into any detail. Let me just say that we currently support development on Windows and MacOS. If you work in Visual Studio you should also install our UnrealVS plug-in.
- The first thing that throws off a lot of people is that compilation is not actually performed by Visual Studio or Xcode, but by our Unreal Build Tool. Visual Studio and XCode solutions are only generated for your convenience. We do this so that we can support different IDEs. In combination with Unreal Header Tool, it also performs a lot of code preprocessing. I will talk about this in more detail in another presentation later today.
- Another thing that is confusing at first is our generous use of acronyms. Don’t worry, you will quickly learn and get used to all of them.
- If you have experience writing your own little game from scratch, you may be tempted to locate our Engine’s main entry point in an attempt to learn something about how it all works. This is the wrong way to get started. You cannot possibly learn anything useful from this, because most of the code you will use on a daily basis are on a much higher conceptual layer… with about two million lines of code in between. Read and watch our programming tutorials instead!
- You will notice that we use prefixes for all our types, and there is no logical explanation for it. The story goes that Tim Sweeney, when he started working on Unreal Engine in his Garage, added FVector for 3d vectors with floating point components and a U-prefix for Unreal game classes. When other programmers joined they assumed that these were naming conventions… and that’s how snowflakes turn into yellow snow. We also use upper-case for all names in C++. It may really upset your OCD, but please don’t try fight this convention, as it will help keeping everything consistent. It may take a while, but you will get used to it.
- C++ is great for a lot of things, but it is also lacking useful features that are present in more modern programming languages, such as Java or C#. We would still like to use those features, so we implemented them on top of C++ with the help of dummy macros that will be parsed by Unreal Header Tool.
- Here is an example for how a C++ struct may be marked up in this way. From this, Unreal Header Tool will generate all the glue code, and you will most likely never have to see it.
- All our fundamental types are declared in the Core module. We have platform specific type definitions for numeric types and strings. We don’t actually use the standard numeric types that are part of the C++ standard.
- The Core module also contains many generic container types. We also have an implementation for various kinds of event delegate.
- There is also a library of smart pointer implementations. The most common ones are shared pointers, which also come with thread-safe variants. Shared pointers simplify the lifetime management of objects, and we use them often.
- String types can be a bit confusing at first, because we distinguish between regular and localized string objects. They each come with special macros for string literals. We also have so called FNames, which store string values in a global hash table. These are very heavily used in the UObject sub-system to identify classes, functions and properties.
- This is a screenshot of the Localization Editor, running in an Unreal Editor instance that is itself localized in Japanese.
- Oh, and FNames are case-insensitive. If you ever wondered why you can’t correct spelling errors in the Editor right now, that is the reason why. And if you think that’s crazy… well… that’s because it is!
- We also use macros for less complicated things, such as logging and assertions. These are some macros you will likely encounter when working on your project.
- We have an internal coding guidelines document that is not yet posted on the Wiki. Since so many programmers from inside and outside Epic are involved, you will find some inconsistencies in style. And, of course, choosing good names for modules, classes, functions and variables is by far the most difficult task for programmers, but also one of the most important ones.
- We also try follow proper software design principles more often now. You may already be familiar with some of them, such as KISS and YAGNI. One of the most important, but less known object-oriented principle is SOLID. [more details here, if time available] We also use common patterns from Gang-Of-Four and Enterprise Integration Patterns. Other methodologies we’re experimenting with are Domain-Driven Design, Test-Driven Development and Aspect Oriented Programming for cross-cutting concerns. Those are all huge topics in themselves. I will have to talk about them another time.
- The three main tools you will come across are Unreal Header Tool, Unreal Build Tool, and Unreal Automation Tool. If you visited one of our programming talks earlier today, you will already know that mark up our C++ classes and functions with special macros. UHT is the tool that parses your header files and converts these macros into magic glue code to enable powerful features like Run-time Reflection and Network Replication. All code generated by UHT is stored in the Intermediates directory, and you will likely never have to look at it, ever.
- Unreal Build Tool is our custom solution to compiling and building the Engine and your projects. When you compile in Visual Studio or XCode, we don’t actually use either one to perform the compilation, but everything goes through UBT instead. UBT also contains the logic for generating your project and solution files when you run the GenerateProjectFiles.bat batch file.
- UBT uses the special C# files *.Build.cs and *.Target.cs that you create inside your module directories to figure out what it needs to do. Here you can see the build and target rules files for QAGame, a special game that our QA team uses for internal testing.
- Another little known feature of UBT is the ability to generate dependency graph data that can be visualized. Here is a screenshot of a JavaScript based visualizer that shows a section of our code base.
- Last but not least, we have Unreal Automation Tool. Like UBT, it is currently written in C#, and you can think of it as batch files on steroids. We use it for a wide range of tasks that we wish to automate, and you can easily add your own scripts, too. The most frequently used automation tasks have to do with building, cooking, packaging, deploying and launching your projects. UAT also performs a number of other common tasks, such as generating documentation and automated testing.
- The full source code for all three tools is included with the Engine. You can find it in the Engine’s “Programs” directory on GitHub.
- At Epic we use these three tools to automate our daily production workflows. For our internal version control system we use Perforce, but we also support Subversion out of the box. Programmers in the community are working on plug-ins for other source code control systems, such as Git and Mercurial. In case you wonder how the Engine files get from our Perforce to GitHub: we have scripts for automatically converting between the two.
- Every single code or content check-in made to our Perforce server will be verified by our Continuous Integration System (CIS). It consists of a farm of build servers that run special software for making and verifying distributed builds. In the past we have created our own CIS software, but we recently switched to off-the-shelf products. With thousands of daily build tasks for several games across up to ten different platforms, only Electric Commander was able to meet our needs. We also experimented with open source systems, such as CruiseControl and Jenkins, which will likely be sufficient for small to medium sized studios.
- Our build engineers recently developed a special build process called the Grand Unified Build Process. It intelligently minimizes the number of modules that have to be built for the various build configurations and build targets. For most game studios a regular build script will suffice.
- Some of the builds that pass CIS are then picked up by our QA team to search for bugs and other problems. When a build is found to be stable enough for distribution, the QA team will mark it as ‘promoted’ in our build system.
- Promoted builds are made accessible to our artists through a small tool called UnrealSync. UnrealSync runs in the system tray and notifies our artists when a new promoted build is available. Artists can then use it to sync their branches to the latest promoted label.
- Our QA team has to perform a large number of very repetitive tests, especially during regression testing. One common, time consuming and extremely boring task is to ensure that all maps still load without errors in the Editor. Our Automation Test Framework can be used to automate these kinds of tasks. We are also starting to embrace Test-Driven Development (TDD) with the help of C++ unit tests. The Automation Test Framework can handle these as well.
- Here is a simple (and not very useful) automation test for setting and resetting the screen resolution.
- It’s not a cat, it’s a dragon! [demo of Automation Tests in the Editor’s Session Frontend UI]