Programming on Windows 8.1: The New Stream and Storage Paradigm (Raffaele Rialdi)

Premium community conference on Microsoft technologies itcampro@ itcamp14#
The new stream and storage
paradigm
twitter: @raffaeler
email: raffaeler@vevy.com
blog: http://www.iamraf.net
Programming on Windows 8.1:

Huge thanks to our sponsors & partners!

• ActivateInstance API
to create objects
• Native types
• IVector<T> and
IMap<T, K>
• HRESULT for errors
.NET Framework Projection magics
• Familiar new
keyword
• Familiar BCL types
• List<T> and
Dictionary<T, K>
• Exceptions
WinRT .NET projection
But projections and mappings can’t fix all the frictions

• The need to enforce the async paradigm
–“fast and fluid” means a better UX
–Blocking apps are crappy!
• Solutions:
–async/await
–Promise pattern
The asynchronous problem

• async / await language feature is easy
• Promise is a ‘continuation’
Async quick recap
var content = await FileIO.ReadTextAsync(storageFile);
lblResult.Text = result;
var content = File.ReadAllText(path);
lblResult.Text = result;
Callback
Sync
Async
create_task(FileIO::ReadTextAsync(storageFile))
.then( [ = ] (String^ content) {
lblResult->Text = result;
}
C++ Promise

• The need to enforce a sandbox
–Protecting the system and user’s resources
from the Apps
• A sandbox is substantially a process with
low privileges
–App process is created with an very poor token
–Leverage the Integrity Levels
–A high number of APIs cannot be used
• sockets, System.IO, Streams, http api, …
The security problem

• System.IO
– Let you browse/navigate the disk
– Let you access absolute path
– Use the old-school synchronous pattern
• Storage API
– Must restrict the access to few well-known locations
• Can access arbitrary locations picked from the user
– Must never block the main thread
• Security and Async are two requirements not
allowing mapping or conversion
The I/O APIs have both those problems

• Application Package
– The place where EXE and DLL lives in
• Using MEF or an IoC? This is your code repository
– Xaml Uri: "ms-appx:///"
• If the manifest specified special folders …
… and the Store certification approved it
– KnownFolder contains the special folders
• Documents, Pictures, Music, etc.
– Note: Documents capability is hidden in VS and its
usage is permitted only to company accounts
What can an app access to?
Windows.ApplicationModel.Package.Current.InstalledLocation

• ApplicationData.Current.LocalFolder
– Local assets, no storage limits
– Xaml Uri: "appdata:///local/"
• ApplicationData.Current.RoamingFolder
– Synced content, limited in size
• ApplicationData.RoamingStorageQuota (100K)
– Xaml Uri: "ms-appdata:///roaming/"
• ApplicationData.Current.TempFolder
– Use only for short duration operations
– Xaml Uri: "ms-appdata:///temp/"
Application-specific folders

• Files shipped with the package
–Mark them as 'content' in Visual Studio
–"Copy Always" ensure fresh updated content
–From XAML use "ms-appx:///" or "/"
–From code use
• Windows.ApplicationModel.Package.Current.
InstalledLocation
• var img = new Uri("ms-appx:///Assets/Logo.png");
– Uri as only absolute in WinRT
Resources
<Image Source="/Assets/Logo.png" Height="100"/>
<Image Source="ms-appx:///Assets/Logo.png" Height="100"/>

DEMO:
A LAP AROUND STORAGE API

• There are still missing methods in some classes
of the WinRT API
• .NET users can use Extension Methods in the
shared code to fill the gap
– Add a method which is identical to the one
available in Win8.1
• Trick! Extension methods do not prevail over
real ones
– Win8.1 will execute the real WinRT one
– WP8.1 will execute the extension method
Windows Phone 8.1 API differences

• Do you remember the security constraints?
– Poor token, low integrity level, restricted APIs
• Sometimes we need to bypass the limitations
– Accessing a file outside the ones owned by the
App
• Pickers use the Runtime Broker to bypass the
limitation
– Accesses the resources for the requesting App
– User is always in control of the Broker requests
Pickers

Pickers and the Runtime Security Broker
Kernel services
WinRT
Runtime
Security Broker
COM / Win32 filtered Complete COM / Win32
Component
Device
access
Picker
Host
…
Component
Broker UI
Process
AppContainer
Process
Restricted token
Low Integrity Level
Standard token
Medium Integrity Level

WORKING WITH STREAMS

• Streams can block the client code
–Until you read/write, you will never know
• Working in chunks makes sense
–Files, network, … typically send/receive chunks
• Buffering limits the use of async to loading
or writing the buffer
• WinRT use a low level buffering concept
–IBuffer interface representing byte arrays
The need of buffering

• It's made only of Capacity and Length
– Creation via Buffer class (no read/write methods)
– Write access is possible via
• WindowsRuntimeBuffer.Create
• Low level access via COM interface
– IBufferByteAccess (C++ only)
– Conversions/mappings by:
IBuffer and friends
WinRT Type .NET Type .NET  WinRT WinRT  .NET
IBuffer Byte[] AsBuffer, CopyTo ToArray, CopyTo
IBuffer Byte N/A GetByte
IBuffer
Stream
MemoryStream
AsStream,
GetWindowsRuntimeBuffer AsStream
WindowsRuntimeBufferExtensions

• DataReader can read IBuffer
• IInputStream and IOutputStream
– Are backed up by IBuffer based buffers
IBuffer is used from other APIs
byte[] blob = Utiltities.CreateSampleBlob();
byte[] blob2 = new byte[blob.Length];
IBuffer buffer = blob.AsBuffer();
using (DataReader reader = DataReader.FromBuffer(buffer))
{
reader.ReadBytes(blob2);
}
Debug.Assert(Utiltities.IsSameDataAndDifferentReference(blob, blob2));
using (DataReader reader = DataReader.FromBuffer(buffer))
{
buffer2 = reader.ReadBuffer((uint)blob.Length);
blob2 = buffer2.ToArray();
}
Debug.Assert(!buffer.IsSameData(buffer2));
WindowsRuntimeBufferExtensions

Streams hierarchy
IInputStreamReference
IRandomAccessStream
Reference
IAsyncOperationWithProgress<IBuffer, uint>
ReadAsync(IBuffer buffer,
uint count,
InputStreamOptions options);
IAsyncOperationWithProgress<uint, uint>
WriteAsync(IBuffer buffer);
IAsyncOperation<bool> FlushAsync();
string ContentType {get; }
IRandomAccessStream CloneStream();
IInputStream GetInputStreamAt(
ulong position);
IOutputStream GetOutputStreamAt(
ulong position);
void Seek([In] ulong position);
bool CanRead { get; }
bool CanWrite { get; }
ulong Position { get; }
ulong Size { get; set; }
Sequential access interfaces
IInputStream IOutputStream
IRandomAccessStream
IRandomAccessStream
WithContentType
IClosable
(IDisposable)

• Maintain internally an IBuffer
• LoadAsync load the buffer
• ReadXYZ methods hit the buffer, not the stream!
– They can be safely synchronous
DataReader puts IBuffer and streams together
Stream
I/O
Latency
0 Size
LoadAsync LoadAsync LoadAsync
IBuffer
DataReader
ReadBoolean
ReadByte
ReadBytes
…
var file = await Package.Current
.InstalledLocation
.GetFileAsync(@"AssetsLogo.png");
using (var stream = await file.OpenReadAsync())
{
using (var reader = new DataReader(stream))
{
await reader.LoadAsync((uint)stream.Size);
Debug.WriteLine("0x" +
reader.ReadByte().ToString("x2"));
} // stream is disposed here ...
// ... unless reader.DetachStream() is not called
}

• Writes are against the IBuffer
• The stream is written only at StoreAsync time
• FlushAsync can be used to flush the underlying
stream
DataWriter use the same schema
Stream
I/O
Latency
0 Size
StoreAsync StoreAsync StoreAsync
IBuffer
DataWriter
WriteBoolean
WriteByte
WriteBytes
…
// ...
using (var writer = new DataWriter(stream))
{
writer.WriteInt32(1);
written = await writer.StoreAsync();
writer.DetachStream();
}
// ...

WinRT Type .NET Type .NET  WinRT WinRT  .NET
IInputStream Stream AsInputStream AsStreamForRead
IOutputStream Stream AsOutputStream AsStreamForWrite
IRandomAccessStream Stream AsRandomAccessStream AsStream
Converting streams between .NET and WinRT
byte[] blob = Utiltities.CreateSampleBlob();
byte[] blob2 = new byte[blob.Length];
using (var memStream = new MemoryStream())
{
await memStream.WriteAsync(blob, 0, blob.Length);
memStream.Seek(0, System.IO.SeekOrigin.Begin);
using (var reader = new DataReader(memStream.AsInputStream()))
{
await reader.LoadAsync((uint)blob.Length);
reader.ReadBytes(blob2);
}
}
Debug.Assert(Utiltities.IsSameDataAndDifferentReference(blob, blob2));

TIPS

• RandomAccessStream static class expose
helper methods to copy a stream
–CopyAsync, CopyAndCloseAsync
• C++ will take advantage of async/await
–they makes a huge difference!
• IBufferByteAccess
Useful tips to remember
Microsoft::WRL::ComPtr<Windows::Storage::Streams::IBufferByteAccess>
GetByteBuffer(Windows::Storage::Streams::IBuffer^ buffer)
{
Microsoft::WRL::ComPtr<Windows::Storage::Streams::IBufferByteAccess> ibba;
Microsoft::WRL::ComPtr<IUnknown> unknown(reinterpret_cast<IUnknown*>(buffer));
unknown.As<Windows::Storage::Streams::IBufferByteAccess>(&ibba);
return ibba;
}
HRESULT Buffer([out] byte** value);

Q & A

Programming on Windows 8.1: The New Stream and Storage Paradigm (Raffaele Rialdi)

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Programming on Windows 8.1: The New Stream and Storage Paradigm (Raffaele Rialdi)

Similar to Programming on Windows 8.1: The New Stream and Storage Paradigm (Raffaele Rialdi) (20)

More from ITCamp

More from ITCamp (20)

Recently uploaded

Recently uploaded (20)

Programming on Windows 8.1: The New Stream and Storage Paradigm (Raffaele Rialdi)