The journey towards resumable downloads on Windows Phone

First of all, most of you would advise me to use the Background Transfers that are available in Windows Phone. And you are right! Yes you are.

But we can’t initiate a Background Transfer from a Background Agent. I still want do download files in the Background Agent. I know the standard Background Agents won’t help, because they can’t be run any longer than 25 seconds. However we still have the Resource Intensive Agents. And yes the constraints for Resource Intensive are even harder to meet. But when the constrains are met, it can run for 10 minutes. For your information these are the hard constraints to meet.

  • External power is required
  • A WiFi connection, or connected to the internet through a pc
  • The device’s battery level is greater than 90%
  • The screen is locked
  • No active phone call

 

Yes I know, in practice these constraints are only met when the phone is on charger during the night. But still it will happen. So I want to download large files, sometimes the download won’t even complete in the 10 minutes that are available. Which could be cause by a slow or medium WiFi/Internet connection in combination with very large files. So I would like to continue the unfinished downloads the next time the Resource Intensive agent gets to run. So I’ve been working on a solution.

Resumable Downloads

Yes I found a solution, you can find it at the end of this post, but you can read my journey as well. There are interesting parts in my journey, but that’s up to you to read or ignore.

Cheapest request to know the size of the download

I started with a little bit of experimentation, just a simple console app. I know that’s not completely equal to a Windows Phone app, specially a Windows Phone Background Agent. But it’s easy experimenting and because I wanted to use the HttpClient, I expected them to be largely equal. They were equal for the API parts, but not the implementation. But back to my idea, I want to know the size of the download I’m about to start. The cheapest way to get to know this, is by a HTTP HEAD request, instead of a HTTP GET. The HEAD request is identical to GET, but it doesn’t contain the body, being the file in my case.

So how do we do a HTTP Head request, and get the Content Length Header? It’s as easy as the following piece of code.

public static async Task<long?> GetRemoteSize(HttpClient client, Uri uri)
{
    var headRequest = new HttpRequestMessage(HttpMethod.Head, uri);
    HttpResponseMessage response = await client.SendAsync(headRequest);
    return response.Content.Headers.ContentLength;
}

 

It’s interesting code, however for Windows Phone you can immediately forget this. It works in the Command line app, but the ContentLength header is somehow not available for Windows Phone when doing a HEAD request, it’s always returning 0. This could be caused by the HttpClient implementation, or it’s maybe at a lower level, I don’t know.

If I modify the method above, to a HTTP GET request where I ask the method to complete as early as when the headers are read. I don’t want to wait for the whole 80 MiB file to be downloaded, just to know the size of the Download.

private async Task<long?> GetRemoteSize(HttpClient client, Uri uri)
{
    var headRequest = new HttpRequestMessage(HttpMethod.Get, uri);
    HttpResponseMessage response = await client.SendAsync(headRequest, HttpCompletionOption.ResponseHeadersRead);
    return response.Content.Headers.ContentLength;
}

That’s working, nice! So I can at least get the length of the download when I start the download. That will enable me to check if they bytes I’ve written to the file are the complete download. I will not make this call as a separate method call, like the method above. This was just experimentation.

The Range Header, or better the Header that enables a partial download aka resume

Let’s say we’ve been downloading for 10 minutes, but the download did not finish yet. We’ve already downloaded 45 MiB of the in total 80 MiB file. We want to continue with the download as soon as our Resource Intensive Agent gets run. So we need to tell the remote server which of the bytes we want from them. Here we have the HTTP Range header to help us out. It’s not available via method or properties, but we can set it very easily. So in below example I wanted to download all the content starting from 45 MiB. The HttpRequestMessage would read like this.

var readRequest = new HttpRequestMessage(HttpMethod.Get, originToDownload);
readRequest.Headers.Add("Range", string.Format("bytes={0}-", 45*1024*1024));

 

The range that you add to the Range header can end with a dash when you want to get all the remaining bytes. This works fine on the Console application, but it fires an InvalidOperationException telling “Nullable object must have a value” on Windows Phone. Sounds like the upper range being empty isn’t supported on Windows Phone.

I did investigate it a little bit, but not too much. I now fill the upper part of the range with long.MaxValue and that seems to work.

var readRequest = new HttpRequestMessage(HttpMethod.Get, originToDownload);
readRequest.Headers.Add("Range", string.Format("bytes={0}-{1}", 45*1024*1024, long.MaxValue));

Be aware that the ContentLength header you’re getting in the response will contain the actual length of bytes that are sent. So in the above case it will show the amount of bytes remaining 45 MiB done of the 80 MiB, means 35 MiB in the Content and 36700160 (35*1024*1024) as ContentLength.

The Solution

If you combine the Range header with the knowledge we gained about getting the ContentLength we can write a method like this:

private async Task<bool> Download(HttpClient client, Uri originToDownload, string targetToDownloadTo)
{
    var readRequest = new HttpRequestMessage(HttpMethod.Get, originToDownload);

    StorageFile fileToDownloadTo = await ApplicationData.Current.LocalFolder
        .CreateFileInPathAsync(targetToDownloadTo, CreationCollisionOption.OpenIfExists);
    using (Stream writeStream = await fileToDownloadTo.OpenStreamForWriteAsync())
    {
        writeStream.Seek(0, SeekOrigin.End);
        long currentLength = writeStream.Length;
        if (currentLength > 0)
            readRequest.Headers.Add("Range",
                string.Format("bytes={0}-{1}", currentLength, long.MaxValue));
        using (HttpResponseMessage response =
            await client.SendAsync(readRequest, HttpCompletionOption.ResponseHeadersRead))
        {
            long? remoteSize = response.Content.Headers.ContentLength;
            long totalRead = 0L;
            if (remoteSize == 0)
            {
                //All bytes have already been read.
                return true;
            }
            using (Stream stream = await response.Content.ReadAsStreamAsync())
            {
                var buffer = new byte[1024*64];
                int bytesRead;
                do
                {
                    bytesRead = stream.Read(buffer, 0, buffer.Length);
                    writeStream.Write(buffer, 0, bytesRead);
                    await writeStream.FlushAsync();
                    totalRead += bytesRead;
                } while (bytesRead != 0);
            }
            if (remoteSize == totalRead)
            {
                return true;
            }
            return false;
        }
    }
}

If you want to copy paste, you will need the helper method to CreateFileInPathAsync.

public static async Task<StorageFile> CreateFileInPathAsync(this StorageFolder parentFolder, string path,
                                                            CreationCollisionOption collisionOption)
{
    string folderPath = path.Substring(0, path.LastIndexOf('/'));
    string fileName = path.Substring(path.LastIndexOf('/') + 1);
    StorageFolder targetFolder = await EnsureFolderExistsAsync(parentFolder, folderPath);
    if (targetFolder != null)
    {
        return await targetFolder.CreateFileAsync(fileName, collisionOption);
    }
    return null;
}

public static async Task<StorageFolder> EnsureFolderExistsAsync(this StorageFolder parentFolder, string path)
{
    StorageFolder currentFolder = parentFolder;
    var pathElements = path.Trim('/').Split('/');
    foreach (string name in pathElements)
    {
        currentFolder = await currentFolder.CreateFolderAsync(name, CreationCollisionOption.OpenIfExists);
    }
    return currentFolder;
}

Memory Usage

It’s very nice we now have a way to download large files, and when required resume them. However be careful around memory usage. In my application I already have the SQLite db-engine in memory, so there’s not much memory left. So even though you can download multiple files after each other the memory usage increases. I stop downloading more files when I’ve less than 1.5 MiB of memory left. Not sure if this will be the sweet spot for your app, I’ve tried keeping that thresshold smaller but got Out of Memory exceptions quite often. So you’ve been warned.

Final notes

Now I created a method to download large files and even resume downloads, I can’t recommend to use it in every place. Use it where appropriate and where the better alternative Background Transfers can’t be used. I can’t think of any place other than a background agent to use this. But there might be others.

In the end this is something that can be used in any other .NET application, with a small set of modifications maybe to suit the targeted platform.

Better reaction on the activation by the Universal Volume Control

As most of you know, you can always start or activate the audio application from which the background audio originates by tapping the Universal Volume Control. Have you noticed that it rarely activates the screen you expect? I did!

To be even honest my own audio playing app P | Cast failed here as well. Specially with the secondary tiles. When you’ve activated the app and playback with from the car mode secondary tile, the reactivation by tapping the Universal Volume Control would always launch the main page.

How can we change the page that’s activated through the Universal Volume Control?

We can’t change it really. It’s just how the Universal Volume Control works. It activates the main form. However, it does add something, an URI parameter. It’s adding the parameter tag and the value is equal to the tag you can provide to the Audio Track you ask the Background Audio to play.

So very interesting we should be able to for example activate a details page that’s belonging to the playing track. I did something differently, I wanted to make sure that the last tile that’s used to activate the app (primary or secondary) is used when activated by the Universal Volume Control.

My app already contains an implementation of the UriMapperBase which allows to convert a request URI into a new URI based on my own mapping rules. So I only had to add a specific rule when the URI contains tag=. I would like to remind you that the implementation inside the if-statement can be modified to your own needs.

internal class AssociationUriMapper : UriMapperBase
{
    public override Uri MapUri(Uri uri)
    {
        Debug.WriteLine("MapUri {1}: {0}", uri, DateTime.Now.ToLongTimeString());
        if (uri.ToString().Contains("tag="))
        {
            App currentApp = (App)Application.Current;

            Debug.WriteLine("Activation through Universal Volume Control, so map to: {0}", currentApp.LastActivationUrl);
            if (currentApp.LastActivationUrl != null)
                return currentApp.LastActivationUrl;
            return uri;
        }
        // Eventually add some other mapping rules.
        // Otherwise perform normal launch.            
        return uri;
    }
}

Multiple URI schemes in one app, is it possible?

This week Robert Irving, maker of Car Starter wrote a very interesting article about Community Uri Schemes. He specifically proposes a generic URI Scheme for Podcast apps. Interestingly I added a similar but app-specific URI scheme to P | Cast in the most recent release.

There are currently two apps that actually make use of the custom URI scheme I created, so I can’t really replace it with a generic URI scheme. So I was wondering if it was possible to add multiple URI schemes to one app. I googled around, but couldn’t really find anything. So there wasn’t much left, just try, and see what happens.

So I opened the WMAppManifest.xml and changed the Extensions element to read like this:

<Extensions>
  <Protocol Name="pcast" NavUriFragment="encodedLaunchUri=%s" TaskID="_default" />
  <Protocol Name="wp-podcast-play" NavUriFragment="encodedLaunchUri=%s" TaskID="_default" />
</Extensions>

 

Then I tested if I could launch the app via both URIs. Don’t forget to modify the UriMapper class you created to also support the additional URI scheme, otherwise the app will hang.

So you can actually add multiple URI schemes to one app. That’s interesting, so I could add the community URI scheme without removing the propriety URI scheme. Now it’s time to think about Robert Irving suggestion, and maybe propose some changes, but that’s for another post.

Update:

According to Scott Lovegrove it’s possible to have up to ten URI schemes for your app. That should be enough I would say.

P | Cast now has a custom URI scheme

When you want your app to become more popular you could enable other apps to launch your app. You can even go beyond just launching and even activating special features.

As you’ve probably read, P | Cast now has a Car Mode UI built in. So there are apps that are focused on Car Mode, which would want to launch P | Cast in it’s Car Mode. Even more, they might want to even start playing the music directly after launching the app.

That’s why adding a custom URI scheme for P | Cast was on my mind from the beginning, and now it’s there.

The basics of launching P | Cast

You can launch P | Cast in it’s normal UI, but also in Car Mode.

/// <summary>
/// Launch P | Cast in Normal UI
/// </summary>
private void Normal()
{
    Windows.System.Launcher.LaunchUriAsync(new Uri("pcast:NormalPlayer"));
}

/// <summary>
/// Launch P | Cast in Car Mode
/// </summary>
private void Car()
{
    Windows.System.Launcher.LaunchUriAsync(new Uri("pcast:CarPlayer"));
}

 

Launch P | Cast and start playing immediately

When we go beyond the basic launch, we can add a couple of additional URI elements to enable immediate playback on app launch.

To start playing the most recent track upon app launch use the below URIs.

/// <summary>
/// Launch P | Cast in Normal UI and immediately start playing the most recent track
/// </summary>
private void NormalPlayRecent()
{
    Windows.System.Launcher.LaunchUriAsync(new Uri("pcast:NormalPlayer?Play=Recent"));
}

/// <summary>
/// Launch P | Cast in Car Mode and immediately start playing the most recent track
/// </summary>
private void CarPlayRecent()
{
    Windows.System.Launcher.LaunchUriAsync(new Uri("pcast:CarPlayer?Play=Recent"));
}

 

And if you want the app to start playing a random track, use these.

/// <summary>
/// Launch P | Cast in Normal UI and immediately start playing a random track
/// </summary>
private void NormalPlayRandom()
{
    Windows.System.Launcher.LaunchUriAsync(new Uri("pcast:NormalPlayer?Play=Random"));
}

/// <summary>
/// Launch P | Cast in Car Mode and immediately start playing a random track
/// </summary>
private void CarPlayRandom()
{
    Windows.System.Launcher.LaunchUriAsync(new Uri("pcast:CarPlayer?Play=Random"));
}

 

If you’ve questions about integration of P | Cast in your app, please leave a message in the comments or contact me on the P | Cast twitter account.

Make your app friendly to use in the car!

Yes the title already says what I want to say. When you have an app that is interesting to use in the car, such as an audio player you should provide an user interface that can be used in the car. All buttons have their extended touch area, but for in car operation the standard button size is too small I think.

34

Alternative user interface with large buttons for in car usage

So I thought about creating a user interface for P | Cast that’s not packed with all the functionality in the normal user interface, but meant to be operated in car or even with your gloves on.

78

As you can see the main functionality play/pause is operated by a very large button. Other functionalities, like an overview of the downloads is not available.

My recommendation for your app if your app is something that should be operable in the car do something similar.

Make the car mode an alternative entry point

Besides the alternative user interface, you can go one step further. You can make the alternative ui an alternative entry point. Add a secondary tile to your app which can activate your car mode from a tile. Something like this.

var tile = new StandardTileData
            {
                Title = AppResources.TileCarTitle,
                BackgroundImage = new Uri("Assets/Tiles/InCarFlipCycleTileMedium.png", UriKind.Relative),
            };
ShellTile.Create(new Uri("/Pages/InCarPlayer.xaml", UriKind.Relative), tile);