Async File Uploads with MVC, WebAPI and Bootstrap

This is a ‘how-to’ aide-memoire on my research on file uploading within MVC/client applications.

Our application has a form with a couple of fields and we want to allow the user to upload a file as part of the form submission. Uploading a file in the old WebForm days was pretty simple. But now we want asynchronous uploads and send via WebAPI, so we need to handle things better.

My requirements are as follows:

  1. We don’t want a traditional post-and-redirect model. It must use an async upload method
  2. We need the whole form (file+form fields) in one go: file-only uploads are of no use in this case
  3. We might want KnockoutJS integration if appropriate..
  4. We don’t want the uploaded data stored in files on the webserver: we want memory streams written to our database object
  5. Will be on a Bootstrap page so ideally needs to blend in (although Bootstrap 2.3 does not itself render these controls)

Server-Side

Initial Investigation

There are two parts to the operation: server-side and client-side.

A bit of Googling for the server-side gets us to a blog post by Henrik Nielsen (from 2012):

Asynchronous File Upload using ASP.NET Web API. This ticks two of our boxes:  first it uses WebAPI to handle the post, and secondly it does so asynchronously. It’s not Knockout-bound but that’s not such a big issue at this stage. The example he provides runs as a console app (!) self-hosted web server, so it’s a bit.. interesting.

Except it won’t work. It’s an old beta-version sample with no complete version and it’s missing a lot of using directives, and.. the list goes on.

There are other versions of this around and most of them either don’t work or store the uploads to files, which is contrary to requirements, and not something I’d ever do. I needed something that used memory or streams.

Knockout

I also found a Knockout solution at Khayrov’s Github project. This was a great tool using Knockout to upload files on the client – and in theory then post onward to the server. The only problem was it used the FileAPI which won’t work on most older browsers. Strictly speaking KO isn’t needed for posting but it helps if the form has behaviours on it.

Back to WebAPI and Memory-only

Much more Googling and I found MultipartMemoryStreamProvider which reads the form data into memory. One issue here is it treats normal form values and files the same, so you have to differentiate. A quick Google suggests a solution. This looks like a much better approach, and uses async methods. I amended the code to make it easier to get the files out directly, by parsing the HttpContent list into a list of files. I have attached a source listing at the end of this article, along with a sample Post method that uses this.

Client Side

That seems to take care of the server side. The client side for testing was a standard html form and a submit method, which is not very async. I needed to use a jQuery .ajax call to post the form. Posting forms via AJAX with file uploads isn’t supported directly with .ajax, but after several experiments I settled on the jQuery.Form plugin. This seems to work very well on the versions I tested: IE8, IE10, Chrome v28, and Firefox v22.

I did have a couple of errors on Firefox but that was because I tried to upload a 22MB file as a test: IIS was set to accept the default maximum upload size of 4096K, which is only 4MB. Setting maxRequestLength to a higher value fixed this:

<configuration>

  <system.web>

    <httpRuntime maxRequestLength="32768" />

  </system.web>

</configuration>

The only thing the form plugin does not do well is handle errors: there seems to be no error code available in the .ajax model. We can handle an error event from the jQuery ajax call, but I wasn’t able to determine which error code was being returned.

Binding the Form Values to the Model

As we are processing the multi-part form ourselves, we are getting a list of file and a list of form values. This means we have to manually process the values. We could extract one-by-one using the name, but we should really use a Model Binder..

It does not seem to be possible to use the DefaultModelBinder from MVC as this takes controller contexts as the constructors, so I added a simple one to the provider. This uses a simple reflection-based name match and set operation.

Source

Source code of the amended WebAPI post and supporting classes:

using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Collections.Specialized;
using System.IO;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Threading.Tasks;
using System.Web.Http;
using System.Web.Mvc;

namespace MyWebApp.Api
{
    public class UploadController : ApiController
    {

        /// 
        /// Accept form post with files
        /// 
        /// 
        [System.Web.Mvc.HttpPost]
        public async Task Post()
        {
            if (!Request.Content.IsMimeMultipartContent("form-data"))
                throw new HttpResponseException(HttpStatusCode.BadRequest);

            var provider = await Request.Content.ReadAsMultipartAsync(new InMemoryMultipartFormDataStreamProvider());

            //access form data
            FormCollection formData = provider.FormData;

            //access files
            IList fileContentList = provider.Files;

            var fileDataList = provider.GetFiles();

            // get the files 
            var files = await fileDataList;

            // formulate the response
            if (files.Any())
                return string.Join("; ", 
                    (from f in files 
                     select string.Format("uploaded: {0} ({1} bytes = '{2}')", f.FileName, f.Size, f.ContentType)).ToArray());
            else
                return "No files attached";
        }

       

        public class InMemoryMultipartFormDataStreamProvider : MultipartStreamProvider
        {
            private FormCollection _formData = new FormCollection();
            private List _fileContents = new List();

            // Set of indexes of which HttpContents we designate as form data
            private Collection _isFormData = new Collection();

            /// 
            /// Gets a  of form data passed as part of the multipart form data.
            /// 
            public FormCollection FormData
            {
                get { return _formData; }
            }

            /// 
            /// Gets list of s which contain uploaded files as in-memory representation.
            /// 
            public List Files
            {
                get { return _fileContents; }
            }

            /// 
            /// Convert list of HttpContent items to FileData class task
            /// 
            /// 
            public async Task GetFiles()
            {
                return await Task.WhenAll(Files.Select(f => FileData.ReadFile(f)));
            }

            public override Stream GetStream(HttpContent parent, HttpContentHeaders headers)
            {
                // For form data, Content-Disposition header is a requirement
                ContentDispositionHeaderValue contentDisposition = headers.ContentDisposition;
                if (contentDisposition != null)
                {
                    // We will post process this as form data
                    _isFormData.Add(String.IsNullOrEmpty(contentDisposition.FileName));

                    return new MemoryStream();
                }

                // If no Content-Disposition header was present.
                throw new InvalidOperationException(string.Format("Did not find required '{0}' header field in MIME multipart body part..", "Content-Disposition"));
            }

            /// 
            /// Read the non-file contents as form data.
            /// 
            /// 
            public override async Task ExecutePostProcessingAsync()
            {
                // Find instances of non-file HttpContents and read them asynchronously
                // to get the string content and then add that as form data
                for (int index = 0; index < Contents.Count; index++)
                {
                    if (_isFormData[index])
                    {
                        HttpContent formContent = Contents[index];
                        // Extract name from Content-Disposition header. We know from earlier that the header is present.
                        ContentDispositionHeaderValue contentDisposition = formContent.Headers.ContentDisposition;
                        string formFieldName = UnquoteToken(contentDisposition.Name) ?? String.Empty;

                        // Read the contents as string data and add to form data
                        string formFieldValue = await formContent.ReadAsStringAsync();
                        FormData.Add(formFieldName, formFieldValue);
                    }
                    else
                    {
                        _fileContents.Add(Contents[index]);
                    }
                }
            }

            /// 
            /// Remove bounding quotes on a token if present
            /// 
            /// Token to unquote.
            /// Unquoted token.
            private static string UnquoteToken(string token)
            {
                if (String.IsNullOrWhiteSpace(token))
                {
                    return token;
                }

                if (token.StartsWith("\"", StringComparison.Ordinal) && token.EndsWith("\"", StringComparison.Ordinal) && token.Length > 1)
                {
                    return token.Substring(1, token.Length - 2);
                }

                return token;
            }
        }

        /// 
        /// Class to store attached file info
        /// 
        public class FileData
        {
            public string FileName { get; set; }
            public string ContentType { get; set; }
            public byte[] Data { get; set; }

            public long Size { get { return (Data != null ? Data.LongLength : 0L); } }

            /// 
            /// Create a FileData from HttpContent 
            /// 
            /// 
            /// 
            public static async Task ReadFile(HttpContent file)
            {
                var data = await file.ReadAsByteArrayAsync();
                var result = new FileData()
                {
                    FileName = FixFilename(file.Headers.ContentDisposition.FileName),
                    ContentType = file.Headers.ContentType.ToString(),
                    Data = data
                };
                return result;
            }

            /// 
            /// Amend filenames to remove surrounding quotes and remove path from IE
            /// 
            /// 
            /// 
            private static string FixFilename(string original)
            {
                var result = original.Trim();
                // remove leading and trailing quotes
                if (result.StartsWith("\""))
                    result = result.TrimStart('"').TrimEnd('"');
                // remove full path versions
                if(result.Contains("\\"))
                    // parse out path
                    result = new System.IO.FileInfo(result).Name;

                return result;
            }
        }

    
    }
}

POSTing Knockout JSON data to WebAPI HttpPost methods

This article describes how to get JSON data from JavaScript to the server, using a WebAPI HttpPost method, with strongly-typed .NET values converted from the JSON data.

Why?

I decided to write this article as there isn’t much clarity on the Internet (shock!) on this subject. There are endless articles on WebAPI and using it with KnockoutJS to GET data. This is simple because WebAPI turns your .NET objects into the requested format (JSON, XML etc.).

What isn’t that simple is posting data back to a WebAPI method via HttpPost. There are lots of little things to do (or not do) to make it work. I’m sharing this so you can learn from our experiments!

ViewModel

Let’s say I have a Knockout viewmodel that represents a shopping cart, so we have something everyone will recognise. We have an array of Items, and a shipping method in the cart.

var Cart = (function () {
    function Cart() {
        this.Items = ko.observableArray([]);
        this.ShippingType = ko.observable("Standard");
        this.Items([
            { Code: 1, Name: "Apple" }, 
            { Code: 2, Name: "Banana" }
        ]);
        this.ShippingType("Next-Day");
    }
    Cart.prototype.SaveCart = function () {
        var data = ko.toJSON(this);
        $.ajax({
            url: "/api/Cart/Save",
            type: "POST",
            data: data,
            datatype: "json",
            processData: false,
            contentType: "application/json; charset=utf-8",
            success: function (result) {
                alert(result);
            }
        });
    };
    return Cart;
})();

We can ignore the Html side, just assume there is a “Save” button that calls the SaveCart method, and this will make an AJAX HttpPost to the /api/Cart/Save method.

In the example above I’ve loaded a couple of items into the array and changed the ShippingType. Normally we’d load all this from outside the viewmodel.

So what are the key things to note here?

1) Don’t Use $.post

The first thing we learned is don’t use jQuery’s $.post() function. This will set the content type as application/x-www-form-urlencoded, but this is not the case: we are sending JSON data.

Instead you should use the lower-level $.ajax() function, which allows you to customise the request.

2) Don’t Send the ViewModel

Note that we do not send the viewmodel object directly: the properties Items and ShippingType are Knockout functions, not values. We need to unwrap these to get the values.

Fortunately Knockout has this solved: if you call ko.toJSON() it will unwrap any observable values into a plain JSON object.

3) Don’t ‘ProcessData’ and Specify ContentType

We need to tell WebAPI the nature of the body content. We set two important values:

processData: false,

contentType:
"application/json; charset=utf-8",

processData tells jQuery not to encode the data as form-urlencoded, which is the default action. Instead it sends up the JSON.

contentType specifies that the data is JSON. It’s this that WebAPI reads to be able to figure out what it’s getting, and how to read it.

Finally you can specify dataType to specify what the POST request will return.

 

4) Ensure WebAPI Has a Matching Signature

So now it’s time to switch to our server and see how we ensure we handle the data correctly.


public class CartController : ApiController
    {
        [HttpPost]
        public string Save(MyCart cartData)
        {
            // TODO: save
            return (cartData.Items.Count() == 0 ?
                "No items found" : 
                string.Format("I saved {0} items", cartData.Items.Count());
        }
    }

    // properties match our viewmodel
    public class MyCart
    {
        public List<CartItem>[] Items { get; set; }
        public string ShippingType { get; set; }
    }

    // same signature as our
    public class CartItem
    {
        public int Code { get; set; }
        public string Name { get; set; }
    }

Note how we have not made any mention of JSON whatsoever. The MVC ModelBinder will try it’s best to interpret the data it gets, and bind it to the data in your method.

Our Save method takes a single parameter, of type MyCart . The name of the type is not important; what is key is that the properties match the JSON data we are sending. This allows the ModelBinder to convert the incoming data into our .NET object.

Note that I specified a List<CartItem> for the item array: I could have specified CartItem[] array type as well – both are okay and ModelBinder will fill it up for you, provided the properties match up.

5) ModelBinder Tries Hard!

ModelBinder is very tolerant. If your capitalisation is different from the JSON it will use a matching one if it can. So changing MyCart.ShippingType to MyCart.shippingtype won’t cause it to stop working. Well done ModelBinder. However, if you specify both variations, ModelBinder will use the one that exactly matches the JSON name.

Order is not critical: you don’t have to ensure the order of properties is the same in your .NET class, ModelBinder works with names, not the order of them.

6) Fiddler Is Your Friend!

Whenever you are debugging these sort of actions, you simply have to have Fiddler about. It allows you to see what is going back and forth, and if it’s been mangled, transformed or translated.