Uploading files through OData

Introduction

OData is a new protocol beeing supported by Microsoft to make REST a standard in web api’s. This means that your data is being made available through the web so external parties and applications can work with your data without needing to use your application.

This ensures for a controlled access and fast access to the data without going through the overhead of the entire web application. Note that OData doesn’t imply that you simply throw your entire database wide open on the Internet. You can still protect yourself from unwanted access by implementing authentication systems to prevent access.

This document will cover a specific portion of the entire OData protocol, namely the uploading of files through the OData system to create attachments on entities, or create entities that represent a file on the system. Because the code in this document is taken from TenForce bvba, it speaks for itself that none of the sample code can be redistributed or used without the consent of the owners.

Creating the DataService

OData in C#.NET relies on the WCF principles to host the service. We’re not exactly dealing with a real WCF service as much of the logic under the hood operates differently than a normal WCF service. Also during all the code examples, usage has been made of the toolkit developed by Jonathan Carter (http://wcfdstoolkit.codeplex.com/)

The first step is creating a class that inherits from the ODataService class located in the toolkit. This ensures that we’re dealing with the correct conventions of the toolkit. I’m not discussing how to setup the project etc, there are enough tutorials on the Internet on how to do this, nor am I discussing how to use the toolkit in your project.

The code for our base class looks like this:

[ServiceBehavior(IncludeExceptionDetailInFaults = true, InstanceContextMode = InstanceContextMode.PerSession)] public class Api : ODataService<Context>
{
public static void InitializeService(DataServiceConfiguration config)
{
config.UseVerboseErrors = true;
config.SetEntitySetAccessRule("*", EntitySetRights.All);
config.SetServiceOperationAccessRule("*", ServiceOperationRights.All);
config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;
Factory.SetImplementation(typeof(Api2.Implementation.Api));
}

protected override void OnStartProcessingRequest(ProcessRequestArgs args)
{
try
{
string uri = DetermineUrl();
bool authSuccess = Authenticate(args.OperationContext, uri);
args.OperationContext.ResponseHeaders.Add(@"TenForce-RAuth", authSuccess ? @"OK" : @"DENIED");

if (!authSuccess) throw new AuthenticationException(@"Invalid username and/or password");
base.OnStartProcessingRequest(args);
}
catch (Framework.TexecExceptions.TexecException te)
{
throw new AuthenticationException(string.Format("No account exists for the url '{0}'.", System.Web.HttpContext.Current.Request.Url.AbsoluteUri), te);
}
}

private static string DetermineUrl()
{
if (OperationContext.Current.IncomingMessageProperties.ContainsKey("MicrosoftDataServicesRequestUri"))
{
var uri = (OperationContext.Current.IncomingMessageProperties["MicrosoftDataServicesRequestUri"] as Uri);
if (uri != null) return uri.ToString();
throw new InvalidOperationException("Could not determine the Uri for the DataService.");
}

if (OperationContext.Current.IncomingMessageHeaders != null) return
OperationContext.Current.IncomingMessageHeaders.To.AbsoluteUri;

if (System.Web.HttpContext.Current != null) return System.Web.HttpContext.Current.Request.Url.ToString();
throw new InvalidOperationException("Could not determine the Uri for the DataService.");
}

private static bool Authenticate(DataServiceOperationContext context, string url)
{
string header = context.RequestHeaders["TenForce-Auth"];
if (string.IsNullOrEmpty(header)) return false;
header = Encoding.UTF8.GetString(Convert.FromBase64String(header));
string[] components = header.Split('|');
return (components.Length >= 2) && Authenticator.Authenticate(components[0], components[1], Authenticator.ConstructDatabaseId(url));
}
}

This class makes sure that your application can now respond to OData calls. Because we mentioned security in the introduction, we have inserted a special requirement for our class. People using our OData Api are required to submit a special header during their requests that contains a username and password that allows them to use the OData Api.

Our class has been extended with some additional functions that extract this information from the headers and verifies this information against the database.

The Attachment entity

OData requires each entity to be represented by a class. Because we’re dealing with entities that need to be downloadable for the users, we need to make a few special adjustments to the class that allows the service to provide the entity as binary data. The code for our class looks like the following:

namespace TenForce.Execution.Api2.Objects
{
using Microsoft.Data.Services.Toolkit.Providers;
using System;
using System.Data.Services.Common;

[HasStream]
[DataServiceKey("Id")]
public class Attachment : IStreamEntity
{
public int Id { get; set; }
public string LinkReason { get; set; }
public DateTime Created { get; set; }
public User Creator { get; set; }
public DateTime Modified { get; set; }
public User Modifier { get; set; }
public string Filename { get; set; }
public int Filesize { get; set; }
public string ExternalReference { get; set; }
public Item Item { get; set; }

#region IStreamEntity Implementation

public string GetContentTypeForStreaming()
{
return "multipart/mixed";
}

public Uri GetUrlForStreaming()
{
return new Uri(Factory.CreateApi().Attachments.GetAttachmentPath(this, false));
}

public string GetStreamETag()
{
return string.Empty;
}

#endregion
}
}

What’s important to note in this class:

· The [HasStream] attribute. This attribute marks the class as a streamable entity according the WCF standards. It means that the class can be streamed on request.

· IStreamEntity interface. This interface is part of the toolkit and defines the required functions that need to be implemented to represent the information of the entity as a stream. This mostly reflects to information for headers.

Basicly you implement the IStreamEntity interface and add the [HasStream] attribute to the class.

The AttachmentRepository

The repository is a class used by the toolkit for translating the incoming requests on the service into internal API calls. There is no base interface or class that needs to be implemented. The toolkit relies on the principle of “convention over configuration” and assumes that there is always a Repository class available for each entity. This is implemented by the RepositoryFor method in the toolkit, which will use reflection to find a class that matches this pattern.

The code for the RepositoryFor method looks like this:

public override object RepositoryFor(string fullTypeName)
{
string typeName = fullTypeName.Replace("[]",string.Empty).Substring(fullTypeName.LastIndexOf('.')+1);
Type repoType = Type.GetType(string.Format("TenForce.Execution.Api2.OData.{0}Repository", typeName));
if(repoType == null) throw new NotSupportedException();
return Activator.CreateInstance(repoType);
}

This code is located in the Context class that needs to be implemented. If you look back at the Api class we wrote, you’ll see that it inherits from the ODataService class. Context is the class that represents the “Context” for the service on where to retrieve all the data from for each supported entity.

When dealing with the entity framework, this would be a wrapper class that maps all the calls to the related database table. In case of the toolkit, we use the reflection provider to map all the calls to the relevant properties on the class, which in return direct the call to the correct Repository implementation.

A property for the Attachment entity looks like this on the Context class:

///

/// Gets a queryable collection containing Attachment entities for the OData WebService.
///

public IQueryable<Attachment> Attachments { get { return CreateQuery<Attachment>(); } }

The property calls the CreateQuery method of the context, which dives into the toolkit for finding out what exactly needs to be done. I’m not going to dive into the toolkit too deeply here, but basicly the CreateQuery statement analyzes the query being submitted and checks if there needs to be paging applied, filtering or other OData supported operations.

A final note, the Context class itself inherits from the ODataContext class, which can be found in the toolkit.

Going back to the Repository class itself for our Attachment entity, we need to follow several conventions from the toolkit. To make it easy for ourselves, we implemented these conventions in an abstract base class where we inherit from. The benefit here is that all logic is stored in a single class.

GetProvider()

GetProvider is a method we have implemented on every class that allows a quick call to the correct ICrud<> implementation in our case. We do this, because the ICrud implementation represent our internal API, and the set implementation depends on who’s calling it. We rely on dependency injection here to figure that out for us, and just program against the interface:

///

/// Returns the IAttachmentsProvider implementation that will handle all the requests.
///

///
protected override IAttachmentsProvider GetProvider()
{
return Factory.CreateApi().Attachments;
}

IStreamRepository implementation

Because our Repository needs to work with Streamable entities, we need to inform the OData Service that we’re dealing with this kind of entities. This is done by implementing the interface IStreamRepository. Again, this interface is present in the toolkit and defines a specific set of functions that we’re going to have to implement.

DeleteStream

This function basicly deletes the stream of the entity, and the physical source of the stream on the server. In our case this means that we actually delete the file on disk that is linked to the given Attachment entity.

///

/// Deletes all the resources of the given attachment
///

/// The entity who's resources need to be deleted.
public void DeleteStream(object entity)
{
if (entity as Attachment == null) return;
Factory.CreateApi().Attachments.DeleteFile(entity as Attachment);
}

GetWriteStream

This function performs actually the bulk of the work. The responsibility of the function is to provide a Stream that allows the service to store the data being sent by the client. Important to know here is the correct working of the toolkit.

This means having a good understanding of the workings of the OData protocol:

• The initial upload contains the binary data of the file being sent.
• Thus we need to create a dummy instance first and link the file to it
• In a subsequent PUT/MERGE call, we need to update the created entity.

The function keeps track of this in our case, but doesn’t handle MERGE requests. We rely on a secondary PUT from the user to update the linked entity.

///

/// Returns a valid Stream implementation where the StreamProvider can write the data in.
///

/// The entity who's binary data needs to be saved.
/// Information about the current operation context.
/// A valid Stream implementation pointing to the correct location.
/// Invalid entity supplied for this Repository.
public Stream GetWriteStream(object entity, System.Data.Services.DataServiceOperationContext context)
{
if(entity as Attachment == null) throw new ArgumentException("Invalid entity supplied.");
// Handle the POST Request. The POST request means that a new file is beeing uploaded to
// create a new attachment.
// This means we need to retrieve the filename from the slugheader if present, or
// generate a dummy one.
// Handle the POST Request. The POST request means that a new file is beeing uploaded
// to create a new attachment.
// This means we need to retrieve the filename from the slugheader if present,
// or generate a dummy one.
if (operationContext.RequestMethod == "POST")
{
// Save the required properties of the Attachment entity
(entity as Attachment).Item = new Item {Id = int.Parse(operationContext.RequestHeaders["Slug"])};
(entity as Attachment).Filename = string.Format("{0}_{1}{2}{3}{4}{5}{6}{7}_newfile",
(entity as Attachment).Item.Id,
DateTime.Now.Year,
DateTime.Now.Month,
DateTime.Now.Day,
DateTime.Now.Hour,
DateTime.Now.Minute,
DateTime.Now.Second,
DateTime.Now.Millisecond);
(entity as Attachment).Created = DateTime.Now;
(entity as Attachment).Modified = DateTime.Now;

// Return the filestream to temporary store the file.
return new FileStream(GetProvider().GetAttachmentPath(entity as Attachment, true), FileMode.OpenOrCreate);
}

// Handle the PUT request.
// The Attachment should have been saved right now in the database.
// So all properties should have correct values.
return new FileStream(Factory.CreateApi().Attachments.GetAttachmentPath(entity as Attachment, false), FileMode.Create, FileAccess.ReadWrite);
}

That’s actually all you need to properly handle the upload of a file. If we run this through OData we can simply send a file in a POST request and later retrieve it by going to our URL like this: http:///Api.svc/Attachment(id)/$value which will downloads the file for us.