File Uploader Demo 'Under the Hood'

By Justin Gattuso
Posted on Aug 26, 2008

Comments (1)

Recently we rolled out our Silverlight File Uploader Demo out on our Sandbox. We also submitted it to the Silverlight 2 Beta 2 Gallery over at the official Microsoft Silverlight site. Within about a week it was reviewed and accepted and published out to the site, since that time we've received a few comments on the control, specifically asking for us to explain how it works.

Since the File Uploader control was developed for commercial purposes I cannot post the code or delve too deeply into the internals so I will instead rundown the overall architecture of the control and how it's able to do what it does. Please be aware that I will mostly be ignoring the .xaml side of things and focus instead on the underlying architecture, code and logic happening behind the scenes.

Front-End Design

First, the control is actually composed of two distinct pieces: (1) the first piece is the main control itself, with the column titles, the scrollbar, the submit and clear buttons etc. This is all static information in its own .xaml file and is where the client-side of the logic is house (in the code-behind). (2) The second piece is in the individual rows in the control, which are dynamic based upon what the user chooses to upload. Rows can be added and removed at will, and they have their own .xaml file, but are dynamically added to the parent uploader control as needed.

Back-End Design

There are also two logical parts to the control itself as well. On the one hand there is the client-side Silverlight front-end display along with the client-side Silverlight logic that controls displaying the Open File dialog, obtaining information from the selected files, removing files, basically all of the user interaction parts. Then there is the server-side logic in the form of a .NET Web Service. This web service is where the real uploading is occurring.  Because of this separation we have been able to write a couple different web services that hook into the standard upload payload and process and handle the uploaded files in drastically different ways; hence the control affords a high degree of re-use since the real processing logic is only handled at the server. The control itself does it's job of uploading files, only focusing on obtaining the files from the user, reporting the status of the upload and giving the user rich interactive functionality along the way; how those files are treated is up to the back-end server logic.

Feature - Concurrent Uploading

One of the biggest questions we've received is, "How does the multiple file uploading work?". Basically, an internal class was created to house all the information that the control needs to do it's job of uploading the file. For the most part this has to do with retrieving and storing the byte count of the file stream. Each time a user adds a new file to the queue for uploading, a new row object is instantiated and all of the required information is stored. When the user tells the control to upload the files, the control loops through all the files in the queue and begins sending byte chunks of the file to the associated web service along with any other information we need to pass along (such as the filename). The web service then takes over, taking the payload of byte information, writing it to disk and returning back to the control with the index where it left off, so when the control sends the next payload, it knows which offset in the stream to retrieve the next chunk of bytes. This process continues repeatedly over the wire as fast as the network speed will allow until finally the web service receives an indication that this payload is the last payload and this file is finished uploading. This entire process happens for each of the files in the queue, hence we gain the ability to send multiple files simultaneously. It's worth noting that all this file uploading does take place simultaneously. When a file is kicked off for uploading and returns back from the web service, information is sent back to identify which upload just finished, so the control can immediately retrieve the next chunk of information and call back to the web service. Therefore the main bottleneck in how many files can be uploaded at the same time depends mostly on the server and network speed.

Feature - Responsiveness

It's worth noting that calls to the web service are done asynchronously. This is necessary for the control to behave the way it does and manage so much concurrent functionality at the same time. If it was done synchronously, the call would block until it was returned, which might be fine (albeit silly) for uploads of a single file, but would dramatically reduce the performance of uploading many files at once. Aside from these asynchronous calls to the web service and back, the control only makes use of threading in one other area, to check if all files have been uploaded in order to kick off an event for custom handling, which I'll get back to in a little bit.

Feature - Efficiency

There is some added logic for leveraging calls to the web service. Currently it allows only so many uploads at a given time (say 10 or 20) for sheer purposes of efficient management. Since the control is essentially disconnected from the web service, while it is in the middle of uploading a batch of 50 files for example, it's still ready to be fed more files into the queue, it will simply keep them in the pending state until a slot or two opens up in the queue after some files that are currently uploading are finished. This functionality was tested with about a 1000+ individual files (all fairly small) and they easily flew through the system. As X files finished, X slots opened up and another X started uploading, until all files were successfully uploaded.

Feature - Client-Side Event Handling

Another interesting aspect of the control is that it's able to notify the client application when all of the files in the queue have been uploaded. This is actually done through Javascript. When all the files are uploaded an event is fired off in the Silverlight control itself. This event was also marked as a scriptable property using the [ScriptableMemberAttribute] attribute. This is a powerful feature, as it effectively invokes the event both in Silverlight as well as in the Silverlight Javascript API, so when the event fires, if there is a Javascript handler for the event, then that fires. The end result is that in applications where we use the file uploader control and we want something special to happen when all files have uploaded, we can do so very easily by simply hooking into this Javascript event and performing any actions we require, again furthering the usefulness of the control in various situations.

Feature - Customization & Control Re-use

Probably the last significantly interesting aspect of the control is that it has some customization options available to it. While the web service back end is fully customizable for processing purposes and there is a hook or two via Javascript for handling special events, there is still a little more customization that can be done to instantiations of the actual control itself. For example, since my favorite color is orange, I originally wrote the control statically to make use of orange colors; however, that isn't the color everyone on the planet will want. So I wrote a couple different style's for the control that are invoked upon instantiating the control with a special style parameter, so if we want a blueish File Uploader we can do so. This slightly leverages Silverlight styling and templates as well. There are also additional parameters for setting the web service it connects to, setting the maximum file upload size, setting the amount of concurrent file uploads, file filtering on the Open File dialog, etc. This is all done to promote re-use of the control. Lastly, it also accepts a user defined payload of additional information that is used exclusively by the back end web service. This is critical because in order for the control to serve a variety of processing needs at the web service level, it might be necessary for the control to send custom information, so this is possible as well.

These instantiation parameters are possible because of the InitParameters parameter when creating a new File Upload control in an HTML page. This just accepts a string, but as we all know, we can pack anything into a string, so in the end the control is capable of accepting an unlimited amount of initialization parameters and data to do it's job.

Feature - File Upload Size

Currently the control has no internal limit on the file size to be uploaded, I tested with files up to just over 1GB in size, on my local system, which still took a little while. While it would certainty work over the Internet, it just might take quite a while. There really isn't any reason it shouldn't accommodate any file size, our goal was for it to easily send files of around 20mb and it does so quite well, again, mostly at the limitation of network speed.

Probably the best thing about this control is that since it is developed in Silverlight, it provides a very useful, easy to use and fun way to upload files over the Internet and shows just one way of performing an old task in a new way. Ultimately I think it shows off the exciting potential of what Silverlight has to offer designers, developers and most importantly all of us, as users.