Can’t upload (empty) files to a SharePoint document library

I feel so stupid now, but I was doing some testing for a SharePoint application that I’ve been working on and couldn’t work out why I was having trouble uploading test files to a document library.

This is a slightly long and pretty embarrassing tale of how I made a simple mistake that took a while to figure out. I’m reproducing the details hoping that you’ll find it useful in debugging similar SharePoint problems and to help you if you made a similarly silly mistake (unlikely!).

My application uses item-level auditing and I’ve got a custom page to show audit information for a bunch of documents in a library. I configured auditing and created my custom page. Then I created a test document in Windows Explorer (File ? New ? Text Document) and copied and pasted it a few times to create a series of documents I could upload to my document library.

image

My attempts at uploading any of these documents failed with an error “The file name is invalid or the file is empty. A file name cannot contain any of the following characters: \ / : * ? " < > | # { } % ~ &”. WTF? My filename is “C:\Temp\Test Document1.txt”. It seems to have the right characters. I tried some of the other files, all failed. What’s going on???

Error when uploading a test document

Naturally I start wondering what I’ve done wrong. I’m working with a newly configured development environment – is that the problem? My solution is based on a custom list definition and custom content types and fields – did I mess something up there? Windows Update recently installed Internet Explorer 8 and the /_layouts/Upload.aspx page uses ActiveX – is there something wrong with my security configuration?

No. No. And, erm, NO! I’m going bald from the head scratching right now…

 

Debugging the problem

Hmmm… let’s try and see what the /_layouts/Upload.aspx page is doing. Opening the page in a text editor (location C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\TEMPLATE\LAYOUTS\Upload.aspx) we can see that it inherits from a class called UploadPage which is defined in the assembly Microsoft.SharePoint.ApplicationPages.

<%@ Assembly Name="Microsoft.SharePoint.ApplicationPages, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" %>
<%@ Page Language="C#" Inherits="Microsoft.SharePoint.ApplicationPages.UploadPage" ... %>

 

At this point I whip out every .NET developer’s best friend, Reflector, and start spelunking around at the UploadPage. class. Hmm… it’s a little hard to tell what’s going on. The OnLoad event is obfuscated, so there’s not much joy there.

OK, how about another approach. All SharePoint messages are stored in the resource files so maybe I can find out which resource key name is associated with my error message.

It turns out that the text “The file name is invalid or the file is empty” is defined in the file wss.resx located in (C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\CONFIG\Resources) with a resource name of upload_document_file_invalid. A bit more searching around in Reflector and the 12 Hive files reveals that this resource is used in Upload.aspx.

We’re back to where we started but at least we have more information. The message is displayed by an ASP.NET CustomValidator control:

<asp:CustomValidator ControlToValidate="InputFile"
    Display = "Dynamic"
    ErrorMessage = "<%$Resources:wss,upload_document_file_invalid%>"
    OnServerValidate="ValidateFile"
    runat="server"/>

 

See the OnServerValidate attribute? That refers to a method in the super-class (or code-behind class if you prefer). Going back to Reflector we can see the definition for the ValidateFile method:

protected void ValidateFile(object source, ServerValidateEventArgs args)
{
    CustomValidator validator = (CustomValidator) source;
    HtmlInputFile file = (HtmlInputFile) validator.FindControl(validator.ControlToValidate);

    if (SPUrlUtility.IsLegalFileName(this.GetLeafName(file.PostedFile.FileName)))
    {
        if ((file.PostedFile != null) && (file.PostedFile.ContentLength > 0))
        {
            args.IsValid = true;
            return;
        }
    }
    else
    {
        args.IsValid = false;
        return;
    }
    args.IsValid = false;
}

 

Now we’re getting somewhere! And, if you haven’t already worked out what I did wrong, you might have an idea of what’s wrong. It took me a little longer though!

I decided to implement my own class as a base class for Upload.aspx. That way I could interactively debug what’s going on while this code is executing. To do this I:

  • Created a class in my solution that derives from Microsoft.SharePoint.ApplicationPages.UploadPage,
  • Copied the ValidateFile method from the disassembled version in Reflector,
  • Added the new modifier to the method declaration to override the non-virtual method in the base class,
  • Searched in Reflector for and added definitions for other methods and types that ValidateFile depends on (but are marked as internal and therefore cannot be called directly. These included GetLeafName, IsLegalFileName, IsLegalCharInUrl and s_LegalUrlChars.
  • Made a backup copy of Upload.aspx and modified the original Upload.aspx to inherit from my class *

My CustomUploadPage class file looked like:

using System;
using System.Web.UI.HtmlControls;
using System.Web.UI.WebControls;

using Microsoft.SharePoint.ApplicationPages;

namespace Mratovich
{
    public class CustomUploadPage : UploadPage
    {
        protected new void ValidateFile(object source, ServerValidateEventArgs args)
        {
            CustomValidator validator = (CustomValidator) source;
            HtmlInputFile file = (HtmlInputFile) validator.FindControl(validator.ControlToValidate);
            if (IsLegalFileName(this.GetLeafName(file.PostedFile.FileName)))
            {
                if ((file.PostedFile != null) && (file.PostedFile.ContentLength > 0))
                {
                    args.IsValid = true;
                    return;
                }
            }
            else
            {
                args.IsValid = false;
                return;
            }
            args.IsValid = false;
        }

        private string GetLeafName(string s)
        {
            int num = s.LastIndexOf('\\');
            if (num < 0)
            {
                return s;
            }
            return s.Substring(num + 1);
        }

        internal static bool IsLegalFileName(string name)
        {
            int num = -1;
            if (name == null)
            {
                throw new ArgumentNullException("name");
            }

            if (name.Length == 0)
            {
                return false;
            }

            int num2 = 0;

            while (true)
            {
                if (num2 < name.Length)
                {
                    if (!IsLegalCharInUrl(name[num2]))
                    {
                        return false;
                    }

                    if (name[num2] == '.')
                    {
                        if ((num2 == 0) || (num2 == (name.Length - 1)))
                        {
                            return false;
                        }

                        if ((num != -1) && (num == (num2 - 1)))
                        {
                            return false;
                        }

                        num = num2;
                    }
                }
                else
                {
                    return true;
                }

                num2++;
            }
        }

        internal static bool IsLegalCharInUrl(char ch)
        {
            if (ch >= '\x00a0')
            {
                return true;
            }

            while (ch == '/')
            {
                return false;
            }

            return s_LegalUrlChars[ch];
        }

        private static bool[] s_LegalUrlChars = new bool[] {
            false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false,
            false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false,
            true, true, false, false, true, false, false, true, true, true, false, true, true, true, true, true,
            true, true, true, true, true, true, true, true, true, true, false, true, false, true, false, false,
            true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true,
            true, true, true, true, true, true, true, true, true, true, true, true, false, true, true, true,
            true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true,
            true, true, true, true, true, true, true, true, true, true, true, false, false, false, false, false,
            false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false,
            false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false
        };
    }
}

 

And here are the first few lines of my modified Upload.aspx file. I only added a reference to my custom assembly containing CustomUploadPage and the Page directive that indicates that the page inherits from my custom class. The rest is ommitted for brevity.

<%@ Assembly Name="Microsoft.SharePoint.ApplicationPages, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"%>
<%@ Assembly Name="Mratovich, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cba245c263929c41"%>
<%@ Page Language="C#" Inherits="Mratovich.CustomUploadPage" MasterPageFile="~/_layouts/application.master" %>
<%@ Import Namespace="Microsoft.SharePoint.ApplicationPages" %>
...

Now I could deploy my solution and place a breakpoint on my CustomUploadPage class’s ValidateFile method. When debugging, the name of the file turned out to be valid and the HtmlInputFile instance was created as expected. So where was this breaking down? Well in CustomUploadPage, the expression (file.PostedFile.ContentLength > 0) was evaluating to false.

Sure enough, when I double-checked the HTTP request in Fiddler the content for the file was empty!

Content-Disposition: form-data; name="ctl00$PlaceHolderMain$ctl01$ctl02$InputFile"; filename="C:\Temp\Test Document (3).txt"
Content-Type: text/plain
-----------------------------7d922d34101ac

 

 

Feeling pretty stupid right now…

At this point I slapped my forehead and nearly started to cry! How stupid! The file I was uploading was empty! The browser couldn’t send any content to the server because there was nothing to send! Remember the test files I created? Well look a little more carefully at the Size column!

image

0 KB! As soon as I entered some text into one of the files, saved it and tried to upload it, it worked!

Man I’m an idiot! :-)  I hope you haven’t made as dumb a mistake as this. Hopefully I’ve passed on some tips on how to go about debugging this in SharePoint and using some tools that should be essential to your development arsenal. If nothing else you can have a laugh at my expense!

- Dario

 

* This isn’t a best practice of course. Modifying files in /_layouts is a bad idea, but this was just for debugging and I restored the backup of the original once I was finished.

Boot from VHD: Blue screen error on startup

In a recent post I was pimping the boot from VHD feature in Windows 7 and Windows Server 2008 R2. However, recently I encountered a problem: when booting from a newly created VHD I kept on getting a blue screen and my machine would immediately restart.

I couldn’t work out what was wrong and thought I’d take a closer look at the message in the blue screen of death window. To do that I had to stop Windows from restarting automatically after displaying the blue screen.

To stop Windows from automatically restarting after a blue screen:

  • Select the operating system you want to start from the Windows Boot Manager screen (don’t press ENTER):
    Windows Boot Manager screen
  • Press F8 on your keyboard to open the Advanced Boot Options for your selected operating system:
    Advanced Boot Options screen
  • Select the Disable automatic restart on system failure option and press ENTER.
  • Now Windows won’t reboot if it encounters a blue screen.

 

My problem

After completing the above steps, I booted into my VHD and saw the error message “The volume that hosts the VHD does not have enough free space to expand the VHD”:

Blues screen error: The volume that hosts the VHD does not have enough free space to expand the VHD

Oops! I didn’t have enough disk space available for the VHD to grow into.

I created a 50 GB dynamic VHD (which physically only took up about 7 GB after installing Windows) but I only had about 30 GB fee on the physical disk and  kept getting blue screens on boot up.

My recommendation: if you’re booting from a dynamic VHD make sure you have enough free space for your disk to grow to its full capacity. I think you can technically get away with less as I was able to boot up a 50 GB dynamic VHD after freeing up 40 GB.

This does suggest that it might be better to just create fixed-size VHDs and not bother with dynamic disks – but copying and compressing a 10 GB dynamic VHDs is a little easier than a 50 GB fixed-size VHDs…

Do what you think is best I guess!

- Dario

Windows 7 and Windows Server 2008 R2 Boot from VHD: My new favourite SharePoint VM environment

If, like me, you’re a SharePoint developer you’re probably developing using a virtual machine environment. Another option is to run a Windows Server operating system natively or hack SharePoint to run on Vista/Windows 7 (definitely unsupported!). Either way it’s never felt truly comfortable for me: it’s always a trade-off between performance and flexibility.

Running Windows natively gives you full access to all your hardware – SharePoint runs at full speed. Going into a virtual environment you give up some of that performance for flexibility: being able to suspend/start/backup/copy/move/trash a whole machine at the click of a button and quickly change things around as you need to.

I’ve tried various different virtualisation solutions: VMWare Server, VMWare Workstation, Virtual PC, Virtual Server and Hyper-V. I thought I’d found my happy medium with Hyper-V: you get a good balance of performance versus flexibility, nice tooling and it’s baked right into Windows Server 2008. I still think this is a great approach and it works well for many scenarios, but I’ve discovered something that works better – at least for me – your mileage may vary.

 

Booting from a VHD in Windows 7 and Windows Server 2008 R2

I heard about this feature coming in Windows 7 before the public beta and I thought it sounded cool. When the Windows 7 Beta came out I “discovered” that it would only boot a Windows 7 image and only off a system with a Windows 7 boot loader. I dismissed this out of hand thinking that it wouldn’t be all that useful to me since I really want to run a server operating system.

Well I should have “discovered” a bit more. I didn’t realise, until recently, that this works with Windows Server 2008 R2 too. This means you can run SharePoint on Windows 2008 R2 by booting off a VHD. You get:

  • Great performance – everything except the disk is “bare metal”. The disk is running in a virtual layer, but all your devices: USB goodies, graphics cards, sound, etc. are all talking directly to the hardware. There’s no hypervisor or device emulation going on. You get access to all available RAM and all CPU cores. Heck you can even get Aero going on if that’s important to you (try that in a “regular” VM!).
  • Flexibility – you still have your entire environment contained in a VHD file. You lose the ability to suspend a virtual machine but I can live with that trade off.

As a result, this is now my preferred choice for developing on SharePoint and for trying things like Visual Studio 2010 Beta before it’s ready to be installed in my main machine. I have backup copies of Windows 7 and Windows Server 2008 R2 VMs with all drivers and settings installed and configured. I can fire up a fresh VM ready for messing around in no time and they run at pretty much full speed. Life is good!

Windows Server 2008 R2 with access to all available RAM and CPU cores and Aero effects at 1900 x 1200

 

How to do it

There are already a number of guides out there for getting this up and running so I won’t re-hash things here. These are the guides that I followed:

 

Some considerations

While this solution works for me, it’s not for everyone. Consider:

  • This only works with Windows 7 and Windows Server 2008 R2. These are still not fully baked. I’m happy to run these full time, you might not be. You might not even be allowed to.
  • There are implications with running SharePoint on Windows Server 2008 R2: you need to be running SharePoint bits with SP 1. If you can’t (perhaps you’re developing for a client who isn’t on SP 1 yet) this isn’t for you. There are some folks who have been able to Windows Vista to boot off a VHD (by copying the Windows 7 boot loader to Vista) so it might be possible to get Windows Server 2003 booting but I can’t vouch for support or stability. It’s not something I need right now so I haven’t fiddled with it.
  • You can only run one VHD at a time. If you need to run multiple virtual machines, this isn’t for you.

If you can deal with these limitations you might find this a great way to work with virtual machines. I think it rocks!

- Dario

Devs4Devs Presentation: Pushing Data to Silverlight over HTTP

On 9 May 2009 I presented at a developer event called Devs4Devs. It’s an event run by Microsoft that invites new speakers to present at a small forum to help unearth new speaking talent for events like DevDays and TechEd.

Public speaking has always been something of a challenge for me, so I decided to push myself and give it a go. I presented a session on pushing data to a Silverlight client over HTTP. I’d worked with this on a project for a customer recently and thought this would be an interesting topic.

The session went pretty well and, despite the nerves, I actually enjoyed myself. I got some good feedback from the attendee evaluations and from Eben (the event organizer). I’m hoping this will help build my confidence and experience in public speaking.

 

About the Presentation

The idea I wanted to talk about is how to use a new WCF binding provided in the Silverlight 2.0 SDK to create a service that can establish a two-way (i.e. duplex) communication with a client over HTTP. HTTP is a request-response protocol and doesn’t allow data to be transferred from the server to the client unless the client has requested it.

I wanted to explore more deeply how this is done with Silverlight (the binding is essentially doing some intelligent polling under the covers for you) but the session was only 20 minutes long, so I had to be brief.

To demonstrate the topic I wrote a service to simulate a random IPL cricket match* (since that’s all the rage at the moment in South Africa). The service generates events for a cricket game and sends out scoreboard updates to any clients that have connected to the service. The client simply binds the scoreboard data to UI elements to display batting and bowling statistics and provide a ball-by-ball commentary (like CricInfo).

Devs4Devs Silverlight Scoreboard 1 Devs4Devs Silverlight Scoreboard 2

My session walked through creating the service and the client in Silverlight 2.0 and showed the awkward asynchronous channel programming model required to get the client to communicate with the service.

I then walked through the improvements that will be coming in Silverlight 3.0 to improve the programming model by showing how the code as implemented using Silverlight 3.0 Beta.

 

Slide Deck and Source Code

I know it’s a little late but if I’m making the slide deck and source code available online for anyone who is interested. Go ahead and grab ‘em:

  • Slides
  • Code – This includes two solutions: one for Silverlight 2.0 and another using Silverlight 3.0 Beta (version 3.0.40307.0).

Please note that the code is NOT production quality. I purposely tried to keep things simple for the presentation by using synchronous service-side methods and I’ve omitted a lot of error and exception handling code. There are also a few bugs in the cricket match simulator and probably plenty more that I haven’t discovered. Use it to learn, but not for anything real and use it at your own risk!

To open the solutions and run the code you’ll need: either Silverlight Tools for Visual Studio 2008 SP1 (for opening and compiling the Silverlight 2.0 solution) or Silverlight 3 Beta Tools for Visual Studio (for opening and compiling the Silverlight 3.0 Beta solution).

Note: Silverlight 3.0 projects are not compatible with Silverlight 2.0 projects. If you install the Silverlight 3.0 Beta Tools for Visual Studio you will be unable to open Silverlight 2.0 projects. I recommend installing the Silverlight 3.0 tools in a separate environment.

 

Other Resources

I’ll try and post a few articles about the code to explain the technique a bit more, but there is a lot of great content out there already:

 

- Dario

 

* The simulation was fun to write (although the code is butt ugly)! I didn’t want to spend too much time on it and after I got started I realized that simulating a cricket game is harder than I originally thought. There are just so many different events in a cricket game and different things that can happen. The simulation engine takes some short-cuts and definitely isn’t complete. I make no apologies, it was written in a hurry and wasn’t really the focus of the session.

Look Ma, I’m blogging!

(Actually, I really doubt she’s reading this!)

It’s been a long time coming and something I’ve been meaning to do for ages, but I’m finally getting off my lazy butt and blogging. I’m not sure how all of this will turn out, but I’m going to give it a go and, well, we’ll see…

 

A little about me

I’ve been a geek for as long as I can remember and have always been fascinated about how things work and making things work. I’ve been interested in computers and software since I was about 9 years old and enjoy pretty much all aspects of technology.

I’m currently working as a developer consultant for Microsoft in South Africa. My main technology focuses are .NET and SharePoint development, but throughout my career I’ve touched a variety of technologies and tools and I’m interested in pretty much anything out there.

 

What you’ll find here

This is my space and a place where I plan to write about the things that I find interesting. This will mostly be technology related and specifically about software development. I’ll also have some personal content – it’s my blog, I can do what I like! I’ll always tag personal and technical content appropriately so that if my Ma does decide to follow my personal posts she won’t have to muddle through the tech stuff and if you’re not interested in stuff my Ma might want to see, you can follow my technology category posts.

I’ll try post interesting things that I’ve learnt about in the work that I do and I’ll try and help others learn from mistakes I’ve made or problems I’ve encountered and found solutions to.

So sit tight, enjoy (hopefully!) and let’s see where this goes…

- Dario