Marquee de Sells: Chris's insight outlet for category '.net' via ATOM 1.0 csells on twitter

You've reached the internet home of Chris Sells, who has a long history as a contributing member of the Windows developer community. He enjoys long walks on the beach and various computer technologies.




REPL for the Rosyln CTP 10/2011

I don’t know what it is, but I’ve long been fascinated with using the C# syntax as a command line execution environment. It could be that PowerShell doesn’t do it for me (I’ve seriously tried half a dozen times or more). It could be that while LINQPad comes really close, I still don’t have enough control over the parsing to really make it work for my day-to-day command line activities. Or it may be that my friend Tim Ewald has always challenged csells to sell C shells by the sea shore.

Roslyn REPL

Whatever it is, I decided to spend my holiday time futzing with the Roslyn 2011 CTP, which is a set of technologies from Microsoft that gives you an API over your C# and VB.NET code.

Why do I care? Well, there are all kinds of cool code analysis and refactoring tools I could build with it and I know some folks are doing just that. In fact, at the BUILD conference, Anders showed off a “Paste as VB” command built with Roslyn that would translate C# to VB slick as you please.

For me, however, the first thing I wanted was a C# REPL environment (Read-Evaluate-Print-Loop). Of course, Roslyn ships out of the box with a REPL tool that you can get to with the View | Other Windows | C# Interactive Window inside Visual Studio 2010. In that code, you can evaluate code like the following:

> 1+1 2
> void SayHi() { Console.WriteLine("hi"); }
> SayHi();
hi

Just like modern dynamic languages, as you type your C# and press Enter, it’s executed immediately, even allowing you to drop things like semi-colons or even calls to WriteLine to get output (notice the first “1+1” expression). This is a wonderful environment in which to experiment with C# interactively, but just like LINQPad, it was a closed environment; the source was not provided!

The Roslyn team does provide a great number of wonderful samples (check the “Microsoft Codename Roslyn CTP - October 2011” folder in your Documents folder after installation). One in particular, called BadPainting, provides a text box for inputting C# that’s executed to add elements to a painting.

But that wasn’t enough for me; I wanted at least a Console-based command line REPL like the cool Python, JavaScript and Ruby kids have. And so, with the help of the Roslyn team (it pays to have friends in low places), I built one:

RoslynRepl Sample Download

Building it (after installing Visual Studio 2010, Visual Studio 2010 SP1, the Visual Studio 2010 SDK and the Roslyn CTP) and running it lets you do the same things that the VS REPL gives you:

RoslynRepl

In implementing my little RoslynRepl tool, I tried to stay as faithful to the VS REPL as possible, including the help implementation:

replhelp

If you’re familiar with the VS REPL commands, you’ll notice that I’ve trimmed the Console version a little as appropriate, most notably the #prompt command, which only has “inline” mode (there is no “margin” in a Console window). Other than that, I’ve built the Console version of REPL for Roslyn such that it works just exactly like the one documented in the Roslyn Walkthrough: Executing Code in the Interactive Window.

Building a REPL for any language is, at you might imagine, a 4-step process:

  1. Read input from the user
  2. Evaluate the input
  3. Print the results
  4. Loop around to do it again until told otherwise

Read

Step 1 is a simple Console.ReadLine. Further, the wonder and beauty of a Windows Console application is that you get complete Up/Down Arrow history, line editing and even obscure commands like F7, which brings up a list of commands in the history:

replbeauty

The reading part of our REPL is easy and has nothing to do with Roslyn. It’s evaluation where things get interesting.

Eval

Before we can start evaluating commands, we have to initialize the scripting engine and set up a session so that as we build up context over time, e.g. defining variables and functions, that context is available to future lines of script:

using Roslyn.Compilers;
using Roslyn.Compilers.CSharp;
using Roslyn.Compilers.Common;
using Roslyn.Scripting;
using Roslyn.Scripting.CSharp;
...
// Initialize the engine
string[] defaultReferences = new string[] { "System", ... }; string[] defaultNamespaces = new string[] { "System", ... }; CommonScriptEngine engine = new ScriptEngine(defaultReferences, defaultNamespaces);
// HACK: work around a known issue where namespaces aren't visible inside functions
foreach (string nm in defaultNamespaces) {
  engine.Execute("using " + nm + ";", session);
}

Session session = Session.Create();

Here we’re creating a ScriptEngine object from the Roslyn.Scripting.CSharp namespace, although I’m assigning it to the base CommonScriptEngine class which can hold a script engine of any language. As part of construction, I pass in the same set of assembly references and namespaces that a default Console application has out of the box and that the VS REPL uses as well. There’s also a small hack to fix a known issue where namespaces aren’t visible during function definitions, but I expect that will be unnecessary in future drops of Roslyn.

Once I’ve got the engine to do the parsing and executing, I creating a Session object to keep context. Now we’re all set to read a line of input and evaluate it:

ParseOptions interactiveOptions =
new ParseOptions(kind: SourceCodeKind.Interactive,
languageVersion: LanguageVersion.CSharp6);
... while (true) { Console.Write("> "); var input = new StringBuilder(); while (true) { string line = Console.ReadLine(); if (string.IsNullOrWhiteSpace(line)) { continue; } // Handle #commands ... // Handle C# (include #define and other directives) input.AppendLine(line); // Check for complete submission if (Syntax.IsCompleteSubmission(
SyntaxTree.ParseCompilationUnit(
input.ToString(), options: interactiveOptions))) {
break;
}
Console.Write(". "); } Execute(input.ToString()); }

The only thing we’re doing that’s at all fancy here is collecting input over multiple lines. This allows you to enter commands over multiple lines:

replmultiline

The IsCompleteSubmission function is the thing that checks whether the script engine will have enough to figure out what the user meant or whether you need to collect more. We do this with a ParseOptions object optimized for “interactive” mode, as opposed to “script” mode (reading scripts from files) or “regular” mode (reading fully formed source code from files). The “interactive” mode lets us do things like “1+1” or “x” where “x” is some known identifier without requiring a call to Console.WriteLine or even a trailing semi-colon, which seems like the right thing to do in a REPL program.

Once we have a complete command, single or multi-line, we can execute it:

public void Execute(string s) {
  try {
    Submission<object> submission = engine.CompileSubmission<object>(s, session);
    object result = submission.Execute();
    bool hasValue;
    ITypeSymbol resultType = submission.Compilation.GetSubmissionResultType(out hasValue);

    // Print the results
    ...
  }
  catch (CompilationErrorException e) {
    Error(e.Diagnostics.Select(d => d.ToString()).ToArray());
  }
  catch (Exception e) {
    Error(e.ToString());
  }
}

Execution is a matter of creating a “submission,” which is a unit of work done by the engine against the session. There are helper methods that make this easier, but we care about the output details so that we can implement our REPL session.

Print

Printing the output depends on the type of a result we get back:

ObjectFormatter formatter =
new ObjectFormatter(maxLineLength: Console.BufferWidth, memberIndentation: " ");
...
Submission
<object> submission = engine.CompileSubmission<object>(s, session); object result = submission.Execute(); bool hasValue; ITypeSymbol resultType =
submission.Compilation.GetSubmissionResultType(out hasValue); // Print the results if (hasValue) { if (resultType != null && resultType.SpecialType == SpecialType.System_Void) { Console.WriteLine(formatter.VoidDisplayString); } else { Console.WriteLine(formatter.FormatObject(result)); } }

As part of the result output, we’re leaning on an instance of an “object formatter” which can trim things for us to the appropriate length and, if necessary, indent multi-line object output.

In the case that there’s an error, we grab the exception information and turn it red:

void Error(params string[] errors) {
  var oldColor = Console.ForegroundColor;
  Console.ForegroundColor = ConsoleColor.Red;
  WriteLine(errors);
  Console.ForegroundColor = oldColor;
}
public void Write(params object[] objects) {
  foreach (var o in objects) { Console.Write(o.ToString()); }
}

void WriteLine(params object[] objects) {
  Write(objects);
  Write("\r\n");
}

replerror

Loop

And then we do it all over again until the program is stopped with the #exit command (Ctrl+Z, Enter works, too).

Where Are We?

Executing lines of C# code, the hardest part of building a C# REPL, has become incredibly easy with Roslyn. The engine does the parsing, the session keeps the context and the submission gives you extra information about the results. To learn more about scripting in Roslyn, I recommend the following resources:

Now I’m off to add Intellisense support. Wish me luck!

0 comments




Spurious MachineToApplication Error With VS2010 Deployment

Often when I'm building my MVC 2 application using Visual Studio 2010, I get the following error:

It is an error to use a section registered as allowDefinition='MachineToApplication' beyond application level. This error can be caused by a virtual directory not being configured as an application in IIS.

On the internet, this error seems to be related to having a nested web.config in your application. I do have such a thing, but it's just the one that came out of the MVC 2 project item template and I haven't touched it.

In my case, this error in my case doesn't seem to have anything to do with a nested web.config. This error only started to happen when I began using the web site deployment features in VS2010 which by itself, rocks (see Scott Hanselman's "Web Deployment Made Awesome: If You're Using XCopy, You're Doing It Wrong" for details).

 If it happens to you and it doesn't seem to make any sense, you can try to fix it with a Build Clean command. If you're using to previous versions of Visual Studio, you'll be surprised, like I was, not to find a Clean option in sparse the Build menu. Instead, you can only get to it by right-clicking on your project in the Solution Explorer and choosing Clean.

Doing that, however, seems to make the error go away. I don't think that's a problem with my app; I think that's a problem with VS2010.

0 comments




The performance implications of IEnumerable vs. IQueryable

It all started innocently enough. I was implementing a "Older Posts/Newer Posts" feature for my new web site and was writing code like this:

IEnumerable<Post> FilterByCategory(IEnumerable<Post> posts, string category) {
  if( !string.IsNullOrEmpty(category) ) {
return posts.Where(p => p.Category.Contains(category));
}
}
...
  var posts = FilterByCategory(db.Posts, category);
  int count = posts.Count();
...

The "db" was an EF object context object, but it could just as easily been a LINQ to SQL context. Once I ran this code, it failed at run-time with a null reference exception on Category. "That's strange," I thought. "Some of my categories are null, but I expect the 'like' operation in SQL to which Contains maps to skip the null values." That should've been my first clue.

Clue #2 was when I added the null check into my Where expression and found that their were far fewer results than I expected. Some experimentation revealed that the case of the category string mattered. "Hm. That's really strange," I thought. "By default, the 'like' operation doesn't care about case." Second clue unnoticed.

My 3rd and final clue was that even though my site was only showing a fraction of the values I knew where in the database, it had slowed to a crawl. By now, those of you experienced with LINQ to Entities/SQL are hollering from the audience: "Don't go into the woods alone! IEnumerable kills all the benefits of IQueryable!"

See, what I'd done was unwittingly switched from LINQ to Entities, which takes my C# expressions and translates them into SQL, and was now running LINQ to Objects, which executes my expressions directly.

"But that can't be," I thought, getting hot under the collar (I was wearing a dress shirt that day -- the girlfriend likes me to look dapper!). "To move from LINQ to Entities/SQL to LINQ to Objects, I thought I had to be explicit and use a method like ToList() or ToArray()." Au contraire mon fraire (the girlfriend also really likes France).

Here's what I expected to be happening. If I have an expression like "db.Posts" and I execute that expression by doing a foreach, I expect the SQL produced by LINQ to Entities/SQL to look like this:

select * from Posts

If I add a Where clause, I expect the SQL to be modified:

select * from Posts where Category like '%whatever%'

Further, if I do a Count on the whole thing, e.g.

db.Posts.Where(p => p.Contains(category)).Count()

I expect that to turn into the following SQL:

select count(*) from Posts where Category like '%whatever%'

And that's all true if I keep things to just "var" but I wasn't -- I was being clever and building functions to build up my queries. And because I couldn't use "var" as a function parameter, I had to pick a type. I picked the wrong one: IEnumerable.

The problem with IEnumerable is that it doesn't have enough information to support the building up of queries. Let's take a look at the extension method of Count over an IEnumerable:

public static int Count<TSource>(this IEnumerable<TSource> source) {
...
int num = 0;
  using (IEnumerator<TSource> enumerator = source.GetEnumerator()) {
    while (enumerator.MoveNext()) { num++; }
  }
  return num;
}

See? It's not composing the source IEnumerable over which it's operating -- it's executing the enumerator and counting the results. Further, since our example IEnumerator was a Where statement, which was in turn a accessing the list of Posts from the database, the effect was filtering in the Where over objects constituted from the following SQL:

select * from Posts

How did I see that? Well, I tried hooking up the supremely useful SQL Profiler to my ISP's database that was holding the data, but I didn't have permission. Luckily, the SQL tab in LinqPad will show me what SQL is being executed and it showed me just that (or rather, the slightly more verbose and more correct SQL that LINQ to Entities generates in these circumstances).

Now, I had a problem. I didn't want to pass around IEnumerable, because clearly that's slowing things down. A lot. On the other hand, I don't want to use ObjectSet<Post> because it doesn't compose, i.e. Where doesn't return that. What is the right interface to use to compose separate expressions into a single SQL statement? As you've probably guessed by now from the title of this post, the answer is: IQueryable.

Unlike IEnumerable, IQueryable exposes the underlying expression so that it can be composed by the caller. In fact, if you look at the IQueryable implementation of the Count extension method, you'll see something very different:

public static int Count<TSource>(this IQueryable<TSource> source) {
  ...
  return source.Provider.Execute<int>(
Expression.Call(null,
((MethodInfo) MethodBase.GetCurrentMethod()).
MakeGenericMethod(
new Type[] { typeof(TSource) }),
new Expression[] { source.Expression }));
}

This code isn't exactly intuitive, but what's happening is that we're forming an expression which is composed of whatever expression is exposed by the IQueryable we're operating over and the Count method, which we're then implementing. To get this code path to execute for our example, we simply have to replace the use of IEnumerable with IQueryable:

IQueryable<Post> FilterByCategory(IQueryable<Post> posts, string category) {
  if( !string.IsNullOrEmpty(category) ) {
    return posts.Where(p => p.Category.Contains(category));
  }
}
...
  var posts = FilterByCategory(db.Posts, category);
  int count = posts.Count();
...

Notice that none of the actual code changes. However, this new code runs much faster and with the case- and null-insensitivity built into the 'like' operator in SQL instead of semantics of the Contains method in LINQ to Objects.

The way it works is that we stack one IQueryable implementation onto another, in our case Count works on the Where which works on the ObjectSet returned from the Posts property on the object context (ObjectSet itself is an IQueryable). Because each outer IQueryable is reaching into the expression exposed by the inner IQueryable, it's only the outermost one -- Count in our example -- that causes the execution (foreach would also do it, as would ToList() or ToArray()).

Using IEnumerable, I was pulling back the ~3000 posts from my blog, then filtering them on the client-side and then doing a count of that.With IQueryable, I execute the complete query on the server-side:

select count(*) from Posts where Category like '%whatever%'

And, as our felon friend Ms. Stewart would say: "that's a good thing."

0 comments




Data Binding, Currency and the WPF TreeView

I was building a little WPF app to explore a hierarchical space (OData, if you must know), so of course, I was using the TreeView. And since I'm a big fan of data binding, of course I've got a hierarchical data source (basically):

abstract class Node {
public abstract string Name { get; }
  public abstract IEnumerable<Node> { get; }
public XDocument Document { get { ... } }
public Uri Uri { get { ... } }
}

I then bind to the data source (of course, you can do this in XAML, too):

// set the data context of the grid containing the treeview and text boxes
grid.DataContext = new Node[] { Node.GetNode(new Uri(uri)) };
// bind the treeview to the entire collection of nodes
leftTreeView.SetBinding(TreeView.ItemsSourceProperty, ".");
// bind each text box to a property on the current node
queryTextBox.SetBinding(TextBox.TextProperty,
  new Binding("Uri") { Mode = BindingMode.OneWay });
documentTextBox.SetBinding(TextBox.TextProperty,
  new Binding("Document") { Mode = BindingMode.OneWay });

What we're trying to do here is leverage the idea of "currency" in WPF where if you share the same data context, then item controls like textboxes will bind to the "current" item as it's changed by the list control. If this was a listview instead of a treeview, that would work great (so long as you set the IsSynchronizedWithCurrentItem property to true).

The problem, as my co-author and the-keeper-of-all-WPF-knowledge Ian Griffiths reminded me this morning, is that currency is based on a single collection, whereas a TreeView control is based on multiple collections, i.e. the one at the root and each one at sub-node, etc. So, as I change the selection on the top node, the treeview has no single collection's current item to update (stored in an associated "view" of the data), so it doesn't update anything. As the user navigates from row to row, the "current" item never changes and our textboxes are not updated.

So, Ian informed me of a common "hack" to solve this problem. The basic idea is to forget about the magic "current node" and explicitly bind each control to the treeview's SelectedItem property. As it changes, regardless of which collection from whence the item came, each item control is updated, as data binding is supposed to work.

First, instead of setting the grid's DataContext to the actual data, shared with the treeview and the textboxes, we bind it to the currently selected treeview item:

// bind the grid containing the treeview and text boxes
// to point at the treeview's currently selected item
grid.DataContext = new Binding("SelectedItem") { ElementName = "leftTreeView" };

Now, because we want the treeview to in fact show our hierarchical collection of nodes, we set it's DataContext explicitly:

// set the treeview's DataContext to be the data we want it to show
leftTreeView.DataContext = new Node[] { Node.GetNode(new Uri(uri)) };

Now, the treeview will show the data we wanted it to show, as before, but as the user changes the selection, the treeview's SelectedItem property changes, which updates the grid's DataContext, which signals the textboxes, bound to properties on grid's DataContext (because the DataContext property is inherited and we haven't overridden it on the textboxes), and the textboxes are updated.

Or, in other words, the textboxes effectively have a new idea of the "current" item that meshes with how the treeview works. Thanks, Ian!

0 comments




.NET Source Code Mass Downloader

On 1/16/08, Microsoft announced the ability to download some of the .NET Framework source code for debugging. This download process was only supported inside of a properly configured Visual Studio 2008.

21 Days Later: Kerem Kusmezer and John Robbins released a tool to download the source code en mass. Frankly, I'm surprised it took so long. : )

0 comments




Microsoft needs you to build Emacs.Net

Interested? Drop Doug a line.

0 comments




Duck Typing for .NET!

For structural typing fans (and they'll be more of you over time -- trust me), David Meyer has posted a duck typing library for .NET. There are many reasons this is cool, but in summary, it allows for many of the dynamic features of languages like Python and Ruby to used used in any .NET language. Very cool.

0 comments




Lutz's Silverlight 1.1 Alpha Samples

Lutz has ported some of his .NET code to use the Silverlight 1.1 alpha, which includes the mini-CLR (or whatever we're calling it these days : ). Enjoy.

0 comments




WPF/E == Silverlight

If you haven't already heard about Microsoft's new high fidelity, cross-platform application development platform, you just haven't been paying attention. Silverlight is the new name for WPF/E (although it's still XAML-based) and does some *amazing* things.

And, if you can wait just a little while longer, you can read about Silverlight in Programming WPF (available now in Rough Cut format and for pre-order from Amazon) in an appendix by my friend and yours, Shawn Wildermuth. Shawn's been doing a ton of Silverlight work lately, including doing a bunch of Silverlight presentations for Microsoft, so he knows of what he speaks.

0 comments




The Potential of WPF/E

Savas turned me onto an amazing WPF/E application. I don't speak the language of the web site, but the screenshot on Savas's site is worth a look...

P.S. I don't smoke (except for the occasional cigar) and I definitely don't want to smell smoke while I eat or in my clothes, but the fact that smokers are no longer allowed to smoke most places strikes me as a violation of an important liberty. Have those studies about the effects of 3rd party smoke been verified?

0 comments




WPF XBAP App: British Library Books Online

"The British Library is one of the world's leading libraries and the national library of the United Kingdom. By charter, it holds a copy of every book ever published in the UK, along with 58 million newspapers, 4.5 million maps, and 3.5 million sound recordings. They hold some of the most priceless literary treasures in existence, including the Codex Sinaiticus (one of the oldest New Testaments in existence), the Lindisfarne Gospels, one of Leonardo Da Vinci's notebooks, the first atlas of Europe by Mercator, the original illustrated manuscript Lewis Carroll's Alice's Adventures in Wonderland, Jane Austen's History of England and Mozart's musical diary. ...

"Enter a fantastic new application, developed in partnership between the British Library and Armadillo Systems. The British Library have digitized the pages of fifteen of their most valuable works and created Turning the Pages, a browser-based WPF application that allows you to interact with these books in a virtual environment from the comfort of your home."

Wow. This is literally the only way to interact with some of this material and it's enabled with WPF. Nice.

0 comments




API Usability

Don has a piece up about something that I've always called "API Usability." The idea when building libraries is to write client code first against some pretend API that you wish existed and then to implement that API. Another good name for this approach would be "RAD API Design," simply because it's the same way I prefer to design UI -- layout the UI the way you'd like it to look and then implement it that way. Of course, I have to admit to preferring Don's name for this style of programming (I like what he calls my conferences, too : ).

BTW, the comments to Don's piece mention to startling similarity between this approach and Test-Driven Development (TDD). I'm a huge fan of TDD (NUnit is a wonderful tool I use all day every day). I'd say that TDD is a generalization of my little "API usability" technique in that you can use it for all kinds of things, e.g. code coverage, perf testing, stress testing, etc, including API usability.

P.S. If we fix the atmosphere, clean up the water, stop polluting the soil and learn to live in harmony with our environment, what's to motivate us to move off this rock before we lose our aggressive drive and then, when we're sipping Mai Thais, the sun explodes? Consuming this planet until nothing's left but an empty husk and we're forced, like locusts to move on to the next one, may well be the only thing that keeps our species alive (assuming we survive the coming ice age, of course).

0 comments




XAML Design Tools from Microsoft

If you're a designer looking for XAML design tools, and you're a designer type (you know -- beret, turtle neck, a wardrobe that extends beyond jeans and t-shirts), you've got two choices from Microsoft. You can use Expression Design [1] for static XAML graphics or Expression Blend (aka Sparkle) for dynamic XAML interfaces. Both are available in pre-release form (CTP for Design and Beta 1 for Blend), so give them a try.

[1] "Expression Design" is not to be confused with "Expression Designer", which is the old name for Sparkle (Blend).

P.S. Can someone give Scoble a hug? He's obviously a little worked up if he's recommending physical violence for disagreements about web site design. : )

0 comments




Holy Cow! An Entire WPF/E DevCenter!

I expected eventually to get a WPF/E (Windows Presentation Foundation Everywhere) download or two, but these guys went crazy; they just released an entire WPF/E DevCenter this morning. Here're just a few highlights:

Enjoy!

P.S. Regardless of whether humans caused global warming or whether it's part of a natural cycle, it's happening and we're going to have to deal with it.

0 comments




.NET Framework 3.0 (aka WinFX) Has Shipped!

After a monumental amount of work, the .NET Framework 3.0 has been completed! It ships out of the box in Vista, but for down level clients (Windows XP and Windows Server 2003) and developer tools, see the links below:

Wahoo!

0 comments




754 older posts       No newer posts