Category Archives: C#

Antlr 4 with C# and Visual Studio 2012

It’s been more than a year since I posted anything to Programming Pages, so I figure I should rectify that (been busy mainly with my alter-ego, Physics Pages).

Recently, I had another look at the parsing package Antlr and discovered that a whole new version has come out (Antlr 4), along with a book The Definitive Antlr 4 Reference, written by Antlr’s author, Terence Parr. However, as with Antlr 3, the book is written exclusively in Java, so a fair bit of detective work is needed to discover how to use it with C# in Visual Studio. The code required both to specify the lexer and parser, and to write the supporting C#, has pretty well completely changed from Antlr 3, so a fresh tutorial is needed.

Installation in Visual Studio

Support for Antlr4 exists for Visual Studio 2012 (and presumably 2010, although I no longer have this installed so I can’t test it), but not, it seems, for Visual Studio 2013. To get the files you need, visit the Antlr download page and get the ‘ANTLR 4 C#.dll’ zip file. Next, click on the ‘C# releases’ link, and from the page that loads, scroll down to ‘Visual Studio 2010 Extensions’ and click on the ‘ANTLR Language Support’ link, then ‘Download’ the Visual Studio plugin from that page, then install it, making sure you’ve selected the correct version of Visual Studio in the dialog.

To incorporate Antlr 4 into a VS project, create a new project in VS. In the project directory, create a folder named Reference and within the Reference folder create another folder called Antlr4. Unzip the entire contents of the ‘ANTLR 4 C#.dll’ zip file into the Antlr4 folder. You’ll need to do this for each project you create.

Then, in Solution Explorer, right-click on the project and select ‘Unload project’. Then right click on the project name (it’ll say ‘unavailable’ after it, but that’s OK) and select ‘Edit <Your project name>.csproj. Scroll to near the end of the file where you should find the line:

  <Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />

Insert the following lines directly after this line:

  <PropertyGroup>
    <!-- Folder containing Antlr4BuildTasks.dll -->
    <Antlr4BuildTaskPath>$(ProjectDir)..\Reference\Antlr4</Antlr4BuildTaskPath>
    <!-- Path to the ANTLR Tool itself. -->
    <Antlr4ToolPath>$(ProjectDir)..\Reference\Antlr4\antlr4-csharp-4.0.1-SNAPSHOT-complete.jar</Antlr4ToolPath>
  </PropertyGroup>
  <Import Project="$(ProjectDir)..\Reference\Antlr4\Antlr4.targets" />

Then right-click on the project name and select ‘Reload project’.

Finally, you’ll need to add a reference to the Antlr4 runtime in your project to provide access to the API. Right-click on References and select the correct version of the runtime dll from the Antlr4 folder you created above. The version should match the version of .NET that you’re using in your project, so if you’re using .NET 4.5, load the file ‘Antlr4.Runtime.v4.5.dll’. Now your project should be all set.

One final note: you’ll need Java to be installed in order to run Antlr4! This is true even if you’re coding in C#, since Antlr4 is written in Java and calls it to generate the code for your lexer and parser.

Writing an Antlr 4 grammar

You should now be in a position to start work on the actual code. If you installed the VS extension above, you can add an Antlr4 grammar to your project by right-clicking on the project name and selecting Add –> New Item. In the dialog you should see 7 Antlr items; 3 of these are for Antlr 4 files, with the other 4 being for Antlr 3. Since we’ll be working with both a lexer and a parser, select ‘ANTLR 4 Combined Grammar’, change the name of the file to whatever you like, and click ‘Add’.

As an illustration, we’ll generate a grammar for the good old four-function calculator, so we’ll call the project Calculator. In Solution Explorer you should find a file called Calculator.g4; this is where you write your parser and lexer rules.  The initial code in this file looks like this:

grammar Calculator;

@parser::members
{
	protected const int EOF = Eof;
}

@lexer::members
{
	protected const int EOF = Eof;
	protected const int HIDDEN = Hidden;
}

/*
 * Parser Rules
 */

compileUnit
	:	EOF
	;

/*
 * Lexer Rules
 */

WS
	:	' ' -> channel(HIDDEN)
	;

Don’t worry about the stuff up to line 12. What we’re interested in are the parser and lexer rules. The syntax for these has changed significantly from Antlr 3, to the extent that any grammar files you may have written for the earlier version very probably won’t work in Antlr 4.  However, the new syntax is much closer to the more standard ways of representing these rules in other systems like lex and yacc (if you’ve never heard of these, don’t worry; we won’t be using them).

Here’s the grammar file as modified for our calculator:

grammar Calculator;

@parser::members
{
	protected const int EOF = Eof;
}

@lexer::members
{
	protected const int EOF = Eof;
	protected const int HIDDEN = Hidden;
}

/*
 * Parser Rules
 */

prog: expr+ ;

expr : expr op=('*'|'/') expr	# MulDiv
	 | expr op=('+'|'-') expr	# AddSub
	 | INT					# int
	 | '(' expr ')'			# parens
	 ;

/*
 * Lexer Rules
 */
INT : [0-9]+;
MUL : '*';
DIV : '/';
ADD : '+';
SUB : '-';
WS
	:	(' ' | '\r' | '\n') -> channel(HIDDEN)
	;

First, look at the lexer rules. The INT token is defined using the usual regular expression for one or more digits. The four arithmetic operations are given labels that we’ll use later. Finally, we’ve modified the WS (whitespace) label so it includes blanks, returns and newlines. The ‘-> channel(HIDDEN)’ just tells the lexer to ignore whitespace.

Now look at the parser rules. The first rule is ‘prog’, which is defined as one or more ‘expr’s. The ‘expr’ is defined a single INT, or a single expr enclosed in parentheses, or two exprs separated by one of the four arithmetic operators.

Note that each line in the expr declaration has a label preceded by a # symbol at the end. These are not comments; rather they are tags that are used in writing the code that tells the parser what to do when each of these expressions is found.

This is where the biggest difference between Antlr 3 and Antlr 4 occurs: in Antlr 4, there is no code in the target language (C# here) written in the grammar file. All such code is moved elsewhere in the program. This makes the grammar file language-independent, so once you’ve written it you could drop it in to another project and use it support a Java application, or any other language for which Antlr 4 is defined.

Using the grammar in a C# program

So how exactly do you use the grammar to interpret an input string? Before we can use the lexer and parser, we need to write the code that should be run when each type of expression is parsed from the input. To do this, we need to write a C# class called a visitor. Antlr 4 has provided a base class for your visitor class, and it is named CalculatorBaseVisitor<Result>.  It is a generic class, and ‘Result’ is the data type that is returned by the various methods inside the class. Your job is to override some or all of the methods in the base class so that they run the code you want in response to each bit of parsed input.

In our case, we want the calculator to return an int for each expression it calculates, so create a new class called (say) CalculatorVisitor and make it inherit CalculatorBaseVisitor<int>. Your skeleton class looks like this:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Calculator
{
  class CalculatorVisitor : CalculatorBaseVisitor<int>
  {
  }
}

To see what methods you need to override, it’s easiest to use VS’s Intellisense. Within the class type the keywords ‘public override’ after which Intellisense should pop up with a list of methods you can override. Look at the methods that start with ‘Visit’. Among these you should find a method for each label you assigned in the ‘expr’ definition in the grammar file above. Thus you should have VisitInt, VisitParens, VisitMulDiv and VisitAddSub. These are the methods you need to override. The other Visit methods you can ignore, as the versions provided in the base class will work fine.

Here’s the complete class. We’ll discuss the code in a minute:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Calculator
{
  class CalculatorVisitor : CalculatorBaseVisitor<int>
  {
    public override int VisitInt(CalculatorParser.IntContext context)
    {
      return int.Parse(context.INT().GetText());
    }

    public override int VisitAddSub(CalculatorParser.AddSubContext context)
    {
      int left = Visit(context.expr(0));
      int right = Visit(context.expr(1));
      if (context.op.Type == CalculatorParser.ADD)
      {
        return left + right;
      }
      else
      {
        return left - right;
      }
    }

    public override int VisitMulDiv(CalculatorParser.MulDivContext context)
    {
      int left = Visit(context.expr(0));
      int right = Visit(context.expr(1));
      if (context.op.Type == CalculatorParser.MUL)
      {
        return left * right;
      }
      else
      {
        return left / right;
      }
    }

    public override int VisitParens(CalculatorParser.ParensContext context)
    {
      return Visit(context.expr());
    }
  }
}

First, notice that the argument of each method (called ‘context’) is different in each case, and corresponds to the tag used to define the method. Each ‘context’ object contains information required to evaluate it, and this content can be determined by how you defined the various expr lines in the grammar file above. (BTW, you may need to rebuild the project to get VS’s Intellisense to work here.)

Take the VisitInt() method first. This is called when the object being parsed is a single integer, so what you want to return is the value of this integer. We can get this by calling the INT() method of the context object, and then GetText() from that. This returns the integer as a string, so we need to use int.Parse to convert it to an int.

Now look at the VisitParens() method at the bottom. Here, the contents of the parentheses could be an expr of arbitrary complexity, but we want to return whatever that expr evaluates to as the result. This is what the inherited Visit() method does: it takes an expr as an argument and calls the correct method depending on the type of the expr. The expr() method being called from the ‘context’ returns this expr (which will have a tag attached to it to say whether it’s an Int, or an AddSub, or whatever) and Visit will call the correct method to evaluate this expr.

Finally, VisitAddSub and VisitMulDiv work pretty much the same way. Both of these represent binary operators, so there are two subsidiary exprs to evaluate before the operator is applied. In each case, we evaluate the left and right operands by calling Visit(context.expr(0)) and Visit(context.expr(1)) respectively. Then we check which operator is in the ‘context’. Note that in the grammar file above we defined a parameter called ‘op’ as the operator. Also note that the labels we gave to the four arithmetic operators in the lexer rules show up as fields within the Calculator Parser class, so we can compare the Type of ‘op’ with these labels to find out which operator we’re dealing with. Once we know that, we can return the correct calculation.

At long last, we’re ready to look at the code that uses all this stuff. Here’s the Main() function:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.IO;
using Antlr4.Runtime;
using Antlr4.Runtime.Misc;
using Antlr4.Runtime.Tree;

namespace Calculator
{
  class Program
  {
    static void Main(string[] args)
    {
      StreamReader inputStream = new StreamReader(Console.OpenStandardInput());
      AntlrInputStream input = new AntlrInputStream(inputStream.ReadToEnd());
      CalculatorLexer lexer = new CalculatorLexer(input);
      CommonTokenStream tokens = new CommonTokenStream(lexer);
      CalculatorParser parser = new CalculatorParser(tokens);
      IParseTree tree = parser.prog();
      Console.WriteLine(tree.ToStringTree(parser));
      CalculatorVisitor visitor = new CalculatorVisitor();
      Console.WriteLine(visitor.Visit(tree));
    }
  }
}

Note the ‘using’ statements at the top, required to access Antlr4.

We create a StreamReader from the console on line 17 so we can type in our expressions. Then on line 18 we pass the string read from the stream (via inputStream.ReadToEnd()) to an AntlrInputStream.

Important note: AntlrInputStream is supposed to accept a raw Stream object for its input, but I couldn’t get this to work. The program just hung up when attempting to read from the Stream directly. It seems this is a bug in Antlr  4 so may be fixed in a future release.

The lexer is then created with the AntlrInputStream, the tokens from the lexer are saved in a CommonTokenStream, which is then passed to the parser. The parser is then run by calling parser.prog(), which calls the ‘prog’ rule (defined in the grammar file above) that initializes the program. The output from the parser is saved in an IParseTree, which is then passed to a CalculatorVisitor to do the evaluations. Finally, the result of the parse + evaluation is printed.

To run the program, type one or more expressions in the console (you can separate them by newlines or blanks). When you’re done, type control-Z on a line by itself and the output from the program should then be displayed.

Advertisements

JSON with ASP.NET MVC 4

As a newcomer to JavaScript and jQuery, I wanted to write a (supposedly) simple little web site which used JSON (JavaScript Object Notation) to transmit information from a database to a web page. Surprisingly there didn’t seem to be any self-contained example or tutorial (that I could find) that showed how to do this basic operation. So here is the result of my attempt. It turns out that it is actually quite simple, once you know what to do.

First off, what is JSON? It is an alternative to XML as a way of sending information from one device to another. It’s a subset of the notation that can be used in JavaScript to define an object literal, which we saw in the last post. As an example, we might have some information on the numbers of various titles of comic books that are stored in a database. After we’ve retrieved this data from the database, we can encode it into JSON, so that it looks like this:

[
  {"Title":"Batman","Count":1},
  {"Title":"Fantastic Four","Count":2},
  {"Title":"Spider-man","Count":2},
  {"Title":"Superman","Count":1},
  {"Title":"Thor","Count":2}
]

That is, the data are stored as name-value pairs. In this case, each element in the array is actually a compound object consisting of two name-value pairs (Title and Count). We could have a simple list of single name-value pairs, or we could have deeper nesting of objects. There are various restrictions on the syntax for JSON objects, but as we’ll see, in MVC 4 you won’t usually have to write your own JSON, since there is a library routine that does it for you.

Now that we know what JSON is, how do we use it as a mediator between a database and a web page in ASP.NET MVC 4? We’ve seen that the procedure for generating web pages in MVC is as follows.

  1. Write a controller that contains a method that is called when a client requests a web page from the server.
  2. Write a view that is linked to the controller method, and which generates the markup that is returned to the client.

The controller method will usually collect some data from some other source, such as a database or some other C# code (which is usually part of the Model in MVC). This data is then packaged up, possibly in a C# object, and sent to the View, where the View formats the data and generates the HTML which is interpreted by the browser.

The procedure for using JSON doesn’t quite fit this pattern. Basically what happens is this:

  1. Client requests a web page.
  2. Controller method is called to process this request.
  3. The corresponding View is returned to the browser.
  4. The View contains some JavaScript code that makes another request to the server, this time asking for some data in JSON format.
  5. different Controller method from that called in step 2 is called to generate the JSON, which is returned to the browser.
  6. The JavaScript code processes the JSON and generates some HTML which is displayed.

The key point is that the JSON is not meant to be displayed as raw data by the browser; rather, it is retrieved by the browser and then processed on the client side before it is displayed.

We’ll illustrate this by showing how the data above can be obtained from the database and sent to a web page using JSON. First, we’ll add a controller to our ComicShop project that we’ve been using to demostrate things:

using System.Web.Mvc;
using ComicShop.Models;
using System.Collections.Generic;

namespace ComicShop.Controllers
{
  public class JsonController : Controller
  {
    //
    // GET: /Json/

    public ActionResult Index()
    {
      return View();
    }

    private IComicRepository comicRepository;

    public JsonController()
    {
      comicRepository = new ComicRepository();
    }

    public JsonController(IComicRepository comicRepository)
    {
      this.comicRepository = comicRepository;
    }

    public ViewResult JsonSummary()
    {
      return View();
    }

    public ActionResult Summary()
    {
      IEnumerable<ComicSummary> summaries = comicRepository.GetSummaries();
      return Json(summaries, JsonRequestBehavior.AllowGet);
    }

  }
}

This controller uses the IComicRepository interface and ComicRepository class that we introduced earlier. This controller contains two methods of interest here. The one called JsonSummary() simply returns a View, which is the View that requests the JSON data (we’ll get to this View in a moment).

The Summary() method is the one that produces the JSON data. The first line in this method requests the data from the database (see the earlier post for details of how this is done; all that’s important here is that we do get this data). The data is returned as an IEnumerable<ComicSummary> list. The ComicSummary class is defined as:

namespace ComicShop.Models
{
  public class ComicSummary
  {
    public string Title { get; set; }
    public int Count { get; set; }
  }
}

That is, it contains two data fields named Title and Count.

The second line in Summary() calls the library method Json() to create the JSON data. It is passed the list of ComicSummary objects in the variable summaries. The second parameter allows Summary() to provide JSON data in response to an HTTP Get request (by default, such a response is blocked for security reasons, so this parameter is needed). This returns the JSON string we gave above.

That’s all there is to producing JSON data (as promised, you didn’t have to actually write any raw JSON yourself, since the Json() method does it for you). If you want to verify that Summary() actually works, you can call it directly from a web browser by using the URL http://localhost:36195/Json/Summary (the port number for your localhost server will probably be different; what’s important is the path /Json/Summary), and you should see the raw JSON data printed in the browser.

Note that we did not associate a View with the Summary() controller method. As far as the browser is concerned, it is receiving raw text as a response to its request to Summary, so it just displays that. We need to do something special if we want it to interpret the JSON as data and incorporate it into a ‘real’ web page.

To that end, we turn our attention to the JsonSummary() controller method, and have a look at its View. This is in the file JsonSummary.cshtml:

<h2>JsonSummary</h2>
<script>
    $.getJSON('/Json/Summary',  function (data) {
        var items = [];
        $.each(data, function (key, val) {
            items.push('<li>' + key + ': ' + val.Title + ' (' + val.Count + ')</li>');
        });

        $('<ul/>', {
            html: items.join('')
        }).appendTo('body');
    });
</script>

As you can see, most of this file is JavaScript. However, it’s not all primitive JavaScript; it makes use of jQuery, so you need to make sure your web project has jQuery installed. If you started with an Empty project, you’ll need to do this yourself; if you started with a full Internet project, it will already be included.

To add in jQuery, right-click on the Solution in SolutionExplorer and select Manage NuGet Packages. Search in the Online section and find jQuery, then click Install. JSON should already be installed – you can check this by opening up References in SolutionExplorer and seeing that Newtonsoft.Json  is included in the list.

NuGet should take care of installing these packages and should find some jQuery JavaScript files in your Scripts folder (if you didn’t have a Scripts folder, it will be created by the installation).

We’re not quite done yet, though. In order for jQuery to be used in your View, that View needs to run the jQuery script. You could include a <script> tag in the View file itself, but if you plan on using jQuery in a lot of Views, it makes more sense to put it in the common layout file which we considered earlier. This file is called _Layout.cshtml and is in the Shared folder within the Views folder. Add the call to the jquery script in the <head> section of the layout. The modified _Layout.cshtml file looks like this:

@{
    Layout = null;
}

<!DOCTYPE html>

<html>
<head>
    <script src="~/Scripts/jquery-1.8.2.js"></script>
    <meta name="viewport" content="width=device-width" />
    <title>Comics</title>
</head>
<body>
    <h2>Comic Shop</h2>
    <div>
        @RenderBody()
    </div>
</body>
</html>

The call to the script is on line 9. The actual version number of jquery may differ from 1.8.2 if you install it later on, but just insert whatever version you got from NuGet.

That’s all we need to get the JavaScript to run, but there’s one more thing that’s very useful, and that’s to get Intellisense working for jQuery. Complete instructions for doing this are here, provided you’re using VS 2012. My _references.js file looks like this:

/// <reference path="jquery-1.8.2.intellisense.js" />
/// <reference path="jquery-1.8.2.js" />

Right, now we can look (finally) at JsonSummary.cshtml. A lot of the code there relies on a knowledge of JavaScript and jQuery syntax, so I won’t go through it in detail. However, I’ll mention a few key points so you can adapt it for your own purposes.

First, the $.getJSON() call is a jQuery method which makes a request to the URL given as its first argument for some JSON data. In our case, we use a relative URL /Json/Summary (the leading / is important: don’t leave it out!). If this call is successful, the JavaScript function given as the second argument to $.getJSON is called. (If the call fails, this function isn’t called, and you’ll get nothing on your web page apart from the HTML that is outside the <script> tag. This can make debugging a nightmare, since you won’t know what’s wrong, but never mind.)

The argument ‘data’ sent to this function is the JSON code. This function then constructs an array called ‘items’ by iterating through ‘data’ using the jQuery $.each() method. The second argument to $.each() is another function, this time taking two arguments. The ‘key’ argument is the integer index of the element from ‘data’ that is being processed, and ‘val’ is that actual element from ‘data’. Remember from the JSON above that each element in ‘data’ is a compound object of type ComicSummary, so it contains a Title and a Count field. We access these fields to create a HTML list entry on line 6, and push this onto the items array.

The last bit of code on lines 9 to 11 creates a <ul> tag and adds the items list to it, then it adds the <ul> tag to the body of the page. The final result looks like this:

LINQ to XML: Creating an XElement

In the last post, we showed how to create a simple XML document in which the data were entered from a DataGrid. We gave the code for constructing the XML as follows:

    private XElement XmlFromLibrary()
    {
      ObservableCollection<Book> library = (ObservableCollection<Book>)((ObjectDataProvider)FindResource("LibraryGrid")).Data;
      XElement libraryElement =
        new XElement("LIBRARY",
          library.Select(book =>
            new XElement("BOOK",
              new XElement("AUTHOR", book.Author),
              new XElement("TITLE", book.Title),
              new XElement("PRICE", book.Price))));
      return libraryElement;
    }

The ‘library’ is fetched from a Windows resource defined in the XAML, with this resource being bound to the DataGrid (see earlier post for full details).

In writing this code, we glossed over some of the details of how the XElement is built. In fact, we used several techniques in this code that could do with further explanation.

The basic form of an XElement constructor is

XElement(XName name, params object[] content);

The first parameter gives the name of the XElement, which is used as the tag when writing out the XML. Usually, we’ll just enter a string here, and rely on the fact that the XElement constructor will convert this into an XName internally so we don’t need to worry about it.

The second parameter uses C#’s params keyword, which allows a variable number (one or more) of arguments to be passed to the constructor. As the data type of the content is just ‘object’, any data type  can be passed as the content of an XElement, and it’s here that the richness of the XElement class comes into play.

There are 8 specific data types that are handled in special ways when passed in as the content.

  1. A string is, as you might expect, just used as is as the content of the XML tag. (In fact, a string is converted into an XText object before it is used.)
  2. XText: This is a special class which is added as a child node of the XElement, but its value, which is a string, is used as the XElement’s text content.
  3. XCData: This allows insertion of the XML CData type, which consists of unparsed character data. Such strings may contain characters such as > and &, which ordinarily have a special meaning in XML syntax, but would be ignored here.
  4. XElement: The content can be another XElement, which is added as a child node to the parent XElement.
  5. XAttribute: This object is added as a child node, and represents an attribute of the parent node.
  6. XComment: Allows a comment to be attached to the XElement.
  7. IProcessingInstruction: Allows a processing instruction to be added to the XElement. (You don’t need to worry about these for most XML that you’ll write, but I may get back to them at some point.)
  8. IEnumerable: This is the magic data type, since it allows collections of data, such as those produced by LINQ query operations, to be passed in as content. The elements in the collection are iterated over, and each element is treated as a separate parameter. We used this feature in the code above to insert a list of Book objects into the XML using a LINQ Select() call.

In addition, you can also pass a null as the content (which does have its uses, though we won’t go into that here).

Finally, if the content is any other data type, the XElement will call the ToString() for that data type and use that as the content. This can cause some confusion, since there are some other LINQ to XML classes (such as XDocument) that are used to attach properties to the XML file that will be accepted as content for XElement, but rather than having the expected effect, XElement will just call its ToString() method and use that as content.

As a simple example, here’s some code that creates an XElement using most of the data types above as content:

using System;
using System.Xml.Linq;

namespace LinqXml03
{
  class Program
  {
    static void Main(string[] args)
    {
      XElement document = new XElement("Library",
        new XComment("This is a test library"),
        new XElement("Program", new Program()),
        new XElement("Book",
          new XElement("Author", "Isaac Asimov"),
          new XElement("Title", "I, Robot"),
          new XAttribute("Pages", 357)),
        new XElement("Book",
          new XElement("Author", "Samuel R. Delaney"),
          new XElement("Title", "Nova"),
          new XAttribute("Pages", 293)),
        new XCData("This contains a > and a & character"),
        new XText("This also contains a > and a & character"));
      Console.WriteLine(document);
   }
  }
}

This produces the output:

<Library>
  <!--This is a test library-->
  <Program>LinqXml03.Program</Program>
  <Book Pages="357">
    <Author>Isaac Asimov</Author>
    <Title>I, Robot</Title>
  </Book>
  <Book Pages="293">
    <Author>Samuel R. Delaney</Author>
    <Title>Nova</Title>
  </Book><![CDATA[This contains a > and a & character]]>This also contains a &gt; and a &amp; character</Library>

The top level XElement has the name ‘Library’. Its first content is a comment, which is written with the <!–…–> delimiters. Next, we’ve added a content object of type Program (that is, the class in which this program is written). The output is produced as a normal XElement tag, but the ToString() method is called from the Program class since it’s not one of the data types that has special meaning as an XElement content. The default ToString() method for a class just produces that class’s full pathname, which in this case is LinqXml03.Program.

Next, we add a couple of Book elements, each of which contains a couple of other XElements for the author and title. We’ve also added an XAttribute for the number of pages in the book.

The last two lines demonstrate the difference between XCData and XText. The XCData reproduces the given text exactly, and encloses it within the <![…]]> delimiters used for CData. The XText places the text as the content of the Library tag, and translates special characters into the XML code, so that > become &gt; and & becomes &amp;.

We’ve already seen an example of using IEnumerable in the code fragment at the top of this post.

LINQ for XML – the basics

LINQ provides a library of classes and methods that allow XML to be generated and imported quite easily (certainly more easily than with previous .NET libraries).

We’ll assume the reader is familiar with the basics of XML syntax and dive in with a simple little program that allows the user to enter some details for books in a library, then store this data to a disk file as XML (and of course to read in data from an XML file and display it).

The GUI is a WPF DataGrid and a menu for handling file operations, as shown:

We’ll represent the data internally using a Book class to represent each book, and an ObservableCollection to represent the collection of books. The data structures are similar to those that we used in discussing data binding to lists and combo boxes. The Book class is a bit simpler than it was there:

using System.ComponentModel;
namespace LinqXml02
{
  public class Book : INotifyPropertyChanged
  {
    public event PropertyChangedEventHandler PropertyChanged;
    protected void Notify(string propName)
    {
      if (this.PropertyChanged != null)
      {
        PropertyChanged(this, new PropertyChangedEventArgs(propName));
      }
    }

    string author;

    public string Author
    {
      get { return author; }
      set
      {
        author = value;
        Notify("Author");
      }
    }

    string title;

    public string Title
    {
      get { return title; }
      set
      {
        title = value;
        Notify("Title");
      }
    }

    decimal price;

    public decimal Price
    {
      get { return price; }
      set
      {
        price = value;
        Notify("Price");
      }
    }

    public Book() { }
    public Book(string author, string title, decimal price)
    {
      this.author = author;
      this.title = title;
      this.price = price;
    }
  }
}

The ObservableCollection is created in a special class called Library:

using System;
using System.Collections.ObjectModel;

namespace LinqXml02
{
  public class Library
  {
    Random rand = new Random();
    private decimal BookPrice()
    {
      decimal price = rand.Next(0, 5000) / 100m;
      return price;
    }

    public ObservableCollection<Book> GetLibrary()
    {
      ObservableCollection<Book> library = new ObservableCollection<Book>();
      return library;
    }
  }
}

This class serves as a resource in the XAML file:

<Window x:Class="LinqXml02.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:local="clr-namespace:LinqXml02"
        Title="MainWindow" Height="350" Width="525">
    <Window.Resources>
        <ObjectDataProvider x:Key="LibraryGrid"
                            ObjectType="{x:Type local:Library}"
                            MethodName="GetLibrary"/>
    </Window.Resources>
    <Grid DataContext="{StaticResource LibraryGrid}" HorizontalAlignment="Stretch">
        <Grid.RowDefinitions>
            <RowDefinition Height="Auto"/>
            <RowDefinition/>
        </Grid.RowDefinitions>
        <Menu VerticalAlignment="Top">
            <MenuItem Header="_File">
                <MenuItem x:Name="saveMenuItem" Header="_Save" HorizontalAlignment="Left" Width="145" Click="saveMenuItem_Click"/>
                <MenuItem x:Name="saveAsMenuItem" Header="Save _as" HorizontalAlignment="Left" Width="145" Click="saveAsMenuItem_Click"/>
                <MenuItem x:Name="openMenuItem" Header="_Open" HorizontalAlignment="Left" Width="145" Click="openMenuItem_Click"/>
                <Separator HorizontalAlignment="Left" Width="145"/>
                <MenuItem x:Name="exitMenuItem" Header="E_xit" HorizontalAlignment="Left" Width="145" Click="exitMenuItem_Click"/>
            </MenuItem>
        </Menu>
        <DataGrid x:Name="bookGrid" Grid.Row="1" ItemsSource="{Binding}" AutoGenerateColumns="False"  HorizontalAlignment="Stretch">
            <DataGrid.Columns>
                <DataGridTextColumn Header="Author" Binding="{Binding Author}" Width="45*"/>
                <DataGridTextColumn Header="Title" Binding="{Binding Title}"  Width="45*"/>
                <DataGridTextColumn Header="Price" Binding="{Binding Price}"  Width="10*"/>
            </DataGrid.Columns>
        </DataGrid>

    </Grid>
</Window>

On lines 7 to 9 we create the resource, then use it as the data context for the Grid on line 11. The DataGrid defined on line 25 uses this data context as the binding for its ItemsSource property, and then we define the three columns, each bound to a property in the Book class. We could have used the auto-generate column feature of a DataGrid, but that doesn’t allow us to customize the widths of the columns, which we’ve done here by assigning each of the Author and Title columns 45% of the horizontal width, with Price getting the remaining 10%.

With the data structures set up and the binding in place, we could run the program and enter some book data, and the data binding will automatically update the ObservableCollection as we enter data into the DataGrid. However, at this stage we have no way of saving the data thus entered. For that we introduce the XML.

First, we’ll have a look at the event handlers for the Save and Save As menu items.

    string saveFilename = "";
    private void saveAsMenuItem_Click(object sender, RoutedEventArgs e)
    {
      SaveFileDialog saveDialog = new SaveFileDialog();
      saveDialog.Filter = "XML file|*.xml";
      saveDialog.Title = "Save library";
      if (saveDialog.ShowDialog() == true)
      {
        saveFilename = saveDialog.FileName;
        saveMenuItem_Click(sender, e);
        Title = "Library - " + saveDialog.FileName;
      }
    }

    private void saveMenuItem_Click(object sender, RoutedEventArgs e)
    {
      if (saveFilename.Equals(""))
      {
        saveAsMenuItem_Click(sender, e);
      }
      else
      {
        XElement saveLibraryXml = XmlFromLibrary();
        saveLibraryXml.Save(saveFilename);
      }
    }

The SaveFileDialog (and OpenFileDialog) classes are in the old Microsoft.Win32 namespace, but they still seem to work well enough. In order to allow us to save changes to a currently open file, we have an auxiliary string called saveFilename. If this string has zero length, then we open the SaveFileDialog to get the user to select a filename. The dialog has a filter that displays only .xml files.

Once a file has been chosen, the saveMenuItem_Click() handler is called, and the method XmlFromLibrary() is called. We’ll consider this in a moment, but first we need to describe the XElement class.

In LINQ’s handling of XML, all XML tags are represented by XElement objects. There is no need for a separate, top-level document object in which to place the XElements; XElement itself can serve as the top level, and all lower levels.

Nested tags in the XML are represented simply as nested XElement objects. This gives the C# code a structure that is easy to understand for the human reader.

Now we can have a look at XmlFromLibrary():

    private XElement XmlFromLibrary()
    {
      ObservableCollection<Book> library = (ObservableCollection<Book>)((ObjectDataProvider)FindResource("LibraryGrid")).Data;
      XElement libraryElement =
        new XElement("LIBRARY",
          library.Select(book =>
            new XElement("BOOK",
              new XElement("AUTHOR", book.Author),
              new XElement("TITLE", book.Title),
              new XElement("PRICE", book.Price))));
      return libraryElement;
    }

After retrieving ‘library’ from the Windows resources, we create the XML representation of the library with a single C# statement. The top level object is libraryElement, which is given the tag LIBRARY. The second argument to its contructor is built using a LINQ Select() call on library. Remember that library consists of a list of Book objects, so we simply iterate through each Book in the list, and construct a new XElement for each Book. Within the Book’s XElement, we add 3 more XElements for the Author, Title and Price fields.

And that’s it. The code is very clean. Back in saveMenuItem_Click(), we simply call the Save() method from the XElement object to save the file to disk. The resulting file for the books shown in the picture above is:

<?xml version="1.0" encoding="utf-8"?>
<LIBRARY>
  <BOOK>
    <AUTHOR>Asimov, Isaac</AUTHOR>
    <TITLE>I, Robot</TITLE>
    <PRICE>3.50</PRICE>
  </BOOK>
  <BOOK>
    <AUTHOR>Niven, Larry</AUTHOR>
    <TITLE>Ringworld</TITLE>
    <PRICE>4.95</PRICE>
  </BOOK>
  <BOOK>
    <AUTHOR>Asimov, Isaac</AUTHOR>
    <TITLE>Foundation</TITLE>
    <PRICE>2.25</PRICE>
  </BOOK>
  <BOOK>
    <AUTHOR>Simak, Clifford D.</AUTHOR>
    <TITLE>Buckets of Diamonds</TITLE>
    <PRICE>5.00</PRICE>
  </BOOK>
</LIBRARY>

The Save() method produces the usual first line of an XML file, and then writes out the XML itself, all neatly indented.

To read the XML file back into the program, we need to construct the internal ObservableCollection from the XML. This is almost as easy as producing the XML in the first place. Here’s the code for the Open menu item, and the associated LibraryFromXml() method that reads the XML:

    private void openMenuItem_Click(object sender, RoutedEventArgs e)
    {
      OpenFileDialog openDialog = new OpenFileDialog();
      openDialog.DefaultExt = ".xml";
      openDialog.Filter = "XML documents (.xml)|*.xml";
      bool? result = openDialog.ShowDialog();
      if (result == true)
      {
        XElement libraryXml = XElement.Load(openDialog.FileName);
        Title = "Library - " + openDialog.FileName;
        LibraryFromXml(libraryXml);
        saveFilename = openDialog.FileName;
      }
    }

    private void LibraryFromXml(XElement libraryXml)
    {
      ObservableCollection<Book> library = (ObservableCollection<Book>)((ObjectDataProvider)FindResource("LibraryGrid")).Data;
      library.Clear();
      var bookElements = libraryXml.Elements("BOOK");
      foreach (XElement book in bookElements)
      {
        Book addBook = new Book(
          (string)book.Element("AUTHOR"),
          (string)book.Element("TITLE"),
          (decimal)book.Element("PRICE"));
        library.Add(addBook);
      }
    }

In the openMenuItem_Click() handler, we use the static XElement.Load() method to read the XML from the file into an XElement.

In LibraryFromXml() we again retrieve the library resource and clear it of existing data. Then we call the Elements() method on the XElement to retrieve a list of BOOK tags. This produces an IEnumerable list of XElements for the BOOK objects in the original XML. For each of these, we simply create a Book object by extracting the AUTHOR, TITLE and PRICE XElements for each BOOK, and then add this Book object to the library. The data binding takes care of the rest, so the DataGrid is automatically updated to display the list of books we read in.

There’s a lot more that can be done with LINQ and XML, but this little example should show you that for saving and reading basic XML, LINQ is easy to use.

Code for this post available here.

LINQ: ToLookup

We’ve seen how to create a Dictionary using LINQ. A Dictionary is a hash table in which only one object may be stored for each key. It can also be useful to store more than one object for a given key, and for that, the C# Lookup<> (part of the System.Linq namespace) generic type can be used.

LINQ provides the ToLookup() method for creating Lookups. It works in much the same way as ToDictionary(), except that as many objects as you like can be attached to each key.

Returning to our example using Canadian prime ministers, we can create a Lookup in which the key is the first letter of the prime minister’s last name. The code is

      PrimeMinisters[] primeMinisters = PrimeMinisters.GetPrimeMinistersArray();
      var pmLookup01 = primeMinisters.ToLookup(pm => pm.lastName[0]);
      var keys01 = pmLookup01.Select(pm => pm.Key).OrderBy(key => key);
      Console.WriteLine("----->pmLookup01");
      foreach (var key in keys01)
      {
        Console.WriteLine("PMs starting with {0}", key);
        foreach (var pm in pmLookup01[key])
        {
          Console.WriteLine("  -  {0}, {1}", pm.lastName, pm.firstName);
        }
      }

This is the simplest version of ToLookup(). The method takes a single argument, which is a function specifying how to calculate the key. In this case, we just take the first char in the string pm.lastName.

For some reason, the Lookup class doesn’t contain a property for retrieving the list of keys, so we need to use a roundabout method to get them. Line 5 uses a Select() to retrieve the keys and an OrderBy() to sort them into alphabetical order. We can then iterate over the keys and, for each key, we can iterate over the prime ministers for that key. Note that the object pmLookup01[key] is not a single object; rather it contains a list of all prime ministers whose last name begins with the letter contained in the key.

The output from this code is:

----->pmLookup01
PMs starting with A
  -  Abbott, John
PMs starting with B
  -  Bowell, Mackenzie
  -  Borden, Robert
  -  Bennett, Richard
PMs starting with C
  -  Clark, Joe
  -  Campbell, Kim
  -  Chrétien, Jean
PMs starting with D
  -  Diefenbaker, John
PMs starting with H
  -  Harper, Stephen
PMs starting with L
  -  Laurier, Wilfrid
PMs starting with M
  -  Macdonald, John
  -  Mackenzie, Alexander
  -  Meighen, Arthur
  -  Mackenzie King, William
  -  Mulroney, Brian
  -  Martin, Paul
PMs starting with P
  -  Pearson, Lester
PMs starting with S
  -  St. Laurent, Louis
PMs starting with T
  -  Thompson, John
  -  Tupper, Charles
  -  Trudeau, Pierre
  -  Turner, John

We can do the same thing using the second form of ToLookup(), which allows us to specify an EqualityComparer to be used in determining which keys are equal. The comparer class looks like this:

using System.Collections.Generic;

namespace LinqObjects01
{
  class LookupComparer : IEqualityComparer<string>
  {
    public bool Equals(string x, string y)
    {
      return x[0] == y[0];
    }

    public int GetHashCode(string obj)
    {
      return obj[0].GetHashCode();
    }
  }
}

This comparer compares two strings and says they are equal if their first characters are equal. Using this class, we can apply the second form of ToLookup():

      PrimeMinisters[] primeMinisters = PrimeMinisters.GetPrimeMinistersArray();
      var pmLookup02 = primeMinisters.ToLookup(pm => pm.lastName,
        new LookupComparer());
      var keys02 = pmLookup02.Select(pm => pm.Key).OrderBy(key => key);
      Console.WriteLine("----->pmLookup02");
      foreach (var key in keys02)
      {
        Console.WriteLine("PMs starting with {0}", key[0]);
        foreach (var pm in pmLookup02[key])
        {
          Console.WriteLine("  -  {0}, {1}", pm.lastName, pm.firstName);
        }
      }

The first argument to ToLookup() now passes the entire pm.lastName, and the second argument to ToLookup() is the comparer object.

When we print out the results, we have to remember that the key for each entry in the Lookup is now the full last name of the first prime minister encountered whose name starts with a given letter. Thus if we printed out the full key, we’d get a full last name. That’s why we print out key[0] on line 8; that way we get the first letter of the name.

The third version of ToLookup() allows us to specify a custom data type to return. If we wanted just the first and last names of each prime minister, for example, we could write:

      PrimeMinisters[] primeMinisters = PrimeMinisters.GetPrimeMinistersArray();
      var pmLookup03 = primeMinisters.ToLookup(pm => pm.lastName[0],
        pm => new
        {
          lastName = pm.lastName,
          firstName = pm.firstName
        });
      var keys03 = pmLookup03.Select(pm => pm.Key).OrderBy(key => key);
      Console.WriteLine("----->pmLookup03");
      foreach (var key in keys03)
      {
        Console.WriteLine("PMs starting with {0}", key);
        foreach (var pm in pmLookup03[key])
        {
          Console.WriteLine("  -  {0}, {1}", pm.lastName, pm.firstName);
        }
      }

We’ve returned to using the first letter of the last name as the key (that is, there’s no comparer), and passed in an anonymous data type as the second argument to ToLookup(). Apart from that, the code is the same as in the first example.

Finally, the fourth version allows us to specify both a custom data type and a comparer, so we can combine that last two examples to get this:

      PrimeMinisters[] primeMinisters = PrimeMinisters.GetPrimeMinistersArray();
       var pmLookup04 = primeMinisters.ToLookup(pm => pm.lastName,
       pm => new
        {
          lastName = pm.lastName,
          firstName = pm.firstName
        },
        new LookupComparer());
       var keys04 = pmLookup04.Select(pm => pm.Key).OrderBy(key => key);
       Console.WriteLine("----->pmLookup04");
       foreach (var key in keys04)
       {
         Console.WriteLine("PMs starting with {0}", key[0]);
         foreach (var pm in pmLookup04[key])
         {
           Console.WriteLine("  -  {0}, {1}", pm.lastName, pm.firstName);
         }
       }

Now we’re back to using the full last name as the key, since the comparer does the checking for matching first letters. Just make sure you pass in the arguments in the right order: (1) choose key; (2) choose custom data type; (3) choose comparer.

MVC: user input model

In the last post, we saw how restructuring the code so that a view model is used to prepare the data for the view helps decouple the view from the calculations required to process the data before displaying it. In this post, we’ll have a look at how to apply a similar approach to user input.

We’ll modify the ComicShop site by adding in a boolean parameter specifying whether we’ve read a particular comic. To do this, we’ll need to modify the underlying database by adding a Read field. You can do this by opening the database in Visual Studio’s Server Explorer (double-click on the database file in Solution Explorer to do this), then navigating to the Books table, then right-clicking and selecting Edit Table Schema. Add in a column called Read, and set its type to ‘bit’. Set its default value to 0.

We also need to modify the Book class by adding a bool property called Read.

To make things easier in what follows, we will also modify the home page so that it displays the Read status of each comic book. We can do this by modifying the IndexModel.cshtml code to this:

@using ComicShop.Models;
@model List<Book>

<h2>Comics</h2>
<p><a href="/Home/Add">Add new comic</a></p>
<p><a href="/Home/Summary">Summary</a></p>
<p><a href="/Home/Read">Comics read</a></p>

<ul>
    @foreach (var comic in @Model)
    {
        <li>
                @comic.Title: <b>@comic.Volume</b> (@comic.Issue) [@(comic.Read ? "Read" : "Unread")]
        </li>
    }
</ul>

Now we’re ready to add a page on which the user can change the Read status of one or more comics. We’d like to display a table of the comics in the database, with each row in the table showing the title, volume and issue of the comic, and a checkbox allowing the Read status to be edited. The controller method for this is very simple:

    public ViewResult Read()
    {
      var model = comicRepository.GetComicList();
      return View(model);
    }

Before we construct the view, we need to stop and think about what the Read view will return. All we really need to identify the read status of a comic is the comic’s primary key (the Id column in the database) and the read status itself. The input model for this page is then the simple class:

namespace ComicShop.Models
{
  public class ComicReadStatus
  {
    public int Id { get; set; }
    public bool Read { get; set; }
  }
}

This just uses the same code we had before, since the view requires the same data as the home page. The Read.cshtml file looks like this:

@using ComicShop.Models;
@model IEnumerable<Book>

@{
    ViewBag.Title = "Read";
}

<h2>Comics Read</h2>
<div>
    <form action="@Url.Action("UpdateReadStatus")" method="post">
        <table>
            <tr>
                <th>Title</th>
                <th>Volume</th>
                <th>Issue</th>
                <th>Read?</th>
            </tr>
            @{ int index = 0; }
            @foreach (var comic in Model)
            {
                <tr>
                    <td>@comic.Title</td>
                    <td>@comic.Volume</td>
                    <td>@comic.Issue</td>
                    <td>
                        @Html.CheckBox("comicRead[" + index + "].Read", comic.Read)
                        <input name="@("comicRead[" + index + "].Id")" value="@comic.Id" type="hidden" />
                    </td>
                </tr>
                index++;
            }
        </table>
        <button name="submit">Update</button>
    </form>
</div>

The page constructs a table in the same way as we did for the ComicSummary page in the last post. The one notable feature here is the addition of the checkbox. Rather than use bare HTML for this, we’ve used one of MVC’s Html helper functions. This function takes 2 parameters; the first is the name of the checkbox and the second is its initial value.

Since we’re displaying a table of comics, there will be a checkbox for each comic, so we name the checkbox in such a way that the table builds an array. The name of the checkbox is translated dynamically into a data type, so we build up the array by defining an ‘index’ parameter to number the rows in the table and insert this into the ‘comicRead’ array for each row in the table. It’s important here to make sure that the properties of the array are the same as those in the data type we are to export from the form. In this case, we’ll be exporting a list of ComicReadStatus objects, so each array element must have a bool Read and an int Id property. The Read property will be set to the value in the checkbox when the form’s submit button is pressed.

If you’re familiar with basic HTML, you might know that a form returns a value for the checkbox only if it is checked, so that if the checkbox is cleared, there would be no corresponding data sent in the post from the form. The Html.Checkbox() function gets around this problem by defining an additional hidden HTML control with the same name as the checkbox, and a permanent value of ‘false’. If the checkbox is true, it overrides this hidden field, but if the checkbox is false it is ignored and the hidden field then gets sent back from the form. Thus we will have a definite value for each checkbox whether it is true or false.

The explicit hidden field in the Read.cshtml code passes the comic’s Id value back from the form. Thus the output of the form is a list of ComicReadStatus objects, each of which contains a primary key of a comic and its Read value as specified by the user on the form.

In the form definition on line 10, we see that the action called by submitting the form is UpdateReadStatus, so we’ll need to write that before we try to use the form (although you can run the site now and go to the /Home/Read page to see the form if you like).

The action method in HomeController is again very simple:

    public ActionResult UpdateReadStatus(List<ComicReadStatus> comicRead)
    {
      comicRepository.UpdateReadStatus(comicRead);
      return RedirectToAction("Index");
    }

The input parameter is a List<ComicReadStatus>. You might wonder how the program knows to convert the data sent back from the form into this structure. This is a feature of MVC’s model binding technology, and for now it’s probably safer just to accept that it works. One thing is important though: you must ensure that the name of the parameter (‘comicRead’ here) is the same as that of the array you defined in Read.cshtml’s checkbox. Unlike the usual C# parameter names, where the parameter in the function definition doesn’t have to match that of the variable that is sent into the function, here it does matter, and if you make the names different, the object received by the action method is null (with all the pain that that generates when you try to run the method).

We’ve delegated the work done by UpdateReadStatus() to a method called UpdateReadStatus() in the ComicRepository. This allows us to decouple the controller from the access to the data source, as we’ve done so far in order to enable unit testing. The last line of the action method redirects the browser back to the home page where you can see the results of your edits.

In the ComicRepository class that we’ve been using for interaction with the database, we therefore need to write some code which saves any changes made by the user back to the database. Herein lies a bit of a problem, since the Read view has no connection with, or knowledge of, where its data comes from (which is correct). However, as such, there’s no way to tell from the data sent back by the form which comics have had their Read status changed. There’s really only one way we can be sure of saving all the changes to the database: we’ll have to iterate over all the comics that were listed on the Read view and check to see which ones have had their Read status changed. This obviously isn’t very efficient, but the only other way of doing this involves responding directly to the user’s clicks on the Read page, and that will take us too far afield. So we’ll content ourselves with a fairly brute force method for updating the database. (Actually, a proper view wouldn’t have that many lines in a table anyway, so this probably isn’t all that inefficient, but still…)

Here’s the code in the ComicRepository class:

    public void UpdateReadStatus(List<ComicReadStatus> comicReadStatus)
    {
      foreach (var comic in comicReadStatus)
      {
        Book book = database.ComicBooks.Find(comic.Id);
        if (book.Read != comic.Read)
        {
          book.Read = comic.Read;
          database.Entry(book).State = System.Data.EntityState.Modified;
        }
        database.SaveChanges();
      }
    }

For each comic in the list, we use the DbSet’s Find() method to look up the full comic object in the database. Remember that the comicReadStatus list contains only ComicReadStatus objects, which contain only the Id and Read fields for a given comic book. The Find() method uses an object’s primary key to look it up in the database and then returns the full object.

We then check to see if the Read field has changed and if so, we change the Read field in the full Book object, and then mark this object’s State as Modified. Finally SaveChanges() will save all the rows that have been marked as modified.

MVC: the view model

The ComicShop application we’ve presented so far has been quite basic, in that it does nothing more than allow the user to store comics in a local database, and display a list of these comics on the home page. In the process of creating the application, though, we’ve shown the division of labour between the model (where the data are stored and manipulated), the controller (which handles requests from the user and decides which view to display), and the view (which creates the actual HTML that is sent back to the browser).

You might think that, having covered all three components for which MVC is named, we’re finished with the basics. However, there is another logical component that forms part of the design when the application becomes anything more than very simple. This is the view model. (The view model occupies a more central position in another design system known as MVVM (thanks Stuart!), but we’ll leave a discussion of that until later.)

To illustrate the need for a view model (and to explain what it is), suppose we wanted to add a web page that displayed a summary of the comics in the database by listing each unique title followed by the number of comics of that title in the database. Ultimately, we’d like to display this information as a table in HTML, where each row in the table consists of a string (the title) and an int (the count for that title). To obtain this information, we need to query the database and do a count of each title returned. Where should this work be done?

One option is that the controller queries the database for a complete list of comics (as it does for the page that displays the complete list), and sends this list as is to the view, leaving the view to count up the number of comics for each title and then display the results. This isn’t very efficient, since the view is being forced to do a lot of stuff that has nothing to do with the display of the page.

Another option is that the controller does the calculations itself and then packages up the results and sends them to the view for display. That seems a little better, since at least the view is being cluttered with calculations. However, it isn’t really the job of the controller either, since its job is mainly to route the page request from the user to the correct view.

The solution currently accepted as best practice is to have the controller create a view model which is then sent to the view. The view model is a mirror in C# code of the data that the view requires in order to produce the HTML. In the case of our ComicShop summary page, we’d like a list of rows for the table, where each row contains the title and count for a group of comics. We therefore create a class representing a single row in the table, which looks like this:

namespace ComicShop.Models
{
  public class ComicSummary
  {
    public string Title { get; set; }
    public int Count { get; set; }
  }
}

The idea is now to have the controller construct the view model as an IEnumerable<ComicSummary> list (that is, a collection of ComicSummary objects that implements the IEnumerable interface), and that is what gets sent to the view. This list is the view model for the view that displays the summary information. It contains only the information that the view needs, and doesn’t require the view to do any further calculation.

Where should this list be constructed? To keep the controller clean, the construction shouldn’t be done directly inside the controller. Farming the process out to a separate class also fits in with the design we adopted when implementing unit testing, since we’d like to be able to test the summary-generation code in the controller without directly accessing the database. A sensible place to put the list-building code, then, is in the classes (one for testing and one for running normally) that implement the IComicRepository interface that we designed in the post on unit testing.

When we’re extracting the information from the database, we can actually do all the calculations required using LINQ. Here’s the method in the ComicRepository class:

    public ComicContext database = new ComicContext("ComicContextDb");
    public IEnumerable<ComicSummary> GetSummaries()
    {
      var summaries = database.ComicBooks.Select(book => book).OrderBy(book => book.Title).
        GroupBy(comic => comic.Title,
        (title, titleGroup) => new ComicSummary
        {
          Title = title,
          Count = titleGroup.Count()
        }
        );
      return  summaries;
    }

The first line in the method is the LINQ code that selects all the comics, orders them by title, and then applies the GroupBy() method to sort them into groups where the key to each group is the Title field. The second argument to GroupBy() is a function which takes two parameters: the first is the title of the comic, and the second (titleGroup) is the group of comics with that title. We then construct a ComicSummary object containing the title and the count of comics in that group. The final returned object is the required IEnumerable<ComicSummary> list.

Back in the HomeController class, we add an action for the Summary page:

    public ViewResult Summary()
    {
      IEnumerable<ComicSummary> summaries = comicRepository.GetSummaries();
      return View(summaries);
    }

(The comicRepository object is initialized in the constructor as before.) The controller code is still very clean, since the work of building the view model is done in ComicRepository.

Finally, the view for the Summary page is also very simple:

@using ComicShop.Models
@model IEnumerable<ComicSummary>

<div>
    <h2>Comic summary</h2>
    <table>
        <tr>
            <th>Title</th>
            <th>Count</th>
        </tr>
        @foreach (var summary in Model)
        {
            <tr>
                <td>@summary.Title</td>
                <td>@summary.Count</td>
            </tr>
        }
    </table>
</div>

The model directive at the top is set to an IEnumerable<ComicSummary> data type, making this a strongly typed view. The rest of the view builds the table by iterating over the elements in the model.

Finally, we can add a GetSummaries() method to our test repository class ComicRepositoryTest. It might look something like this:

    public IEnumerable<ComicSummary> GetSummaries()
    {
      return new[]
      {
        new ComicSummary
        {
          Title = "Superman",
          Count = 3
        },
        new ComicSummary
        {
          Title = "Thor",
          Count = 12
        }
      };
    }

We manually add a couple of ComicSummary objects to a list and return it. To use this, we can add some tests to HomeControllerTests. For example, we can test that the number of entries in the table is 2:

    [TestMethod]
    public void Summary_Count()
    {
      HomeController homeController = new HomeController(
        new ComicRepositoryTest());
      var summary = homeController.Summary();
      var table = summary.Model as ICollection<ComicSummary>;
      Assert.IsTrue(table.Count == 2, "Count is {0}", table.Count);
    }

Since IEnumerable <T> doesn’t support a Count property, we’ve cast the Model field of ‘summary’ to an ICollection, which does have a Count. ICollection inherits IEnumerable, so the cast is valid providing the underlying object implements ICollection, which in this case it does.

Unit testing: decoupling the database

In the last post, we saw how to add a simple unit test to an MVC4 project. At the time, we noted that our ComicShop project wasn’t in the best form for unit testing, since the HomeController accessed the database (in which the comic books are stored) directly. This means that any test of HomeController required interacting with the database, so we’re not testing just the code in the controller in isolation.

Here, we’ll see how to decouple the database from the controller, and in the process, how to add in a localized test for the controller that is independent of the database.

The idea is to define a C# interface whose job it is to provide the connection to the data source. In the functioning application, this interface is implemented by a class that does connect to the database, but for testing purposes, we use another class that also implements this interface, but which provides a test set of data without referring to a database.

To design this interface, we look at the HomeController code, and note that there are, at the moment, only two methods that interact with the database: Index(), which retrieves the list of Books in the database, and the HttpPost version of Add(), which takes data off a form submitted by the user and adds this to the database. The other Add() method just displays a form in which the user enters data; no connection to the database is used here.

Our interface, then, should contain two methods which, in classes implementing the interface, are used to connect to the database in one case and to some local data source in the testing case. Here’s the definition of the interface:

using System.Collections.Generic;

namespace ComicShop.Models
{
  public interface IComicRepository
  {
    List<Book> GetComicList();
    void AddComic(Book book);
  }
}

We’ve added this interface to the Models namespace, since it deals with data storage, and classes that implement it are model classes.

We can now define a class called ComicRepository which implements IComicRepository and move the code relying on the database from HomeController to ComicRepository. This class is as follows:

using System.Collections.Generic;
using System.Linq;

namespace ComicShop.Models
{
  public class ComicRepository : IComicRepository
  {
    public ComicContext database = new ComicContext("ComicContextDb");
    public List<Book> GetComicList()
    {
      var comics = database.ComicBooks.Select(book => book).OrderBy(book => book.Title);
      return comics.ToList();
    }

    public void AddComic(Book book)
    {
      database.ComicBooks.Add(book);
      database.SaveChanges();
    }
  }
}

GetComicList() contacts the database and retrieves the list of books using the same LINQ code as we had in HomeController before. However, it converts the result to a List<Book> and returns it. There is no mention of a View, since that’s the job of the controller; this class should concern itself purely with manipulating the data.

In a similar vein, AddComic() accepts a Book and adds it to the database.

The new version of HomeController now looks like this:

using System.Web.Mvc;
using ComicShop.Models;

namespace ComicShop.Controllers
{
  public class HomeController : Controller
  {
    private IComicRepository comicRepository;

    public HomeController()
    {
      comicRepository = new ComicRepository();
    }

    public HomeController(IComicRepository comicRepository)
    {
      this.comicRepository = comicRepository;
    }

    public ViewResult Index()
    {
      var model = comicRepository.GetComicList();
      return View("IndexModel", model);
    }

    public ActionResult Add()
    {
      return View();
    }

    [HttpPost]
    public ActionResult Add(Book book)
    {
      comicRepository.AddComic(book);
      return Content("New comic added.");
    }
  }
}

All explicit references to the database have been eliminated. The interaction with the data source is done entirely through the comicRepository object.

A couple of important additions have been made: we now have two constructors in HomeController.

The argumentless constructor initializes the comicRepository object to ComicRepository, the class that accesses the database. We want this to be the default behaviour of the controller, since this is what is done when we are running the web site.

The second constructor allows us to specify the repository used by HomeController. We’ll use this constructor when we write the test class.

First, we need to define a test class that implements IComicRepository. This class should use a local data source without any reference to the database. Since the web site uses a List<Book> as its working data structure, we can use this as the test data source. We therefore define ComicRepositoryTest like this:

using System.Collections.Generic;
using ComicShop.Models;

namespace ComicShop.UnitTests.Models
{
  public class ComicRepositoryTest : IComicRepository
  {
    private List<Book> comicRepository;

    public ComicRepositoryTest()
    {
      comicRepository = new List<Book> {
        new Book {
          Title = "Incredible Hulk", Volume = 3, Issue = 134
        },
        new Book {
          Title = "Superboy", Volume = 1, Issue = 17
        }
      };
    }

    public List<Book> GetComicList()
    {
      return comicRepository;
    }

    public void AddComic(Book book)
    {
      comicRepository.Add(book);
    }
  }
}

We’ve put this class in the namespace ComicShop.UnitTests.Models (you’ll need to create a new folder in your tests project in Visual Studio), since it’s a data class. Since it implements IComicRepository, we need a ‘using ComicShop.Models’ at the top.

This class has a constuctor that initializes the List<Book> and adds a couple of Books to it. Note that although this class is in the UnitTests namespace, it’s not a TestClass itself; it’s just an auxiliary class that is used in the testing.

We then implement the two methods in IComicRepository. GetComicList() just returns the List<Book>, and AddComic() adds the Book to the List<Book>.

We can now go back to HomeControllerTests (that we started in the last post), and rewrite the tests properly. The class now looks like this:

using System.Collections.Generic;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using ComicShop.Controllers;
using ComicShop.Models;
using ComicShop.UnitTests.Models;
using System.Web.Mvc;

namespace ComicShop.UnitTests.Controllers
{
  [TestClass]
  public class HomeControllerTests
  {
    [TestMethod]
    public void Index_ValidView()
    {
      HomeController homeController = new HomeController(
        new ComicRepositoryTest());
      var result = homeController.Index();
      Assert.IsNotNull(result);
    }

    [TestMethod]
    public void Add_AddBook()
    {
      HomeController homeController = new HomeController(
        new ComicRepositoryTest());
      homeController.Add(
        new Book
        {
          Title = "Iron Man",
          Volume = 3,
          Issue = 57,
          Publisher = "Marvel"
        }
      );
      var result = (ViewResult)homeController.Index();
      var modelList = (List<Book>)result.Model;
      Assert.IsTrue(modelList.Count == 3, "Count is {0}", modelList.Count);
    }
  }
}

Each test should be entirely self-contained, so we don’t define any class variables; we create an instance of HomeController within each TestMethod. For the tests, we pass an instance of ComicRepositoryTest to the HomeController constructor, so HomeController uses the local data source rather than the database. The Index_ValidView() test is the same test we wrote in the previous post, except this time there is no interaction with the database.

We’ve added another TestMethod to test adding a Book to the data source. Add_AddBook() calls HomeController’s Add() method with a Book that is created on the spot, rather than using data sent in by the user from a web page. We then retrieve the list of Books by calling Index() and extracting the Model field from the ViewResult. Since the list defined in ComicRepositoryTest had two Books in it and we’ve added another one, the size of the list should now be 3, so that’s what we test in the Assert.IsTrue() statement.

Some of Assert’s methods allow an optional message to be printed out if the test fails. In this case, the last two parameters sent to Assert.IsTrue() are a string and its parameter. If the Count of items in the list is not 3, we’d like to see how many items are in the list, so we print that out. You can test this feature by changing the test so it fails (by testing if Count is 4, say). To see this message, click on the failed test in Visual Studio’s Test Explorer, and the results should show up at the bottom of the panel.

We could go on and add more tests to the ComicShop project, but you should get the idea at this stage.

MVC: unit tests

Having spent all my working life in academia, I never got into the formal-testing-of-code mindset, even though I wrote some fairly hefty programs. Of course I tested my programs, but it was mainly a case of trying every possible scenario by hand and then fixing things that didn’t work.

Testing has turned into something of an industry in its own right. The more extreme testers advocate test-driven development, in which the test for a section of code is written before the code itself, and the code is then written so it passes the test.

Whatever your philosophy on testing, having a set of tests which can be run at any time at the press of a button is surely a nice idea, and Visual Studio has tools that support this form of testing. We’ll have an introductory look at how to include unit tests in an MVC4 project, using Visual Studio 2012 (version 11).

When you create an MVC4 project in VS, there is an option to create a test suite at the same time. However, if you don’t do this and opt instead for the empty project (as we’ve done up to now), it’s easy enough to add in a test project later.

One thing is worth pointing out here, though. Older versions of VS had a feature where you could right-click on a class or method in your code and add a unit test on the fly. For technical reasons, this feature has been removed in VS2012, so the only way of adding tests (as far as I can tell) is via the test project itself.

To add a test project to an existing solution, right-click on the existing solution in Solution Explorer and select Add–>New project, then click on the Visual C#–>Test node and select Unit Test Project. You’ll get a starter C# file which looks like this:

using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace ComicShop.UnitTest
{
  [TestClass]
  public class UnitTest1
  {
    [TestMethod]
    public void TestMethod1()
    {
    }
  }
}

Note that a class used for unit testing must have the attribute TestClass, and a method must have the attribute TestMethod.

Even at this stage, we can run the test. In the Test menu, select Windows–>Test explorer, and in the Test Explorer window, click on Run All. Since there are no conditions yet that can fail, the test should pass, and you’ll see a message to this effect.

To see what happens when a test fails, insert the line Assert.IsTrue(1 > 2); in the TestMethod1 method and run the test again. The condition is clearly false, so the test will fail, and the Test Explorer tells you this, along with which test failed.

Since we’ll be testing code we’ve written in the original ComicShop project, the ComicShop.UnitTests project needs access to that code, which it doesn’t have at the moment. To rectify this, right-click on References in the ComicShop.UnitTests project and then Add reference. Expand the Solution node and under Projects, select the ComicShop project and add it.

We now have enough material to be able to add unit tests to our existing code. However, the code as it stands isn’t in quite the right form for ‘proper’ unit testing. The reason is that the controller code we wrote in earlier posts has a direct link with the database used to store the comic book data. Thus any unit test we write for a controller method will involve connecting to the database. You might think that any proper test would have to do this anyway; after all, the program connects to the database so surely any test would also have to do this.

The point is that there are really two distinct operations involved in the controller method as it stands. First, there is the connection to, and interaction with, the database. Second, there is the logic in the controlller method itself; that is, the logic that determines what the controller does with the data it gets from the database (or sends to it, if we’re adding a new comic book). We’d really like to test these two things separately.

We’ll see in the next post how to decouple the controller from the database, but for now, just to get used to writing a unit test, we can write a simple unit test with the controller code as it stands. As a reminder, here’s the code for the Index() method in the HomeController.

    private ComicContext database = new ComicContext("ComicContextDb");
    public ActionResult Index()
    {
      var comics = database.ComicBooks.Select(book => book).OrderBy(book => book.Title);
      var model = comics.ToList();
      return View("IndexModel", model);
    }

We’ll test that the ActionResult returned at the end isn’t null.

It’s a good idea to mirror the the original application’s code structure in the unit test code structure, so we’d like tests on HomeController to be in a Controllers namespace within ComicShop.UnitTests. To do this, right-click on the ComicShop.UnitTests project and select Add–>Add folder, then name this folder Controllers. Right-click on the Controllers folder and then on Add–>Unit test. Name the new test HomeControllerTests. (Do not click on Add–>Class to add the test, since the resulting class won’t be included in the tests.)

Here’s the complete class that does our simple test.

using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using ComicShop.Controllers;
using System.Web.Mvc;

namespace ComicShop.UnitTests.Controllers
{
  [TestClass]
  public class HomeControllerTests
  {
    [TestMethod]
    public void Index_ValidView()
    {
      HomeController homeController = new HomeController();
      var result = homeController.Index();
      Assert.IsNotNull(result);
    }
  }
}

We create a HomeController object, call its Index() method and save the returned value in ‘result’. We then apply one of the static methods in the Assert class (one of the provided UnitTesting classes). As you can see from the Intellisense, there are a lot of tests in Assert. This one does just as it says: it tests if ‘result’ is null.

Before we leave this introductory post, it’s worth pointing out that any method that is to be used as a TestMethod must be void (return nothing) and have no parameters. The unit test must be completely encapsulated within the code for that method.

In order to test anything more significant than this, we should really restructure our code so that we decouple the controller from the database, but that’s a topic for the next post.

Attributes

An often-overlooked feature of C# is the attribute. You might have seen them in code you’ve looked at: they appear as what look like variable names or method calls enclosed in square brackets that appear before method or class definitions (or sometimes in other places). So what are they?

Put simply, an attribute is a way of storing information about the code within the code itself; meta-information if you like. There are several pre-defined attributes you can use, but in this post we’ll look at how you can define your own attributes.

As an example, suppose you want to store creation and modification dates for some of the methods in your code. Typically, a given method has one creation date, but could have any number of modification dates. We can store each of these dates as an attribute attached to each method.

A class used to create attribute objects must inherit the Attribute class (in the System namespace). If the class has a constructor that requires parameters, these parameters must all be primitive types (things like int, float, double, string and so on). This is due to technical aspects concerning the way attributes are stored in the code, so we won’t go into that here; just remember that you can’t pass anything other than primitive data types in an attribute’s constructor.

Since we want to store dates, this means that we can’t use a DateTime structure as a parameter directly; we’ll need to pass the day, month and year as separate int parameters. With that in mind, here’s the definition of the Created attribute class which we’ll use to store the date a method was created:

  public class Created : Attribute
  {
    public Created(int year, int month, int day)
    {
      Date = new DateTime(year, month, day);
    }

    public DateTime Date { get; set; }
  }

You’ll see that Created inherits the Attribute class, and that it has one constructor used to initialize the Date object. (There are a number of other features that can be specified when defining an attribute class, which we’ll get to in a minute. This is the simplest attribute definition, in which we accept the default values for all these features.)

With this definition, we can add a Created attribute to a method.

    [Created(2012, 6, 1)]
    public void Method_1()
    {
      // ........
    }

We put the attribute in square brackets immediately before the method to which it’s attached. The attribute code is just a call to the Created constructor, and the attribute is a Created object.

For the Modified attribute, we need the same data (a DateTime) stored, but we want to allow more than one Modified attribute for each method. The default behaviour of an attribute is to allow only a single instance of that attribute per method. However, we can override this behaviour by giving the attribute class itself an attribute. So we define our Modified attribute as follows:

  [AttributeUsage(AttributeTargets.Method | AttributeTargets.Class, AllowMultiple=true)]
  public class Modified : Attribute
  {
    public Modified(int year, int month, int day)
    {
      Date = new DateTime(year, month, day);
    }

    public DateTime Date { get; set; }
  }

The class definition here is the same as for Created; the difference is the AttributeUsage line at the start. AttributeUsage is a System pre-defined attribute which specifies how the following attribute class is allowed to be used. This code is just another call to the AttributeUsage constructor, this time with two parameters. The first parameter specifies what C# language elements the attribute can be used with. The default is AttributeTargets.All, which applied to our Created attribute above. Here, we’ve specified that Modified can be used methods and classes, though there are several other language elements that could be specified as well.

The second parameter gives a value for AllowMultiple, which defaults to false. Since we want to allow multiple Modified attributes, we set this to true. Note that there’s no way of setting just the AllowMultiple parameter without also specifying the AttributeTargets, since AllowMultiple is always the second parameter in the constructor.

With this definition, we can now add some Modified attributes to our methods. Here is the revised code, this time with two methods:

    [Created(2012, 6, 1)]
    [Modified(2012, 6, 12), Modified(2012, 8, 27)]
    public void Method_1()
    {
      // ........
    }

    [Created(2011, 11, 14)]
    [Modified(2011, 12, 12), Modified(2012, 2, 29), Modified(2012, 4, 30)]
    public void Method_2()
    {
      // .........
    }

All this is fine, but clearly attributes aren’t much use if all we can do is define them. We need a way of accessing their values later on. This is done using C#’s reflection techniques.

Reflection allows us to access properties of the code that is running at the time. We can extract information on pretty well all levels of the code, including classes and individual methods within classes. Here’s the code for accessing and printing the attributes we defined above:

    static void Main(string[] args)
    {
      MethodInfo[] methods = typeof(Program).GetMethods();
      foreach (MethodInfo method in methods)
      {
        WriteHistory(method);
      }
    }

    public static void WriteHistory(MethodInfo method)
    {
      Console.WriteLine("History for {0}", method.Name);
      Attribute[] attrs = Attribute.GetCustomAttributes(method);
      foreach (Attribute attr in attrs)
      {
        PropertyInfo dateInfo = attr.GetType().GetProperty("Date");
        if (dateInfo != null)
        {
          Console.WriteLine("    {0}: {1}", attr.GetType(),
            ((DateTime)dateInfo.GetValue(attr)).ToString("MMM dd yyyy"));
        }
      }
    }

These two static methods are defined in the same class (called Program) as the other methods above, but we could equally well have defined them elsewhere; we’d just need to modify the code a bit to access the information.

The Main() method finds the type of the enclosing Program class and calls GetMethods() to return an array of MethodInfo objects for each method in the class. This array includes entries for the methods we’ve defined here, but it also includes methods inherited from the base class (‘object’ in this case).

We then loop through this array and pass each MethodInfo object to the WriteHistory() method.

The MethodInfo class, as its name implies, contains information on the method which it represents. In WriteHistory(), we first write out the method’s name, then we extract an array of Attribute objects for that method.

In the loop over the Attribute objects, we want to print out the value of the Date property for each attribute (whether it’s Created or Modified). In this special case, since both attributes contain a Date property, we can use  the Type class’s GetProperty() method to extract the Date property, and store the result in a PropertyInfo object.

If it finds the desired property (if the result isn’t null), we can then print out its value, where we’ve cast this to a DateTime and given the ToString() method a formatting string to make the date appears neat.

The output of this program is:

History for Method_1
    AttributeTest02.Modified: Aug 27 2012
    AttributeTest02.Modified: Jun 12 2012
    AttributeTest02.Created: Jun 01 2012
History for Method_2
    AttributeTest02.Modified: Apr 30 2012
    AttributeTest02.Created: Nov 14 2011
    AttributeTest02.Modified: Dec 12 2011
    AttributeTest02.Modified: Feb 29 2012
History for WriteHistory
History for ToString
History for Equals
History for GetHashCode
History for GetType

You can see that the attributes don’t always appear in the same order in which they were defined, so you may need to sort the output if the order is important. For example, here we’d usually like the Created attribute to appear before the Modified ones, and the Modified ones to be in chronological order. These are details we can deal with later.

Note also that the GetMethods() method returns all the methods for the Program class (apart from Main()), some of which are inherited.