Angular - Handling promise status using filter

In angular application you can often see a code logically similar to:

function SubmitController($http) {
  var vm = this;

  vm.inProgress= false;
  vm.resolved = false;
  vm.rejected = false;

  vm.submitData = function() {
    vm.inProgress = true;
    return $http.post('....')
      .then(function() {
        vm.resolved = true;
        //Doing something usefull
      })
      .catch(function() {
        vm.rejected = true;
      })
      .finally(function() {
        vm.inProgress = false;
      });
  }
}

And a template

<button ng-disabled='vm.inProgress' ng-click='vm.submitData()'>Submit</button>
<span class='ng-hide' ng-show='vm.inProgress'>Submitting...</span>
<span class='ng-hide' ng-show='vm.resolved'>Submitted succsssfully!</span>
<span class='ng-hide' ng-show='vm.rejected'>Failed to submit!</span>

Looks familiar, isn't it? And it's getting even more messy if you have several actions like this. So following the DRY principal I came across with a very simple and flexible solution, just keep reading :)

The Filter

The filter is applied on the result of the action that returns promise and transforms it to a status object. The object has inProgress, resolved and rejected properties. The filter will be updating those properties on resolve or reject cases. So here is the code:

function promiseStatus() {
  return function(promise) {
    var status = {
      inProgress: true,
      resolved: false,
      rejected: false
    };
    promise
      .then(function() {
        status.resolved = true;
      })
      .catch(function() {
        status.rejected = true;
      })
      .finally(function() {
        status.inProgress = false;
      });
    return status;
  }
}

All you need to do is adjust your template like this

<button 
  ng-disabled='status.inProgress' 
  ng-click='status = (vm.submitData() | promiseStatus)'>Submit</button>
<span class='ng-hide' ng-show='status.inProgress'>Submitting...</span>
<span class='ng-hide' ng-show='status.resolved'>Submitted succsssfully!</span>
<span class='ng-hide' ng-show='status.rejected'>Failed to submit!</span>

As you can see the template is pretty match the same but your controller is crystal clear now:

function SubmitController($http) {
  var vm = this;

  vm.submitData = function() {
    return $http.post('....')
      .then(function() {
        //Doing something usefull
      });
  }
}

Full source code can be found on Github.

Cooking POCOs recipe: T4 and XML based DSL

In my previous post I talked about code generation of DTOs using TSQL script. In this post I will talk about more general solution. I will show how to easily build DSL processor using Visual Studio T4 templates. I will be using XML based DSL.

Source code used in this post can be found on the Github. Please feel free to use it on your projects.

T4 Text Templates in Visual Studio

There is a lot of information on the web about T4 text templates so I will just mention some key points. In general T4 templates are very similar to traditional ASPX templates. The major difference is that a # sing is used instead of %:

Current time is <#= DateTime.Now.ToShortTimeString() #>

Unfortunately Visual Studio doesn't have a built-in syntax highlighting and code completion. Luckily there are third party extensions available. If you're a ReSharper user then you can use a very nice ForTea extension. It's open source and can be installed using ReSharper Extension Manager.

In general that there are two types of T4 templates:

• Run time T4 text template - Used at run time as a name suggests. One of the common uses of the run time templates is to generate a body of an email.
• Design time T4 text template - With design time templates you can generate code files dynamically at design (I would say development) time. Common use of the design time templates is to read some input from another file and produce an output code based on the the input. So in this post I will show how use the design time templates to generate DTOs.

Using T4 Deisgn time templates to generate DTOs

The are a lot of commands and events on my project. At first they were tiny classes with just a set of properties:

public class AddDepartmentCommand
{
    public string ReferenceNumber { get; set; }
    public string Name { get; set; }
    public string Description { get; set; }
}

Then to simplify debugging most commands got ToString overridden. Then to simplify unit testing most commands got IEquatable implemented and so on. So at the moment we have a minimal command similar to below:

public class AddDepartmentCommand : IEquatable<AddDepartmentCommand>
{
    public string ReferenceNumber { get; set; }
    public string Name { get; set; }
    public string Description { get; set; }

    public AddDepartmentCommand(string referencenumber = default(string), string name = default(string), string description = default(string))
    {
        ReferenceNumber = referencenumber;
        Name = name;
        Description = description;
    }

    public bool Equals(AddDepartmentCommand other)
    {
        return
            ReferenceNumber == other.ReferenceNumber &&
            Name == other.Name &&
            Description == other.Description;
    }

    public override string ToString()
    {
        return string.Format("", ReferenceNumber, Name, Description);
    }
}

Initially we kept related DTOs in a single source file but very soon the file became to long to skim through quickly. The file with just five commands like above would be about 200 lines of code! Putting each DTO in a separate code file is also not the best option. There will be a lot of code files just defining your DTOs making it harder to see and analyse the structure of your project.

An approach we are currently using is an XML based DSL file where all the commands are defined:

<?xml version="1.0" encoding="utf-8" ?>
<DepartmentCommands namespace="DotnetT4PocoDsl.Commands">
    <AddDepartment ReferenceNumber="string" Name="string" Description="string" />
    <UpdateDepartment Id="int" ReferenceNumber="string" Name="string" Description="string" />
    <RemoveDepartment Id="int" Reason="string" />
</DepartmentCommands>

This file is processed with the Design time T4 template that generates command classes with aforementioned code structure.

The template

The template is quite long and boring so I'll show just some key parts of it. Full source code can be found on the Github.
At the top of the template there are input DSL file paths. The paths are relative to the project root.

//Define your inputs below
var inputs = new[]
{
   "Commands.xml",
   "Events.xml"
};

Then iterate through the inputs and load corresponding DSL file

var source = Host.ResolvePath(input); //Resolve actual path of the input file
var sourceDoc = XDocument.Load(source);

The rest logic is straight forward. We just iterate through the XML elements and generate corresponding items of our DTO class.
For example to generate the most simple Equals method you can use just this:

public bool Equals(<#= classDefElement.Name #> other)
{
    return <#= String.Join(" && ", propertyDefAttribs.Select(prop => prop.Name + " == " + "other." + prop.Name)) #>;
}

That will produce

public bool Equals(UpdateDepartmentCommand other)
{
    return Id == other.Id && ReferenceNumber == other.ReferenceNumber......
}

However in certain cases you may want the Equals to be smarter and handle complex types. Some of my commands have properties of type IDicitonary and XElement. So let's tweak our Equals method to handle them:

<#
var equalityParts = propertyDefAttribs.Select(property =>
{
    if (property.Value == "XElement")
        return "XNode.DeepEquals(" + property.Name + ", other." + property.Name + ")";
    if (property.Value.StartsWith("IDictionary"))
        return property.Name + ".SequenceEqual(" + "other." + property.Name + ")";
    return property.Name + " == " + "other." + property.Name;
});
#>
public bool Equals(<#= classDefElement.Name #> other)
{
    return <#= String.Join(" &&\r\n           ", equalityParts) #>;
}

So now this will produce:

public bool Equals(AddDepartmentCommand other)
{
    return ReferenceNumber == other.ReferenceNumber &&
           Name == other.Name &&
           Description == other.Description &&
           CustomFields.SequenceEqual(other.CustomFields) &&
           XNode.DeepEquals(FieldsMeta, other.FieldsMeta);
}

ToString is also quite simple to generate:

<# var toStringParts = propertyDefAttribs.Select(property => "\"" + property.Name + ": \", " + property.Name); #>
public override string ToString()
{
    return String.Concat("<<#= classDefElement.Name #> ", <#= String.Join(", \", \" , ", toStringParts) #>, ">");
}

This gives a clear and simple ToString method:

public override string ToString()
{
    return String.Concat("<AddDepartmentCommand ", "ReferenceNumber: ", ReferenceNumber,.........);
}

Regenerating DTOs to reflect input changes

Visual Studio automatically generates the output for you when you save the template, however it knows nothing about your inputs. In order to regenerate the DOTs in case of the input changes you basically have two options: you can open the template and just do Ctrl + S, you can use Run Custom Tool Command (see below)

One of the greatest benefits of using T4 Design time templates for code generation is flexibility. On the other hand editing very long T4 template can be quite challenging. So if you are using some better approach then please share your ideas.

Cooking POCOs recipe: SQL based DTOs

Do you have a lot of POCOs in your projects? I do and it's quite tedious and boring for me to add such stuff by hand. Well ReSharper makes this task a bit easier but anyway a lot of typing has to be done. Most of my projects have two general kinds of POCOs: DTOs and Events or Commands. In this post I'll show you my recipe on cooking DTOs that are used to transfer data that is going to be fetched using SQL (TSQL to be exact).

SQL based DTOs recipe

Many of my DTOs are based on the SQL queries and thus they can be derived from some database table (or at least some fields of it). So simple solution is to create a TSQL script that generates the DTO class from database schema. The script is below:

DECLARE @tableName NVARCHAR(MAX), @schemaName NVARCHAR(MAX), @className NVARCHAR(MAX)
 
--------------- Input arguments ---------------
SET @tableName = 'Incidents'
SET @schemaName = 'dbo'
SET @className = 'IncidentDto'
--------------- Input arguments end -----------

DECLARE tableColumns CURSOR LOCAL FOR
SELECT cols.name, cols.system_type_id, cols.is_nullable FROM sys.columns cols
 JOIN sys.tables tbl ON cols.object_id = tbl.object_id
 WHERE tbl.name = @tableName
 
PRINT 'public class ' + @className
PRINT '{'
 
OPEN tableColumns
DECLARE @name NVARCHAR(MAX), @typeId INT, @isNullable BIT, @typeName NVARCHAR(MAX)
FETCH NEXT FROM tableColumns INTO @name, @typeId, @isNullable
WHILE @@FETCH_STATUS = 0
BEGIN
 SET @typeName =
 CASE @typeId
  WHEN 36 THEN 'Guid'
  WHEN 56 THEN 'int'
  WHEN 61 THEN 'DateTime'
  WHEN 104 THEN 'bool'
  WHEN 231 THEN 'string'
  WHEN 239 THEN 'string'
  WHEN 241 THEN 'XElement'
  ELSE 'TODO(' + CAST(@typeId AS NVARCHAR) + ')'
 END;
 IF @isNullable = 1 AND @typeId != 231 AND @typeId != 239 AND @typeId != 241
  SET @typeName = @typeName + '?'
 PRINT '    public ' + @typeName + ' ' + @name + ' { get; set; }' 
 FETCH NEXT FROM tableColumns INTO @name, @typeId, @isNullable
END
 
PRINT '}'
 
CLOSE tableColumns

All you have to do is set your inputs (table, schema and resulting class name) and invoke the script. It will quickly give you a nice DTO like this:

public class EmployeeDto
{
        public int Id { get; set; }
        public int AccountId { get; set; }
        public string ExternalId { get; set; }
        public string EmailAddress { get; set; }
        public bool IsActive { get; set; }
        public string UniqueId { get; set; }
        public bool IsBuiltIn { get; set; }
        public string LastName { get; set; }
        public string FirstName { get; set; }
        public string MiddleName { get; set; }
        public bool IsAuthorizedOnAllDepartments { get; set; }
}

The script of course is not fully complete but it's easy to extend and adjust it based on your needs. Other kind of POCOs are Events and Commands. I have a completely different solution for them so I'll show it in my next post. Otherwise the post will be getting pretty long :)

Using log4net in your ASP.NET MVC application? Be carefull to use the %aspnet-request{} patterns

I have been using log4net for quite a long time on my ASP.NET MVC project. There is a handy pattern  %aspnet-request{ASP.NET_SessionId} out there. It helps to track actions made by particular user if there are several active users at the same time.

The pattern has been there in my log.xml for years and was working great until I finally migrated from MVC 3 to the most recent MVC 5. After the migration I've noticed a strange issue related to the ASP.NET request validation. If unsafe request is sent to some action that has the validation enabled then the server sends an empty response with status 200 and there is nothing in the log file???

To investigate the behavior an empty ASP.NET MVC project was created with a very simple Echo controller:

    public class EchoController : Controller
    {
        private static readonly ILog Log =
            LogManager.GetLogger(typeof (EchoController));

        public ActionResult Index(string data)
        {
            //This line is required to force a new session.
            Session["dummy-key"] = "dummy-value";
            Log.Debug("Index action got data: " + data);
            return Content("Got data: " + data, "text/plain");
        }

        protected override void OnException(ExceptionContext c)
        {
            Log.Error("Unhandled error occured", c.Exception);
        }
    }

And log config:

    <log4net>
        <appender name="FileAppender" type="log4net.Appender.FileAppender">
            <file value="logs/application.log" />
            <layout type="log4net.Layout.PatternLayout">
                <conversionPattern value="(session-id: %aspnet-request{ASP.NET_SessionId}) %m%n" />
            </layout>
        </appender>
        <root>
            <level value="ALL" />
            <appender-ref ref="FileAppender" />
        </root>
    </log4net>

So simple request with safe data will give you a response and log output you would expect:

c:\>curl --request POST http://localhost:55747/echo --data "data=hello" --verbose
* About to connect() to localhost port 55747 (#0)
*   Trying 127.0.0.1... connected
> POST /echo HTTP/1.1
> User-Agent: curl/7.23.1 (x86_64-pc-win32) libcurl/7.23.1
> Host: localhost:55747
> Accept: */*
> Content-Length: 10
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 10 out of 10 bytes
< HTTP/1.1 200 OK
< Cache-Control: private
< Content-Type: text/plain; charset=utf-8
< Server: Microsoft-IIS/8.0
< X-AspNetMvc-Version: 5.2
< X-AspNet-Version: 4.0.30319
< Set-Cookie: ASP.NET_SessionId=1z2kxgj3b1cqxsvwpd3tdigq; path=/; HttpOnly
< X-SourceFiles: =?UTF-8?B?RTpcamVueWFccHJvamVjdHNcTG9nNG5ldFJlcXVlc3RWYWxpZGF0aW9uXExvZzRuZXRSZXF1ZXN0VmFsaWRhdGlvbi5NdmNcZWNobw==?=
< X-Powered-By: ASP.NET
< Date: Wed, 01 Jul 2015 11:45:49 GMT
< Content-Length: 15
<
Got data: hello* Connection #0 to host localhost left intact
* Closing connection #0

Log output:

(session-id: (null)) Index action got data: hello

The session-id is null because it has just been started. Repeating the request with session cookie results in a log output with the session-id assigned:

c:\>curl --request POST http://localhost:55747/echo --data "data=hello" --verbose --cookie ASP.NET_SessionId=1z2kxgj3b1cqxsvwpd3tdigq

Log output:

(session-id: (1z2kxgj3b1cqxsvwpd3tdigq)) Index action got data: hello

So far everything works as expected. Now let's try some unsafe data:

c:\>curl --request POST http://localhost:55747/echo --data "data=<hello>" --verbose --cookie ASP.NET_SessionId=1z2kxgj3b1cqxsvwpd3tdigq
* About to connect() to localhost port 55747 (#0)
*   Trying 127.0.0.1... connected
> POST /echo HTTP/1.1
> User-Agent: curl/7.23.1 (x86_64-pc-win32) libcurl/7.23.1
> Host: localhost:55747
> Accept: */*
> Cookie: ASP.NET_SessionId=1z2kxgj3b1cqxsvwpd3tdigq
> Content-Length: 12
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 12 out of 12 bytes
< HTTP/1.1 200 OK
< Server: Microsoft-IIS/8.0
< X-AspNetMvc-Version: 5.2
< X-SourceFiles: =?UTF-8?B?RTpcamVueWFccHJvamVjdHNcTG9nNG5ldFJlcXVlc3RWYWxpZGF0aW9uXExvZzRuZXRSZXF1ZXN0VmFsaWRhdGlvbi5NdmNcZWNobw==?=
< X-Powered-By: ASP.NET
< Date: Wed, 01 Jul 2015 11:49:27 GMT
< Content-Length: 0
<
* Connection #0 to host localhost left intact
* Closing connection #0

Wat???  HTTP/1.1 200 OK and Content-Length: 0 and there is nothing in the log file???

Some digging has led me to a webpages:Enabled app setting. Starting from ASP.NET MVC 3 it should be set to false. As mentioned here it will prevent Razor templates from being directly accessible. But it looks like it does something else. On my project there was no that option so I've added it and tried again:

c:\>curl --request POST http://localhost:55747/echo --data "data=<hello>" --verbose --cookie ASP.NET_SessionId=qvrc2hkukx0iywp1qudps1xb
* About to connect() to localhost port 55747 (#0)
*   Trying 127.0.0.1... connected
> POST /echo HTTP/1.1
> User-Agent: curl/7.23.1 (x86_64-pc-win32) libcurl/7.23.1
> Host: localhost:55747
> Accept: */*
> Cookie: ASP.NET_SessionId=qvrc2hkukx0iywp1qudps1xb
> Content-Length: 12
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 12 out of 12 bytes
< HTTP/1.1 500 Internal Server Error
< Cache-Control: private
< Content-Type: text/html; charset=utf-8
< Server: Microsoft-IIS/8.0
< X-AspNet-Version: 4.0.30319
< X-SourceFiles: =?UTF-8?B?RTpcamVueWFccHJvamVjdHNcTG9nNG5ldFJlcXVlc3RWYWxpZGF0aW9uXExvZzRuZXRSZXF1ZXN0VmFsaWRhdGlvbi5NdmNcZWNobw==?=
< X-Powered-By: ASP.NET
< Date: Wed, 01 Jul 2015 11:56:22 GMT
< Content-Length: 11192
<
<!DOCTYPE html>
<html>
    <head>
        <title>A potentially dangerous Request.Form value was detected from the client (data=<hello>).</title>

I've posted just some part of the response and as you can see the status code is 500 and output has exception details. Let's check the log file:

(session-id: 

The line is trimmed! Looks like something goes wrong inside of the log4net. Luckily log4net has internal debugging support (see details here). To activate it there is an app setting:

<add key="log4net.Internal.Debug" value="true"/>

and also a trace listener should be added:

    <system.diagnostics>
        <trace autoflush="true">
            <listeners>
                <add 
                    name="textWriterTraceListener" 
                    type="System.Diagnostics.TextWriterTraceListener" 
                    initializeData="logs\log4net.log" />
            </listeners>
        </trace>
    </system.diagnostics>

Repeating the request and checking the log4net.log finally makes it clear:

System.Web.HttpRequestValidationException (0x80004005): A potentially dangerous Request.Form value was detected from the client (data="&lthello&gt").
   at System.Web.HttpRequest.ValidateString(String value, String collectionKey, RequestValidationSource requestCollection)
   at System.Web.HttpRequest.<>c__DisplayClass5.b__3(String key, String value)
   at System.Web.HttpValueCollection.EnsureKeyValidated(String key)
   at System.Web.HttpValueCollection.GetValues(Int32 index)
   at System.Collections.Specialized.NameValueCollection.Add(NameValueCollection c)
   at System.Web.HttpRequest.FillInParamsCollection()
   at System.Web.HttpRequest.GetParams()
   at System.Web.HttpRequest.get_Params()
   at log4net.Layout.Pattern.AspNetRequestPatternConverter.Convert(TextWriter writer, LoggingEvent loggingEvent, HttpContext httpContext)
   at log4net.Layout.Pattern.AspNetPatternLayoutConverter.Convert(TextWriter writer, LoggingEvent loggingEvent)
   at log4net.Layout.Pattern.PatternLayoutConverter.Convert(TextWriter writer, Object state)
   at log4net.Util.PatternConverter.Format(TextWriter writer, Object state)
   at log4net.Layout.PatternLayout.Format(TextWriter writer, LoggingEvent loggingEvent)
   at log4net.Appender.AppenderSkeleton.RenderLoggingEvent(TextWriter writer, LoggingEvent loggingEvent)
   at log4net.Appender.TextWriterAppender.Append(LoggingEvent loggingEvent)
   at log4net.Appender.FileAppender.Append(LoggingEvent loggingEvent)
   at log4net.Appender.AppenderSkeleton.DoAppend(LoggingEvent loggingEvent)

AspNetPatternConverter is used extract tokens from request and append them to the log. Further digging into the ASP.NET sources shows that the HttpRequestValidationException is raised when Request.Params are accessed. Request.Params are lazy populated with QueryString, Form, Cookies and ServerVariables. Request.Form will be validated and raise the HttpRequestValidationException. 

Probably the issue should be fixed by handling this exception inside of the AspNetPatternConverter and I will send a pull request some time soon. But production is waiting and the fastest solution I've came up with is to create a custom converter. It's quite simple and takes just a few lines of code. In my case I just need to log the session-id so here is the code I have:

    public class Log4NetSessionIdConverter : PatternLayoutConverter
    {
        protected override void Convert(TextWriter writer, LoggingEvent loggingEvent)
        {
            string sessionId;
            if (HttpContext.Current == null || HttpContext.Current.Session == null) 
               sessionId = SystemInfo.NotAvailableText;
            else 
               sessionId = HttpContext.Current.Session.SessionID;
            writer.Write(sessionId);
        }
    }

All you have to do is to add the converter declaration for your layout:

      <converter>
        <name value="session-id" />
        <type value="Log4netRequestValidation.Mvc.Log4NetSessionIdConverter" />
      </converter>

And then session-id pattern can be safely used:

<conversionPattern value="(session-id: %session-id) %m%n" />

So let's try the request again:

c:\>curl --request POST http://localhost:55747/echo --data "data=<hello>" --verbose --cookie ASP.NET_SessionId=qvrc2hkukx0iywp1qudps1xb
* About to connect() to localhost port 55747 (#0)
*   Trying 127.0.0.1... connected
> POST /echo HTTP/1.1
> User-Agent: curl/7.23.1 (x86_64-pc-win32) libcurl/7.23.1
> Host: localhost:55747
> Accept: */*
> Cookie: ASP.NET_SessionId=qvrc2hkukx0iywp1qudps1xb
> Content-Length: 12
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 12 out of 12 bytes
< HTTP/1.1 500 Internal Server Error
< Cache-Control: private
< Content-Type: text/html; charset=utf-8
< Server: Microsoft-IIS/8.0
< X-AspNet-Version: 4.0.30319
< X-SourceFiles: =?UTF-8?B?RTpcamVueWFccHJvamVjdHNcTG9nNG5ldFJlcXVlc3RWYWxpZGF0aW9uXExvZzRuZXRSZXF1ZXN0VmFsaWRhdGlvbi5NdmNcZWNobw==?=
< X-Powered-By: ASP.NET
< Date: Wed, 01 Jul 2015 11:56:22 GMT
< Content-Length: 11192
<
<!DOCTYPE html>
<html>
    <head>
        <title>A potentially dangerous Request.Form value was detected from the client (data=<hello>).</title>

Finally it works correctly. The response is 500 and log output:

(session-id: qvrc2hkukx0iywp1qudps1xb) Unhandled error occured
System.Web.HttpRequestValidationException (0x80004005): A potentially dangerous Request.Form value was detected from the client (data="<hello>").

It's time to click a TeamCity button to trigger a production build.