Switching the client side build library in visual studio 2013 MVC template to gulp and bower

Why?

A lot of people use Mads Kristensen’s absolutely awesome Web Essentials plugin for Visual Studio – we use it for less compilation, and bundling of our less/js.  It does however fall down when you need to use it in a continuous integration context, so we find that we keep the compiled/bundled output in our repository.

Couple that with the fact that in the next release of visual studio, gulp/grunt/bower are becoming first class citizens in terms of it’s support out of the box.

Scott Hanselman’s point in that post is a valid one – nuget is a superb addition to the .net ecosystem, and compare it to the dark days of ‘download a DLL from somewhere and hope’, it’s revolutionised .net development.  But there are other, arguably far better, and certainly far richer ecosystems out there for client side build, which on the one hand is absolutely awesome (npm is easy to build for and publish modules to), and on the other hand, daunting (I counted at least 15 modules that would simply minify my css for me).  Thankfully, the community talks/blogs a lot about this, so finding commonly used packages is as easy as reading from a number of sources and seeing which one comes out on top.

Microsoft are to be applauded for taking this approach and opening up the pipeline in this way – their whole approach recently with OSS of the .net clr, as well as the potential promise of a reliable .net on linux via vNext, and it’s a great time to be a .net dev.

All code for this example post is available at https://github.com/terrybrown/node-npm-gulp-bower-visual-studio

What is Gulp?

I won’t go into detail, as many other posts cover it well.  Essentially, it is a streaming build system written in node that allows people to create tasks and build up a pipeline of activities such as transforming less, copying files, validating javascript, testing, etc.  It is a more recent addition to the market (grunt, a tool with similar aims, though a different approach is another in the same arena).

What is Bower?

Essentially, a package manager for front end libraries (be they javascript, css, etc.) – think of it at a rudimentary level like nuget for client libraries.  There is a very good short video on egghead.io

Holy wars solved early – Gulp vs Grunt

Clever people have written about this.  I personally prefer the streams approach and the code over configuration driven nature of gulp over the ‘temp file all the things’ and config based approach of grunt.

Getting Setup – local dev machine + visual studio

Machine needs to be running node and gulp (gulp needs to be installed globally)

Node has just hit v 0.12 which has a number of updates (not least to streams3 and away from the somewhat interesting streams2)

node --version

Will confirm which version of node you’re running.  You don’t need the latest version, though the update in 0.12 has been a long time coming.

Setting up gulp/bower

npm install gulp -g
gulp --version
npm install bower -g
bower --version

TRX – Task Runner Explorer: This will give you a custom task runner for gulp within visual studio.

NPM/NBower Package Intellisense: Who doesn’t like intellisense right?

Grunt Launcher: Not ideally named, but a great little add on to give you right click support for gulp/bower and grunt.

You may also want to follow the steps in http://madskristensen.net/post/grunt-and-gulp-intellisense-in-visual-studio-2013 to get full intellisense.

Note: Switch off build in web essentials (it’s being used purely for intellisense)

File > New Project – and a tidy up

We want to hand over all JS and CSS handling to gulp.  This includes bundling and minification, as well as hinting/linting. We’ll start with the default MVC template from Visual Studio as the basis of our work.

Remove asp.net bundling/optimization

In the current template for MVC sites, Microsoft provide a handy bundling mechanism that although fine for smaller sites, still maintains the same problems as above and doesn’t give you separate control over your ‘distribution’ JS/CSS.  We’ll remove:

Microsoft.AspNet.Web.Optimization (and dependencies WebGrease, Antlr, Newtonsoft.Json)

This will also involve a few changes to web.config and the codebase (see https://github.com/terrybrown/node-npm-gulp-bower-visual-studio/commit/5cfb58b8e57faa4c518a067fa473d740e43725a3)

Remove client side libraries (we’ll replace these later)

  • bootstrap 3 (bower: bootstrap)
  • jquery (bower: jquery)
  • jquery validation (bower: jquery-validation)
  • jquery unobtrusive validation (bower: jquery-validation-unobtrusive)
  • modernizr (bower: modernizr)
  • RespondJS (bower: responsd)

Setting up Bower

bower init

This will lead you through a number of questions (accept defaults throughout for now, though you can read up on the options here)

You will end up with a bower.json file that will look something like:

image

Re-installing javscript and css dependencies

Take all of the package references above that we removed (the bower versions) and run the following on the command line:

bower install bootstrap jquery jquery-validation jquery-validation-unobtrusive modernizr respond --save

Do NOT forget the ‘- -save’ postfix at the end – this will ensure that your bower.json is updated with the local dependencies.

This will start the download and install, and you will end up with a new folder in your solution called ‘bower_components’ folder which contains all of the local dependencies.  Ensure you add this folder to your .gitignore (or source control ignore file of choice).

As a temporary step, switch to visual studio – add the ‘bower_components’ folder to your solution, and re-map all of your js/css files from the default template to the newly downloaded versions.

image

Setting up the build with Gulp

Firstly, we need to get this local solution ready to receive npm packages as dependencies (gulp + the other supplemental libraries we’ll be using are available via npm.

npm init

Again, accept all of the defaults really, or whatever you fancy in each field.

The examples from here down will be somewhat contrived – your own use case can dictate what you do at each step here, but for the purposes of example, what we want to achieve is:

  • Deliver all jquery and jquery validation libraries into a single request
  • Deliver bootstrap and respond as a single request
  • Create a basic more modularised structure for our CSS using less and then concatting/minifying as part of the build

In our real use cases at work, our needs are far more complex, but the above will serve as an example for this post.

Setting up a default ‘gulpfile.js’.

var gulp = require('gulp');

// define tasks here
gulp.task('default', function(){
  // run tasks here
  // set up watch handlers here
});

You can name and chain tasks in gulp really easily – each one can act independently or as part of an overall build process, and TIMTOWTDI (always) – what I’ll put forward here is the version that felt easiest to read/maintain/understand.

Deliver multiple vendor libraries into a single request

var gulp = require('gulp');
var del = require('del');
var concat = require('gulp-concat');

var outputLocation = 'dist';


gulp.task('clean', function () {
	del.sync([outputLocation + '/**']);
});

gulp.task('vendor-scripts', function () {
	var vendorSources = {
		jquery: ['bower_components/jquery/dist/jquery.min.js',
	'bower_components/jquery-validation/dist/jquery.validate.min.js',
	'bower_components/jquery-validation-unobtrusive/jquery.validate.unobtrusive.min.js']
	}

	gulp.src(vendorSources.jquery)
		.pipe(concat('jquery.bundle.min.js'))
		.pipe(gulp.dest(outputLocation + '/scripts/'));
});


gulp.task('default', ['clean', 'vendor-scripts'], function(){});

Ok, there are a number of things in here – key points:

  1. Read from the bottom up over – if you issue a straight ‘gulp’ command on the command line, you wil always run the ‘default’ task.  In this case, it doesn’t do anything itself (the empty function as the third param), but instead has a chained dependency – it’ll run ‘clean’ first, then (upon completion) run ‘vendor-scripts’ tasks.
  2. ‘clean’ task uses the ‘del’ npm module to clean out the output folder we will be pushing the built scripts/css to.
  3. ‘vendor-scripts’ uses the ‘gulp-concat’ npm module to simply join an array of files together (in this case, the jquery + jquery validation files)

if you switch to a command prompt window and run ‘gulp’ on it’s own, you will see output similar to:

image

And in visual studio, you will now see a hidden ‘dist’ folder there with the output of what you have just generated (remember to update your .gitignore – you do not want to commit these)

Disabling Web Essentials

Less has been our tool of choice for our CSS for some time now, and web essentials really did/does rock as a VS plugin to aid your workflow on those (nice inbuilt bundling, compilation, etc.  That said, now that we’re moving to a more customised build process, we need to switch the compilation side of it off.

Tools > Options > Web Essentials

Switch everything in ‘Javascript’ and ‘LESS” to false.

Deliver minified and concatenated CSS from LESS

We contrived a number of .less files in order to create the proof of concept:

_mixins.less

@brand_light_grey_color: #EFEFEF;

.border-radius(@radius: 4px) {
	-moz-border-radius: @radius;
	-webkit-border-radius: @radius;
	border-radius: @radius;
}

layout.less

@import "_mixins.less";

body {
    padding-top: 50px;
    padding-bottom: 20px;
}

/* Set padding to keep content from hitting the edges */
.body-content {
    padding-left: 15px;
    padding-right: 15px;
}


/* Override the default bootstrap behavior where horizontal description lists 
   will truncate terms that are too long to fit in the left column 
*/
.dl-horizontal dt {
    white-space: normal;
}

div.rounded {
	.border-radius(4px);
}

forms.less

@import "_mixins.less";

/* Set width on the form input elements since they're 100% wide by default */
input,
select,
textarea {
    max-width: 280px;
}

Nothing complex, though it’ll let us at least build a workflow around them.

There are a couple of key tasks we want to perform here:

  1. Grab all less files and compile them over to css
  2. Compress that css
  3. Push them all into a single file in our dist folder

Thankfully, the ‘gulp-less’ plugin performs the first two tasks, and we have already achieved the other for our JS so it’s just a repeat of those steps.

Integration into Visual Studio and tying it all together

We now have a basic working build that we can add to as and when our process demands – node and the node package manager (npm) have a massive ecosystem of libraries to support all sorts of tasks (generaily, gulp- prefixed for gulp related build tasks), so you can start to build from this point forward.

Key thing now is tying this workflow into Visual Studio, and this is where the cool happens.  The Task Runner Explorer gives us a lot of extensibility points.

image

Each of these tasks/sub-tasks can be right clicked and ran as you would do from the command line easily, but you also have a nice option to ‘bind’ certain actions in Visual Studio to steps within your grunt build.

E.g.

image

In this instance, we have bound our ‘clean’ gulp task to a ‘clean solution’ within visual studio.

Tying it all together – watching the solution

Web essentials was awesome at monitoring your work real time and updating bundled files (both less and js) into their respective outputs, but thankfully, gulp comes to the rescue in the guise of ‘gulp-watch’ – this is a highly configurable module that allows you to perform actions on changes to files.

Thankfully, now that we have all of the other tasks, the watch workflow is simply a matter of matching up targets to watch, and tasks to run when things happen to those targets.

var watch = require('gulp-watch');

gulp.task('watch', function () {
	gulp.watch('bower_comonents/**/*', ['vendor-scripts', 'vendor-css']);
	gulp.watch('Content/**/*.less', ['css']);
});

gulp.task('default', ['clean', 'vendor-scripts', 'vendor-css', 'css', 'watch'], function(){});

Once we have that, we can go back to the task runner explorer, right click the ‘watch’ task, and set it to run on solution open.

We now have our solution in watch mode permenantly and any changes to our less or the vendor scripts will trigger the appropriate tasks.

What’s next?

We’ve solved the problem (compiled css/js needing to be in our repo with web essentials), so the next steps really are incorporating this gulp build task into our CI server (TeamCity), though we’ll leave that for a follow up post.

Now that we have a whole set of automation going, we may as well re-introduce linting/hinting of our less and javascript too – some configuration will be needed here to ensure we’re happy with the outcomes, but fundamentally the ‘right thing to do’.

Testing our JS workflow is the next natural step, and there are plenty of gulp+other task runners to sit within this workflow that will let you validate your scripts either at build time or at save.

Visual Studio 11 (Beta) – Two week review

I’ve been using VS11 as my primary dev environment now for a few weeks so thought I’d write up my findings on the IDE as a replacement for VS2010.  It’s worth stating my general usage and projects so that you get a feel for where my thinking is coming from – if I get waffly (anyone who knows me knows I can tend to) then feel free to skip to the TL;DR section.

Machine Spec

Running under Win 7 Professional on an Dell M6500 Precision laptop – I5 2.66 with 8GB ram.  Both OS and Visual Studio are running on an SSD (though it’s not a particularly fast one, it’s still considerably faster than a 7200RPM disk).

Plugins

My predominant development is MVC front end to c# domain/services/repositories (yup, you heard it, repositories Smile with tongue out) – for front end work we use Chirpy (css concat/minification) and Web Workbench (for working with SCSS files).  Source control is sorted with AnkhSVN.

On top of that, which installation of Visual Studio would be complete without Resharper, so I’ve been keeping that up to date via the Resharper 7 EAP.

User Interface

This is the element that seems to have generated the most discussion and the most feedback from the community.  When I first saw the screenshots, I thought the whole thing looked a bit bland, though very much saw where they were trying to go with it – it just didn’t look as ‘pretty’ as VS2010.

First few days after install while I was still using both IDEs I was still in that mindset and found the UI a little bland.

Over the past two weeks though, the more I’ve worked in the environment, the more I see the point of it – the tooling just blends into the background.  It still very much feels like it’s working as hard for me (if not harder) than VS2010 was, but it’s not so apparent.  I can very much just get on with the job of coding.

I love the more minimalist UI – I’ve now grabbed a plugin for VS2010 to hide the file/edit etc. menu’s as it feels nicer not having them (like other products, a quick tap of the alt key brings you back into a comfort zone).

It felt odd that the toolbars seemed (at least initially) to be separate/jumpy – the toolbars differed between solution explorer having focus and the editor having focus, and I had to customise this from the default in order to get a UI that I was happy with, though after two weeks I’ve gone to having no toolbars and I don’t miss them (I never used to use much on them that often, and keyboard shortcuts are indeed ftw!)

Solution Explorer

imageI still have uncertainty about the solution explorer – it’s the one area where things feel like a step backwards.  The glyph approach in VS11 certainly keeps the UI subtle, but in a very large solution (is 81 projects very large? feels it!) it feels very much like the distinction between projects, solution folders, folders, files, modified files, etc. etc. etc. all blends into one.

This is definitely an area whereby when someone skins up a ‘vs2010 solution explorer’ for VS11 I’ll install it.

Don’t get me wrong, I can work with it, and the search tools make it far easier to find something (though with Resharper that was never hard anyway = CTRL+T for the win), but overall it doesn’t feel as much of a win in this one window.

 

Performance

Prefixing this by saying it’s very much a beta, though I don’t know why I’m bothering as the performance on the whole is far better.

Startup time from ‘open solution’ to being ready I’ve timed as being a smidge longer (it’s a technical term) – not enough to worry me, but certainly noticeable.

I really like the concept of what they seem to be trying to achieve (the projects loaded counts down as VS works through the loading, and you get a visual indicator in Solution Explorer on which projects are loaded and which aren’t – I suspect the intention is to let you get cracking on those projects that are loaded while it loads up the others.  In reality though, the UI is a little too sluggish at this point, so I find it better to just wait until the solution has loaded.

Once you have everything loaded/available though, the story changes entirely – everything feels more responsive, quicker to navigate and just generally ‘better’ – because of the cleaning of the UI mentioned above, I feel far more ‘in the code’ than I ever have done in VS2010.

Build Times

If there is one reason and one reason only to upgrade to VS11, this is it.  If you ever have a boss that talks about the cost/benefit of anything, demonstrate to him a big project build via VS2010 versus VS11.

Initial/Clean build on our 81 project solution in VS2010 takes between 40 and 60 seconds depending upon what else the machine has going on and how long VS2010 has been running.  VS11 is faster on initial/clean build, but not massively – 30-45 seconds.

Where we really gain is in subsequent builds – the build manager in VS11 seems to have taken parallelization very seriously.  VS2010 and we’re still looking at around 40 seconds on a build after a code change down in the guts of the solution.  VS11 and that’s averaging around 8-10 seconds.

This is the productivity carrot that makes it easy to sell this as a product really.  I find myself building more often simply because it’s not so impactful, and I find myself spending more time just ‘Getting Things Done’.

There is a caveat to this – I noticed a few occasions whereby in a sequential build all was tickety boo, but on a parallel build we were getting occasional (but not consistent) build errors with unmet reference dependencies.

Turns out it was our fault, and the project really didn’t have a reference to that dependency, but because in a sequential build it was getting addressed before it got to that project, it never generated an error, whereas I suspect in the parallel build world we were getting something akin to a race condition.

I can’t confirm this – it could be just my lack of understanding (most likely!) but anyway, adding the references to the project that was generating the intermittent build errors has resolved the problem.

Tooling

I’ve very much tortured myself here – part of me wishes I hadn’t, though I thought I’d see fully what it had to offer, so I’m trialing the Ultimate SKU of VS11 (I only have a license for Professional in 2010).

I know that some of this tooling exists within VS2010, though I can’t comment on it in there so this is a ‘clean’ review of it in VS11.

Code Clones

I was dreading running this – much as I’m very active in ensuring code quality, and we have a cracking team working on the product, it’s a product that is coming up to 2 years old, so I expected this to find some laziness dotted here and there.

Overwhelmingly, there wasn’t as much as I’d thought, and a lot of the issues reported were around some of our commonality in exception handling/logging (which in fairness should be AOP’d at some point, but it’s not what I’d consider duplication in the traditional sense).

First run through builds up an index so that subsequent runs through are a lot faster (I assume it takes a delta of changes to the codebase or something similar).

The way it highlights the issues is very elegant and it picked up a few issues that were indeed laziness/unawareness of devs (myself included!) that we’ve managed to refactor nicely and improve upon the code quality without any real cost to productivity – larger scale code reviews for things like this can take an age!

You can look through your ‘Exact’ or ‘Strong’ matches in no time at all and it just all feels very much more of a streamlining and an easing of quality management.

Love it, and will be sure to run it now periodically as part of my ‘ensure the boy scouts have been in the code base’ reviews.

Calculate Code Metrics for Solution

I’m aware that this at least is in VS2010 Ultimate SKU too, so I don’t know what (if anything) is different between the versions, but again, this is another tool to really allow me to see the codebase at a ‘big picture’ level very easily – I don’t have to so readily monitor checkins because the code metrics will identify code that has a bad maintainability index, areas where cyclomatic complexity has gone a bit awry, etc. – bloody useful, and again one that will come into my regular reviews to save me time.

Unit Test Code Coverage

I’ve used other solutions for this in the past, so again, it’s nice to see it baked in – though it’s a shame it’s Ultimate only.  I very much take the approach with Unit Testing of ‘ensure the functionality is covered’ while not worrying too much about % (other than as an indicator where the former may be suffering), and the interface on this makes it very easy to look through and pick out areas that are lacking testing.

I’ve added > 30 // TODO UNIT TEST comments to the codebase today, and again, it took no time at all to find those.

TL;DR

With this iteration of Visual Studio it feels very much like the workflow/lifecycle of what we do as developers has been at the forefront, and it’s difficult to find anything (other than the solution explorer) where it doesn’t feel like a significant improvement over the previous iteration.  I’ve only scratched the surface over the past two weeks of running with it, but I will very much be convincing our management to upgrade when it releases, and will do my best to attempt to get them to justify the cost of a few SKUs of Ultimate for all of the bells and whistles that are brought about within that SKU.

Positives

  • Build times have to sit at number one – productivity, productivity, productivity!
  • Tooling is superb in assisting you in ‘getting things done’
  • The UI chrome just blends into the background letting you ‘get things done’
  • Did I mention that I feel far more able to just ‘get things done’? Smile

Negatives

  • Solution Explorer on the whole is better/faster/more responsive, though this is one area where the lack of chrome (imo) makes things a bit harder
  • Startup time – although not significantly slower is certainly a little slower