Monthly Archives: February 2013

Create New Folder Hierarchies For TFS Projects using Git SCM

If, like a lot of people who’ve worked heavily with TFS you may not have spent much time working with Git or any of its DVCS bretheren.

Firstly, a few key things:

1. Read and absorb the tutorial on how best to work with Git from the guys over at Atlassian.
http://atlassian.com/git/tutorial/git-basics

2. Install the Visual Studio 2012 Update 2 (currently in CTP, possibly in RTM by the time you read this).
http://www.microsoft.com/en-us/download/details.aspx?id=36539 (grab just vsupdate_KB2707250.exe)

3. Install the Git Tools for Visual Studio http://visualstudiogallery.msdn.microsoft.com/abafc7d6-dcaa-40f4-8a5e-d6724bdb980c

4. Install the most recent Git client software from http://git-scm.com/downloads

5. Set your default Visual Studio Source Control provider to be “Microsoft Git Provider”.

6. Setup an account on Team Foundation Service (https://tfs.visualstudio.com/), or if you’re lucky enough maybe you can even do this with your on-premise TFS instance now…

7. Make sure you enable and set alternative credentials in your TFS profile:

alt-credentials

8. Setup a project that uses Git for source control.

At this stage you have a couple of options – you can clone the repository using Visual Studio’s Git support

gitclone

OR you can do it right from the commandline using the standard Git tooling (make sure you’re at a good location on disk when you run this command):

git clone https://thesimpsons.visualstudio.com/defaultcollection/_git/bart milhouse
Cloning into 'milhouse'...
Username for 'https://thesimpsons.visualstudio.com/': homer
Password for 'https://thesimpsons.visualstudio.com/':
Warning: You appear to have cloned an empty repository.

I tend to setup a project directory hierarchy early on and with Git support in Visual Studio I’d say it’s even more important as you don’t have a Source Control Explorer view of the world and Visual Studio can quickly create a mess when adding lots of projects or solution elements.  The challenge is that (as of writing) Git won’t support empty folders and the easiest work around is to create your folder structure and drop an empty file into each folder.

Now this is where Visual Studio’s Git tools won’t help you – they have no concept of files / folders held outside of Visual Studio solutions so you will need to use the Git tools at the commandline to affect this change. Once have your hierarchy setup with empty files in each folder, at a command prompt change into the root of your local repository and then do the following.

git add -A
git commit -m "Hmmmm donuts."

Now, at this point, if you issue “git push” you may experience a problem and receive this message:

No refs in common and none specified; doing nothing.
Perhaps you should specify a branch such as ‘master’.
Everything up-to-date.

Which apart from being pretty good english (if we ignore ‘refs’) is pretty damn useless.

How to fix? Like this:

git push origin master

This will perform a forced push and your newly populated hierachy should be pushed to TFS, er Git, er TFS. You get the idea. Then the others on your team are able to clone the repository (or perform a pull) and will receive the updates.

HTH.

Update: A big gotcha that I’ve found, and it results in a subtle issue is this: if you have a project that has spaces in its title (i.e. “Big Web”) then Git happily URL encodes that and will write the folder to disk in the form “Big%20Web” which is all fine and dandy until you try to compile anything in Visual Studio. Then you’ll start getting CS0006 compilation errors (unable to find metadata files).  The fix is to override the target when cloning the repository to make sure the folder is validly named (in my example above this checks out the “bart” project to the local “milhouse” folder).

Tagged

Save Bytes, Your Sanity and Money

In this day of elastic on-demand compute resource it can be easy to lose focus on how best to leverage a smaller footprint when it’s so easy to add capacity. Having spent many a year working on the web it’s interesting to see how development frameworks and web infrastructure has matured to better support developers in delivering scalable solutions for not much effort. Still, it goes without saying that older applications don’t easily benefit from more modern tooling and even newer solutions sometimes fail to leverage tools because the solution architects and developers just don’t know about them. In this blog post I’ll try to cover off some and provide background as to why it’s important.

Peak hour traffic

We’ve all driven on roads during peak hour – what a nightmare! A short trip can take substantially longer when the traffic is heavy. Processes like paying for tolls or going through traffic lights suddenly start to take exponentially longer which has a knock-on effect to each individual joining the road (and so on). I’m pretty sure you can see the analogy here with the peaks in demand that websites often have, but, unlike on the road the web has this problem two-fold because your request generates a response that has to return to your client (and suffer a similar fate).

At a very high level the keys to better performance on the web are:

  • ensure your web infrastructure takes the least amount of time to handle a request
  • make sure your responses are streamlined to be as small as possible
  • avoid forcing clients to make multiple round-trips to your infrastructure.

All requests (and responses) are not equal

This is subtle and not immediately obvious if you haven’t seen how hosts with different latencies can affect your website. You may have built a very capable platform to service a high volume of requests but you may not have considered the time it takes for those requests to be serviced.

What do I mean?

A practical example is probably best and is something you can visualise yourself using your favourite web browser. In Internet Explorer or Chrome open the developer tools by hitting F12 on your keyboard (in IE make sure to hit “Start Capturing” too) – if you’re using Firefox, Safari, et al… I’m sure you can figure it out ;-). Once open visit a website you know well and watch the list of resources that are loaded. Here I’m hitting Google’s Australia homepage.

I’m on a very low latency cable connection so I have a response in the milliseconds.

Network view in Internet Explorer

Network view in Google Chrome

This means that despite the Google homepage sending me almost 100 KB of data it serviced my entire request in under half a second (I also got some pre-cached goodness thrown in which also makes the response quicker). The real interest beyond this is what is that time actually made up of? Let Chrome explain:

Request detail from Google Chrome

My client (browser) spent 5ms setting up the connection, 1ms sending my request (GET http://www.google.com.au/), 197ms waiting for Google to respond at all, and then 40ms receiving the response. If this was a secure connection there would be more setup as my client and the server do all the necessary encryption / decryption to secure the message transport.

As you can imagine, if I was on a high latency connection each one of these values could be substantially higher. The net result on Google’s infrastructure would be:

  • It takes longer to receive the full request from my client after connection initialisation
  • It takes longer to stream the full response from their infrastructure to my client.

Both of which means my slower connection would use Google’s server resources for longer thus stopping those resources servicing another request.

As you can see this effectively limits the infrastructure to run at lower capacity than it really could and also demonstrates why performing load testing requires that you run test agents that utilise different latencies so you can gauge realistically what your capacity is.

Some things you can do

Given you have no control over how or where the requests will come from there are a few things you can do to help reduce the effect of low latency clients will impact your site.

  1. Reduce the number of requests or round trips: often overlooked but is increasingly becoming easier to achieve. The ways you can achieve a reduction in requests include:
    1. Use a CDN for resources: Microsoft and Google both host jQuery (and various jQuery plugins) on their CDNs. You can leverage these today with minimal effort. Avoid issues with SSL requests by mapping the CDN using a src attribute similar to “//ajax.googleapis.com/ajax/libs/jquery/1.6.1/jquery.min.js” (without the http: prefix). Beyond jQuery push static images, CSS and other assets to utilise a CDN (regardless of provider) – cost should be no big issue for most scenarios.
    2. Bundle scripts: most modern sites make heavy use of JavaScript and depending on how you build your site you may have many separate source files. True, they may only be a few KB but each request a client makes will need to go through a process similar to the above. Bundling refers to the combining of multiple JavaScript files into a single download. Bundling is now natively supported in ASP.Net 4.5 and is available in earlier versions through third-party tooling for either runtime or at-build bundling. Other platforms and technologies offer similar features.
    3. Use CSS Sprites: many moons ago each individual image reference in CSS would be loaded as an individual asset onto your server. While you can still do this the obvious net effect is the need to request multiple assets from the server. CSS sprites combine multiple images into one image and then utilise offsets in CSS to show the right section of the sprite. The upside is also client-side caching means any image reference in that sprite will be serviced very quickly.
    4. Consider inline content: there I said it. Maybe include small snippets of CSS or JavaScript in the page itself. If it’s the only place it’s used why push it to another file and generate a second request for this page? Feeling brave? You could leverage the Data URI scheme for image or other binary data and have that inline too.
  2. Reduce the size of the resources you are serving using these approaches:
    1. Minification: make sure you minify your CSS and JavaScript. Most modern web frameworks will support this natively or via third-party tooling. It’s surprising how many people overlook this step and on top of that also don’t utilise the minified version of jQuery!
    2. Compress imagery: yes, yes, sounds like the pre-2000 web. Know what? It hasn’t changed. This does become increasingly difficult when you have user generated content (UGC) but even there you can provide server-side compression and resizing to avoid serving multi-MB pages!
    3. Use GZIP compression: there is a trade-off here – under load can your server cope with the compression demands? Does the web server you’re using support GZIP of dynamic content? This change, while typically an easy one (it’s on or off on the server) requires testing to ensure other parts of your infrastructure will support it properly.
  3. Ensure you service requests as quickly as possible – this is typically where most web developers have experience and where a lot of time is spent tuning resources such as databases and SANs to ensure that calls are as responsive as possible. This is a big topic all on its own so I’m not going to dive into it here!

If you’re a bit lost were to start it can pay to use tools like YSlow from Yahoo! Or PageSpeed from Google – these will give you clear guidance on areas to start working on. From there it’s a matter of determining if you need to make code or infrastructure changes (or both) to create a site that can scale to more traffic without needing to necessarily obtain more compute power.

Hope you’ve found this useful – if you have any tips, suggestions or corrections feel free to leave them in the comments below.

Tagged , , , ,