GitHub Copilot Workspace - First Impressions

Since 2021 the software development industry has increasingly been headed in a direction where AI-assisted software development is the norm. Many developers already build solutions using assistants such as GitHub Copilot in their code editor of choice.

While the impact of tools like Copilot has already been substantial we are really only at the beginning of a broader change in how software is designed and delivered, with GitHub's future vision taking shape in GitHub Copilot Workspace which is currently in limited technical preview.

I've been lucky enough to gain access to the tech preview, so in this post I am going to review its current capability.

The Vision

First off, if you're not familiar with GitHub Copilot Workspace, here's the launch video that gives a good overview of what it is and how it is expected to function.

The Reality

That launch video is pretty slick, right? But how does GitHub Copilot Workspace actually work in practice? Let's take a look!

I've used Copilot Workspace in a couple of different scenarios - both in the context of demonstrating its capability to others and in my own personal projects. Here are the two scenarios:

Senario 1: Add Cosmos DB support to an OSS project

I had someone ask about support for Cosmos DB for a Json Data Generator project they found on GitHub. I forked the repository and created a new GitHub Issue as a starting point for the solution.

You will see there is a pending PR attached to that Issue. The PR contains the (mostly) Copilot Workspace generated code. The only gap in the generated solution was missing / outdated package references to the Cosmos DB SDK. Once these were updated, the code ran as expected and was submitted back to the original repository.

I expected a pretty high degree of success with this scenario, as the code generation was relatively simple and the solution was well defined and constrained in scope.

There a few things that I love about the workflow:

1. The PR that is created uses the same capabilities as GitHub Copilot Enterprise's Pull Request Summaries feature. This is a great way to quickly include what changes have been made.

2. There is a link to the Copilot Workspace session so I can review the process used to create the PR. This important from a human oversight perspective, thougn it isn't currently clear how long these sessions are retained for.

One item that I think needs improving is a clear guide that when you are finished with your changes and want to commit them back your codebase, that Copilot Workspace will create a new branch and PR for you. Up until this point I thought everything was going to be pushed directly to the main branch!

Scenario 2: Modifying my blog to simplify the URL

I wanted to modify my blog to remove the posts from a URL path - something I've wanted since I migrated from Wordpress.

I'm not a massively experienced JavaScript developer, and while the excellent Next.js site template I use is well organised, I really wanted to avoid a "hack at it until it works" situation and to ensure I update all places that required changes.

Like with my previous scenario, I created a new GitHub Issue that described the problem and the desired outcome. I then created a new Copilot Workspace session and started to work through the problem. As my blog lives in a private repository I'll post some pictures to help explain the process.

Starting with the Issue

Here's the original issue, and I've hightlight the GitHub Copilot Workspace button that you click to get started.

Building the specification

Once I'm logged into GitHub Copilot Workspace, it automatically starts to build a specification that summarises the changes required and how they relate to existing behaviour/code. It took less than a minute to generate this specification, but I think it's safe to say the bigger or more complex your codebase/repository or the broader your change, the longer this will take.

I can add or edit items in the specification if it doesn't look accurate or inclusive enough. If I don't think the specification is correct, I can click the "Regenerate specification" button to see what changes Copilot Workspace recommends, though I'd likely want to go back and change my inputs as well to help.

Generating the plan

Once I'm happy with the specification I can hit the Generate Plan button to have Copilot Workspace generate the list of code changes required to meet the specification. I've snipped the list below to just show the top of the list of what will need to be modified. If you have a big change, expect this step to take a while.

Again, like the specification, I can review and edit the plan to ensure it meets my needs. I can also regenerate the plan if I think it's not quite right, potentially excluding files.

Making the changes

Once I'm happy with the plan, I can start to make the changes by hitting the Implement Selected Files button. At this point the screen layout will change and you'll see the file changes in real-time as Copilot Workspace makes them, along with a diff of the change that you can review while Copilot changes files.

At time of writing this process is sequential and does batches of lines in source files, so depending on the number of files and the complexity of the changes, this could take a while. I think it would be great if this could be parallelised in the future to speed up the process, though file inter-dependency would be challenging to manage.

GitHub Copilot Workspace can also do filesystem operations, so in my case it's moving files around to meet my requirement because the URL behaviour is tied to the filesystem folder structure.

I've done this scenario a few times and at least once I experienced a time-out which resulted in a cancelled job. I suspect this is because the job ran for too long, or exceed the allowable token limit. I think this is a preview limitation and I expect it to be resolved in the future.

On the plus side you can use the file view on the left side of the Copilot Workspace UI to select the remaining files and restart the implementation.

Reviewing the changes

You have a few options to review the resulting changes - file explorer, terminal or a Codespace, all of which render in the browser.

I love this idea, and use Codespaces regularly, but it does limit the value of the review feature in Copilot Workspace for any application that doesn't support these access methods (mobile apps, desktop apps, code that requires hardware/emulators). You can always commit the changes to a new branch and then review there using your existing workflow.

As my blog is a Next.js site, I can run the site in the Codespace to see the changes in action.

What was the result? Unfortunately the changes weren't implemented as needed, though in this instance the site still ran fine. Oddly, the created Codespace doesn't show in my list of Codespaces, though I suspect that's a preview limitation tied to how Copilot Workspace is currently hosted.

I've been through this scenario a few times and always run into an issue that means my change either only partially works, or doesn't work at all, failing deployment.

Summary and thoughts

I think GitHub Copilot Workspace is a great idea and has a lot of potential, but it's not quite there yet.

For simple, focused changes, or maybe for building proof of concepts or prototypes it's going to be giving value to all users right now, but for more complex changes, it's not quite ready. I think is more-so the case if the individual using Copilot Workspace is unfamiliar with the programming language or solution they are using.

This same situation also applies to the GitHub Copilot extensions in the code editor - it's great for simple changes or for getting you started, but it's not a replacement for an experienced developer who knows the programming language and codebase. Pair Copilot with an experienced developer and watch them fly!

Right now, the highest value feature of the GitHub Copilot suite is in the natural language exploration and explanation of languages, frameworks and codebases - whether this is via Copilot Chat in the code editor, or Copilot Enterprise in the browser.

We are still early days in this journey, and it's safe to say that AI-assisted software development is here to stay, and is a valuable tool in the developer's toolbox. GitHub Copilot Workspace will supercharge developers, allowing them push left in the software delivery lifecycle, but that vision is a little way off yet.

Having said that... you had better get polishing your prompt engineering skills!

😎