Saturday, March 8, 2014

Docker Vagrant TeamCity

In this post I'll introduce the brand new TeamCity.Virtual plugin that supports execution under virtual environments from Vagrant or Docker

Software is the problem

Every build running under CI requires some software/libraries/JVMs/* to be pre-configured on the machine. When your setup has only one build agent everything is simple. But if you need an easy recreatable environment things get more complicated. Every time you need to install some software on every build agent you have to do lots of routine work. When there is a pool of build machines its getting overcomplicated

There exists at least one way to cope with this complexity. Virtual machines. LinuX Containers. Thanks to Vagrant it became an easy, repeatable and scriptable solution. With LinuX Containers (for example Docker) things are even faster

TeamCity.Virtual plugin makes using Docker and Vagrant in TeamCity as easy as using a build runner. So for example, you may run your nodejs builds in a fresh environment of required version without any extra per-agent pre-configuration work

TeamCity.Virtual plugin

The build runner is called Docker / Vagrant. On the runner settings page you may specify the virtual environment configuration to start and the script to execute inside it:

Vagrant
 
Docker

Implementation Details

The plugin works as follows:

  • The plugin detects the installed on the build agent Vagrant and/or Docker (so only compatible build agents will be used)
  • It starts the virtual environment (box/container) on the build agent
  • Mounts the build checkout directory into the started machine
  • Maps the working directory into machines path
  • Executes the provided script in the started virtual environment in the working directory
  • Destroys the environment wiping all the state

Settings

Vagrant box is specified via Vagrantfile. For Docker you need to specify image name only

Run experiments

This is what you see in the build log of Vagrant VM command:

Similar builds under Docker:

What is is for

In the next posts I'll cover details on how to use TeamCity.Virtual plugin to run builds with Nodejs, JVM, Android and much more. Remember, all you need to configure for build agents is Docker or Vagrant. Here are links for publicly available images: Vagrant Boxes and Docker Images

Download and Run

Plugin is implemented under Apache 2.0. Sources are on GitHub. Builds are set up and running in TeamCity.

For more details, see README.md

Your feedback is welcome! Share what you think

Wednesday, December 18, 2013

Try-With-Resource in Java

Starting from Java 1.7 we have new syntax: try-with-resource. Recently I had a chance to use it in a project. Main usage I had was for logging contexts.

This code looks feasible for small examples, but things got complicated when one has big code block under try/finally. Next approach is to use Java 1.7 try-with-resource. The approach is similar to using construction in C#.

CloseableResource is an interface inherited from Java's standard AutoCloseable. The only reason for it is to avoid declared exceptions. The implementation of an interface is following:

In Java 1.7 it's unfortunately not allowed to use expression as resource in the try-with-resource statement. So I was forced to have a dummy variable declaration there. So I named the variable with underscores to make it be unusable. I with I could write something like this: try(foo()) { /*code*/ }. Nowadays this is not supported.

Meanwhile the pattern was good to use and I start using it in many places. In some places I added catch clause for the try.

This code looks good, but it does not work as it was expected! I expected from it to call logger.error under ProjectMDC.mdc() resource but it turned out the logger.error() method is called AFTER ProjectMDC.mdc() context is closed. The problem is that logging was expected to be logged with ProjectMDC but it was not. So the only right code should be patched as follows

This code is similar ugly as pre 1.7 code:

The only lack in the older code is that one have to declare CloseableResource variable and than explicitly close it at end. There is one extra variable in code to take care of.

Outcome

The sad outcome here is that brand new try-with-resource statement has the following misses

  • It's not clear that resource is disposed before catch block call
  • It's not allowed to put expression(s) as resources (remember using in C#)
  • Real usage code turns out to be quite ugly (still)

The Kotlin Way

All this time I was recalling how easy once could create one's own using in Kotlin. For example:

Here I declared my own construction that is more useful in my project! So the usages are following

For more Kotlin details, you may follow to the documentation.

The only thing is that this code looks more easy-to-understandable. In the long run, I'd say, this code is more unlikely to contain bugs. Again this is only because simplicity of code

Done

Be careful with new Java syntax and read manuals with more attention. Happy coding and Happy Christmass and New Year

Thursday, August 22, 2013

Node.js and NVM and Grunt in TeamCity.Node

In the previous posts I wrote about Jonnyzzz.Node plugin that provides support for Node.js, NPM, Grunt and Phantom.js in TeamCity. Today I'm here to introduce NVM support in the plugin. I'd also shed light on Grunt runner improvements.

NVM

In the latest builds of the plugin you'll find the build runner called Node.js NVM Installer. This build runner would install Node.js and Node Package Manager to your build agent machine with no pre-setup at all. As the implementation I use the amazing script from creatonix/nvm. The build runner fetches the latest version of the script from GitHub. Note, the script only works under Linux and MacOS.

The new runner provides very simple settings. All you need is to specify version of Node.js. After build runner is executed, you may simply use Node.js or Node.js NPM build runners or even call node or npm command from any of you scripts/runners.

You may use the Node.js NVM Installer more than once within the build configuration. So you may test/run your code under several Node.js version at once!

Grunt

There were many requests in feedback to let Grant build runner use system-wide grunt instead of NPM fetched local grunt. Starting from now you may select the preferred way you like Grunt runner to be executed!

It's handy to refer to TeamCity properties from build scripts. Grunt scripts are not an exception to the rule. Starting from now Grunt build runner would add --teamcity.properties.all=<file.json> parameter with path to a .json file that contains all TeamCity parameters serialized to JSON hash. So now you may easily integrate your builds with TeamCity! You may use the following code to parse parameters in Grunt.js:

Downloading TeamCity.Node

This plugin is provided under Apache 2.0 license. Sources (in Kotlin) are found on GitHub. You may download compiled build from TeamCity