CloudBees Blog

Jenkins IRC Plugin

2012-02-19T17:32:00.000-05:00

Overview
Jenkins is a continuous integration switchboard centralizing the communication with many different protocols, such as SSH, Git and SVN. This blog entry will focus on the IRC protocol and how one can turn Jenkins into IRC bot with the IRC plugin.

When the IRC plugin is installed and configured, Jenkins can send notification messages to a chat room (or channel), such as build information, and users can send messages to Jenkins to obtain information and perform actions, such as builds.

Stable Release Versions
The latest release of the IRC plugin is 2.18 and was released in November 2011. It has known issues.

Requirements for Plugin-Use
Jenkins 1.392 or newer and the utility plugin "instant-messaging" (called "Instant Messaging plugin").

How to Use
To demonstrate the use of the IRC bot plugin I first set up a test channel at FreeNode called ##jenkins-irc-bot-test. Then I configured Jenkins, from the main configuration page, to connect to FreeNode and that channel:

Using the Mac OS Colloquy chat client, I can connect to the same channel and also connect to Jenkins itself for the configured nickname, jenkins-irc-bot-nick, and send commands, such as !jenkins help that lists all the commands:

The command prefix !jenkins identifies a chat message as a command and is configurable from the IRC configuration section of the main Jenkins configuration page.

I set up a job called test and then "chatted" a few commands:

The transcript shows that I first listed all the jobs and also obtained the status of the job test. Then I built job test, checking the queue, and finally checking the status again. The chat room (or channel) ##jenkins-irc-bot-test shows that there are two new messages:

Jenkins is sending messages that the test job was started and completed. I configured the job test to send such messages:

How to Use the Plugins on DEV@cloud/RUN@cloud?

The IRC plugin is available to install on a DEV@cloud Jenkins instance on CloudBees Platform as a Service (PaaS) and it may be used in the same manner as with any other Jenkins deployment.

Relevant Documentation

--Paul Sandoz, Developer

CloudBees

www.cloudbees.com

Solving the Private vs. Public PaaS Dichotomy: Discover CloudBees AnyCloud!

2012-02-14T08:30:00.000-05:00

Demonstrating CloudBees' leadership in PaaS innovation, today we are announcing AnyCloud: a unique offering that will redefine and further broaden the PaaS market. AnyCloud provides the ability to have a unified view of your applications managed by CloudBees' PaaS, while physically deploying those on any infrastructure: public, hosted provider or even on-premise, in your own datacenter. Any cloud, anywhere.

A growing number of companies are increasingly turning to Platform as a Service (PaaS) as a way to improve their time-to-market, increase their agility and reduce their costs. The CloudBees Platform, for instance, provides application development, deployment and management services and eliminates much of the operations cost and IT hassles normally encountered by developers. And because it is available as a SERVICE, there is no need for you to setup either infrastructure or any software.

Unfortunately, for some companies, the promise of PaaS bashes up against the reality of enterprise constraints:

How can I use a PaaS and get low-latency access to data that sits on-premise and that I cannot move to the cloud?
How can I abide by security and privacy rules and still use PaaS?
How can I keep my processing and data in a specific region?

One solution to these problems is to a deploy a private PaaS – or “Platform as a Software” – a PaaS that you install, configure, patch and upgrade much like you do your existing middleware products. In doing so, IT is taking on another layer of software to maintain, update and pay for support on.

The truth is that a key part of the promise of PaaS lies in the “S”, as in SERVICE. When you choose a private PaaS path, you are forced into delivering the service for the PaaS yourself! Your IT team hopes it can forget everything it has learned about operating system and middleware configuration and support, but in reality it will have to be re-trained for something entirely new! You therefore continue to bear the infrastructure capital costs, in addition to the operational costs for maintaining the infrastructure. Thus, as illustrated conceptually above, the potential operational efficiencies of PaaS compared to IaaS are masked by the continued burden for both the infrastructure itself as well as your continued operational obligations associated with it.

In contrast, when you use a service provider, as opposed to a software vendor who is “enabling” you to provide service, you get a number of benefits that lead to lower cost and better service:

The service provider reduces your operational cost. The expertise to manage the technical intersection between complex middleware and complex PaaS software is part of the provider’s responsibility, not yours. You can focus better on your core business.
Service providers get better economies of scale. The expertise they have in-house is leveraged across thousands of customers.
Service providers have an incentive to invest in operational efficiency. Any vendor has an incentive to improve its margins.

Getting Rid of the Public vs. Private Dichotomy!

Consequently, companies considering using a PaaS face what has become a typical dichotomy:

Either they can opt for a Public PaaS and enjoy all of the benefits of increased agility, much improved time-to-market and reduced costs, but not be able to move some of their applications onto that PaaS for technical/latency or privacy issues;
Or they can opt for a Private “PaaSoftware” in order to satisfy their technical and privacy requirements, but they will most probably never see the full benefit from increased agility, time-to-market or reduced-cost compared to their existing middleware setup (au contraire).

Announcing CloudBees AnyCloud

As a solution to this recurring problem, today, as the Java PaaS innovation leader, we are announcing CloudBees’ AnyCloud.

The AnyCloud architecture solves the above dichotomy by providing the best of both worlds: a fully-serviced PaaS (by CloudBees) that enables a multitude of deployment scenarios at the same time: on public cloud infrastructure (IaaS) in multiple regions, on traditional hosting providers (on vSphere for example) as well as on-premise, on your own servers, inside your own data center!

(NOTE: More information on the AnyCloud architecture is available in our CloudBees AnyCloud white paper.)

By opting for a supported CloudBees AnyCloud hosting provider or by registering servers in your data center, CloudBees AnyCloud delivers the operational cost savings and cloud efficiency of a fully-managed PaaS to your business – on any cloud – while satisfying your technical and privacy constraints.

With AnyCloud, your developers and lines of business get the benefits of zero-friction development and deployment and faster-time-to-market while getting low-latency access to your most important on-premise resources and existing investments. Same if you’re concerned about keeping your applications and data in a specific region due to privacy or regulatory concerns: AnyCloud is the answer. And if you already have a trusted hosting provider or preferred infrastructure cloud vendor, AnyCloud helps you enjoy the benefits of a fully serviced PaaS with the vendor relationship and investment you’re comfortable with.

More information on the AnyCloud architecture is available in our CloudBees AnyCloud white paper.

In addition to our existing production US-based offerings, we are currently rolling out support for AWS in Europe, OVH in Europe, as well as AnyCloud support in your own datacenter. If you are interested in AnyCloud, just contact us.

Onward,

Sacha

Sacha Labourey, CEO and Founder

CloudBees

www.cloudbees.com

New RUN Pricing (and Venti Chai Latte)

2012-02-08T12:09:00.007-05:00

Since we started CloudBees, we understood one essential part of the value of a PaaS was to abstract IT infrastructure away from developers. Developers shouldn’t have to worry about server provisioning, operating systems, application servers, firewalls, load-balancers, etc. But PaaS offerings are not just about provisioning IT resources, they are also about how your applications get mapped onto those resources.

To that end, at CloudBees, we have defined our own unit of compute, an app-cell. The app-cell is a much more fine-grain concept than what a CPU core is, yet provides enough capacity for decent work to be performed (our testing shows that a basic web application accessing a database can handle up to 250 requests per second on a single app-cell). That’s for the “how much” part. But we can also finely define the time those resources get allocated to your application, based on your application requirements. As an analogy, while IT infrastructure providers like AWS could only rent you a bus for an entire week, CloudBees is able to rent you a single seat, by the minute.

This technical leadership, coupled with the high number of accounts using our platform, obviously leads to great economies of scale.

Consequently, a few weeks ago, we announced that we were significantly dropping DEV@cloud prices and were providing a fully serviced “Jenkins as a Service” continuous integration offering for LESS than you would pay doing it yourself, on-premise, or by renting machines at AWS. Just check, our prices start at … 0$. That’s a hard price to beat.

Today, we are rolling out our new pricing structure to RUN@cloud and as you will see, it is a significant drop! Our new pricing will be available at two subscription levels:

Base: this subscription level is fine if you mostly need a place to run your application, but do not care about sophisticated clustering setup, support tickets, etc.
Enterprise: this offers all of the CloudBees features, sophisticated clustering, bigger memory sizes, technical support, etc.

While the Base level will be available in a couple of weeks, the Enterprise level is available today. Our current customers have been converted to this new Enterprise level and will automatically benefit from a 62% cost reduction. Our new Base level (once available), will show a 74% discount compared to our current pricing!

This new pricing puts the deployment of a simple Java application with a database at $9.50/month, and a fully clustered HA application at $19/month, much lower than what you could achieve by directly renting EC2 servers for example (without even talking about the time you’d have to spend supporting this environment).

We are also announcing the ability to deploy the CloudBees PaaS on your own dedicated machines on AWS, and let us manage your applications on that pool of resources - contact us if you are interested in this. We have customers doing this today.

This year, the CloudBees team has set itself the objective to show the Java world that using CloudBees’ PaaS is cheaper, better and faster. This announcement clearly demonstrates the first one. As my colleague Michael Neale put it: “If you find something cheaper, it is probably a Venti Chai Latte.”

Onward,

Sacha

Sacha Labourey, CEO

CloudBees

www.cloudbees.com

A Festival of Emerging Technology in Philly

2012-02-02T17:42:00.000-05:00

CloudBees is proud to be sponsoring the 7th Annual Philly Emerging Tech Conference, April 10 & 11th in Philadelphia. If you're in the vicinity, check out this popular conference, hosted by our friends at Chariot Solutions. It sells out most years, so don't wait to sign up for this 2-day, 5-track technology extravaganza.

The Emerging Tech Conference (a.k.a. Emerging Technologies for the Enterprise) covers topics ranging from Java EE in the cloud to the Play Framework to cross-platform mobile development to NoSQL to HTML5 to functional programming patterns. Check out the full session list here and the list of expert presenters here.

The Early Bird rate of $375 per person is available through February 15. If you can take advantage of the group discount -- 4 colleagues or friends, not necessarily from the same company –- register at the same time and the price drops to $281.25 per person.

Sessions covering a wide array of technologies complement the Java and JVM-based focus. In addition to keynote speakers Chad Fowler (VP Engineering, LivingSocial) and Alex Payne (CTO, Simple Finance), the conference features Yukihiro Matsumoto (creator of the Ruby Programming Language); Douglas Crockford (Senior JavaScript Architect, Yahoo); Yehuda Katz (Ember.js and jQuery core developer); Elika Etemad (W3C CSS Working Group); James Shore (author of The Art of Agile Development); and many others. Keep an eye on the site, because more speakers and session abstracts will be added over the next few weeks.

Register and attend!

Tasks and Warnings with Jenkins

2012-02-02T09:01:00.012-05:00

Overview
Jenkins is a highly extensible platform and, in some cases, plugins are the foundation for further plugins.

One such example is the plugin for static code analysis. This plugin provides the necessary foundation for reporting and presenting of static analysis produced from building jobs, but does not in and of itself do any analysis. That is the responsibility of additional plugins which analyze files in the build workspace, such as source code and, if appropriate, the results of building that source code.

For example, there is the FindBugs plugin, that runs FindBugs which looks for bugs in Java code. FindBugs is an excellent utility for identifying problems in Java code. Combine FindBugs with Jenkins and those problems can be easily viewed, monitored and tracked over builds.

However, this blog entry will focus on two other static analysis plugins: the Task Scanner plugin; and the Warnings plugin.

Stable Release Versions
The latest release of the Task Scanner plugin is 4.26 and was released in December 2011. It has no known issues. The latest release of the Warnings plugin is 3.27 and was released in January 2012; it has known issues.

Requirements for Plugin-Use
Jenkins 1.409 or newer and the utility plugin "analysis-core" (called "Static Analysis Utilities").

How to Use

The Task Scanner plugin will scan the workspace for files and check if lines in those files contain certain strings or tags.

I created a job on Jenkins to build Jenkins and configured the Task Scanner for the job as follows:

after building, the results can be viewed by clicking on the "Open Tasks" link on the left hand side of the screen:

We can observe that the Jenkins code base has 15 high priority tasks ("FIXME"), 330 normal priority tasks ("TODO") and 425 low priority tasks ("@deprecated"). It is not surprising that there are so many @deprecated tasks, Jenkins has maintained backwards compatibility for a long time.

Given a particular task, it is possible to drill down and view the source:

The (Compiler) Warnings plugin can scan, using selected parsers, the build console log and workspace files and report compiler warnings. There are many parsers, if Cobol is your language of choice (or not as the case is highly likely to be) there is a parser for that.

In this case, I have configured the previous Jenkins job to scan for compiler warnings in the console log:

To ensure that such warnings are generated by the Java compiler, I need to tweak the Jenkins pom file:

<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<showDeprecation>true</showDeprecation>
<showWarnings>true</showWarnings>
<source>1.5</source>
<target>1.5</target>
</configuration>
</plugin>

After building, a summary of the results can viewed from the build status page:

The plugin detected 213 warnings. Notice that the status of the open tasks from the Task Scanner plugin are also displayed.

A summary of the warnings can be viewed by clicking on the "Compiler Warnings" link on the left hand side:

Looks like some classes in the Jenkins core module could be cleaned up to use non-deprecated features. Drilling down we can see where certain deprecated features are used. For example:

As presented these plugins are rather useful to track tasks and warnings in source code. They are quick to set up, even for a large project such as Jenkins.

Finally, the results of both the tasks and compiler warnings can be combined with the Static Analysis Collector plugin. Combine it with the DashBoard View plugin and a summary can be presented:

How to Use the Plugins on DEV@cloud/RUN@cloud?

All plugins previously mentioned are available to install on a DEV@cloud Jenkins instance on CloudBees' Platform as a Service (PaaS) and they may be used in the same manner as with any other Jenkins deployment.

Relevant Documentation

--Paul Sandoz, Developer

CloudBees

www.cloudbees.com

Dependency Graph Viewer for Jenkins

2012-02-01T10:29:00.000-05:00

In Jenkins, jobs are the smallest unit of computation. For simple use of Jenkins, one job (and its checkout/build/report cycle) can suffice, but more sophisticated automation almost always involves multiple jobs that are working together.

The Dependency Graph Viewer plugin for Jenkins, by Stefan Wolf, makes it easy to deal with such multi-job workflow/activity by visualizing the relationship, as you can see below:

Setting Up

This plugin uses GraphViz, a C++ application that performs the actual layout of nodes and edges in the graph. This tool does a pretty good job of figuring out the "intuitive" layout most of the time.

Because this is a native application, GraphViz needs to be installed separately. Windows, Mac, and OS X users can do so from the GraphViz website. Linux users should be able to install this from their package manager, since it's a very popular tool. Other than GraphViz, this plugin requires no configuration. A plugin not requiring any configuration also means it can be removed side-effect free, so you can try this plugin out in your production environment without worrying about possible data loss.

How to Use It

Once installed, you'll see a "dependency graph" action link on the left in every job and view, which displays the relevant upstream/downstream jobs for the current job in question (or jobs in the current view in question). The graph is clickable, and if you do click on it, you can jump to the top page of that job. This makes it easy to navigate around in a complex job setup.

Upcoming Features

The upcoming version 0.3 expands this plugin to recognize other kind of inter-job relationships that go beyond upstream/downstream relationships, such as the "subroutine" relationship induced by the parameterized build trigger plugin, or the "producer/consumer" relationship induced by the copy artifact plugin.

Eventually, I think it should be turned into an extension point to enable other plugins to add all kinds of interesting relationships here.

- Kohsuke Kawaguchi
Jenkins Founder

Elite Developer & Architect, CloudBees

Controlling What You See (with the View Job Filters Jenkins Plugin)

2012-01-30T14:39:00.000-05:00

Overview
When you first start using Jenkins for continuous integration, you will most likely have only a few projects, and the default main screen will suit your needs perfectly.

After a while, however, you will have more and more jobs in your Jenkins server and the default screen may no longer be adequate.

Stable Release Version
The latest release is 1.18 which was released in September 2011 and has no known issues.

Requirements for Plugin-Use
Jenkins 1.398 or newer

Step-by-Step Instructions on How to Use the View Job Filters Plugin:

Installation

Go to your Jenkins instances root page.
If your Jenkins instance has security enabled, login as a user who has the Overall | Administer permission.
Select the Manage Jenkins link on the left-hand side of the screen.
Select the Manage Plugins link.
On the Available tab, select the View Job Filters Plugin and click the Download and Install button at the bottom of the page.
(If you are using a version of Jenkins prior to 1.442) Restart Jenkins once the plugins are downloaded.

Configuration

This plugin does not have any global configuration options, instead it adds additional functionality to the views available within Jenkins.

If you have not created any views, then your system will be using the default “All” view. This view is read only (see the links at the bottom of this page for how to edit your “All” view). So in order to get started with this plugin, you first need to create a view.

From your Jenkins instance's root page, there is a tab called “+” at the end of all the tabs. Click on that tab to create a new view.
Give the view a name, and select the type of view you want to create. View types are an extension point that other plugins can contribute. Jenkins has one built in view type, List View.

If you already have created a view, just select that view and click on the “Edit View” button. In either case you should now be at the following screen:

View Configuration screen with the View Jobs Filters plugin installed, while in the process of adding a view job filter.

Jenkins has some rudimentary filtering built in, essentially: the status filter; the manual set of check-boxes; and the use a regular expression. So by way of comparison, and so that you know where any issues you might have are originating from, here is the same screen without the View Jobs Filters plugin installed:

View Configuration screen without the View Jobs Filters plugin

To start filtering the jobs in your view, you click on the Add Job Filter button, and select the type of filter to add. There are multiple types of filters, which I will describe in a moment, but first I need to explain how the filters work.

A view starts off with an empty set of jobs. To this is added all the selected jobs, followed by any jobs selected by the regular expression and then this initial set of jobs is filtered based on the built-in status filter. The View Job Filters then proceeds to operate on the set of jobs, adding and removing jobs in sequence. Most of the filters provided by the View Job Filters have a “Match Type” field, which determines whether they add or remove jobs from the set of jobs. The filters apply in the order in which you define them (and you can re-order them by drag and drop). Once all the filters have been applied, you have a final set of jobs that will be displayed in the view.

The Match Types are:

Include Matched - adds to the set of jobs, any jobs not already in the set which the filter rule matches.
Include Unmatched - adds to the set of jobs, any jobs not already in the set which the filter rule does not match.
Exclude Matched - checks all the jobs in the set of jobs, removing any which match the filter rule.
Exclude Unmatched - checks all the jobs in the set of jobs, removing any which do not match the filter rule.

This is a very powerful mechanism that allows you to create a view containing only the jobs you want to see... though at times it can seem like you are back in school doing set theory while trying to figure out exactly how to cut those Venn Diagrams to get at the set of jobs that you want to see.

The View Job Filters plugin exposes an extension point for other plugins to add additional types of filters, but here are the types of filters that are currently built into the plugin:

All Jobs: This adds all the jobs to this view. You could select all the checkboxes, you could specify a regex of “.*”, but this is by far the easiest way to start with all the jobs when either the filters you will be adding will be trimming back, or when you want to customize the “All” view.
Build Statuses Filter: This adds/removes jobs based on their current status, i.e. whether they are currently building; were never built; or are currently in the build queue. By using the Include/Exclude Unmatched match type you can invert the selection, i.e. whether they are currently not building; have been built at least once; or are not in the build queue.
Build Trend Filter: This adds/removes jobs based on recent events. In some cases the Build Statuses Filter or the Job Statuses Filter will be a simpler way to get the same result. Some of the criteria you can construct with this filter are:

All jobs that ran in the last 5 hours;
All jobs that have been unstable for the last 7 builds.
All jobs that have at least one stable build in the 10 days.
All jobs that have not been run in the last 30 days.
All jobs that have been triggered by a SCM change within the last week.

Job Statuses Filter: This adds/removes jobs based on the job status, i.e. Stable; Failed; Unstable; Aborted; Disabled.
Job Type Filter: This adds/removes jobs based on the type of job. For example, I use this filter to identify any Maven 2/3 project type jobs so that I can ~~beat-up~~ give out to the people creating those jobs thereby continuing my long standing disagreement with Kohsuke over whether the Maven 2/3 project type is a good idea or not ;-)
Logged-in User Relevance Filter: This adds/removes jobs based on their relevance to the logged in user. For example: matching jobs that were started by the user, or where the user committed changes to the source code of the job; matching jobs with a name that contains the user's name or login id.
Other Views Filter: This adds/removes jobs based on whether they are in a different view's set of jobs. Note: you can create a circular logic of death if View A has another views filter based on View B which has another views filter based on View A. There are longer and shorter ways to such a circular logic, but it all amounts to a dog chasing its tail.
Parameterized Jobs Filters: This allows adding/removing parameterized jobs based on whether regular expressions match the job parameters. If you need to use this one, you are doing very fancy stuff altogether. An example use case would be where you have a job parameter that selects the database that the job uses for running tests against, you could create a view which selects all the jobs targeting a specific database.
Regular Expression Job Filter: This allows using a regular expression to match against one of: the job name; the job description; the SCM configuration (i.e. select a specific branch); the email recipients; the Maven configuration; or the job schedule.
SCM Type Filter: This allows filtering based on the SCM type, for example to identify all the projects still using CVS or all the projects that have migrated to GIT.
Unclassified Jobs: This allows finding all the jobs that are not in a view already. Note: you can create a circular logic of death, so if you are using this filter, make sure you put it in one and only one view.
User Permissions for Jobs: This allows filtering jobs based on the logged in user's permissions for the job, i.e. whether they can: configure; build; or access the job's workspace

Once you have created your chain of filters, you can just save the view to see the set of jobs in that view.

Tips & Tricks, How to Use it on DEV/RUN

Start simple - this is a very powerful plugin, and while you can build very complex chains of filters, e.g. a view of All Matrix build jobs which are in the build queue and have at least one non-stable build in the last week, are currently stable, have been started by the current user, are not in view B, have a build parameter with the name matching “[dD]atabase” and a value of “test2” where the SCM is matching “svn:.*/foobar/.*” and which the currently logged in user has not got the ability to configure may make perfect sense to you, but maybe you should start with something a little simpler first, and work your way up to such complexity... you really need to have 100's of jobs before very complex filters become necessary.
Unclassified Jobs and Other Views Filter should be used with care. Both of these filters have the capacity to create circular logic loops. If you use one of them in one and only one view, you have nothing to worry about, but once you think you need to use another one, break out your set theory math book from school to make sure you won't be creating a circular logic loop.
When you have a system with more than about 25-50 jobs, the default “All” view can become useless, as it can be hard to find the jobs you want. The Jenkins Wiki describes how you can edit or replace the “All” view. Alternatively you can keep the “All” view but just pick a different view as the default view from the main Jenkins configuration screen. It can be useful to have the default view just show jobs that are relevant to the logged in user.
The View Filters plugin is not currently available on DEV@cloud, but will be shortly. Installation will be just as it is for a standalone Jenkins instance.
If you are creating views which are filtered based on job parameters, or based on being relevant to the currently logged in user, you may want to use the Build Filter Wrapper column feature. Some examples might help:

You create a view which consists of all jobs that the currently logged in user committed to in the past week. You use the Build Filter Wrapper column to replace the default Status, Weather and Last Success/Failure columns so that they only display the results from the builds that the currently logged in user committed to. That way the logged in user can see which builds they broke, as opposed to builds that they committed to recently but that didn't break.
You create a view which consists of all jobs that have pushed code into the production server (i.e. where the “deployment target” parameter was equal to “production” or some such criteria). You use the Build Filter Wrapper column to replace the default Status, Weather and Last Success/Failure columns so that they only display the results from the actual deployments into production and the subsequent deployments into test are filtered out of the view... that way if the screen is full of blue balls and sunny skies, you are a happy camper!

Known Issues

None.

Relevant Documentation

--Stephen Connolly, Developer

CloudBees

Jenkins Support Comes to WANdisco’s uberSVN Platform

2012-01-30T12:35:00.000-05:00

If you use uberSVN, we have good news - now it's even easier to use Jenkins with it! Last year, our friends at WANdisco announced that Jenkins is fully integrated with uberSVN and available in the uberAPPS Store.

WANdisco's uberSVN is a freely available, open ALM tool that transforms Apache Subversion into a versatile platform designed for social coding. uberApps provides an array of tested and certified developer tools – both free and paid – in one user-friendly location. Here you can quickly download a version of Jenkins that comes pre-configured to work with Subversion and uberSVN (if you want to see a great overview/demo, go here). uberSVN also gives you "one throat to choke" for products, services, and support.

Now, backed by CloudBees expertise, uberSVN users can add advanced Jenkins support to the mix, so they know they have peace of mind when they depend on Jenkins for their crucial development processes. Since 82% of the Jenkins users we recently surveyed told us they consider Jenkins mission-critical, backing your Jenkins with formal support is a very good thing!

WANdisco's Jenkins support includes:

24-by-7 worldwide coverage
Online, email and phone support
Named support contacts
Online case tracking
Access to highly experienced Subversion and Jenkins support staff
One-hour response times with a Platinum or Platinum Plus package

With our team of world-renowned Jenkins experts, including Jenkins founder Kohsuke Kawaguchi, CloudBees provides higher-level support for any advanced technical issues that uberSVN/Jenkins users may encounter.

Not using uberSVN? No worries, you can still get support for Jenkins through a Jenkins Enterprise by CloudBees subscription.

Want to learn more? Visit the WANdisco blog or view the press release. Also, we'll be announcing a Jenkins Tips training webinar soon, so stay tuned!

Writing Programs that Drive Jenkins

2012-01-26T08:00:00.007-05:00

One of the interesting discussions during SCALE 10x was about using Jenkins as a piece of a bigger software. This person was interested in using Jenkins to run some business analysis operations, and wanted to have a separate interface for business oriented people. This is actually an emerging but common theme I hear from many users. Another company in San Jose actually built the workflow engine that uses Jenkins as a piece in a bigger application (aside from the actual build and test, this workflow involves reviews, approvals, etc.), and GitHub Janky can be classified as one such app, too.

This is something I always believed in — that every piece of software needs to be usable but a layer above. Or put another way, every software should be usable as a library.

So in this post I'm going to discuss various ways you can programmatically drive Jenkins.
Let's start with the REST API of Jenkins. For most of the data Jenkins renders as HTML, you can access its XML version and JSON version (as well as a few other formats, like Python literal fragment.) You do this by adding /api to the page (see http://ci.jenkins-ci.org/api for example.) Those pages discusss other REST API where applicable. For example, you can POST to a certain URL and it'll create/update job definitions, etc.

If you are going to use REST API, you might find the auto-discovery for Jenkins useful. You can discover Jenkins on the local subnet via UDP broadcast, or DNS multi-cast. There's also a distinctive HTTP header "Jenkins-Version" on the top page of Jenkins that allows your application to verify that it's talking to a real Jenkins server, as well as an instance ID that allows you to identify Jenkins instanecs. These featuers allow you to build smarter applications.

For Jenkins protected by some authentication mechanism, you can use the user name + API key in the HTTP basic auth (and I want to add OAuth support here.)

REST API is great -- its programming language agnostic. It is also convenient that neither the server nor the client has to trust each other. But those APIs are bound by the request/response oriented nature of the HTTP protocol.

Another great integration point for Jenkins is the CLI. This uses the same underlying technology that drives master/slave architecture, which enables your command line clients to be a lot more intelligent. For example, REST API exposes an URL that you can post to get a build started. But the equivalent command in CLI can have you block until the build is complete (and exit code indicates the status), or run the polling first and proceed to build only when the polling detects a change, or allow you to perform a parameterized build with multiple file uploads very easily. For protected Jenkins, CLI supports SSH public key authentication to securely authenticate the client.

A slightly different version of the CLI is "Jenkins as SSH server". Jenkins speaks the server side of the SSH protocol, and allows regular SSH clients to execute a subset of CLI commands. In this way, you don't need a Java runtime installed on the client side to drive Jenkins.

These two integration APIs are often much easier to script than REST API.

Those APIs are available for non-privileged users, and they are great for small scale integrations. But for more sophisticated integration needs, we have additional APIs.

One is the REST API access to the Groovy console, which allows administrator users to run arbitrary Groovy scripts inside the Jenkins master JVM (and you can submit this script as POST payload, and get the response back as the HTTP response). This allows you to tap into all the Jenkins object models. Unlike the REST API, in this way you can ship the computation, so in one round-trip you can do a lot. You can do the same with CLI, which also lets you access stdin/stdout of the CLI.
The other sophisticated integration API I wanted to talk about is the remoting API that does Java RPC (not to be confused with the remote API, which is the synonym for the REST API). The remoting API is the underlying protocol that we use for master/slave communications, and it revolves around the notion of shipping a closure (and code associated with it) from one JVM to another, executing it, and getting the result back. If your application runs elsewhere, you can establish the remoting API channel with Jenkins master, then prepare a Callable object. You can then have Jenkins master execute this closure, and the result is sent back to your JVM.

There's an example of this available. You bootstrap this in the same way the CLI client talks to the master, then you "upgrade" the communication channel by activating the remote code download support (which requires the administrator privilege, for obvious reasons).

The great thing about this is that your data structure is rich Java object model all the way, and you never have to translate your data to externalizable serialization data format like XML or JSON. This greatly simplifies your program.

I think this list covers all the major integration APIs that Jenkins offers. If you are building any interesting applications that use Jenkins as a building block, please share your experience so that we can make it better!

Kohsuke Kawaguchi
Jenkins Founder,
Elite Developer & Architect, CloudBees

Postgres in the Cloud Goodness with CloudBees

2012-01-25T17:17:00.011-05:00

Today, EnterpriseDB announced the availability of Postgres Plus Cloud Database. Some of the advantages of this DBaaS (Database as a Service) are point-and-click provisioning, online backups, automatic scaling and failover.

Developers who prefer Postgres can now use this service with the CloudBees Platform as a Service (PaaS) to build database-backed applications.

Configuring web applications (on CloudBees) to talk to the database requires minimal configuration changes. Developers end up just changing their datasource configuration settings.

I'd recommend that developers change an existing application to play with this database. The delta upload feature in CloudBees (where only touched changes are uploaded during redeploys) makes this process fairly painless (dare I say enjoyable :-)).

A detailed article that builds a web application from scratch, configures Maven to pull in the right Postgres libraries and uses JPA and JDBC is available here. The source code of the application is available on github. Feel free to use the code to get started quickly.

- Harpreet Singh, CloudBees

Painless Maven Builds with Jenkins

2012-01-24T17:42:00.007-05:00

Overview

One of the great things about Maven is that it provides a standard project and build layout, along with a standard set of "goals." Not only does this make it easier for developers to get up to speed on a new project, but it also allows Jenkins to provide special support for Maven projects, reducing the configuration needed while enhancing the build report automatically. That is the goal of the Maven plugin for Jenkins.

The central feature of the Maven plugin for Jenkins is the Maven 2/3 project type. Thanks to the Maven project object model (POM), this project type can automatically provide the following features:

Archive artifacts produced by a build
Publish test results
Trigger jobs for projects which are downstream dependencies
Deploy your artifacts to a Maven repository
Breakout test results by module
Optionally rebuild only changed modules, speeding your builds

All of the above can be accomplished in Free style builds, but this requires more configuration on your part.

Requirements for Plugin Use

All Jenkins releases have the Maven plugin included. You must also have at least one Maven installation.

How to Use It

First, you must configure a Maven installation (this step can be skipped if you are using DEV@cloud). This can be done by going to the system configuration screen (Manage Jenkins-> Configure System). In the "Maven Installations" section, 1) click the Add button, 2) give it a name such as "Maven 3.0.3" and then 3) choose the version from the drop down.

Now, Jenkins will automatically install this version any time it's needed (on any new build machines, for example) by downloading it from Apache and unzipping it.

Next, create a new Maven job by 1) clicking "New Job" on the left hand menu, 2) give it a name, and 3) choose the "Build a Maven 2/3 project".

You will then be presented with the job configuration screen. On this page, you need to provide 1) the SCM and 2) the Maven goals to call. That's it! Choose the SCM you want to use (we'll use Git), and then specify what Maven targets to call. We'll call the "clean site install" goals so we can see the full effect.

...

Now, just click build now, and click on the progress bar in the left hand "Build Executor Status" to watch Jenkins install Maven, checkout your project, and build it using Maven. The build output should look something like this:

Now that the project is built, we can navigate to the detail page for our Maven module (project page -> Modules link on the left). For each module, Jenkins displays:

A link to the module's workspace
The module's artifacts
A clickable test result trend
Recent changes (just for that module!)

What's unique about the Maven job type is that these links and reports are automatically broken down by module for you. You can even customize notifications for this module by clicking the "Configure" link on the left.

Finally, if we had configured Jenkins to build any downstream dependencies of this project, they would automatically start building after this build completed.

Tips

When running multiple builds simultaneously on the same slave, they will share the same local Maven repository. This can cause problems if two builds are trying to update the same artifact simultaneously, since local Maven repositories are not designed for concurrent access. One long-standing solution to this is to use the "Use private Maven repository" option in the "Advanced" section of the Maven build. This will create an isolated local Maven repository for the job (in $WORKSPACE/.repository) which prevents these problems. Jenkins releases since 1.448 let you specify a Maven repository per executor, which is a more efficient way to solve the same problem.

If you are using the Maven plugin, you should also investigate the M2 Release Plugin for automating releases with one click. The M2 Extra Steps plugin will let you run arbitrary build steps before and after your build. Finally, the new Config File Provider plugin lets you maintain different settings.xml which can be referenced by your Maven builds.

Relevant Documentation

Maven is a trademark of the Apache Software Foundation.

Ryan Campbell

Developer

CloudBees

Securing the Cloud: Part 2 - Managing Security Around Remote Login and Development

2012-01-24T01:47:00.014-05:00

In Securing the Cloud: Part 1, we looked at the ways in which developers at CloudBees manage credentials. In today's post, we'll look at how we manage security around remote login and remote development.

Like the previous post, because a large portion of our infrastructure is in the Amazon Web Services environment (AWS), this post will specifically focus on that platform.

Remote Server Login
One major advantage in using the CloudBees Platform as a Service (PaaS) is that you do not have to manage servers anymore. Using our platform, developers develop, deploy and scale applications with minimal server interaction.

However, behind the scenes, CloudBees engineers do need to manage server lifecycle. Not only for instances that run customer code, but for web proxying layers, databases, Git/SVN repos, and many other administrative things. In the previous post, we discussed the credentials that allow developers to see, and perhaps manage, the lifecycle of these servers. However, we also need to manage the ability to remotely login to these machines to perform maintenance or fix problems that may occur. In addition, we need to limit traffic from the outside world in a way that allows applications to work, but does not allow malicious attempts to break into the systems.

Locked Down Access
Our first strategy is to make prodigious use of EC2 security groups and rules. Each of our instances has a particular role it serves, and as such is tied to a specific security group that reflects that type of role. Our application servers, our proxying layer and our databases each have separate EC2 security groups attached to them. On the DEV@cloud side, our Jenkins master instances, the executor machines and the proxying layer also have their own EC2 security groups.

It is within these security groups that we can restrict outside traffic to only the ports needed, and then also limit inside traffic between the EC2 security groups where things need to "talk" internally. For example, our web proxying layer allows outside traffic from ports 80 and 443 - and that's it. Our application servers don't allow outside traffic at all, and only allow connections to specific ports coming from the web proxying layer. This tiered and locked down approach ensures we don't succumb to attackers looking for a backdoor into our environment.

Backdoors
Of course, we still DO need backdoors into the systems in order for our own team to get in and perform administrative tasks. Most commonly this includes remote login (SSH) to a server, but also includes access to backend web interfaces to monitor application health or observe application metrics in order to solve issues.

To ensure we maintain as much security around these backdoors as possible, we hide them all behind a Virtual Private Network (VPN) that is accessible only to CloudBees developers. We use openvpn, which is a userspace-based SSL VPN that tunnels traffic over UDP. Each developer who has the need for access is given a private key to access the VPN. Once established on the VPN, the developer now has access to the ports needed to get into the system.

Note that doesn't mean they automatically have access INTO the systems, it just means they have access to the mechanisms to get into the systems. Case in point: once on the VPN, developers have access into port 22 (SSH) on our various machines. However, this still doesn't mean they have the access keys to actually login to those various systems - this is a separate credentialing and distribution mechanism that is handled on an as-needed basis.

This two layer approach gives us a high level of security, while still maintaining usability for our development team.

Handling Problems
While it provides security, the VPN system can still be a source of friction. Maintenance, or an unplanned outage on the VPN system itself, can halt developer progress across the entire system. In a way, the VPN becomes a single point of failure for our team to be able to handle system level issues, should they occur.

To handle this, we allow our administrators to make temporary rule changes to the EC2 security groups. This facilitates work on system issues if the VPN system, itself, becomes a bottleneck to progress. As an example, they can open SSH access to a specific external IP address a developer may be using in order to let them login while bypassing the VPN. This change can only be facilitated by an administrator.

In addition, our security group rules are monitored by an external script on a nightly basis. A script matches the state of the security group rules with a known state stored in a Git repository; any deviations are noted and an email is generated. This allows all administrators to keep tabs on rule changes and ensure "temporary" changes get reverted, or made permanent by adding them to the Git repository of "good" rules.

We feel that our VPN approach, coupled with continuous auditing of security group rules against a known standard, provides us with a very high level of overall security around external facing access into our critical infrastructure. This, in turn, provides our customers with the highest levels of security against intrusion and potential data theft.

In my third and final post on the topic of security, we'll look at how we manage credential access to external services that developers may need to use.

Caleb Tennis, Elite Developer
CloudBees
www.cloudbees.com

Read Parts 1 and 3 in Caleb's Securing the Cloud blog series:

Securing the Cloud: Part 1 - Managing Credentials

2012-01-16T20:23:00.010-05:00

In the cloud world, companies must be vigilant to the types of risk that exist when placing their code and infrastructure into the hands of others. Anytime you place your business data in the hands of a third party, there's risk. While providers like Amazon, Rackspace or VMware have a certain amount of credibility behind their names - inherently, there's still some sort of risk involved. This is especially true with cloud providers, where the implementation and security behind the scenes is usually not visible to the end customer. You have to trust the vendor.

At CloudBees, we have a number of security measures in place to help safeguard your applications and code against external threats. Some of these are practices we've honed over the past few years, others are best practices that everyone should be doing. In this post, I'll share a few of the things we're doing to help safeguard our customers.

While we have equipment in multiple data centers worldwide, a good portion of our Platform as a Service (PaaS) cloud environment runs on Amazon Web Services (AWS) so this will be the focus of my article.

AWS Credential Management

Our service offerings have been developed over the past few years and, like many others who have spent a few years in Amazon's cloud, the evolution of credential management has improved. Originally, Amazon offered one set of credentials that were universal across your AWS account. To tackle security, a lot of people (CloudBees included) had multiple Amazon accounts for a layer of separation between services and access needs. In 2010, Amazon released a more fine-grained credential and access management system called AWS Identity and Access Management (IAM).

We have spent a significant amount of time revising our access management system to take advantage of this system. As noted, previously there was one centralized set of credentials for a specific AWS account. This meant that every developer who needed access had to be given these credentials. In addition, all of our services that utilized the EC2 API also had to have these credentials distributed to the instances they ran on.

I don't think I need to explain why having a single key to the kingdom is not the greatest. For starters, if a developer left the company this would necessitate a forced change of all of the credentials out there. This means that every developer would need to be given a new set of credentials, and every application would need to be updated to have the new set of credentials in place. While we make use of automation, this is certainly not a desired scenario.

When the company first started, the number of developers was small, and everyone did everything. However, as more people joined the team and had different access needs, it became not only a security threat, but also just a development threat for everyone to have full access to all systems.

Over the past few months, we've done an extensive audit and redeployment using both developer and service specific credentials throughout our system. Not only can we have specific credentials now for each developer and each service, but we can also lock those credentials down to minimize security risks.

One such example is our DNA service. This is an internal facing service that we use to monitor and manage instance and service health. The DNA application needs the ability to access instance lists, start and stop instances, and update IP address information, amongst other things. Not only does DNA now have its own credentials that are specific to its service, but those credentials are locked to a single fixed IP address. AWS will not accept commands using those credentials unless they originate from that single IP address. Now we worry much less about those credentials being used maliciously.

With developer-specific credentials, comes the ability to much more easily rotate and disable access as needed. Now we can easily/quickly remove access for a specific set of developer credentials, without impacting other services or developers in the process. As well, we can limit developer access to the pieces of infrastructure they need in order to do their work. Our major concern isn't a rogue developer causing issues, it's much more concerning if someone's laptop gets stolen, or someone at a coffee shop oversees login credentials on the screen. Restricted access also limits what a developer can do accidentally, if they target the wrong thing or try something when they don't completely understand the potential outcome.

Rolling out new credentials to all developers and services is not easy. This process has required considerable planning and execution, and you have to be ready for some of the potential downfalls. For one, when people have restricted access to the system, they now are not able to react to major system issues that may creep up. When service access becomes limited, it can cause future potential issues. For example, if new features are added that make use of restricted API calls - nobody may remember they are restricted and a significant amount of debugging time may be spent trying to figure out why things don't work.

You must also think through scenarios when developers still have access to change other developers or services permissions. For example, locking down our DNA service to a specific IP address increases security, but if any developer can go in later and change that lockdown, it may not be obvious that change ever happened. During some of our initial audits, we found a lot of security-related changes that were done ad hoc to quickly get something that had been broken working again, but then the security change was never later reversed.

As a result, part of our policy now is to disallow developers or services from making Identity Access Management (IAM) changes. Those changes are handled by a group of three administrators, and via an administrative account only. This account is the only one able to make IAM changes (controlling access to this account will be discussed in another blog post).

By distributing credentials in this manner, we feel we have much better protection of our infrastructure in the cloud which, in turn, allows us to keep our customers data more secure.

Stay tuned for Part 2, where I will discuss how we manage remote access and network traffic security.

Caleb Tennis, Elite Developer

CloudBees

www.cloudbees.com

Read Parts 2 and 3 in Caleb's Securing the Cloud blog series:

A Big Welcome to "Continuous Information" – The CloudBees Newsletter for Jenkins

2012-01-16T19:49:00.002-05:00

A Welcome from Jenkins Founder Kohsuke Kawaguchi

One of the take-aways for me from October's Jenkins User Conference (JUC) is that people do want to hear us talk about Jenkins -- not just about the user-level features but sometimes about gory inner details and what's being cooked.

Kohsuke

So I'm naturally very excited about the launch of a Jenkins newsletter. It will hopefully be a means for us to give you an overview of what's going on in the noisy bazaar that is the Jenkins project, for those who are too busy to go through all the e-mails, all the new plugins, and IRC conversations that are occurring every day.

Now, pulling this info together is hard work and takes considerable effort, so I wanted to thank the editors who are stepping up to take this on, and I hope some of you will consider participating in the effort by submitting articles, suggesting what you want to hear, and so on. I'm really hoping that this will be a useful community resource.

This is the inaugural newsletter! Sign up to receive future newsletters directly or view this issue in full…

-- Kohsuke Kawaguchi, Jenkins & Hudson Project Founder,
and Elite Architect at CloudBees

PS -- Watch Kohsuke’s Keynote at the JUC to get the latest status of the Jenkins project. In fact, all of the JUC presentations are available here. We've also just published a JUC highlights video - see who came, who spoke...who had the best shoes and what the most coveted JUC swag was!

PPS -- Call for Papers for JUC 2012 is out! Please help us spread the word...

Amazon 1 - Microsoft 0

2012-01-16T11:53:00.002-05:00

The cloud is redefining the IT industry as we’ve known it. Amazon’s recent announcement to provide free Windows micro-instances on EC2 is another event that underscores this revolution occurring from the move to the cloud and how even a giant like Microsoft has probably been forced in such a move.

However, I see several roadblocks to truly leveraging “free” on EC2 with the Windows platform. Let’s explore this further…

For as long as IaaS providers have existed, using a Windows machine has always been more expensive than using a Linux machine. As an illustration, you can see below the current prices for Amazon Web Services and Rackspace, two leading IaaS providers.

Rackspace Pricing

Amazon EC2 pricing

Using Microsoft Windows is about 33% to 40% more expensive than running Linux in the cloud (from just a per-hour pricing). Note that at that price point, in both cases you are not going to get support from any vendor (you would have to either get support from Microsoft for Windows or pay for a Linux subscription from Red Hat or another provider). This leads to several observations.

First, this demonstrates that current software licensed-based business models are ill-suited for the cloud. In order to be competitive, cloud providers are looking for ways to scale their business at the lowest possible cost, which, most of the time, means they’ll always prefer relying on Open Source software if it is feature-competitive with its proprietary counterparts. At the infrastructure layer, this is almost always true: virtualization, routing, firewall, load-balancing, storage, DNS, SSL, etc. are king in Open Source land. This makes it very hard for vendors like Microsoft and even VMware to remain competitive when up against cloud-providers’ offerings aimed at commoditizing the market. Consequently, in the new cloud era, especially at the infrastructure layer, either you are able to provide a full-fledge SERVICE offering (that is the “S” in “XaaS”) that can be packaged in a way that makes sense to your market, or times will be hard, very hard.

While this post focuses on Microsoft, it doesn't mean that things will necessarily be rosier for Open Source vendors down the road. Let's open a quick parenthesis on this aspect. Once a cloud provider has decided to move with Open Source software, the next question is whether they need to rely on a third-party company to get support for that open source components they are using? While they might need help initially, providers will typically have grown the required expertise in-house as they expand, with no need for a high-friction process involving a third-party vendor. Furthermore, since their infrastructure is very homogeneous (hundreds of thousands of identical servers!), they can support their open source software at a much lower price point than any generic subscription provider ever could: they do not have to run gazillions of tests on gazillions of different motherboards, chipset, LAN cards, controllers, disks, CPU, etc. Their domain problem isn’t a problem – as their infrastructure is hyper-focused. This point is illustrated by the recent launch of Amazon’s own Linux distribution, based on RHEL/CentOS: Amazon Linux. Not only is support for that version of Linux provided as part of the default IaaS support cost (and as we know, FREE is a tough price point to beat), but it probably saves costs to Amazon since those customers are using a stack that Amazon understands extremely well, hence reducing the probability of hard-to-diagnose support tickets.

The bottom line is that any infrastructure subscription/license business, open source or proprietary, is a hard business in the cloud era.

But let’s go back to our point with regard to Microsoft Windows. So, Windows is not only up to 40% more expensive to run in the cloud, but the tools available to build a Windows AMI (vs. a Linux AMI), customize it, provision it, build a JEOS (Just Enough OS) are much weaker on Windows than on Linux. Anybody who has tried to properly automate the usage of Windows and Linux in the cloud will tell you that none of this is trivial, and that Windows is particularly hard. Second roadblock to Windows' adoption in the cloud: Windows' cloud DNA is weak, very weak.

Let's pursue on the on-ramping of new cloud users. About a year ago, Amazon launched their “free usage tier” offering. The idea was to provide free compute capacity to new subscribers so they could experience and learn the cloud at no cost. Yet, because this was a free offering, it was only made available for Linux-based AMIs. This meant that for about a year, tens of thousands of developers, DevOps, sys admins, etc. have learned how to use the cloud on Linux, not on Windows. Third roadblock: Windows is not a good candidate for on-ramping new users to the cloud, both for technical and pricing reasons.

Microsoft has probably tried to ignore this for while but this wasn't sustainable: Amazon announced today that Windows is now also eligible on these free tier instances. For this to happen, MSFT probably had to provide free pass-through licenses to Amazon for that very purpose and maybe even more as this is really a way for Microsoft to tap into AWS's on-ramping funnel. As we all well know, whenever we read “Microsoft Windows” and “free” in the same sentence, we know that something big had to force that situation.

One last roadblock? Well, now you can try to do something meaningful on Windows with 613MB of RAM, which is what AWS’ free micro instances provide...

The cloud is redefining the IT industry as we’ve known it and re-shuffling power among vendors. This is an opportunity for new entrants such as CloudBees to make a difference as well as for dominant positions to shift base. My humble opinion? Amazon will become the new Microsoft and today’s news might be one of the first symbolic events that signal that shift. It will be very hard for Microsoft to turn their ship in a new direction.

Oh, and remember, you can test whatever the cloud revolution means to you, as a Java Developer, on the CloudBees PaaS. Just try us for free! ;)

Onward,

Sacha

Better Integration Between Jenkins and GitHub (with the GitHub Jenkins Plugin)

2012-01-11T12:19:00.002-05:00

Overview

Jenkins (and in fact any CI systems) works best when builds are triggered after each and every commit. That provides you, the developer, with the most instant feedback possible. That, in turn, means you get fix(es) to the problems you introduced before you leave the context of what you were working on.

There are two general techniques for detecting commits in any source control system:

Polling (the "Dad, are we there yet... are we there yet... are we there yet!" technique) where you ask the source control system at regular intervals.
Push, where the source control system tells you there has been a change.

The first technique requires no support from the source control system but puts a lot more load on both the CI system and the source control system. As well as that, it can only pick up changes as often as the polling frequency.

The second technique requires support, in both the source control system and the CI system, for the sending and receiving of notifications.

The Jenkins GitHub plugin provides support for receiving push notifications from Github post-commit hooks and using those notifications to trigger jobs within Jenkins. It can set-up the post-commit hooks in GitHub for you.

In addition, GitHub decorates the Jenkins “Changes” pages with links to the pretty GitHub commit, issues pages and adds a sidebar link to the corresponding GitHub project page.

Stable Release Version

The current stable release is version 0.9

Requirement for Plug-in Use

There are not too many requirements for using this plugin:

Jenkins 1.400 or newer
Jenkins Git Plugin version 1.1.12 or newer
At least one GitHub hosted project to build ;-)

Step-by-Step Instructions on How to Use it

Installation

Go to your Jenkins instances root page.
If your Jenkins instance has security enabled, login as a user who has the Overall | Administer permission.
Select the Manage Jenkins link on the left-hand side of the screen.
Select the Manage Plugins link.
On the Available tab, select the Github Plugin and click the Download and Install button at the bottom of the page (if you do not got the Git Plugin installed, do not worry, Jenkins is smart enough to install/upgrade the Git plugin, where required).
Restart Jenkins once the plugins are downloaded (Note: users of Jenkins 1.442 or newer should be aware that the plugin currently requires a restart to function correctly).

Configuration

If all you want to do is enable the hyperlinks in the Recent Changes for the build, it is just a case of providing the GitHub project URL in the Jenkins job configuration, e.g.

Goto your Jenkins instance job.
Select the Configure link on the left hand side of the screen.
In the GitHub project field, enter the URL of the GitHub project. If your GitHub project's git URL looks like: git@github.com:username/project.git,

then the GitHub project should be: http://github.com/username/project/

or if the project is private, you can get faster navigation with: https://github.com/username/project/

If you want to enable build triggering, you need to configure your Jenkins instance for receiving the push notifications from GitHub. There are two ways you can achieve this. The first way is to let Jenkins manage the Post-Receive URLs for you:

Go to your Jenkins instances root page.
If your Jenkins instance has security enabled, login as a user who has the Overall | Administer permission.
Select the Manage Jenkins link on the left hand side of the screen.
Select the Configure System link.
In the GitHub Web Hook section select the Let Jenkins auto-manage hook URLs option.
Ensure you have provided at least one username and password for connecting to GitHub (the password is required as GitHub does not expose an API for managing the Post-Receive URLs).

The second way is to manage the Post-Receive URLs yourself:

Go to your Jenkins instances root page.
If your Jenkins instance has security enabled, login as a user who has the Overall | Administer permission.
Select the Manage Jenkins link on the left hand side of the screen.
Select the Configure System link.
In the GitHub Web Hook section select the Manually manage hook URLs option.
For each project that you want to have triggering builds, you need to open the Repository Administration screen on that GitHub project's page.
Select the Service Hooks tab.
Select the Post-Receive URLs hook.
Add the URL, which will be the root URL of your Jenkins instance with /github-webhook appended.

Once you have configured your Jenkins instance for receiving the push notifications, you can enable jobs being triggered via the push notifications:

Goto your Jenkins instance job.
Select the Configure link on the left hand side of the screen.
Select the Build when a change is pushed to GitHub checkbox and save the configuration.

Tips & Tricks, How to Use it on DEV/RUN

The plugin is identical to configure on DEV@cloud.

Any Known Issues

The recent GitHub refresh of their site's look-and-feel broke automatic Post-Receive URL management for versions of this plugin prior to 0.9

Relevant Documentation

Cheers!

Stephen Connolly

Elite Developer & Architect

CloudBees

Global Build Stats Plugin

2012-01-10T09:00:00.002-05:00

Overview

For every job, Jenkins maintains a set of past builds that it uses to compute the "weather" of a project as well as trends for unit tests, build time or performance metrics. This is an awesome feature, as it goes beyond the base concept of continuous integration (CI) and brings the project to life, as a mirror of the team development practices.

The drawback is that job history consumes a large amount of disk space and slows Jenkins initialization, so you have to configure a retention strategy to reduce the footprint of your jobs' history. This also only gives you a per-job view, not the whole Jenkins instance.

Jenkins Global Build Stat plugin has been designed to get usage statistics at an upper level. As Jenkins does for jobs, it gathers data for every build but doesn't keep detailled bits for each run. Storing build history as synthetic information, it can keep a far longer history of Jenkins builds and compute a synthetic view of your CI activity.

Stable Release Version

Global Build Stats plugin was created by Frédéric Camblor for his personal use, and has reached a 1.2 release with various progressive improvements, thanks to active Jenkins community contribution. This is a typical example of how the Jenkins open source community works.

Requirement for Plug-in Use

Global Build Stats plugin is a self-contained plugin, that can be installed on top of any 1.398+ Jenkins instance. The graph engine is pure Java-based, which means it does not require the native library to be installed.

Step-by-Step Instructions on How to Use It

After plugin installation, Global Build Stats is available on the Jenkins administration page. First, initialize data for the existing builds that Jenkins has maintained as jobs history. This will create the initial dataset for Global Build Stats, that can used to set up statistical charts and will enrich with new builds.

Next, create charts with various criteria. Charts only report build time, but can filter by job name, slave node used for building or status. A chart can easily be created to monitor a specific build slave or successful builds of a set of jobs related to team activity. Here is a sample chart of jobs running on DEV@cloud to build...DEV@cloud:

As it doesn't use Jenkins jobs build history (but for data initialization), Global Build Stats will store build history in custom files; that may increase the data storage needed by gigabytes. For this reason, it's important to configure a dedicated retention strategy. The simplest and most useful is to set up the number of days to keep builds history. You can then compute and compare CI usage for, say, the last six months, without requiring a huge hard disk, and generate a large set of charts based on usage.

Tips & Tricks, How to Use it on DEV/RUN

Global Build Stats can be used to compute the total cumulative build time for selected jobs. This can be very useful if you consider migrating your CI to CloudBees DEV@cloud service, but you need an estimate of hosting cost.

Running a few test builds on CloudBees infrastructure to determine the ratio between our build slave velocity and your own infrastructure, you can then estimate the build minutes required, per month, for your usage of Jenkins. Thanks to job filtering, you can do this calculation on a subset of your jobs that can be run on a DEV@cloud FREE subscription with the limited set of essential plugins.

Known Issues

Global Build Stats plugin uses a dedicated set of files to store build statistics. Even if access is performed in an asynchronous way, on large Jenkins instances this may result in a long initialization period. For huge Jenkins build farms with a single master, a wait period of a few minutes for Jenkins to initialize on startup may be required. Improvement is planned in 1.3 via a more flexible storage engine.

Relevant Documentation

Global Build Stats plugin is well documented on https://wiki.jenkins-ci.org/display/JENKINS/Global+Build+Stats+Plugin. You may also get in touch with the plugin author by joining the Jenkins user mailing list.

Enjoy!

Nicholas de Loof, Senior Engineer

CloudBees

CloudBees Wins a Perfect Trifecta!

2012-01-10T03:03:00.001-05:00

I always start the New Year with plenty of well-intended resolutions. Never mind, most of them don’t last: as soon as I start hitting the post-break reality, resolutions seem harder to satisfy, exceptions to whatever I have committed to seem to stack up and, oh well…maybe next year.

For now, the same logic doesn’t seem to apply at CloudBees. Here’s why.

In 2010, we committed to ourselves that in 2011 we would offer the first end-to-end, development-to-production Java PaaS in GA. Last year, in January 2011, we announced the first end-to-end, development-to-production, Java PaaS in GA. That was a great way to start the year -- without breaking an important New Years resolution.

In 2011, we committed to aggressively communicating to the industry who CloudBees is and what we offer. We did just that. As if it was a mirror image of January 2011, this January--in just a few days—also saw that resolution come to be! In just a few days, CloudBees received three awards!

The first one was NetworkWorld, who recognized CloudBees as one of “9 hot technology startups to watch in 2012.” Wow - not bad for a one-year old company!

The next day, Dr. Dobbs announced the JOLT Awards. CloudBees received an Honorable Mention and, according to Executive Editor Andrew Binstock, “…CloudBees was included for special distinction because we strongly feel it indicates an important new direction in coding.”

Finally, last but not least, today we received an “InfoWorld 2012 Technology of the Year Award,” placing us in good company with products such as … Amazon Web Services, the iPad, Apple Siri, MacBook Air, etc. You can also read the press release we issued today.

With such a great start, I have a very good feeling about CloudBees in 2012...and I can’t wait to see what NEXT January may also bring! After all, good things happen in threes…

Onward,

Sacha

Creating Jenkins: Kohsuke's Story on Chariot Techcast

2012-01-06T11:16:00.000-05:00

Chariot Solutions' Ken Rimple featured our very own Jenkins founder, Kohsuke Kawaguchi, on his Chariot Techcast Podcast this week. Kohsuke created Jenkins as a way to make sure he didn’t forget to check in files for the CVS projects he contributed to…. and now tens of thousands of developers worldwide use the tool for continuous integration. (A survey last month shows that 82% of them consider Jenkins mission-critical to their development process!)

In the interview, Kohsuke discusses…
•    The challenges in developing a tool like Jenkins
•    Motivations for building Jenkins and developing on his own when no one was using it
•    Differences now that Jenkins is such a big hit
•    How he came up with the plug-in system and transparency of project and developer community
•    How the name Jenkins came to be
•    Current efforts to improve Jenkins
•    And more...

Listen and learn!

Jenkins Community Survey Results: 82% Consider Jenkins Mission Critical

2011-12-22T09:00:00.001-05:00

The results from the first Jenkins Community Survey are in! 624 people took the time to share how they use Jenkins and tell us what capabilities matter most to them.

Highlights:

82% of respondents said Jenkins is mission critical to their development process
71% use Jenkins as their only CI tool
Jenkins is useful for companies of any size, but 76% work for organizations with > 10 developers
80% of respondents have 3 or fewer masters and 10 or fewer build agents
65% of companies have 10 or fewer executors
76% have between 2 and 50 projects
Additional reporting capabilities and high availability were the highest-prioritized features

Check out the full survey results here (minimal registration required). You’ll learn which features users thought were most important to develop, which factors are most influential in making Jenkins easier to adopt, and many other insights into a typical Jenkins installation.

Many, many thanks to everyone who took the survey. You have been instrumental in helping the Jenkins community focus on the most needed features! And congratulations to Chris Dolan, who won the drawing for the AppleTV.

Hello Java, Hello iPad, Hello World!

2011-12-21T12:14:00.000-05:00

A CloudBees New Year’s Challenge

Experience the joy of Platform as a Service as we ring in the New Year! Publish an sample “Hello Java in the Cloud” application on the CloudBees PaaS by January 16th, and we’ll enter you to win an iPad 2. We even give you the example code – all you have to do is roll it live, which only takes minutes. And maybe spice it up a bit (you can add your own text to the photo above)!

Bonus for Charity
For every app created and tweeted under the #CloudBeesJava hashtag, we’ll donate $5 to the International Committee of the Red Cross!

Bonus X2
For everyone who ReTweets your app link with the hashtag #CloudBeesJava, we’ll donate an additional $1!

Bonus X3
Any app that is published and tweeted is eligible, but if you make your app fun, interesting, and/or festive, we’ll triple-enter your name to win the iPad.

How It Works

Publish an app on CloudBees. Get started now!
Using the handy button, tweet the app’s URL using the hashtag, #CloudBeesJava. Feel free to post on your Facebook page, blog, etc.
We’ll monitor the hashtag and enter each app to win the iPad. Each RT will garner $1 more to charity (up to $1000 for RTs, but there's no donation limit for apps deployed). If you modify the sample app to make something cool, we’ll enter you three times. No worries if you write an app but don’t have Twitter – just email me the URL and I’ll get your name in the pot (lisa AT cloudbees.com).
Win! You could win an iPad 2. The International Committee of the Red Cross wins more resources to help the world. And everyone wins the satisfaction of successfully launching a Java an app in the cloud! Read the contest details here.

Give it a try!

PS - Meme alert: If "I can haz PaaS" doesn't make any sense to you, check out I can has cheezburger and LOLcats...

How Much Does CI Development in the Cloud Cost? Less Than You Think!

2011-12-14T10:24:00.006-05:00

Developing in the cloud is now so cost effective, you will try to justify NOT developing in the cloud! We have some new tools to help you estimate how much it would cost you to develop on the CloudBees PaaS. But first, some background...

To run Jenkins, you need to allocate a machine to be what is called a "Master"; this Master contains your build job definitions as well as the Jenkins job runtime scheduler - which runs at all times. While you can run build jobs on this Master, its capacity is limited and you will typically have to allocate additional computing capacity to the Master by attaching to it additional build machines - also known as "Slaves". A single Master can have tens of Slaves attached to it.

The efficient technology we have developed for the CloudBees Platform allows us to partition and isolate EC2 resources for improved efficiency. Our underlying unit of compute power consumed by any application, including our own services like Jenkins, is an "app cell." Since our processing units are much more fine-grained than Amazon's, we can make use of our own pool more efficiently than just the m1.* levels available to you from Amazon. Consequently, the CloudBees Platform will allocate just enough capacity to run your Jenkins Master, making it a much more efficient setup than on an m1.small setup, for example. Yet, no build jobs will ever run on this Master; your build job will execute on dynamically allocated dedicated machines of your requested size (m1.small, m1.large), and will do so only for the duration of your build, by the minute. Furthermore, if you need to perform several builds in parallel, multiple machines will be dynamically allocated to your Master - but only for the duration of those builds. For example, running five ten-minute builds sequentially or in parallel will lead to the exact same cost on DEV@cloud! Consequently, with DEV@cloud, you end-up paying for a minimal fixed monthly fee for your Jenkins Master as well as for the exact number of build-minutes you've consumed (and only those in excess of your monthly quota).

If you were to do this yourself on Amazon EC2, you would have to decide how many machines of what size to allocate of the different EC2 machine sizes -- both for your Master as well as for your Slaves -- and you'd have to account for concurrent build Slaves upfront by statically allocating the number of machines your want to your Master. Alternatively, you could handle the dynamic provisioning and de-provisioning yourself, as well as manage the Jenkins workspace state.

Thus, not even accounting for the labor associated with managing the EC2 resources directly, as well as the Jenkins setup, CloudBees is able to offer a lower price for its development services like Jenkins than you would be able to achieve yourself using Amazon directly.

How much lower? Check it out - go here and calculate!

Onward,

Sacha

Enhanced Pricing and Packaging for the CloudBees Platform

2011-12-13T12:34:00.028-05:00

Since we launched our DEV@cloud development platform service more than a year ago, we have experienced tremendous success! Today we have thousands of developers using our services for their builds and tests.

In the spirit of giving at this time of year, we are pleased to announce:

Significantly lower pricing for DEV@cloud services
Changes in our technical support offering

...all nicely packaged up, just in time for the holidays. Read below for more about our progress this past year, and these exciting new announcements.

2011 Platform Highlights

In the last six months, we have added lots of new features, including:

The ability to add integrated CloudBees Ecosystem Partner services to your DEV@cloud environment. Ecosystem Partner offerings include popular services, such as JFrog, Sauce Labs, SonarSource and XWiki

Complete integration between the DEV@cloud and RUN@cloud services, with no need for you to set up any credentials or install any plugins

Ability to select a build machine that best fits your needs

We will continue to add more features on an ongoing basis. Additionally, we are working on several performance enhancements.

Lower Pricing for DEV@cloud Services: Efficient Technology and User Adoption

Today, we are announcing some changes to our development services pricing. First of all, we are removing all restrictions to the number of Git and Subversion repositories you can create: you are now only limited by the disk space allocated to your account. This is especially handy if you want to host one application per Git repository, for example.

With regard to Jenkins, we are significantly dropping our build minute prices and extending our build minute quotas! Given the massive number of builds we are now performing, we are dropping build prices by more than 80% and moving from one cent per build minute (or 60 cents per hour) to just 10.6 cents per hour! Our m1.large build prices will benefit from the same discount and will be billed at the equivalent of 42.5 cents per hour (billing still takes place by the minute). If you want to estimate what your new bill will be, we have also added a pricing calculator to our DEV@cloud pricing page.

Also, effective immediately, our Pro and Enterprise subscription customers will benefit from a doubling of the free build quota: from 2,500 free build minutes to 5,000 for Pro subscribers, and from 5,000 to 10,000 build minutes for Enterprise subscribers.

At these prices, doing continuous integration in the cloud should be mandatory!

Changes in Our Support Offering

Finally, on the support front, we are making some immediate changes. We will now provide silver support to all of our customers and only offer premium support options on-demand. Note that if you already benefit from a premium support subscription option, this change will not apply to you; you will be able to continue getting excellent support from CloudBees based on your current support terms and conditions.

We are working on additional improvements and will be communicating more about them very soon. Stay tuned!

P.S. Just as we were getting ready to publish this blog, InfoQ published a round-up/overview of six major PaaS offerings, including CloudBees. CloudBees was ranked very highly! Read the InfoQ article.

Onward,
Sacha Labourey
CEO
CloudBees, Inc.

Follow CloudBees:

Devops in the Cloud – Meetup this Thursday at Netflix

2011-12-12T18:16:00.000-05:00

Netflix, JFrog, and CloudBees are excited to host a Devops in the Cloud Meetup this Thursday! Taking place for the first time at the Silicon Valley, Devops in the Cloud will put the spotlight on cloud computing best practices, tricks and tips from those who have been there.

Where: Netflix campus in Los Gatos, CA

When: Thursday, December 15th, 5:30 PM to 8:30 PM (PT)

Details: Full invitation here, but it’s sold out and the waiting list is full. So…

Live Stream: Attend virtually via live-stream!

You’ll hear about thoughts, solutions, tools and visions of future development practices in the cloud. Even better, a Netflix case study will demonstrate how many of these powerful processes are already possible.

Featured Presentations

Building Cloud Tools for Netflix – Carl Quinn, Joe Sondow, NETFLIX
Where are my runtime artifacts?! Cloud Deployment Using a Smart Repository – Frederic Simon, JFROG
Maintaining reliability in an unreliable world – Jeremy Edberg, NETFLIX
DEV@cloud, Behind the curtain – Kohsuke Kawaguchi, JENKINS founder and CLOUDBEES Architect

Whether you're a Software Developer, a Release Engineer or a DevOps, this event is your opportunity to take a glimpse into the future of CI in the cloud.

Don't miss the live stream!

SendGrid is Now Part of the CloudBees Ecosystem

2011-11-18T09:00:00.000-05:00

This blog is a guest post from Brandon West, Developer Relations Engineer at SendGrid…

SendGrid is happy to announce that we've partnered with CloudBees, a Platform as a Service (PaaS) provider that offers cloud-based services for building, testing and deploying Java web applications. As of today, SendGrid's email service is now part of the CloudBees Ecosystem! CloudBees and SendGrid are both focused on reducing the friction that developers have to deal with when it comes to configuring and building out infrastructure so that they can spend their time building great things.

For those of you who aren't familiar with SendGrid, our service makes it simple to send email from any web application. SendGrid's cloud-based SMTP email infrastructure relieves developers of the cost and complexity of maintaining custom in-house email infrastructures. We provide reliable email delivery, scalability, and real-time analytics along with flexible APIs that make custom integration a breeze. Our service alleviates the email headaches for developers, allowing them to focus on making great apps.

If you've got a CloudBees account, you can now enable SendGrid from the CloudBees Ecosystem page with one simple click. SendGrid is a great choice for CloudBees customers looking to add a robust cloud-based SMTP email solution to their applications, and our partnership will make it easier than ever to do just that. If you're a working on a Java application and are not already using CloudBees, we encourage you to sign up for a free account and give it a try. If you build an app with SendGrid and CloudBees and want to share it with us, we may feature it in a future blog post. We’re excited to see what people build!