How to build a AWS CodePipeline to build and deploy a Lambda function written in Scala

After reading through many tutorials and playing around with tools and concepts I finally managed to build an automated AWS based deployment chain, which deploys my Scala code into a Lambda.

This is what I have in the end: When I push my code changes into my github repository, AWS CodePipeline reacts on the change and downloads the sources into an AWS CodeBuild environment. There everything is built using sbt and the build artifacts are stored in S3. When that’s done AWS CloudFormation takes over to deploy the changes. This creates or updates a Lambda function to use my latest Scala code for event handling.

The code is just a basic http request handler that takes a piece of json containing a “name” key and responding with a hello string using the key’s value in the response body.

I started to work my way up from this tutorial about running Scala in Lambda. But instead of reacting to an S3 event I wanted the handler to react on a HTTP request with some json in the body. Luckily Yeghishe provides some code to handle those requests in Scala.

So I ended up with this MyHandler.class

package example

import io.github.yeghishe.lambda._

// handler io.github.yeghishe.MySimpleHander::handler
// input "foo"
object MySimpleHandler extends App {
  def handler(name: String, context: Context): String = s"Hello $name"

case class Name(name: String)
case class Greeting(message: String)

// handler io.github.yeghishe.MyHandler
// input {"name": "Yeghishe"}
class MyHandler extends Handler[Name, Greeting] {
  def handler(name: Name, context: Context): Greeting = {"Name is $name")
    Greeting(s"Hi ${}. Have a nice day!")

For sbt to build this there are some external library dependencies to manage. Including also the AWS dependencies my build.sbt looks like this now:

import sbt.Keys.libraryDependencies

javacOptions ++= Seq("-source", "1.8", "-target", "1.8", "-Xlint")

lazy val root = (project in file(".")).
    name := "lambda-demo",
    version := "1.0",
    scalaVersion := "2.11.4",
    retrieveManaged := true,
    libraryDependencies += "com.amazonaws" % "aws-lambda-java-core" % "1.0.0",
    libraryDependencies += "com.amazonaws" % "aws-lambda-java-events" % "1.0.0",
    libraryDependencies += "com.amazonaws" % "aws-lambda-java-log4j" % "1.0.0",
    libraryDependencies += "io.github.yeghishe" %% "scala-aws-lambda-utils" % "0.0.2"

assemblyMergeStrategy in assembly :=
      case PathList("META-INF", xs @ _*) => MergeStrategy.discard
      case x => MergeStrategy.first

This adds three amazon libs to let the code run in the AWS environment and the Yegishe’s lib for the request handling. I also had to add a plugins.sbt into the root/project folder containing just the line

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.12.0")

This gives my sbt the ability to execute an assembly command, which builds a jar containing my code an all the needed libs in it. At this point I was able to directly upload my jar into Lambda and run it through the web interface.

But I wanted a fully automated build pipeline. Following this amazon tutorial I created a CodePipeline with four steps. In general everything was straight forward from here, but there still where some traps for me to fall into, since this tutorial assumed JavaScript code to be executed in Lambda.

I had to change the buildspec.yml to deploy my assembled jar file correctly.

version: 0.1

      - echo Build started on `date`
      - echo Run the test and package the code...
      - sbt compile
      - sbt assembly
      - echo Build completed on `date`
      - mkdir deploy
      - mkdir deploy/lib
      - cp target/scala-2.11/lambda-demo-assembly-1.0.jar deploy/lib
      - aws cloudformation package --template-file samTemplate.yaml --s3-bucket testbucket-77 --output-template-file NewSamTemplate.yaml
  type: zip
    - deploy/lib/lambda-demo-assembly-1.0.jar
    - NewSamTemplate.yaml

In the tutorial the call to “aws cloudformation” is done in the install phase before any code is being actually compiled and packaged. So the package command only puts the NewSamTemplate.yaml and all the source files into my S3 bucket. In the end I always ran into ClassNotFoundExceptions in Lambda, since it could not find my handler class. The compiled and packaged build artifacts where actually stored in another S3 bucket, which was created by AWS CodeBuild.

Moving the package command into post_build ensures that it is called after everything is build. It will than upload everything in respect to the CodeUri value in the samTemplate.yaml

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Description: An AWS Serverless Specification template describing your function.
    Type: 'AWS::Serverless::Function'
      Handler: example.MyHandler
      Runtime: java8
      CodeUri: deploy
      Description: ''
      MemorySize: 512
      Timeout: 15
      Role: 'arn:aws:iam::362856109184:role/service-role/lambda_basic_execution'

Here I defined the code artifact location to me the deploy folder. In the buildspec above I made sure this folder exists and that the assembled jar file will be in a lib folder inside that deploy folder. I learned that Lambda can only access jars if they are in a lib folder in the final deploy zip.

Inside the CodeBuild environment I also needed a Docker image, that had sbt and aws-shell on board. The first to build the Scala code and the second to execute the aws cloudformation command. Since I could not find a prebuild image I created my own one and pushed it into dockerhub. You can find it here 7oas7er/sbt-aws-shell.

Don’t Tell – The Missing Column of the Delegation Board

I love Delegation Boards since I first learned about them in Jurgen Appelo’s book “Management 3.0”. If you have not heard about them yet, you can find a good introduction here at the Management 3.0 website.

A couple of month ago I became the head of a department of Agile Coaches and Agile Project Managers. To help me find my own way of leading, especially when it comes to making decisions, I started to set up my delegation board.


Soon after I was installed in the new position my team, other colleges and the whole company expected me to make decisions about anything. I had no chance to do an upfront planning of my delegation levels and so I decided to fill the Delegation Board on the fly.

It was like saying to my team “Hey team, I decided about this, you see and I did it all alone. What do you think about it?” This helped me a lot to reflect about an appropriate delegation levels for my different responsibilities and it was a clear message to my team, that I expect them to challenge me continuously for advancing to more deliberate delegation.

For instance I was asked by HR if I want to have some trainees in my department. We never had any and so I directly said no. This was a decision and I did not even ask my team what they think about having trainees with us.

So at least I put a sticky note onto my delegation board in the “Tell” column with the topic “Do we need trainees”. I told the team I made that decision alone and I’m not sure if it is a good idea to make those decisions alone. If they feel uncomfortable about it too, they should please actively challenge it to a higher delegation level.

At the same time I came to learn that there are decisions I make, that I don’t even tell my people about. It happened when I was hiring a new freelancer as an Agile Coaches for my department. We negotiated salaries and only the candidate and me were part of that discussion. Finally we came to an agreement about the pay.

That was a decision I made that would not even fit into the “Tell” column, because I did not intend to tell my other employees about the result of the decision. Right now we simply do not have that kind of culture where employees know and discuss salaries openly.

Still I wanted to make transparent that I make those kind of decisions. They should know I decide about salaries for external and internal employees. And they should not that I would not even tell them, what the result of the decision was. Knowing this my employees have the chance to question the status quo. They can think about how they want to deal about salary negotiation and suggest me another more open way of handling it.

Ofdont_telll course they can also decide that they are totally fine with the current situation and they like me to take care for that kind of administrative act as a service for them, which is what they are doing at the moment.

In conclusion I’m convinced that the delegation board is a great tool to shape and develop your delegation and leadership style. And if you really believe in it, you should add another column “0 – Don’t Tell”.


Bringing Kanban into Marketing – Part 2

A couple of month ago I started to implement a Kanban system for a marketing team in our company. This team is responsible for planning and running more than 500 internal and external events each year.

As you can read in my previous post they asked me for help because they were missing transparency. They just wanted to see what everyone in the team is doing on a day-to-day basis. They wanted to have their work visible. They wanted to be able to see redundant work as much as being able to recognize if they have forgotten something.

Very soon we set up a Kanban board that visualized the flow of their work beginning with an initial briefing for the event, all the necessary planning, the actual running and finally a closing phase for an event. Every day they met in a stand up in front of the board, updated each other on event preparation task progress, made decisions on how to handle blockers, pulled in new tasks and saw how great they were doing.

It did not took long to remove the primary source of dissatisfaction they had, the lack of transparency. Over time a couple of things emerged, that finally led us to redesign our Kanban board, which I want to share with you here.

One thing that happened really fast in the first weeks after we set up the board, was that transparency decreased again, not because they had submarine projects or did not make proper use of the board and task cards. It just got to filled up with events and tasks.



What you see here is the board filled up with event cards in the second column. Every card presenting a single event that is currently in preparation. At times there were more than 30 events in preparation in parallel. The next three columns “To Do”, “Doing” and “Done” are reflecting the work on a task level that has to be done to prepare an event. Usually we had lots of tasks floating around in “Doing” even over longer periods of time.

Also there are only few tasks in “To Do”. The team simply did not use that column to visualize the work still to be done for the reason that if they did, the board would have been exploded with upcoming tasks. In the end you see how this Kanban board was not longer able to provide a good overview about the work in the system.

Delivering events is different – fast is no goal

I came to realize that the nature of work of preparing and running events is very different from the work of a product development team in respect of how the work flows. Let me explain that. When I saw the filled up board of the event team, I initially thought they just have too much work in progress (WIP). So limiting the amount of events that they prepare at the same time, would reduce the number of cards on the board and increase the board’s ability to provide an overview again.

And limiting WIP in general reduces cycle times, which is exactly what we want in product development, where we seek to push out new ideas to our customers as fast as possible to generate feedback and iterate over it. This is totally different for the event team. There is absolutely no sense in delivering the company’s Christmas party in August. Nearly everything that they work on has a fixed dead line. And success for them means to deliver on time, not late and not early.

I got good feedback from Arne Roock on the issue. He suggested that, if they could not finish earlier, they may be able to start later thus decreasing cycle time in another way. Why shouldn’t we try to start the planning of events later? Good idea I thought, but then I was convinced, that it is not helpful for the team to start with the planning of the Christmas party say in early December. All possible event locations would have already been booked out by that time. Same is true with other external dependencies for events like event speakers, caterers and even attendees.

See my problem? I got a Kanban team, that has to start work items as soon as possible an deliver on time. All my talk about reducing cycle times, limiting WIP, working in serial instead of working in parallel just made no sense to them. And so I was missing goals and arguments to improve the team. Kaizen to what goal?

The nature of the team’s work – waiting is normal

Things got clearer for me when I understood that events are work items, which wait most of the time. When you come to plan a new event, you do some things very soon, like book a location or acquire speakers. When you are done with that early preparation, you wait.
Most of the rest of the preparational work for an event is then done in the weeks directly before the event’s “delivery date”. Waiting is normal in that system. So we decided to build that into the board, showing us which events are currently waiting.


I got the tip from Wolfgang Wiedenroth to add a space to the board, where events are explicitly allowed to wait. He suggested multiple waiting columns one for each month of the year and that’s what we did. You can see the additional waiting columns at the right on the board.

Now when the team starts to work on an event, it does all the early things to do, like booking hotels and contacting speakers. When this is done the event is parked and the team decides when it will continue to work on it. For instance they decide its OK to park the Christmas party till October after location is clear. The event card is then put in the October column.
When October comes they will decide to pull back the event card for the Christmas Party into the system and do all the remaining preparation tasks for the event.

Using this approach the team’s board is much more expressive again. The team sees what events they are currently working on at a glance as the number of events in preparation in parallel is reduced.

One other change to the board was suggested by the team. Instead of having “To Do” and “Doing” columns on the task level, they wanted to have “Tomorrow” and “Today” columns on the board, reflecting what are they working on today and what are they planning to do tomorrow. This column layout suites the reactive nature of their work much better. Though they have a rough plan when they start to prepare a new event, most of the actual work they do is highly dependent on the reaction of external factors and thus very reactive.

In result these two board changes increased the transparency for the team again. Everyone sees easily what’s currently worked on and who is working on what. The daily stand ups are running smooth and in time again and provide opportunities for the team to help each other and to decide how to organize for optimal progress. Still I am missing some kinds of measurable goals for them like faster time to market would be to a product development team. I am looking forward to get your feedback and ideas for system improvements.


Bringing Kanban into Marketing


Three weeks ago one of our marketing teams asked me for some consulting about increasing process transparency for themselves, their stakeholders and their management. Since Monday we are running a Kanban implementation and having Daily Stand Ups in front of what became the company’s biggest Kanban board. In this post I will show you how we got that far and share some first observations with you.

The Team

Part of our marketing department is a team which is taking care for the planning and running of internal and external events and for creating and delivering educational content like webinars. This “Event Team” is responsible for the success of more than 500 events each year many of them being big customer marketing events. The work of the Event Team has great value to our company and the company knows the team is doing a great job.

Nevertheless the team asked for a workshop to talk about the way they work. They had the feeling some things could be improved on the team’s process and workflow level. In an initial meeting with the team they worked out, that their biggest pain is a lack of transparency. They are all working on many things at the same time, but they miss the big picture.

Who works on what? Who needs help? What is left to do to prepare event x? They want to have this transparency for themselves and for their stakeholders and managers. They want to be able to explicitly show how much stuff they get done and what kind of value they deliver to us all. So we discovered what is called a “source of dissatisfaction” in Kanban and what would give us a goal for improvement from now on.

Getting Buy In

After that meeting I had ideas how to tackle the transparency problem. I would recommend to the team and the team’s management that we set up a Kanban system where we start to visualize the flow of the team’s work, thus making transparent who works on what, what the work is and where the work lies.

Introducing Kanban to a team that has no explicit Agile background is not a small change. I knew it would affect how the team members interact with each other, it would call for even more self-organization and for the team to take ownership of their process. For that change, the related time and cost investment and the risk of failing I wanted to have an explicit buy in.

I met with the department lead and spoke with him about the team’s request for help and my ideas for supporting the team, got his go and promised to keep him updated. For the team I scheduled a short Kanban training. In 90 minutes I introduced them to the concept of evolutionary change, the J-Curve effect, the principles and practices of Kanban, the idea of creating a physical team board and the investments and risks evolved. I got their buy for the change and I planned the next step: Creating a Kanban board.

The Board Building Workshop

A week later I met the team to create the team’s Kanban board. I put some blank sheets of flip chart paper on the meeting room’s wall to scribble board designs and I introduced them to our approach of designing their Kanban system. Here are the steps I’ve prepared for them on another flip chart.


Starting with work item types we took a lot of time to discuss the work flow and the discovery process was very iterative. For instance we initially defined the work item types “Offline Event” and “Online Event” as the team said it was doing two different things here. During the work flow discovery we found that cards of both item types would flow the same way and we decided to have just one work item type “Event”.

The work flow we found for planning and running events in this team came down to “Inbox -> Briefing -> Preparation -> Running -> Finishing”. Inbox is of course the place where new event cards come in. Here the team needs to prioritize the events to make sure they prepare more important and more complicated events first. The first thing happening to each event is a briefing session with the event’s stakeholders. There goals, content, audience, expectations and so on are identified and documented by the event team.

An event card will leave Briefing if the team knows what to prepare for the event. The Briefing column is very similar to a task planning column of a software development team and the briefing itself reminds me of a Sprint Planning II in Scrum. The column Preparation has the typical three sub-columns on a task level To Do, Doing, Done. No surprise the work of the Event Team is mostly focused around preparation of events. After an event has been prepared, meaning that all preparation tasks went to Done, the event can take place. The card sits in Running. Finally for some events there is a debriefing or feedback session, which occurs in the Finishing column.

It took us 90 minutes to finish our board design. We did not make it through points 3 to 5 of the system design flip chart but I was fine with it as the problem to solve is the lack of transparency and for that having a visible work flow might be a good solution already. The last thing I wanted to know then was how big the real physical board would need to be. I asked how many events they are currently working on so I could multiply the amount with the sticky note size to get the board dimensions. I was really surprised when the team discovered more and more events they are currently preparing. I think we stopped counting at 30. Enough for me to know that we would need a very big board 😉

We used the team’s lockers as the board background every locker being a column of the Kanban board. I also scheduled a Daily Stand Up every morning with the team in front of the board.

The first week

Now after some days the board fills up with more and more tasks. Some events are “delivered” each day. They go into the Running column and into Finishing the next day. The team uses this to briefly give an update on how the event went. New events are entering the board from left and are pulled into Briefing and Preparation. On the task level the team is having spontaneous discussions. Sometimes someone adds additional information that have not been transparent to all, sometimes a task is trashed as the team notices it is not necessary to do it.

We already faced the limits of our current board design. It seems we are in need of a swim lane for tasks that are not related to a special event and I have the feeling we need one or two additional columns for events that are waiting to be prepared after the briefing and that are waiting to be run after the preparation. We also need to find a way to easily visualize who works on what for about 100 task cards. I scheduled a board iteration meeting and will post about the evolution of the Event Team’s Kanban system.

On other thing that took me some time to realize is that a plain optimization of lead time might not be an appropriate goal for this team. Most of the events they are preparing are scheduled for a fixed date in the future, so most of them have dead lines. For this team delivering an event in March that is expected to take place in April does not make sense, though the lead time would be less. If we seek for any optimization here I think we have to focus on the task level in the event preparation phase, but I’m not sure what the next goal could be and what kind of measuring will support us here. Any ideas and feedback welcome.

Kanban – What’s in for you as a ScrumMaster?

When I first heard something about Kanban several years ago I did not like it at all. Working as a passionate ScrumMaster with a team following good Scrum practices, I was very suspicious about it. The first thing I’ve learned about Kanban was that there are no sprints and there is no commitment and it sounded horrible to me. I just could not imagine that a team will aim for high performance without giving a commitment to the Product Owner and to themselves.

In the last two years I had a lot of touching points with Kanban. I listened to talks, read books, drank coffee with Kanban enthusiasts and finally even worked with several teams making use of Kanban implementations. My perceptions about the usefulness of Kanban in software product development changed completely. Nowadays I would not want to set up a Scrum team without making use of the principles and practices of Kanban. Otherwise I would not set up a Kanban team without using some of the Scrum ceremonies like retrospectives or daily stand ups.

I know ScrumMasters who are still very suspicious about Kanban and only think of it as being Scrum without commitments. If you are one of them, I want to give you a brief list of things Kanban can do for you to make your Scrum better.

Flow management

A Scrum team is self organized. That means after pulling a set of stories into the sprint or after committing onto a sprint goal, they can do whatever they want to have a successful sprint. Kanban has practices for your team to manage their work flow from generating ideas to delivering product increments to customers. It gives your team good guidance on how to use the freedom of self-organization successfully by explicitly visualizing the complete work flow and limiting work-in-progress for instance.

More to see

When set up properly you as a ScrumMaster will be able to learn more things about your team using several metrics and charts of Kanban. Those will add to your Scrum toolbox and your ScrumMaster’s intuition to evaluate the fitness of your team and to guide them to become better.

Experimental learning

There are retrospectives in Scrum. And your team is expected to use them as learning opportunities and to commit to improvements. Kanban in addition wants you to set expectations for each change your team commits to do and later on asses if your change was successful or not. By using learning models like the Deming cycle for instance Kanban shows you how to evaluate you improvement ideas in a complex environment and how to keep track of them.

Greater adaptability if needed

I love Scrum and I believe it is the best fitting framework for many software development teams working on innovative products. But Scrum is also very strict about many things, like its roles and ceremonies. It even expects me as the ScrumMaster to defend it against framework changes. This will sometimes put you as a ScrumMaster into the very uncomfortable situation to foster the team’s self-organization but not allowing your team to modify Scrum to their individual needs.

Kanban gives you a broader perspective. It helps you to see your team from a systems thinking point of view and encourages you to continuously change your process as long as there are sources of dissatisfaction left. Kanban helps you to let go and even to encourage your team to modify their process beyond the borders of Scrum if it increases their business agility. The evolutionary change that Kanban fosters will help you control the risks involved when crossing “safe Scrum borders”.

Less resistance to change

If you want to successfully change how people work you need their acceptance and trust. Many teams which are confronted with Scrum for the first time perceive it as a big revolution. It might initially scare them so much that they don’t even want to start changing at all and even if they start it might take a long time until they really see and feel the benefits.

Kanban encourages you to start with your team right where they are and even accept all roles, job titles and responsibilities. From that starting point you will change continuously and step by step. Most of the time it is much easier for others to follow your change management that way.


From my perception scaling a Kanban system is a very straightforward process, though it might not be easy in terms of getting more teams, departments and management into the system. Starting with the visualized flow of your team you can scale out by taking into account what is happening with the things your team builds and where does all this work for your team come from. You grow your Kanban and integrate more and more of the complete value stream.

By following the same principles and practices you can also manage the flow of many teams, departments or the whole organization as you do with Portfolio Kanban. You just have to choose the appropriate granularity for the work items on your board.

For me there is absolutely no Scrum vs. Kanban question. I see both of them acting in different dimensions and helping each other just the same way I would encourage my Scrum team to make use of XP practices like TDD or pair programming. Kanban made my Scrum better, I believe it will improve yours to.

Scaling The Happiness Index

How we started – the need for more direct feedback

Early in 2012 we (some ScrumMasters, team leads and other IT managers of Germany’s top real estate portal ImmobilenScout24) found value in the idea to have something like a continuous employee happiness feedback.

By the time we already had 500+ employees and we were making use of several annual employee feedback surveys. Those surveys gave us good insights into the thoughts, feelings and wishes of our employees. They had comprehensive and sophisticated questions covering all relevant aspects of work. And of course there were always parts dealing with all kinds of feedback related to leadership and management.

The survey results were useful to us and they still are but they were lacking of prompt feedback. It was hard for us to see how for instance management decisions are taken or how people think about events or new strategic directions by assessing happiness half a year later. Because of the comprehensiveness of the annual surveys it was no option to simply do them more often. We wanted more direct feedback but we don’t wanted people to do surveys all day.

What we were looking for was something lightweight. Something that will take only five minutes to do and still delivers valuable content for us. We found it in the Happiness Index which was mentioned in a blog post by Jeff Sutherland and which was originally invented by Henrik Kniberg at Crisp.

The Happiness Index

If you are not familiar with the Happiness Index, let me introduce you to it. The basic idea is to ask people only few questions about their happiness but regularly and often. This will then enable you to chart happiness over time and recognize trends and impacts of events to the happiness of your team members.

The underlying believe is that happy employees are better employees in terms of creativity, productivity etc. and that one important aspect of good management is to make sure your employees are happy at work. This follows all the findings about motivation that are covered so well by Dan Pink or Simon Sinek for instance.

The Happiness Index originally asks the following questions:

  1. What’s your name?
  2. How happy are you? (on a scale from 1 “very unhappy” to 5 “very happy”)
  3. What makes you feel best right now?
  4. What makes you feel worst right now?
  5. What would increase your happiness?

The second question gives you a simple number as answer, which makes it very easy to aggregate and to do all kinds of fancy statistics with it. The questions three to five are free text and allow people to express their feelings about happiness in natural language.

Adopting the Happiness Index and rolling it out

The Happiness Index was originally used on a team level to visualize a team’s mood over time and to help teams and management to improve their happiness. We wanted to use it for all our IT product development teams and we already had about 25 with a total of 150 people in early 2012 when we started.

The level of trust within a team is naturally higher than the level of trust between 150 people of a whole IT department. So we decided very soon to have our Happiness Index anonymous. No name field would be part of the survey.

One other thing that we wanted to have differently than the original Happiness Index was the general happiness question “How happy are you?”. This was too general for us as we wanted to learn about things that we might be able to influence. We made two distinct questions out of it.

  1. How happy are you in your team?
  2. How happy are you with the company?

Building and maintaining good teams is one of the core management responsibilities and so the question number one was aimed to give us direct feedback about the quality of our work as team leads or ScrumMasters for certain teams.

The second question was intended to give us more insights on how people think about the bigger cultural and strategic topics in or organization. We intentionally avoided to define what “team” or “company” means. We hoped to learn where employees draw the borders here.

After some time we dropped question number four “What makes you feel worst right now?”. The given answers were highly redundant to the answers given to question five “What would increase your happiness?” and the last one seemed us to be better in enabling people to make suggestions for improvements. We find it to be more proactive.

For the amount of data we expected to collect would be too much to be visualized completely and directly we came up with the idea to have a Quarterly Happiness Review meeting with Management and all employees. For that I committed myself to dig into the collected data of the last quarter, analyse it, find trends, visualize everything and then present the results.

Running and maintaining the Happiness Index

For the high number of people we wanted to address regularly and since some of them were working remotely we looked for a web survey solution. Pretty soon we decided to go with a Google Drive survey which writes its input into a table. I spent some time figuring out how to use Google Drive functions to continuously fill in additional data into the table with every newly inserted row. But I got it working and now every survey answer got the right “next even calendar” week assigned.


The survey and the generated charts were then included into our wiki, so that our teams could use the survey and see the feedback in a happiness chart directly on one page. The chart with the aggregated team and company happiness was placed right beside the survey form and updated every couple of minutes.

Google Drive gave us this instant charting of happiness over time and allowed for some spreadsheet analysis and Excel exports to do more comprehensive analysis when needed. When I’m preparing a Quarterly Happiness Review I always end up in Excel. I play around with pivot charts to find new interesting connections in the data we have.

We decided to remind all the teams to give us happiness feedback every two weeks. I do this by simple sending out an email which contains the link to our wiki page with the embedded survey. Every two weeks is the level of granularity we found appropriate for keeping track of our employee happiness. We therefore aggregate all votes given over a period of two weeks into one data point like “calendar week 16”.

When sending reminder emails I sometimes add some ad hoc analysis. For example I give the top or low three topics of the last two weeks. This motivates people to vote as they see something is happening with the data and it is not just stored somewhere.

For the Quarterly Happiness Reviews I try to figure out about trends in the quantitative happiness values and show how they might relate to certain contexts in time. More important to us but more difficult to generate is the qualitative analysis. What is making people happy? What do they want to change? As those survey fields are free text by design I end up reading each end every post. To get a first impression I put all given answers in a tag cloud generator and get an intuitive visualization about the relevant happiness topics.


I then assign clusters to all the given answers. Some answers are more related to the work environment and others relate to management. Some mean something about working Agile and some speak about the teams. Over time I empirically invented a set of clusters that seems to fit the answers of our people very well. The set is precise enough for us to draw conclusions from it and still it contains not more than a couple of clusters, which makes it easy to handle.

What have we learned so far?

From Happiness Review to Happiness Review I was able to better analyse and visualize the data and learned how to interpret it. Now after two years of measuring continuously and after the sixths Quarterly Happiness Review meeting I will give you a brief summary of what we’ve learned so far about ourselves.


We always had the feeling to work in an open, friendly and collaborative company culture which is making us happy and it was no surprise to us that this shared gut feeling is reflected in the data too. We see that all in all our employees are very happy with and in their teams. The happiness with the company is good too, but slightly lower than the team happiness. We interpret this to be an expression of the very team focused Agile working culture that we have established in IT product development. You identify with your team first and then with the company.


From the qualitative data we’ve learned that the main sources of happiness of our IT product development teams are working in a good team, having high performance and working Agile in that order. Nearly three-quarter of all answers to the question “What makes you feel best right now?” go into on of these three clusters. Lately the cluster “Good management” is getting bigger and people give more and more positive feedback in that direction.

As one of the responsibilities of management is to keep and maintain the happiness of their teams, this learning can be seen as an empirical leadership guidance. If we want to have our IT employees happy then we must care for team building and bonding, help them perform and let them live Agile values.

On the other hand we’ve learned that certain things exist that make people unhappy too. Most of the time the change requests go into “Better management”, “Work environment” or “Be more Agile” clusters.


Since a couple of month we also have our colleges from the Sales Department on board. This enables us to recognize what happiness sources we share between IT and Sales employees and where the needs and perceptions differ between people of departments so different.

It showed up that IT and Sales share the importance of good teams and high performance for their happiness. Additionally we could see that performance is more related to reaching goals for the colleges in Sales whereas it is more related to delivering product increments for the IT teams.

In addition to that the people in the IT gain some of their happiness from reaching technical excellence which is not a surprise. The Sales teams on the other hand gain happiness from providing customer excellence as they have direct customer contact each day. Interestingly we never got any happiness feedback related to customer excellence from our IT teams though we work Agile (which the teams love) and customer happiness is an important goal for us.

This way the happiness index helps us to recognize a learning opportunity. I am convinced that we have to make Sales and IT people to talk with each other regularly about the products we build and the quality we deliver as the IT teams can only benefit from the customer excellence point of view the Sales teams have.

One last thing that I want to mention is the way the Happiness Index affected happiness directly. Like in the management wisdom “you get what you measure” just by having it set up and running and reminding people to tell us how happy they are every two weeks, our happiness became something more tangible, something that we took more serious. We had more moments to reflect and talk about it and thus treating it with special appreciation. From an employees point of view, having the Happiness Index shows that our management cares for their happiness.