Last week Confoo took place in Montreal. This has always been a good conference in that there’s quite a bit of variety in talks offered. No matter your interests, there will be talks of interest for you. It’s a pretty pricey conference though but normal compared to many larger conferences. This post covers some of the main points that I took away from the talks I attended. Mainly for my own long term reference but maybe it’ll be useful to others.
I won’t go into details on implementation below as there’s a lot of information out there on each subject already. From the talks at Confoo, I aimed for ones that were more architecture focused with as little language specific info as possible. This provided a lot of theoretical information leaving the practical information flexible for whatever implantation is desired.
Microservices
In the past I’ve worked on a number of very “large” applications where all code and logic is kept under a single codebase/cms/framework which, while simpler at the beginning, can easily get out of control as time goes on. New features are often added to the existing application without fully analyzing whether it really needs to live within that application. Over time, the application grows and grows with features that should probably live outside of it. Customizations of the underlying framework may take place to handle all the new use cases making upgrading more and more troublesome.
I think one of the main hindrances to “defaulting” to microservices instead of the main application is the overhead to setting up a new application every time. There are many skeleton codebases to aid the initial setup but I’ve found that these have been great for reference instead of actually using them as the start of the application. Since everyone has their own way of doing things, after setting up a few microservices, I imagine your own skeleton code base will be formed for the way you or your company works. This will greatly reduce the overhead for future services.
In summary, aiming for multiple smaller applications instead of a monolith should create a more maintainable application over the long run in exchange for some more overhead and setup at the beginning which should get easier with time and practice.
Data Visualization & Dashboards
I’ve used them in the past and it’s fairly common knowledge these days that “application dashboards” are extremely useful in the world of the web. Seeing trends and information at a glance can tell you whether that last deploy made things better or worse or it can highlight an area of improvement for the application. In the past I’ve used Librato which was a great data store and visualizer adhering to the StatsD format of sending data. At the time I used Librato, other open source time series DBs such as Graphite weren’t the best to get set up and the final interface was lacking. I attended a few talks around dashboards specifically to see what has changed in the past few years.
I was happy to learn of Grafana which would replace the visuals of Graphite. It has a very beautiful UI and I’m hoping the installation process will be simpler than Graphite. I’m looking forward to giving this a try in the near future.
I also learned of Logstash which can be installed on servers to automatically parse log files and extract things like response codes and times from web servers. This is a really easy “win” for data gathering as it can be installed and automatically tracked without any changes to an application’s setup.
I’ll also be looking into time series databases to store the data such as InfluxDB. I’d definitely aim to keep the visualization and data storage separate this time around so that if a better visualizer comes along, switching while keeping the underlying data-store will be simpler. I’m starting to lean more and more to this separation of concerns in application so that there’s flexibility in tools and less burden when changing/upgrading one or the other.
I look forward to getting a “stats” server up and running as soon as possible to get back some of that valuable information these dashboard provide.
Circuit Breakers
I had never really heard of or used “circuit breakers” in the past so this was a good concept to be exposed to. Especially when dealing with applications that are using microservices or communicating with 3rd party APIs.
The most basic circuit breaker is to monitor an API call to ensure it succeeded or timed out. If it succeeded, all’s well, and there’s nothing special to do. If it timed out, or is otherwise unreachable, something might be wrong with the endpoint so there’s no point flooding it with calls, potentially making the situation worse. Set a timer for when to try another call. After that time is up, try one more call to the endpoint to check if things have changed. If it succeeded, clear the timer and report the endpoint as healthy. If it is still unreachable, set it to unhealthy and restart the timer for the next retry.
For example, if your user authentication is an external endpoint and calls to it start failing, you can block login attempts and display a message on the login page. Though a user still can’t log in, knowing that something is wrong is much better than endlessly waiting or erring out while the target endpoint is down. This allows for parts of your application to “go down” while not taking down the whole.
Confoo itself
Overall Confoo is a great conference to attend. Personally, there were very few time blocks where I couldn’t find something I was actively interested in. There were even a few where I had to make some compromises and miss a talk I was looking forward to in favour of another one.
One thing that stood out was the food! They had fresh coffee stations for all your latte needs and the lunches provided were top notch. Chicken, pasta, and roast, all very tasty and well prepared.
If you’ve never been to Confoo, and if you have someone to sponsor you a ticket ;), I highly recommend you check it out next year in Montreal. They’ve also recently started hosting one in Vancouver if you’re on the west coast.