At HomeAdvisor, as our ecosystem of Java clients, mobile apps, and microservices continues to grow, we've been thinking a lot more about backwards compatible software. Typically, we think of software compatibility in two forms: intraprocess (source code, compiled libraries, etc) and interprocess (APIs, messaging, etc). With more than a dozen agile teams all writing code and services that have to work together, keeping compatibility in mind is important for every change we make. In this post, the first of a three part series on writing backwards compatible software, we'll look at intraprocess software compatibility. We'll look at the different ways software libraries can introduce breaking changes, from simple source code level changes to more difficult to track logic errors. We'll also look at some of the best practices we have adopted to help us prevent writing software that breaks other teams. Backwards Compatible Software in Source Code Source code (or compiled code you use … [Read more...]
Introducing Kafdrop: Open Source Kafka UI
Four years ago, we introduced Kafka into our technology stack as our preferred messaging system. As we've noted elsewhere, we've embarked on a transition to a microservice based architecture Microservices need to communicate with each other to do anything useful. This communication can happen either synchronously (via REST HTTP calls, for example) or asynchronously, via messaging. We chose Kafka as our message broker because of it's speed, simplicity, and resiliency. If LinkedIn can process 800 billion messages per day, it can certainly handle the volume of messages we intend to ultimately throw at it. However, for all of the things we love about Kafka, one of the things we found lacking was the tooling around it. While it ships with a variety of useful command line tools, they use inconsistent parameters, the parameters are hard to remember, and you often needed to run several commands to get a sense of what is going on in the cluster. Kafdrop: An Open Source Kafka UI For these … [Read more...]
Fedex Friday Spring 2016
On Friday March 18, the HomeAdvisor tech team took a day off from our backlogs to play with new technology. Our semi-annual day of innovation, commonly known as a software hackathon or codefest, allows team members to explore new languages and technologies in the hopes of improving some aspect of development or production, or just learn something new. We've adapted the practice from Atlassian's "Shipit Day" made popular in a Dan Pink TED talk on motivation... because we have one day to deliver the goods. In this post we'll discuss how we run our semi-annual day of innovation at HomeAdvisor, and discuss some of the projects we saw. How It Works We typically pick a Friday that is deconflicted from our normal development schedule. However, because we work in 2 week sprints, every Friday is either a delivery day or the last day of a sprint. While our developers don't directly participate in deliveries, they may be called upon to support any issues that arise. As you would expect, … [Read more...]
From Monolith to Microservices
In our last post, we introduced our villainous software monolith and explained its origin story. To recap, in the early days of moving from a small startup to large software organization, it made sense to clump all our code together. Builds were less complex (always build everything), software dependencies were never an issue (all the code you need is one place), and server space was expensive (deploy everything to a few hosts). Life was good. But times have changed, and just like the pet rock, not all ideas are good forever. We're no longer a small band of developers trying to build a few applications to secure some more VC. We've grown like crazy and hired a ton of people over the past 15 years and our customer base has grown to levels we never imagined in those early days. On our busiest days we connect upwards of 50,000 consumers to service professionals. And with the introduction of products like Instant Connect and Instant Booking, the demand on our systems will continue to … [Read more...]
Multithreading with Java 8 and RxJava
Multithreaded code is something we've been using more and more here at HomeAdvisor. Obviously, running in a modern servlet container we've been dealing with multithreading for years. A couple of years ago we began using multiple threads to handle a single HTTP request. Why would we ever want to deal with the complexities of multithreaded code? Well, we can't say that we were exactly excited to introduce complexity, as we think in simplicity lies elegance. The sole motivation for multithreaded code is performance, and performance is very important to us. There are two main reasons we care about performance: We love our users and want to keep them happy. Bots will crawl more of our site the faster it is to respond. Throughout this article we'll use a simple scenario to highlight different Java multithreading techniques. We have some code that makes two possibly long network calls to get data, then combines the data into a single list to create a new object for displaying. To … [Read more...]