Here at HomeAdvisor, we've recently started taking a closer look at creating more automated tests around our growing suite of microservices. Our testing strategy has typically followed the software testing pyramid. Unit tests have always been the biggest part of our test approach, with integration and component level tests becoming more prevalent as we've migrated to microservices. But all of these testing strategies are within the application, and require almost as much maintenance as the code itself. While all of these testing strategies are just as important in microservices as they are in a software monolith, microservices usually come with some additional testing burden in the form of contract testing. This refers to the public facing API that a microservice offers, and comes with its own set of concerns beyond normal testing methodologies. In this post, we'll talk about how we use Swagger to automatically create API documentation for all of our microservices, so that we … [Read more...]
Api-Tite: An API Automation Framework
In recent years, building a comprehensive API automation framework has become crucial for large software systems. Our digital world has grown to include more and more devices that communicate with each other instead of humans, and most of this is driven by APIs. It's more important than ever to ensure organizations can release code with ease and confidence. Using an API automation framework ensures APIs are thoroughly tested prior to releasing, including boundary and error cases that are tedious to test with every code release. There are several benefits that such a framework will provide: Forces APIs to be documented consistently for both human and machine readability. Faster and more reliable than UI based automation such as Selenium. Can cover a great deal more scenarios that might be impossible to using UI Automation. As a Automation Test Engineer who specializes in REST API testing, I have seen and tried numerous strategies for API testing. In my experience, … [Read more...]
The HomeAdvisor Microservice Stack
We've talked a lot recently about our experience decomposing our monolithic web applications and migrating towards a microservice architecture. In this post, the third part in our ongoing microservices series, we'll talk about the technology and tools we have chosen to build our common microservice stack. Choosing a Microservice Stack As we talked about in our microservices pitfalls post, one of the most important things you should do before building microservices is settle on a microservices stack. A microservice stack is the consistent set of technologies with which all teams will develop their microservices. This is what Mark Richards referred to as a "base image". As we'll discuss below, failure to take this important step early on can have lasting impacts on your organization. The HomeAdvisor microservice stack is based on a variety of open source projects that generally have great community support and proven track records across multiple industries. Choosing your own stack … [Read more...]
Microservice Pitfalls & AntiPatterns, Part 1
Last month I had the pleasure of attending ArchConf 2016 and hearing from some really smart people and their experiences working with microservices. As we've discussed before, we began our migration to microservices awhile back, so I wasn't necessarily there to learn how to "do" microservices. What I was really interested in was hearing how other organizations have approached microservices, what struggles they've had, and any pearls of wisdom they may have to share. We've certainly learned a few lessons along the way, but as they say it sometimes takes a village. One of the talks that really resonated with me was from the great Mark Richards entitled "Microservice Pitfalls and AntiPatterns". Mark started off with two excellent quotes to help explain the difference: An anti-pattern is just like a pattern, except that instead of a solution it gives something that looks superficially like a solution but isn't one. -- Andrew Koenig A pitfall is something that was never a good idea, … [Read more...]
Introducing Kafdrop: Open Source Kafka UI
Four years ago, we introduced Kafka into our technology stack as our preferred messaging system. As we've noted elsewhere, we've embarked on a transition to a microservice based architecture Microservices need to communicate with each other to do anything useful. This communication can happen either synchronously (via REST HTTP calls, for example) or asynchronously, via messaging. We chose Kafka as our message broker because of it's speed, simplicity, and resiliency. If LinkedIn can process 800 billion messages per day, it can certainly handle the volume of messages we intend to ultimately throw at it. However, for all of the things we love about Kafka, one of the things we found lacking was the tooling around it. While it ships with a variety of useful command line tools, they use inconsistent parameters, the parameters are hard to remember, and you often needed to run several commands to get a sense of what is going on in the cluster. Kafdrop: An Open Source Kafka UI For these … [Read more...]