Re-develop an API from a Platform Engineer Team
Creating an API from scratch is something we see many times in books and learning materials, but not often in real life. Most of the time, the API already exists, and we have to iterate on it.
👋🏼Dear Optimistiers! I want to start today by announcing The Optimist Engineer has reached 200 subscribers starting this week!! What to say? Just thank you all!!
In the current issue, I will explain my experience, working in a Platform Team, creating a new API based on an existing one, and sharing the challenges I found, the failures I had, and the successes as well.
In today’s issue, we will cover:
Platform Engineering Team
The Use Case
What Worked Well
What Worked No So Well
Do you think this is something that might be interesting to someone you know? Share it now and continue your reading!👇
Platform Engineering Team
Since this is a quite recent naming, I want to dedicate a couple of sentences to describe what I mean by Platform Team. Here I’m talking about a team of software engineers, with different levels of experience, implementing software/applications/microservices/services to be used by other teams (Feature Teams) which, in the end, deliver a product to the company’s customers.
All these dev teams are agile teams, which means we have a multi-disciplinary set of professionals, and not only SRE-related people. I highlight this fact because most of the literature about Platform Engineering refers to SRE-related work and, in my opinion, that’s just a subset of the concept.
The Use Case
The use case is very simple and, most likely, you experienced the same already;
There is an existing API that was meant to satisfy some use cases.
Some of those use cases are not needed anymore or never were.
Some technical concepts made sense in the past, but not anymore.
There are new use cases that the API needs to satisfy because the company’s product evolve.
The developer experience by using the API is not the best and the Feature Teams struggle to use it.
It’s time to be better; it’s time to do better.
What Worked Well
It’s always a good idea to iterate about the Domain your services and APIs should know. It’s instrumental that you, as a developer, understand the Domain for your API and, at the time of addressing a use case like this, it’s worth it that you revisit if the Domain changed or not. Spoiler: Most likely yes.
Well, that first paragraph was about Domain Driven Development, which is great and, by the book, it’s pretty clear. However, how do you ensure you know your Domain? In this use case, you achieve this by talking to the current users of the API: The Feature Teams. For this, I would recommend performing a first round of interviews with the Tech Leads from all the Feature Teams that are using the existing, and not great, API. Your goal on this first round of interviews is to detect if your Domain changed since the API was created, and if Feature Teams had to be imaginative to solve problems that the API does not solve for them today.
Another thing that worked pretty well in my experience, and actually is the most valuable learning I had, is to keep frequent and fluent communication with your users during the design of the new API. If you talk to users once the design is done, they will ask for something you did not think about, or they request something different/new when they see the final API. You have to think about this re-develop of the API as a new “product”, or version, for your customers, and so agile principles apply as well.
My recommendation to achieve this is by meeting/chatting with some Tech Leads or Senior Software Engineers who are using the existing API, as soon as you have something that makes sense in your brain. Ask them what they think, if the orientation is right, and if they would feel comfortable or better than today by using this new version of the API. Do this in a petite comité at the beginning, otherwise, you take the risk of having a meeting that could be highjacked by the attendees.
Don’t get me wrong: staying at a high level when asking for feedback from technical people about the API you are building for them is not a good idea. Get dirty; go into details about the usage, the performance, open up the difficult questions and the complaints they have nowadays with the existing API… You will see that they are looking forward to helping you with all this.
One of the problems to solve is the Developer's Experience. For this, you have to identify the possible problems. For example, maybe when the API was originally created the usage of the gRPC was a great idea and, maybe today, for the new use cases at least, using REST is a better idea.
One of the most important keys is that the API should be intuitive in the sense that, a human reading the endpoint, and the possible payload/parameters, can understand what the API/endpoint will respond to and even sense the data structure of that response.
Also, the new API should be really simple to set up at the code level. Ideally, with no more than 2 configuration parameters a Feature Team should be ready to use an HTTP client to call your API. If Feature Teams have to write some code as “preparation”, before actually start using your API, that will lead to a poor Developer Experience. Keep this in mind when you have to re-develop an API/endpoint.
What Worked Not So Well
One of the main flaws we, as Software Engineers, have at the time doing a job like this (re-implement an API) is to work in an isolated mode. Working yourself in the design of the new API, without sharing your ideas, and answering your own questions, could lead to a situation in which you are implementing something that is not suitable for your users. Again, keep in mind that this can/should be treated as an agile product. This happened to me sometimes, I have no qualms admitting it.
It happens sometimes as well that, when we are in charge of implementing something “new”, even if it’s a re-develop of something, we are tempted to be too much innovative, open our “magic box” and try to do something dramatically new. Sometimes that is indeed possible, but on the majority of occasions, it’s not the case. Instead, you can orient that “innovative spirit” in the direction of the observability, reliability, and sustainability of the new API, which are topics that change and evolve over time, and maybe you can apply here something new/better.
Summary
Communication is key and it’s the main driver that will lead you to build the right API/service/software. Try to think how you, putting yourself in the users’ shoes, would like to use that re-developed API.
Remember also that you have to build something that fits the needs; which brings to my head the sentence:
You have to build the right thing and not necessarily the best thing.
Did you have to reimplement something? How was your experience? what did work well and not so well for you? What was your main struggle? Share it in the comments!
Best,
Marcos