Integrating With Other Teams

At work a lot of our services talked to each other through SQS. Other teams published into our queues and we published into theirs. A lot of events came in every day.

Drawing that looks easy, and it isn’t. You agree on the payload going in and the one coming out, and then you keep going back and forth with the other team, asking and fixing the docs. And when you publish, you don’t know if it worked. The queue takes the message and that’s it. To find out, you read a reply queue, wait for a webhook, or ask someone on the other team to go look for you.

Then there’s how many queues you end up with. On one product we delivered, the messages came through four different queues, and we read from all four, even though everything ended up in the same service of ours. That’s one product. Picture N products talking to the same service, each wanting a queue of its own. You can make it one shared queue instead, but then that queue carries N contracts from N products, and you keep validating every one of them. I even wrote FastSQS to help with that. We already used FastAPI a lot, so it gives SQS handlers the same routing and validation. It helps, but it can’t fix all of it.

I didn’t like any of it. Then we were reading Fundamentals of Data Engineering in the company book club, and the book mentions Bezos’s API mandate, so I went and read it. It says every team exposes its data and what it does through an interface, and talks to the other teams only through those interfaces.

I’ve always liked practices you can repeat the same way every time, and this felt like the right way: each service owns its data and talks to the world through one API. Digging from there I found CQRS.

CQRS splits a service into two paths: commands, which change something (“do this”), and queries, which only read (“how is it?”). A command doesn’t need to answer much more than “got it”, and a query never changes anything. Event sourcing, separate databases and the rest are things people bolt on, and you can skip them.

So the service got an API on the outside and kept CQRS inside. A command comes in as a POST. The API checks the payload, drops it into the service’s own queue and answers 202 with an id. A broken payload gets a 422 on the spot instead of dying in a dead-letter queue after the retries run out. That early answer is what the queue never gave us. Reads are GETs. Nobody else publishes into our queue anymore.