I want to share something a bit different today.

Over the past months, I’ve been spending a lot of time thinking about The Optimist Engineer. What it is, what it’s becoming, and what I want it to be in the future.

When I started writing, the goal was simple:

👉🏼 Share useful ideas and real-world lessons that could help other engineers grow.

And honestly, I didn’t expect it to resonate the way it has.

Some of your replies, messages, and conversations have made me realize that this is not just a newsletter anymore. It’s something people actually find useful in their day-to-day work.

And that’s where this thought started.

Lately, I’ve been wondering what it would look like to take this one step further.

I’ve also shared these thoughts with , a friend of mine. Talking them through with someone I trust has helped me reflect more honestly on what this could become.

Not in terms of writing more.

But in terms of going deeper. Being more intentional. Sharing more of the thinking, the trade-offs, and the real decisions behind the work.

And with that idea, a question naturally appeared:

Would it make sense at some point to create a paid version of The Optimist Engineer?

To be honest, this is not an easy thought for me.

Part of it is practical.

I’ve never set up something like this before, and things like taxes, fiscal setup, and all the “non-writing” parts feel… unfamiliar and a bit intimidating.

And part of it is more personal.

There’s always that question in the back of my mind:

what if I build it… and no one is actually interested in paying for it?

I don’t have a clear answer to that yet.

What I do know is that I care a lot about keeping this space useful, honest, and worth your time.

So for now, I’m just exploring the idea. Thinking about what it could look like, and whether it would make sense at all.

I’ll share more thoughts on this soon.

Thanks for being here, and for reading The Optimist Engineer.

Marcos
The Optimist Engineer

What does a Tech Lead actually do during the week that a Senior Engineer doesn’t?

It’s easy to get lost in meetings, but true technical leadership is exercised through high-leverage technical tasks.

Today, I want to focus on the two “power moves” that will make you stand out: Architecture Reviews and Strategic Tech Debt Management.

1. Architecture Reviews: From Gatekeeper to Facilitator 🏛️

Many engineers think an Architecture Review is about being the “smartest person in the room” who says “yes” or “no” to a design. Wrong.

A Lead uses reviews to:

• Identify cross-team impacts: Does this change in the Monolith affect the new modular services we are building?

• Teach trade-off analysis: Don’t just point out a flaw; ask the team, “If we choose this database now, how hard will it be to migrate in 6 months?”

👉🏼 Leadership here is about Architecture Incrementalism.

You aren’t looking for the “Final Perfect Version”; you are ensuring the team takes a step forward without creating a legacy nightmare tomorrow.

2. Tech Debt: Stop Complaining, Start Managing ⚖️

We all have technical debt. Let’s not complain about it but manage it.

👉🏼 Growing into leadership means moving away from “we should rewrite this” toward “we will improve this incrementally.”

Here is how you lead through debt:

• The Debt Radar: Don’t just fix bugs. Identify which parts of the code are “high-interest debt”, the ones that slow down every single sprint.

• Negotiation: Your job is to translate “clean code” into “business value” for product managers. “If we spend two days refactoring this module, we will deliver the next three features 20% faster.”

Knowledge Pills 💊

If you want to start acting like a Tech Lead today, pick one of these tasks:

• Shadow a Review: Ask a Lead if you can join their next architecture sync. Don’t just watch the tech; watch how they ask questions to guide the team.

• The “Boy Scout” Rule at Scale: Don’t just leave the code cleaner; leave the architecture clearer. Document one undocumented decision this week.

• Categorize your Debt: Next time you see a mess, don’t just fix it. Tag it. Is it “Deliberate Debt” (we rushed for a deadline) or “Outdated Debt” (the system evolved)?

Wrap up ✨

👉🏼 Technical leadership is about reducing friction for your team.

When you master architecture reviews and debt management, you aren’t just “coding”; you are building an environment where everyone else can code better.

I’d love to hear your thoughts!

Which of these two tasks do you find more challenging: defending “Refactoring Time” to your manager, or giving feedback on a peer’s architecture without sounding like a “hater”?

Drop a comment below. Let’s build a better engineering culture together!

Stay optimistic and keep building.

Thanks for your support and feedback, I really appreciate it!

You’re the best! 🖖🏼

𝘐𝘧 𝘺𝘰𝘶 𝘦𝘯𝘫𝘰𝘺𝘦𝘥 𝘵𝘩𝘪𝘴 𝘱𝘰𝘴𝘵, 𝘵𝘩𝘦𝘯 𝘤𝘭𝘪𝘤𝘬 𝘵𝘩𝘦 💜. 𝘐𝘵 𝘩𝘦𝘭𝘱𝘴!

𝘐𝘧 𝘺𝘰𝘶 𝘬𝘯𝘰𝘸 𝘴𝘰𝘮𝘦𝘰𝘯𝘦 𝘦𝘭𝘴𝘦 𝘸𝘪𝘭𝘭 𝘣𝘦𝘯𝘦𝘧𝘪𝘵 𝘧𝘳𝘰𝘮 𝘵𝘩𝘪𝘴, ♻️ 𝘴𝘩𝘢𝘳𝘦 𝘵𝘩𝘪𝘴

💚 On WhatsApp

💙 On LinkedIn

🩵 Bluesky

One of the most common questions I get from Senior Engineers is:

How do I actually prove I’m ready for a Lead role?

Most people think it’s about writing more code or being the fastest at closing JIRA tickets. But here is the truth:

👉🏼 Leadership isn’t about the code you write; it’s about the decisions you enable.

The transition from Senior to Lead requires a fundamental mindset shift. You need to stop thinking in terms of “tasks” and start thinking in terms of “systems” and “impact.”

In today’s email, I share with you 3 pills of knowledge for you to thrive in this ride.

The Mindset Shift: From “How” to “Why” 🚀

When you are a Senior Dev, you focus on the how:

• How do I implement this function?

• How do I fix this bug?

When you step into Tech Leadership, you must pivot to the why and the what:

• The Individual Contributor (IC) builds the bridge.

• The Tech Lead decides where the bridge should go and ensures the whole team knows how to maintain it.

This shift is where most engineers struggle, but there’s a perfect “training ground” to practice this: System Design.

By the way, if you are looking for good System Design learnings, subscribe for free right now to the work done by 👇🏻

The System Design Newsletter

Download my system design playbook on newsletter signup for FREE

By Neo Kim

Leadership in Action: The ETL Example 🛠️

Let’s look at a practical example we’ve discussed in this other email: Designing an ETL (Extract, Transform, Load) architecture.

A Senior Developer might jump straight into the code, picking the trendiest library to move data from point A to point B. But a Tech Lead approaches it differently. They don’t just build an ETL; they make leadership decisions through design:

1. Contract over Code: They define the “Contract” between systems. They think about how this data pipeline affects the Data Science team or the Frontend.

2. Trade-offs as Strategy: They don’t look for the “perfect” tool. They weigh cost vs. scalability. “Do we need real-time streaming, or is a simple batch process enough for our current business stage?”

3. Mentorship through Architecture: They design the system so it’s easy for other developers to contribute without breaking things.

When you document these decisions, explaining the why behind your architecture, you are already performing the role of a Tech Lead. You are providing clarity, reducing risk, and empowering the team.

Knowledge Pills for Your Growth 💊

If you want to accelerate your path to leadership this month, try these three things:

• Own the “Non-Happy” Path: Don’t just design for when things work. Lead by defining how the system fails (observability, retries, alerts).

• Write for Others: Start writing RFCs (Request for Comments). If you can convince your peers that your architecture is the right one, you are leading.

• Stop Being the Hero: If you are the only one who can fix a specific system, you aren’t leading; you are a bottleneck. True leaders build systems that don’t need them to survive.

Wrap up! ✨

👉🏼 Technical leadership is a muscle.

You build it every time you take a step back from the IDE to look at the bigger picture. Whether you are refactoring a monolith or designing a new data pipeline, ask yourself: “How does this decision make my team better?”

Now, I want to hear from you!

What is the biggest challenge you’ve faced when trying to move from “doing” to “leading”? Is it the lack of time, or perhaps not knowing how to start the conversation with your manager?

Leave a comment below, I read and reply to every single one!

Stay optimistic and keep building.

Thanks for your support and feedback, I really appreciate it!

You’re the best! 🖖🏼

𝘐𝘧 𝘺𝘰𝘶 𝘦𝘯𝘫𝘰𝘺𝘦𝘥 𝘵𝘩𝘪𝘴 𝘱𝘰𝘴𝘵, 𝘵𝘩𝘦𝘯 𝘤𝘭𝘪𝘤𝘬 𝘵𝘩𝘦 💜. 𝘐𝘵 𝘩𝘦𝘭𝘱𝘴!

𝘐𝘧 𝘺𝘰𝘶 𝘬𝘯𝘰𝘸 𝘴𝘰𝘮𝘦𝘰𝘯𝘦 𝘦𝘭𝘴𝘦 𝘸𝘪𝘭𝘭 𝘣𝘦𝘯𝘦𝘧𝘪𝘵 𝘧𝘳𝘰𝘮 𝘵𝘩𝘪𝘴, ♻️ 𝘴𝘩𝘢𝘳𝘦 𝘵𝘩𝘪𝘴

💚 On WhatsApp

💙 On LinkedIn

🩵 Bluesky

Follow-up readings 🔖

☝🏼A Schema Registry acts as the central source of truth for data schemas, helping teams manage schema evolution, enforce compatibility rules, and facilitate seamless communication between distributed services.

In today’s issue, I will address:

• What a Schema Registry is.

• Benefits of Using a Schema Registry.

• Use Cases Where Schema Registries Are Essential.

• When Schema Registries Are Not Worth It

• Major Schema Registry Solutions Comparison.

Let’s start.

🧐 What a Schema Registry is

☝🏼A Schema Registry is a centralized repository for storing and retrieving data schemas.

👉🏼 It is designed to handle version control, ensuring that any evolution in the data structures is tracked and managed.

By maintaining a clear contract between producers and consumers, the registry enables reliable serialization and deserialization of data formats such as Avro, Protobuf, and JSON Schema, among others.

In a microservices architecture, during the sync and async communication, you have to ensure that the data you will write and read is compatible with your services.

👉🏼 Basically, the producer of the data could use a version of the data, version 1.2.0, and the consumers could use the same or an older version, like 1.1.0, without breaking either side. That’s the magic that a Schema Registry does for you.

☝🏼 Thinking about your code. The different Schema Registries offer SDKs for many languages, so you can serialize and deserialize data using the API offered by the SDK.

For example, when using an asynchronous API through Apache Kafka, you will have to configure your consumer with the deserializer class offered by the SDK from the Schema Registry you have chosen.

I already anticipated one of the major benefits of using a Schema Registry, but let’s explore more.

💡 Benefits of Using a Schema Registry

I could enumerate some, like:

• Centralized Management: It provides a single source of truth for all data contracts, simplifying maintenance across microservices. And having a single source of truth in a distributed system means a lot!

• Schema Evolution: The registry supports versioning and controlled evolution, which allows for backward, forward, or full compatibility as business requirements change.

• Data Quality and Consistency: Enforcing compatibility rules helps prevent errors that might occur due to mismatches in data structure between services.

• Reduced risk of breaking change: With clear contracts defined, teams can build services independently while ensuring they communicate correctly using standardized data formats.

• Enhanced Monitoring and Governance: It provides audit trails and version histories, enabling teams to track changes and troubleshoot issues more efficiently.

How does it sound to you?

But still, is a Schema Registry something you really need? Let’s explore the use cases.

🎯 Use Cases Where Schema Registries Are Essential

1. Schema evolution without breaking producers or consumers

In event-driven systems, data models evolve over time. Fields are added, renamed, deprecated, or reorganized. In a distributed architecture where producers and consumers evolve independently, these changes can introduce incompatibilities if they are not managed properly.

👉🏼 A schema registry allows schemas to be versioned and compatibility rules to be enforced so that changes can be introduced without breaking existing services.

Without a registry:

• Each team manages its own data model in code.

• Schema changes can break existing consumers, and problems are often discovered only at runtime.

• Teams need to coordinate deployments to avoid incompatibilities.

With Schema Registry:

• Schemas are stored in a centralized and versioned location.

• Compatibility rules, such as backward or forward compatibility, can be enforced.

• Incompatible changes are detected before the producer can publish messages.

2. Explicit contracts between teams

In microservices architectures or streaming systems, events act as interfaces between different services or teams. For these interactions to be reliable, the structure of the messages must be clearly defined and accessible to all participants.

👉🏼 A schema registry allows the data model to become a formal, versioned contract that producers and consumers can rely on.

Without a registry:

• The schema usually lives in code or informal documentation.

• Consumers need to infer the message structure themselves.

• It is common to see mismatches between documentation and the actual messages being produced.

With Schema Registry:

• The schema becomes a central source of truth.

• Producers and consumers rely on the same contract definition.

• Versioning allows the contract to evolve in a controlled way.

3. Automatic validation of messages

A common issue in data platforms or event-driven systems is the presence of messages with incorrect structures or missing fields. These problems can cause failures in consumers or even break entire data pipelines.

👉🏼 A schema registry enables automatic validation to ensure that messages comply with the registered schema before they are published or processed.

Without a registry:

• Producers can send messages with inconsistent structures.

• Consumers must implement manual validation logic.

• Errors are often detected late, after the message has already entered the system.

With Schema Registry:

• Messages are validated against the registered schema.

• Producers cannot publish invalid data.

• Data quality and consistency across the system are improved.

At this point, you may think:

Damn! I need to put a Schema Registry right now!

Wait, and let’s see when it does not make sense.

❌ When Schema Registries Are Not Worth It

1. Only one producer and one consumer that are tightly coupled

When the producer and the consumer belong to the same system, the same repository, or the same team, schema evolution can be managed directly in the code and through coordinated deployments.

👉🏼 In this situation, introducing a registry adds an additional operational dependency without solving any real decoupling problem.

Without a registry:

• The schema lives in the same repository as the producer and the consumer.

• Changes to the data model are deployed in a coordinated way.

• Integration tests validate that both sides remain compatible.

With Schema Registry:

• An additional service must be operated and maintained.

• Serialization configuration and schema management introduce extra complexity.

• There is little practical benefit because the system is already coordinated.

2. The system is still in a prototype or exploration phase

During the early stages of a project, data models tend to change quickly while ideas and architectural decisions are being validated. At that stage, the priority is iteration speed rather than strict governance of data contracts.

👉🏼 Introducing a registry too early adds friction to development and slows down the evolution of the system.

Without a registry:

• Schemas can evolve rapidly together with the code.

• Changes can be made without formal compatibility processes.

• The team can iterate and refactor freely.

With Schema Registry:

• Schema changes require version registration and compatibility validation.

• Additional steps are added to the development workflow.

• Development speed decreases in a phase where experimentation is important.

3. Messages do not represent structured data

A Schema Registry is designed to manage and validate structured data with clearly defined fields. If messages contain opaque payloads such as binary blobs, compressed files, or encrypted content, there is no meaningful structure to register or validate.

👉🏼 In this situation, introducing a registry does not provide any technical benefit.

Without a registry:

• Messages are treated as opaque payloads.

• Consumers handle the interpretation logic.

• Only basic metadata may be validated if needed.

With Schema Registry:

• There is no real structure to register or validate.

• Compatibility rules cannot provide meaningful guarantees.

• Infrastructure is added without delivering functional value.

Before finishing for today, let me give you a comparison table for the Schema Registries I’ve analyzed recently.

📊 Major Schema Registry Solutions Comparison

This table provides an overview of key features and capabilities of each Schema Registry solution, making it easier to compare and select the best fit for specific needs.

✨ Takeaways

Let’s wrap up for today!

A Schema Registry acts as a central source of truth for data contracts in distributed systems. In microservices or event-driven architectures, where multiple producers and consumers evolve independently, maintaining a centralized repository of schemas allows all services to share a clear and versioned definition of the message structure.

One of its most important benefits is enabling schema evolution without breaking producers or consumers. Through versioning and compatibility rules such as backward or forward compatibility, teams can introduce changes to data models while ensuring that existing services continue to function correctly.

In addition, a Schema Registry significantly improves data quality and consistency. By automatically validating messages against the registered schema before they are published or consumed, it prevents malformed or incomplete data from entering the system and causing failures that are often difficult to detect at runtime.

However, it does not always provide value. When there is only one tightly coupled producer and consumer, or when a system is still in a prototype phase where data models change frequently, introducing a Schema Registry can add operational complexity without solving a real problem.

That’s it for today!

If you are building event-driven systems or microservices, sooner or later you will face the challenge of keeping data contracts consistent across services.

A Schema Registry can solve that problem, but as we saw, it only makes sense in the right context.

Have you used a Schema Registry in your architecture? What worked well, and what challenges did you face?

Leave a comment below, I read and reply to every single one!

Stay optimistic and keep building,

Thanks for your support and feedback, I really appreciate it!

You’re the best! 🖖🏼

𝘐𝘧 𝘺𝘰𝘶 𝘦𝘯𝘫𝘰𝘺𝘦𝘥 𝘵𝘩𝘪𝘴 𝘱𝘰𝘴𝘵, 𝘵𝘩𝘦𝘯 𝘤𝘭𝘪𝘤𝘬 𝘵𝘩𝘦 💜. 𝘐𝘵 𝘩𝘦𝘭𝘱𝘴!

𝘐𝘧 𝘺𝘰𝘶 𝘬𝘯𝘰𝘸 𝘴𝘰𝘮𝘦𝘰𝘯𝘦 𝘦𝘭𝘴𝘦 𝘸𝘪𝘭𝘭 𝘣𝘦𝘯𝘦𝘧𝘪𝘵 𝘧𝘳𝘰𝘮 𝘵𝘩𝘪𝘴, ♻️ 𝘴𝘩𝘢𝘳𝘦 𝘵𝘩𝘪𝘴 𝘱𝘰𝘴𝘵.

Share

🔖 Related readings

System Design Classroom

Event-Driven Architecture (EDA) is exciting until you deal with event versioning.

Or you can use an Adapter but with a twist. Yes, like in 90% of the problems in Software Development, there is a Pattern that will help you find a smart solution…

Read more

2 years ago · 62 likes · 9 comments · Raul Junco

Javarevisited Newsletter

24 Microservices Patterns for Experienced Developers

Hello guys, if you are working in Microservice projects then it make sense to learn different Microservices design pattern and principles, not just to make your architecture more robust but also to understand existing projects…

Read more

2 years ago · 36 likes · javinpaul and Soma

The Software Frontier

System Design Simplified: A Beginner's Guide to Everything You Need to Know (Part 9.1)

Hello everyone, Lorenzo here! (As always, running a bit behind schedule—but I promise, this one will be worth the wait…

Read more

a year ago · 4 likes · Lorenzo Bradanini

Today’s issue features a special guest . Here is Jorge’s introduction.

Hi! I’m Jorge Castaño, a Tech Lead helping software engineers grow technically and step into leadership with confidence. I share lessons from leading engineering teams to help you avoid the common mistakes of your first leadership roles.

I want to recommend Jorge’s newsletter for all those Spanish-speaking Software Engineers seeking growth and Engineering Leaders looking for learning resources.

Vivir del Software

Manual de Supervivencia Developer | Consejos de carrera para destacar en el mundo del Software | ¿PEC? Te suscribes | Gratis incluso si eres DevOps

Por Jorge Castaño

Without taking any more time, I'll hand the word 🎤 over to Jorge.

Congratulations, you made it. You finally have your senior developer position. It’s been a tough road.

But if you thought this was the end, I have bad news: you can’t retire yet. With any luck, you still have 30 years of daily meetings ahead.

Reaching the senior floor is not the last stage of your career, which is why today I want to talk to you about the 3 possible paths that open up in front of you.

The mindset shift

Sorry to keep bringing bad news, but the first thing you need to take the next step is to change the way you think. To reach senior, you learned in depth how programming works, which algorithms are most efficient, and which designs best fit each situation. You focused on the technical side.

Many of us have tried to progress using the same tools: more technology, more specialization, more technical depth. But now that’s no longer enough. #sad

The next level is not about knowing more. It’s about learning that your impact no longer depends solely on what you produce with your own fingers. The next step means making decisions, guiding your team, and multiplying the work of others.

It’s an uncomfortable change that requires letting go of everything that defines us: being good programmers. The challenge is no longer measured by the tickets you close; it’s more diffuse. But it’s also bigger, since every decision you make has a greater impact.

The three paths

Ok, my next role involves a greater impact on my team, but how?

Well, there are 3 possible directions. None is better than the others; you’ll have to figure out which one you want to walk.

The technical track (Individual contributor)

You’re still an engineer, but your influence is no longer limited to your own code. At the early levels — Tech Lead — you still have a concrete team, clear deliverables, and you keep writing code. Enjoy it.

As you move toward Staff / Principal Engineer, the game changes: you define technical direction, reduce uncertainty in problems others don’t even know how to start, and influence without hierarchical authority.

Climbing the technical track may mean you code less and less, but you’ll be very close to the solutions being implemented. And that feels almost the same as building them yourself. Almost.

Engineering Management

This is the path that scares and attracts people the most at the same time. You leave code as your main tool — not necessarily forever, but as a priority — and people become your job. Your success is no longer measured by what you produce, but by what your team produces.

This role requires completely different skills: listening, communication, and difficult conversations. It can be daunting because we go from being programmers to being managers. But watching your team grow and being able to help them directly is also a rewarding job.

This track change requires training — don’t think you’ll suddenly be able to handle such a different role without help. The problem is when your company doesn’t support you through it, which is most of the time. #sadder

Product Manager

Possibly the least obvious path for a developer, but an increasingly common leap in the AI era. If you find yourself asking why things get built, this is your move.

Moving into product means starting to operate in the space of what and why before how. The intersection of what the user needs, what the business wants, and what the team can build. You’ll need to prioritize, say no, and explain why.

Developers who move into product have an advantage that no non-technical PMs can replicate: they understand the real cost of decisions. But be careful: it’s dangerous to be stuck on the how when your job now is to define the what.

How to choose your path?

There’s no right answer, but there are right questions. Take some time with this, because thinking is hard work.

The most common trap is choosing by elimination or by inertia: “I don’t see myself managing people, so technical track” or “I was offered a manager role and took it.” That’s not choosing, that’s going with the flow. #saddest

Three questions to help clarify:

What gives you the most energy today?

Not what you’re good at, but what’s your fuel? Solving a complex technical problem, unblocking someone on your team, or understanding why a product isn’t working.

What are you willing to leave behind?

Every path involves a real trade-off. The technical track means letting go of direct control. Management means letting go of the code. Product means letting go of technical certainty. What loss can you live with?

Where do you want to be in three years?

Picture your day-to-day. What your meetings look like, what problems you’re solving, who you’re talking to. Choose carefully, but don’t delay the decision. Worst case, you can always go back to the technical track.

That’s it! Think about those three questions calmly before you decide your next step.

Here’s a prompt to talk it through with your favorite AI:

I’m a Senior developer thinking about my next career step. I want to explore three possible paths: the technical track (Tech Lead, Staff or Principal Engineer), Engineering Management or Product Manager.

Ask me questions one at a time to help me figure out which path fits me best. Don’t give me the answer upfront. First I want to explore what gives me energy, what I’m willing to leave behind, and where I see myself in three years.

At the end, give me your honest take on which path seems most aligned with what I’ve shared, and why.

Thanks, Marcos, for allowing me to speak to your community.

If you want to talk more about this or anything else, feel free to reach out on Linkedin.

🫂 Thanks Jorge!

Marcos back!

I want to send a deep Thank You to Jorge Castaño for sharing his experience with all of us. To learn more from Gábor, take a look at his newsletter 👇🏻

Vivir del Software

Manual de Supervivencia Developer | Consejos de carrera para destacar en el mundo del Software | ¿PEC? Te suscribes | Gratis incluso si eres DevOps

Por Jorge Castaño

Thanks for your support of my work here, really appreciate it!

If you enjoyed this article, then click the 💜. It helps!
If you know someone else will benefit from this, ♻️ share this post.

How much easier would things be if those decisions were written down, clear, and useful for whoever joins later?

In today’s email, you will learn how to stop your technical decisions from disappearing into Slack threads and endless PRs.

The real problem: technical decisions that disappear

Most problems in software systems don’t come from syntax errors.

They come from forgotten reasoning:

• A decision is made on a call.

• Someone summarizes it in Slack.

• Three months later, nobody remembers why that path was chosen.

Then someone proposes a “better” solution.

And the debate starts again.

Rework. Friction. Endless PR comments.

👉🏼 RFCs are the antidote to collective technical amnesia.

They create memory.

What an RFC actually is in your day-to-day work

Forget Internet standards.

In your team, an RFC is simply a short, structured document that explains:

• Why this change is needed

• What is being proposed

• What trade-offs exist

• What could go wrong

That’s it.

👉🏼 It’s about raising the quality of technical thinking before code is written.

And that alone changes the level of conversation.

Why writing RFCs builds authority and helps you persuade

☝🏼 Let’s be honest: Authority in engineering does not come from seniority alone. It comes from clarity and transparency.

Writing RFCs builds both in public.

👉🏼 You demonstrate structured thinking and invite participation.

Anyone can have an opinion in a meeting.

Very few people can:

• Clearly define the real problem.

• Separate assumptions from facts.

• Lay out alternatives.

• Explain trade-offs.

• Anticipate risks before someone else mentions them.

An RFC forces you to do exactly that.

And there’s something even more important: Many technically sound proposals fail because they arrive fully baked. When developers are not involved early, they don’t feel ownership. And when people don’t feel ownership, they resist.

👉🏼 Sharing an RFC as a draft changes the dynamic. It becomes a conversation instead of a decision announcement.

When you should write an RFC

☝🏼 Not every change needs one.

But write an RFC when the decision:

• Changes the architecture.

• Introduces a new or expensive dependency.

• Modifies a public API.

• Affects multiple teams.

• Has long-term impact beyond a single ticket.

👉🏼 Just to give you a real example: The last time I wrote an RFC was to propose how to implement the AGENTS.md framework in the repos of the Engineering Organization.

AI Native Compass

From AGENTS.md to sub-agents: structure and best practices

If you work in Platform Engineering or on a development team, you need to understand one thing: subagents ≠ MCPs…

Read more

a month ago · 4 likes · Marcos F. Lobo 🗻🧭

Now, an important part.

How to handle feedback without turning it into an opinion war

☝🏼 RFCs don’t eliminate disagreement but they structure it.

To deal with this situation, a few simple rules help:

• Separate facts from preferences.

• Set a time window for feedback.

• Escalate only when needed.

• Document the final decision clearly.

And one important principle: No silent vetoes.

👉🏼 If someone blocks the proposal, ask them to write a counter-proposal using the same structure.

Where to write RFCs

I like to use Git to publish my RFCs because I find it a good tool for:

1. Reach my audience, a.k.a. developers.

2. A Pull Request permits comments and contributions.

So, if you want to start writing RFCs, my suggestion for you is:

• Create a new repository.

• Where the README.md file explains what the repo is and how to create RFCs.

• Also, you will have an RFC template that the people can (must) use to contribute their RFCs.

That’s it!

So far, you have some clarity about the RFCs. Now, let me give you the template I’m using nowadays. Click the button below to receive it for free 👇🏻

🎁 Get RFC template for free

✨ Summary

RFCs are not ceremonies. They are not corporate theater. And they are definitely not paperwork for the sake of process.

They are thinking tools.

They force you to slow down just enough to clarify the problem before jumping to the solution. They turn opinions into arguments. They transform isolated ideas into shared decisions. And over time, they build something most engineering teams lack: technical memory.

Writing RFCs will not make your code compile faster. But they will make your decisions stronger. They reduce friction. They create alignment. They make discussions sharper and shorter. And they make your reasoning visible.

And, look:

• If you want to grow as an engineer, keep writing code.

• If you want to grow in influence and authority, start writing decisions down.

Hope you enjoy today’s email. I know I made some bold arguments, and I hope you consider them when you plan your next proposal.

Thanks for your support and feedback, I really appreciate it!

You’re the best! 🖖🏼

𝘐𝘧 𝘺𝘰𝘶 𝘦𝘯𝘫𝘰𝘺𝘦𝘥 𝘵𝘩𝘪𝘴 𝘱𝘰𝘴𝘵, 𝘵𝘩𝘦𝘯 𝘤𝘭𝘪𝘤𝘬 𝘵𝘩𝘦 💜. 𝘐𝘵 𝘩𝘦𝘭𝘱𝘴!

𝘐𝘧 𝘺𝘰𝘶 𝘬𝘯𝘰𝘸 𝘴𝘰𝘮𝘦𝘰𝘯𝘦 𝘦𝘭𝘴𝘦 𝘸𝘪𝘭𝘭 𝘣𝘦𝘯𝘦𝘧𝘪𝘵 𝘧𝘳𝘰𝘮 𝘵𝘩𝘪𝘴, ♻️ 𝘴𝘩𝘢𝘳𝘦 𝘵𝘩𝘪𝘴

💚 On WhatsApp

💙 On LinkedIn

🩵 Bluesky

In detail, today you will learn:

• The goal of an MCP.

• Components and responsibilities.

• Delivery options.

Before jumping into it, I just wanted to let you know that I am writing another newsletter on how to move from Cloud Native foundations to AI‑powered platforms. There, you will find more details about how to work with AI in the enterprise world. You can subscribe here 👇🏻

AI Native Compass

From Cloud Native to AI Native

By Marcos F. Lobo 🗻🧭

Let’s continue now.

🎯 Goal of an MCP

The MCP defines a standardized way for environments and tools (IDEs, CLIs, agents) to request context, such as files, metadata, secrets, telemetry points, policies, etc., from a Context Provider (an agent or service).

This context is used to feed LLMs/Models that perform assistance tasks (autocompletion, test generation, security analysis, etc.) without exposing the entire repository or unnecessary secrets.

🧱 Components and Responsibilities

• IDE Plugin / MCP Client

  • User interface, event hooks (cursor, selection), UI for context approval.

  • Communicates with the Agent via local socket (Unix socket/named pipe) or localhost HTTP/gRPC.

• MCP Agent

  • Orchestrates context gathering (files, git, build graph, dep).

  • Applies policies, redaction, and minification.

  • Caches results and maintains provenance history.

  • Interacts with secret/telemetry providers.

  • Calls models (local or remote) or forwards the response.

• Model Provider / LLM

  • Can be a cloud endpoint (OpenAI, Anthropic, Azure) or a local model (Llama, Mistral).

  • The Agent must be able to redirect prompts and enforce size limits.

🚀 Delivery options for MCP servers

At the time of delivering an MCP server, the deployment decision is not just technical. The aspects you have to consider are:

• Latency.

• Security.

• Developer experience.

• Operating costs.

• Governance.

The main options are discussed in more detail below.

Native Binary

An executable that the developer installs locally and that runs as a system process.

Advantages:

• Maximum performance and low resource consumption.

• Direct access to the filesystem, Unix sockets, and OS resources.

• Immediate startup (no cold start).

• Better integration with IDEs and local tools.

Disadvantages:

• Need for cross-platform builds (Mac, Windows, Linux).

• More complex version and upgrade management.

• Distribution and binary signing require a robust pipeline.

When to choose it:

• Teams with a strong focus on Developer Experience.

• Need for minimal latency.

• Environments where local access to the repository is critical.

Docker Container

The Agent runs as a Docker container locally.

Advantages:

• Full consistency across operating systems.

• Simplifies Agent dependencies and libraries.

• Easy versioning (image tags).

• Compatible with enterprise environments where Docker is already standard.

Disadvantages:

• Additional containerization overhead.

• Volume permission management (can be tricky).

• Slightly more complex UX (requires an active Docker container).

I personally don’t find these advantages really critical, but I wanted to add them from an objective point of view.

When to choose it:

• If all developers already use Docker.

• If you want to minimize “works on my machine” issues.

Cloud Service

The Agent runs as a shared service (internal or external SaaS). Clients connect remotely.

Advantages:

• Centralized policies and drafting.

• Unified observability.

• Easy to implement RBAC and auditing.

• No complex local installation required.

Disadvantages:

• Higher latency (network round-trip times).

• Privacy risks if the context contains sensitive code.

• Greater security complexity (TLS, IAM, isolation).

When to choose it:

• Large organizations with compliance needs.

• Globally distributed teams.

• When strict traceability is required.

✨ Summary

In today’s email, you get an overview of the MCP server, its architecture, and the best approaches to deliver it to your customers.

Architecturally, a typical MCP-based application has three clear building blocks:

• The IDE plugin/client (UI hooks and event triggers).

• The MCP Agent (context gathering, policy enforcement, redaction, caching, and orchestration).

• The Model Provider (local or cloud-based models).

Each component has distinct but complementary responsibilities.

Choose wisely the way to deliver your MCP. Nowadays, delivering MCP servers via Docker is my preferred choice because it offers a good balance between performance and version management.

I would love to hear about your experiences implementing MCP servers. Reply to this email or put a comment in Substack.

Thanks for your support and feedback, I really appreciate it!

You’re the best! 🖖🏼

𝘐𝘧 𝘺𝘰𝘶 𝘦𝘯𝘫𝘰𝘺𝘦𝘥 𝘵𝘩𝘪𝘴 𝘱𝘰𝘴𝘵, 𝘵𝘩𝘦𝘯 𝘤𝘭𝘪𝘤𝘬 𝘵𝘩𝘦 💜. 𝘐𝘵 𝘩𝘦𝘭𝘱𝘴!

𝘐𝘧 𝘺𝘰𝘶 𝘬𝘯𝘰𝘸 𝘴𝘰𝘮𝘦𝘰𝘯𝘦 𝘦𝘭𝘴𝘦 𝘸𝘪𝘭𝘭 𝘣𝘦𝘯𝘦𝘧𝘪𝘵 𝘧𝘳𝘰𝘮 𝘵𝘩𝘪𝘴, ♻️ 𝘴𝘩𝘢𝘳𝘦 𝘵𝘩𝘪𝘴

💚 On WhatsApp

💙 On LinkedIn

🩵 Bluesky

Your CEO, CTO, and even your CPO are telling you:

Use the AI as much as possible!

Or this other (my favourite)

So will be able to deliver faster than before!

I will not jump into this debate (today), but into another: the development of the in-house services that implement the Model Context Protocol, to be used by AI Agents in the IDE of the developers.

I’ve seen many folks in Platform Engineering teams start writing MCP-based services meant to be used by the dev teams, and I wondered:

Hey, does this actually solve a problem for the dev teams?

In today’s issue, I want to share with you my vision about when an in-house MCP-based service makes sense.

Before jumping into it, I just wanted to let you know that I am writing another newsletter on how to move from Cloud Native foundations to AI‑powered platforms. You can subscribe here 👇🏻

AI Native Compass

From Cloud Native to AI Native

By Marcos F. Lobo 🗻🧭

Now, let’s get back to our business.

First of all, is this an automation?

Platform Engineering teams are meant to provide frameworks, CI/CD workflows, libraries, processes, and best practices to the Software Engineering teams so they can ship value to customers fast.

Platform Engineering teams have a great responsibility: choose the right solution so that Software Engineering teams will embrace and onboard the solution smoothly.

Since Software Engineering teams work with IDEs, and thanks to the existing integration between IDEs (VS Code, IntelliJ) and AI Agents, or existing IDEs like Cursor or Windsurf, one of the steps that could make sense is:

Let’s create an MCP-based service so developers can use it from their IDEs.

This is a great approach.

However, I’ve seen many Platform Engineering teams failing on this. Why?

👉🏼 They implemented an MCP-based solution when the feature teams needed just an automation.

Do not mix AI with automation; they are different things.

So, next time you think the solution is an MCP-based service, ask yourself the following:

Can the work be done by an automation?

Is an in-house MCP-based service needed?

Look at this huuuge list of MCP servers available, many of them are official from their providers.

So, another question to ask yourself?

Will my MCP-based service solve a problem that isn’t already solved by existing MCPs?

I’ve intentionally made that question plural.

Connecting multiple MCP-based services to tools like Cursor or Windsurf already allows an AI Agent to solve problems that span multiple sources.

In fact, AI Agents can already orchestrate across the configured MCPs to resolve those problems.

So the real question is:

Why add another MCP that acts as a wrapper?

Maybe yes!, maybe no!

I’ve raised a couple of reflections that now your brain is trying to plug and play with the rest of your ideas. In order to help you, I will now list when it makes sense to create an MCP and when it does not.

When does an in-house MCP-based service not make sense?

• Small organizations (1–2 teams) where the operational overhead of the MCP outweighs the benefits.

• Top priority: developer speed and minimal friction (you prefer direct adapters in the IDE).

• High team autonomy: with very different rules and an aversion to centralization, the MCP will become a source of political friction.

• When the cost of maintaining adapters/compatibility outweighs the benefits, e.g., too many vendors with unstable APIs.

When does an in-house MCP-based service make sense?

• You have >=10–20 teams with different policies and want global consistency (rulesets, RBAC, auditing).

• You need correlation/dedupe between findings from multiple scanners (SAST, SCA, linters, Sonar, etc.) to prevent one bug from appearing as three.

• Compliance/audit requirements: logging who automated what, blocking pull requests based on corporate policy, and evidence retention.

• You’ll want to enforce cross-functional policies, like legal or privacy rules affecting all repositories, and enforce controlled downgrades when providers fail.

Let’s wrap up for today.

✨ Summary

Keep this in mind:

👉🏼 Just because you can implement an MCP-based service to solve a problem doesn’t mean you should.

From there:

• Make an assessment of the problem you need to solve.

• Is it something you can solve with automation?

• Be sure you have the right expertise in the team to implement these kinds of tools.

Thanks for your support and feedback, I really appreciate it!

You’re the best! 🖖🏼

𝘐𝘧 𝘺𝘰𝘶 𝘦𝘯𝘫𝘰𝘺𝘦𝘥 𝘵𝘩𝘪𝘴 𝘱𝘰𝘴𝘵, 𝘵𝘩𝘦𝘯 𝘤𝘭𝘪𝘤𝘬 𝘵𝘩𝘦 💜. 𝘐𝘵 𝘩𝘦𝘭𝘱𝘴!

𝘐𝘧 𝘺𝘰𝘶 𝘬𝘯𝘰𝘸 𝘴𝘰𝘮𝘦𝘰𝘯𝘦 𝘦𝘭𝘴𝘦 𝘸𝘪𝘭𝘭 𝘣𝘦𝘯𝘦𝘧𝘪𝘵 𝘧𝘳𝘰𝘮 𝘵𝘩𝘪𝘴, ♻️ 𝘴𝘩𝘢𝘳𝘦 𝘵𝘩𝘪𝘴

💚 On WhatsApp

💙 On LinkedIn

🩵 Bluesky

Money is important, more and more every day, and when you are in the cloud, holding the money is a real challenge 💸.

That’s why, in today’s issue, I will give you an overview of OpenCost, a CNCF project.

But first things first, what OpenCost is?

It’s an open-source, vendor-neutral project that brings real-time cost visibility and allocation to Kubernetes and cloud infrastructure so engineering teams and platform teams can make data-driven decisions about efficiency and showback/chargeback.

Nice definition, right? For me, that kind of definition are okeysh, but I really need to understand what it does.

🤔 What OpenCost actually does

• Measures & allocates cloud and Kubernetes costs in near real-time (CPU, memory, storage, network, GPU, etc.).

• Maps cloud billing to Kubernetes concepts, so you can answer “how much did team X spend last week?” or “which deployment is causing the spike?”

• Supports multi-cloud and on-prem via integrations with AWS/Azure/GCP billing APIs and customizable pricing models.

Another thing I see super useful for real life is that OpenCost can give you the money you burn by namespace. Why is it important for me? Because, usually, the resources from the different dev teams are deployed in separate namespaces, hence, each dev team can see how much their stuff costs.

From there, the next step is obvious. Having namespace costs allows you to:

• Allocate expenses directly to teams or products (if each team uses its own namespace).

• Quickly identify which namespace is consuming the most resources or money.

• Integrate with showback/chargeback processes within your platform.

Last but not least, I think it’s important to mention that OpenCost is Kubernetes-native: you typically deploy it into your cluster(s) (Helm charts available), and it relies on existing metrics sources (Prometheus) and cloud billing data for price lookups.

🎯 Problems it solves

When you start deploying your SaaS product into a Kubernetes clusters you face many challenges. Some of the ones related to costs are:

• Opaque Kubernetes spend. Kubernetes abstracts resources, but that abstraction hides who/what is actually driving costs. OpenCost restores accountability.

• Ownership breaks down in shared Kubernetes platforms. Kubernetes clusters are shared by design. Cloud providers bill at the account or subscription level, but teams think in terms of services and products. Without a clear ownership model, questions like “which team is driving this cost?” are hard to answer. Manual mappings and spreadsheets 😱 appear as workarounds.

• Kubernetes hides the true cost of “idle”. Over-provisioning, shared nodes, and autoscaling make it hard to understand idle capacity and inefficiencies. OpenCost surfaces these hidden costs, allowing platform teams to reason about efficiency in concrete, measurable terms.

🧐 State of the project

OpenCost is a CNCF project that progressed from Sandbox into Incubating. It was accepted into CNCF in mid-2022 and promoted to Incubating in October 2024, reflecting growing community adoption and governance maturity.

The project is actively maintained (GitHub repo, releases, and community contributors) and has continued to publish roadmaps and year-in-review notes describing frequent releases and expanding integrations.

Recent public updates indicate priorities around refining cost models, improving supply-chain security for cost data, and adding features to better track emerging workloads (for example: AI/ML usage costs).

✨ Summary

Many companies include the cost assessment before implementing the solution. Others (lucky you) just jump into implementation, and then the problem of understanding the cost comes.

In any case, whether you made an estimation, or you just don’t know, nowadays (always?), you must have visibility on the costs of the applications and resources deployed in your Kubernetes, and OpenCost is one of the options available.

That’s all for today!

I would love to hear about your experiences using OpenCost or any other tool for measuring the costs of cloud applications. Drop me a comment or reply to this email; I will read it and come back to you!

Thanks for your support and feedback, I really appreciate it!

You’re the best! 🖖🏼

𝘐𝘧 𝘺𝘰𝘶 𝘦𝘯𝘫𝘰𝘺𝘦𝘥 𝘵𝘩𝘪𝘴 𝘱𝘰𝘴𝘵, 𝘵𝘩𝘦𝘯 𝘤𝘭𝘪𝘤𝘬 𝘵𝘩𝘦 💜. 𝘐𝘵 𝘩𝘦𝘭𝘱𝘴!

𝘐𝘧 𝘺𝘰𝘶 𝘬𝘯𝘰𝘸 𝘴𝘰𝘮𝘦𝘰𝘯𝘦 𝘦𝘭𝘴𝘦 𝘸𝘪𝘭𝘭 𝘣𝘦𝘯𝘦𝘧𝘪𝘵 𝘧𝘳𝘰𝘮 𝘵𝘩𝘪𝘴, ♻️ 𝘴𝘩𝘢𝘳𝘦 𝘵𝘩𝘪𝘴

💚 On WhatsApp

💙 On LinkedIn

🩵 Bluesky

👉🏼 You need to be seen doing the work of the role you want.

Over the last few months, I’ve written several articles about Staff Engineer, technical leadership, ownership, and breaking out of silos.

I’ve now condensed all that practical advice into a one-page playbook you can actually use.

☝🏼 It’s a set of behaviors, habits, and templates you can start applying this week.

What’s inside the playbook?

• The core mindset behind Staff roles: scope and consistency.

• 8 practical habits Staff Engineers repeat over time (delivery, cross-team impact, metrics, mentoring).

• Ready-to-use templates: weekly stakeholder update, experiment write-up, meeting ritual.

• What to bring to your 1:1s: a simple evidence-based dossier for promotion conversations.

• A weekly checklist to stay focused and avoid “invisible work”.

Why I built it

Because Staff Engineer is not a title you wait for: It’s a pattern of visible behavior you build over time.

And most engineers don’t fail because they lack skill; they fail because their impact is scattered, siloed, or invisible.

👉 Download the playbook

You can download the one-page Staff Engineer Playbook here:

📦 Download the playbook for free

Let’s make it practical

Once you’ve read it, reply to this email with:

“#X → my hardest habit”

(where X is the habit you struggle with the most)

I’ll reply with 3 concrete actions you can take next week to start building Staff-level evidence.

Stop waiting to be “ready”.

Start behaving like the role you want.

Hope you enjoy this playbook!

Be safe,
Marcos

Thanks for your support and feedback, I really appreciate it!

You’re the best! 🖖🏼

𝘐𝘧 𝘺𝘰𝘶 𝘦𝘯𝘫𝘰𝘺𝘦𝘥 𝘵𝘩𝘪𝘴 𝘱𝘰𝘴𝘵, 𝘵𝘩𝘦𝘯 𝘤𝘭𝘪𝘤𝘬 𝘵𝘩𝘦 💜. 𝘐𝘵 𝘩𝘦𝘭𝘱𝘴!

𝘐𝘧 𝘺𝘰𝘶 𝘬𝘯𝘰𝘸 𝘴𝘰𝘮𝘦𝘰𝘯𝘦 𝘦𝘭𝘴𝘦 𝘸𝘪𝘭𝘭 𝘣𝘦𝘯𝘦𝘧𝘪𝘵 𝘧𝘳𝘰𝘮 𝘵𝘩𝘪𝘴, ♻️ 𝘴𝘩𝘢𝘳𝘦 𝘵𝘩𝘪𝘴

💚 On WhatsApp

💙 On LinkedIn

🩵 Bluesky

What can I do for getting promoted for the role I want?

If you are working in a tech product company, with around 800 or 1K employees, you have to know that you will not get a promotion just because “the time came”.

👉🏼 Instead, you have to demonstrate you can do the job before getting promoted.

I will not discuss here whether this is a good practice or not. It’s just the reality.

In today’s email, I will focus on the journey to the Staff Engineer role, and I will share with you 3 actions that are a must for you to get promoted to Staff Engineer.

The prerequisite here is to catch the opportunity to lead a project. In my experience, this can happen in two ways:

This opportunity can come to you from upper management. For example:

Silvia (you), there is a project for reducing costs in our obserbability system, and I want you to lead it.

Or you build the opportunity. For example:

Hey Barbara (your manager), I’ve validated a POC for feature teams own and deliver their own infrastructure, which will make them 2x faster to production with the same quality than today.

In the real world, this last one takes more time than just a sentence like that, but I hope you see the point.

Once the opportunity is on you, these are the 3 actions I found successful in many cases for the person get promoted to Staff Engineer.

Ownership end-to-end

You are THE person for a project, from the beginning to the end.

As simple as it sounds. You will be the person whom managers will ask in case they have questions or require information.

You clearly understand the problem and the expectations of the stakeholders.

You are responsible for everything on the project. If you have dependencies, you will have to manage them and coordinate the priorities across all of them.

To be able to get the full level of ownership, I start asking this 3 questions as many times as I need to really feel I have control of the project.

1️⃣ What is the problem we want to solve?

• What are the use cases?

• What is the pain?

2️⃣ What’s the benefit for the company?

Is it cost saving, efficiency, reducing incidents in production…?

You have to know the priorities at the company level, so you can align the solution in that direction.

3️⃣ Who is the customer and what are the dependencies?

This could be either the customers of the company or internal customers, like developers, marketing, etc.

If you are praying for not having dependencies, sorry. If the project is meant to be done by a Staff Engineer role, for sure, you will have dependencies… and most likely outside of your organization. But this is good in the sense that, once you succeed (because you will 😉), you demonstrate you can coordinate with external people and coordinate successfully.

Communication

Oh dear, this is the most important part of this framework; Bring visibility to your stakeholders and managers.

You can communicate at any time:

• When progress

• When blockers

• …

☝🏼 Overcommunicating is better than a lack of communication.

Ensure you understand how your stakeholders like to get communication, such as:

• When (daily, weekly, …). I would recommend you start weekly and, if your sta

• The format. I usually like to be precise, using the framework:

  • TL;TR

  • Context

  • Current status

  • Next steps

• Via email, Slack, meeting, Confluence page, …

🎁 At the end of this email, I give you a Pro Tip about communication. But, for the moment, let’s continue to the last point of this framework.

Plan

You most likely prepared a plan to deliver work for your team before reaching this big project. The idea here is the same, but on a wider scale, including your external dependencies.

1️⃣ Brainstorming / Kickoff

Join your team and your dependencies (if possible), and explain the project. Also, prepare a brainstorming session for the key problems of the project. This will bring ownership from all.

This part could sound easy, but it could become tricky because you may face some people that generates noise or hijack the session (most likely with no intent to). Polish your human skills to sail the brainstorming session to your north star and so achieve your goal:

1. Communicate.

2. And get feedback.

2️⃣ Technical Story Mapping

This is something that you will find very useful. I have pending an email for a deep dive into Technical Story Mapping. Until then, I can tell you this is an exercise on which:

• Identify the categories of work.

• The work to do on each category.

• Prioritize the work.

• Coordinate with your dependencies.

The previous brainstorming is really useful for this.

I like the tool Miro for Story Mapping, but you can use any other tool that allows you to print “boxes, stickers, and text”.

3️⃣ Continuous sync

For this, return to the part I wrote about communication, because it’s the same, but with a difference: here, you don’t need (or want) the stakeholders.

This is more for the Workforce or Virtual Team (however you call it) that will work on the project you lead.

The format I like the most is a Weekly Ops meeting for the project, where you:

• Explain where we are.

• Check the status of your dependencies.

• Identify blockers, and the owner who can solve them.

• Next steps

✨ Summary

Let’s wrap up for today.

• Promotions to Staff Engineer are not about tenure, they are about demonstrating you can already do the job.

• The journey starts by leading a meaningful project, either assigned by leadership or proactively created by you.

• Success at this level requires three core actions.

• Take full end-to-end ownership, from problem definition to delivery and dependency management.

• Communicate early and often, giving visibility into progress, risks, and next steps.

• Create and execute a clear plan that aligns teams, dependencies, and priorities.

The key idea is simple: act like a Staff Engineer before you get the title.

Don’t forget about the Pro Tip! 👇🏻

🎁 Pro Tip

For every single meeting you have during the development of the project, you should:

1. Prepare an agenda with the goal, context, and expected outcomes.

2. Once the meeting ends, reply with the meeting notes and action items (a meeting without action items is a waste of time, meaning that an email could solve it).

3. Record the meeting notes and action items in a place where you can get back to them in the future, in case you have to remember a decision.

This saved me a couple of times already, mainly when someone comes to challenge your plan when you are in the middle of its implementation.

🔖 Follow-up readings

Thanks for your support and feedback, I really appreciate it!

You’re the best! 🖖🏼

𝘐𝘧 𝘺𝘰𝘶 𝘦𝘯𝘫𝘰𝘺𝘦𝘥 𝘵𝘩𝘪𝘴 𝘱𝘰𝘴𝘵, 𝘵𝘩𝘦𝘯 𝘤𝘭𝘪𝘤𝘬 𝘵𝘩𝘦 💜. 𝘐𝘵 𝘩𝘦𝘭𝘱𝘴!

𝘐𝘧 𝘺𝘰𝘶 𝘬𝘯𝘰𝘸 𝘴𝘰𝘮𝘦𝘰𝘯𝘦 𝘦𝘭𝘴𝘦 𝘸𝘪𝘭𝘭 𝘣𝘦𝘯𝘦𝘧𝘪𝘵 𝘧𝘳𝘰𝘮 𝘵𝘩𝘪𝘴, ♻️ 𝘴𝘩𝘢𝘳𝘦 𝘵𝘩𝘪𝘴

💚 On WhatsApp

💙 On LinkedIn

🩵 Bluesky

Based on that, and taking into account our use case, we decided to implement an AWS Serverless approach, more concretely, using an AWS Step Function.

In today’s email, I want to share with you what the Software Architecture I tend to use in this kind of application is, so that you can apply it as well.

Before continuing, if you like my work and you want to support it, I would appreciate it if you could share this newsletter with your peers via:

💚 On WhatsApp

💙 On LinkedIn

🩵 Bluesky

Or just forward this email. Thanks in advance!

Getting back to our business…

If you read me for a while, you know that I like to reduce complexity as much as possible.

👉🏼 Making things simple is complex.

Input, Output, and Use Case

As simple as it sounds.

For each of the lambda functions within a Step Function, the Software Architecture that works well for me is:

• input. In this package, you will implement the code for reading from the outside world. For example, if your app has to download a configuration file from an S3 bucket, call a REST API endpoint, etc., you will implement it here. Within this input package, I would advise having a sub-package per technology from which your app will read, so you encapsulate each.

• output. In this package, you will implement the code for pushing data outside of your app. For example, if your app needs, after your business logic, to write a file into an S3 bucket, this is the right place. Same as for input, I would advise having a sub-package per technology from which your app will write, so you encapsulate each.

• Use Case. Assuming you have a single use case (since this is an ETL), you can resolve your “orchestration” from one single use case, which uses the different inputs, executes the business logic, and uses the outputs.

A question that I recently got about this Software Architecture was the following:

What happens if, from one of the APIs I use in the input package, I need another dataset? Could I just implement a new function in the input package for that and leverage the existing code?

☝🏼 That means a new domain. It does not matter if you read data from one or many APIs, but the domain matters.

For example:

• If you are gathering information about Observability, it does not matter if you gather that from Datadog or New Relic; what matters is the concept of Observability (the domain) for you.

• If you are gathering information about Applications, it does not matter if you gather that information from Datadog as well; what matters is the concept of Applications (the domain) for you.

Another example:

• If you are gathering information about Bugs, it does not matter if you gather that information from Jira or others; what matters is the concept of Bugs (the domain) for you.

• If you are gathering information about Tests executions, it does not matter if you gather that information from Jira as well; what matters is the concept of Tests executions (the domain) for you.

Then, you need to implement a new Lambda Function (in the context of the Step Function) that resolves the ETL for each of the domains.

If you still want to reuse code (which I would recommend), when using Step Functions, you can leverage AWS Lambda layers. They allow you to share common code across multiple Lambda functions within the same workflow.

As you can see, this is not exactly a hexagonal architecture, but an adaptation from it.

I would love to hear from you and about Software Architecture in general, and for ETLs in particular. Write down a comment, I'll read you.

Cheers!

Marcos, where do you find the time writing, besides working as Tech Lead for a company and have an actual life outside of the keyboard?

I explained how I organize myself for finding time for writing, even though I’m not a writer per se. To that, he told me:

Marcos, you should write about that. I’m pretty sure it can help more people.

And I took his advice, and so here we go today.

In today’s email, I will explain to you why I write about software development and how I find time for writing about it.

Before continuing, if you like my work and you want to support it, I would appreciate it if you could share this newsletter with your peers via:

💚 On WhatsApp

💙 On LinkedIn

🩵 Bluesky

Or just forward this email. Thanks in advance!

Now, let’s start with the Why.

Why do I write about Software Development?

It helps me to settle down my experiences.

Writing about Software Development helps me by:

1. Put in perspective my experiences and learnings.

2. Learn from my own experiences.

And, as far as the feedback I get from you, it helps others who will walk through my same track at some point.

Another question that Ángel raised:

But, Marcos, how do you find to topics to write about?

I write about what happens in my real work life, or about close friends I trust, which makes it very easy to find topics. Fortunately (?), in my daily work, many things happen, many learnings, every single day.

Because of that, the topics flow very smoothly.

How do I organize my time for writing?

In the train, I have 4 hours of commuting home.

I know, it’s a lot. Let me give you context.

Almost three years ago, because of personal decisions, I started commuting longer than before: by train, two days a week (I have a hybrid position), two hours from home to work, and two hours back.

I saw an opportunity in this new situation of my life, and so I started writing this newsletter, The Optimist Engineer.

Yeah, I know, I could use those four hours for something more human, like sleeping, or getting a dopamine boost from the infinite scroll of any social media app.
And I’ll confess that I do that sometimes, just not during the four hours of commuting.

I use my train rides back home to write. I invest two hours, twice a week, in writing a new newsletter. As I write these lines, I’m traveling at 298 km/h by train, in the middle of nowhere on the map of Spain.

I can tell you this aspect of my life has made me become consistent during these 3 years; since 2024, I have been writing a new issue of this newsletter per week. That’s a lot for me, I can tell you. In the past, I started this blog, which I gave up because I was not able to find a regular moment for writing.

The pain of having to commute 4 hours, twice a week, helped me to become constant.

If you want to start writing, my advice is:

1. Write about something that comes naturally to your mind. For me, it’s very easy to write about things that happen on a daily basis.

2. Find your slot. Same as for going to the gym, find a slot where you can write. I write for 2 hours, twice a week, only, that’s why my newsletter issues are no longer than 4 or 5 minutes of reading!

Hope this helps others, as my friend Ángel expected. If yes, I would love to hear from you and about your struggles. Write down a comment, I'll read you.

Happy New Year to everyone!

I usually take the last two weeks of the year to get rest, and I want to take this moment to thank you for this exciting year 2025 and your support along the ride.

Seriously, thank you. I wish you the best for the coming year.

You will receive my next email in the first week of January 2026.

If you have read my newsletter for a while, you know that I’ve been deeply immersed in the Product Platform Engineering world for the last 8 years. In case you want to go deep into what a Product Platform Engineering Team is, I recommend the following reading 👇🏻

In today’s email, I go one step beyond, and you will learn the technical principles on which a Product Platform Engineering Team should be based.

The main idea of today’s email is this:

You need to define a way of working so your team can release their applications, libraries, and frameworks quickly.

Otherwise, the DEV teams will find a way to achieve the same on their own.

To achieve this, you must focus on the following steps, in order:

1. Programming language and framework to use

This is extremely important to me; it can be the difference between taking half a day or a full week to deliver a new feature to teams using the framework.

You have to pick up a language and framework/libraries that enable your team to deliver fast.

A Product Platform team must be pragmatic from day one. The language should let you achieve high performance with minimal code. Likewise, the framework must offer transparent support for observability: logging, metrics, and tracing should be first-class citizens.

How to choose it? I recommend that you make the decision based on:

1. What is the most common language/framework used in the Engineering organization?

2. Are there enough people in the market knowledgeable about that language/framework?

3. Is the language/framework mature enough to provide an easy way to have things like observability out of the box?

☝🏼 Crucial: the team must be proficient in the chosen language and framework. Don’t pick a language that only one person in the company understands.

2. Avoid pointless debates that don’t help the development teams

Some people will argue tabs vs spaces, others will push Google’s linter, and others want a custom rule set. Some prefer squash merges, others don’t.

These are details that add no value to your real customers: the development teams. Why waste time opening debates about it?

☝🏼 Let’s focus on the business value and the priorities.

Development teams want new features or ways to deliver those features, the same goal as your end users.

3. Use the same CI/CD as the development teams

Be close to the development teams’ pain.

☝🏼 Their pain must be your pain.

If they struggle with the CI/CD system, you should experience the same constraints because that alignment will make your platform more useful and realistic.

• If you implement CI/CD functionalities for development teams, your team should use them as well.

• If your team uses something else, you have to use it too.

You have to understand their struggle so you can propose solutions for them.

4. Hold yourselves to the same standards as the development teams

You might not be running business traffic, but that’s not an excuse to lower standards.

Best practices, such as:

• Pair Review / Programming

• TDD

• Software Architecture that speaks (Screaming Architecture mentioned in the Clean Architecture book)

• Security updates

• Among others that are demanded for development teams, so they can deliver high-quality products.

☝🏼 Apply the same level of scrutiny for support, monitoring, and reliability to the Product Platform team as you do to the development teams. That closeness helps you understand their perspective and provide better support.

Some examples of this are:

• Monitoring: development teams have monitors and alerts when services fail, then your QA Enablement team should too.

• Support: development teams rely on a support network to help each other, then build the same for your team.

Those are the minimal commitments to reach in any Product Platform team.

☝🏼 Ensure you put in place a way of working as a team before implementing solutions for the DEV teams.

Hope you find this email useful, and I would like to hear your vision about how to build Platform Engineering teams. I read you in the comments.

Thanks for your support and feedback, I really appreciate it!

You’re the best! 🖖🏼

𝘐𝘧 𝘺𝘰𝘶 𝘦𝘯𝘫𝘰𝘺𝘦𝘥 𝘵𝘩𝘪𝘴 𝘱𝘰𝘴𝘵, 𝘵𝘩𝘦𝘯 𝘤𝘭𝘪𝘤𝘬 𝘵𝘩𝘦 💜. 𝘐𝘵 𝘩𝘦𝘭𝘱𝘴!

𝘐𝘧 𝘺𝘰𝘶 𝘬𝘯𝘰𝘸 𝘴𝘰𝘮𝘦𝘰𝘯𝘦 𝘦𝘭𝘴𝘦 𝘸𝘪𝘭𝘭 𝘣𝘦𝘯𝘦𝘧𝘪𝘵 𝘧𝘳𝘰𝘮 𝘵𝘩𝘪𝘴, ♻️ 𝘴𝘩𝘢𝘳𝘦 𝘵𝘩𝘪𝘴

💚 On WhatsApp

💙 On LinkedIn

🩵 Bluesky

For running your tests, you have:

1. A Docker Compose file

2. Which has the 2 different services that your application talks to

3. Plus the corresponding infrastructure that those 2 services require (database, etc)

4. Plus Kafka, because your service produces/consumes in Kafka.

And sometimes the Docker Compose fails; some service did not start because of some random problem, and you have to restart.

👉🏼 This has a problem: It slows you down for reaching production for every single change you implement.

In today’s email, you’ll learn one key step that will help you deliver your applications to production more quickly.

🎯 The Problem

Dependencies. That’s the underlying problem.

In this context:

A dependency is any other service, application, or system that your application interacts too.

👉🏼 Testing your application is costly because your service has too many dependencies.

I’ll share with you 6 problems caused by having multiple dependencies.

1. Coupling and coordinating changes: If your service depends on many others, any change to an API/contract can force coordinated changes across multiple teams or your deployment. The result: blocked deployments, delays, and coordination meetings.

2. Unpredictable latency and performance. Chained calls to external services increase latency and variability (p99 spikes). This impacts user experience and SLOs.

3. Testing complexity: This was my main point at the beginning of the email. Unit tests are more representative; integration tests become more expensive and slower. This makes it difficult to ensure quality in CI.

4. Operations and observability become more difficult. You need metrics/traces/alerts for each dependency. Identifying the root cause of a problem becomes more complicated.

5. Versioning and compatibility: If you depend on APIs without stable contracts or without semver, managing versions becomes a pain (breaking changes, coordinated migrations).

6. Loss of team autonomy. Small teams lose the ability to iterate quickly if they must coordinate with many external stakeholders.

So, the fewer dependencies your application has, the faster it can be deployed in Production.

🧞‍♂️ The Solution

Reduce dependencies to one, or the most as you can. That’s my recommendation.

I know it sounds tough, but in my experience, it’s the essential step if you want your application to reach production as quickly as possible.

But, Marcos, the business logic of my application needs all those dependencies, how can I reduce my dependencies then?

Decoupling. Decoupling your application from your dependencies. Take advantage of refactoring time to do this.

I wrote a dedicated email explaining how to decouple a monolith here, but I can give a couple of concrete hints now.

1. Implement asynchronous communication with your dependencies.

Example:

• Instead of calling 2 REST APIs and communicating with Kafka.

• Use only Kafka (or another one) for communicating with those 2 APIs.

Coming back to the problem of testing your application, now you don’t need to spin up 2 different services (with the linked infra) plus Kafka; You just need Kafka for your tests.

2. Consume one single dependency and extract logic to new applications/services.

Example:

• Your application reads from AWS S3, applies some business logic, and produces to Kafka.

• Create a new Platform Service, configurable, that is in charge of only reading from AWS S3 and producing into Kafka.

• Then, update your application to read from Kafka (where the other service puts the data), apply the business logic, and produce to Kafka.

With just those two strategies, you’ll turn a ‘hard-to-ship’ application into an independent one that can be deployed to production multiple times a day.

✨ Takeaway

I want you to retain this from today’s email:

👉🏼 Reduce the number of dependencies in your applications. As much as you can.

That’s THE takeaway today. Only that.

If you need more strategies for achieving this, reply to this email, comment if you are in Substack, or ping me in Bluesky or Mastodon. Happy to help!

We are more than ✨1080 Optimist Engineers✨!! 🚀

Thanks for your support and feedback, really appreciate it!

You’re the best! 🖖🏼

𝘐𝘧 𝘺𝘰𝘶 𝘦𝘯𝘫𝘰𝘺𝘦𝘥 𝘵𝘩𝘪𝘴 𝘱𝘰𝘴𝘵, 𝘵𝘩𝘦𝘯 𝘤𝘭𝘪𝘤𝘬 𝘵𝘩𝘦 💜. 𝘐𝘵 𝘩𝘦𝘭𝘱𝘴!

𝘐𝘧 𝘺𝘰𝘶 𝘬𝘯𝘰𝘸 𝘴𝘰𝘮𝘦𝘰𝘯𝘦 𝘦𝘭𝘴𝘦 𝘸𝘪𝘭𝘭 𝘣𝘦𝘯𝘦𝘧𝘪𝘵 𝘧𝘳𝘰𝘮 𝘵𝘩𝘪𝘴, ♻️ 𝘴𝘩𝘢𝘳𝘦 𝘵𝘩𝘪𝘴

💚 On WhatsApp

💙 On LinkedIn

🩵 Bluesky

You don’t.

But this is exactly what happens when you move from one company to another.

In today’s email, I’ll share a strategy you can use to implement successful solutions elsewhere.

Let me describe a scenario for you:

1. You and your team worked for months on a challenging engineering project.

2. You faced many obstacles, refactors, and performance issues.

3. And you finally succeeded.

During that experience you learned and collected multiple:

• Patterns.

• Strategies.

• Processes.

• Best practices.

After some time, it’s time for you to move on to new opportunities. What happens with all that knowledge?

Even if you have a great memory, you won’t remember everything.

And that is the problem: how to remember things that worked for you in the past.

You already put in the effort, so why lose it just because you changed companies?

This happened to me many times, and I struggled to remember what I did in “X company” years ago that I could apply again today.

And yeah, you cannot copy/paste code from your former company into a private repo/cloud-storage; That’s ilegal, unless the code is open sourced.

But you can copy/paste:

• Patterns.

• Strategies.

• Processes.

• Best practices.

Which is exactly what you learnt, and which is exactly the important part.

Do you like what you read? Share it with your friends via 👇🏻

💚 On WhatsApp

💙 On LinkedIn

🩵 Bluesky

Now the magic question:

Marcos, I want to save that knowledge too!

Here’s what I do. I use visual tools to store the knowledge. I’ve used a couple:

• draw.io. This was my first choice before Excalidraw.

• Excalidraw. I like it very much because I’m very fast at creating something. Also, it works really well offline or with low bandwidth (me working on the train).

You draw your knowledge. The knowledge is not copyrighted by companies (unless patents or similar).

So next time you need to remember or browse whether a previous solution might fit your current problem, you just access that visual memory, with the following advantages:

1. It was drawn by you in the past, so you’ll understand it now too.

2. It contains the general idea at a level that’s enough for you to understand what was done. Implementation details are not needed.

Hope this little trick can help you from now on.

Do you have other tips that could be useful for people when switching companies? Write them in a comment!

We are more than ✨1070 Optimist Engineers✨!! 🚀

Thanks for your support and feedback, really appreciate it!

You’re the best! 🖖🏼

𝘐𝘧 𝘺𝘰𝘶 𝘦𝘯𝘫𝘰𝘺𝘦𝘥 𝘵𝘩𝘪𝘴 𝘱𝘰𝘴𝘵, 𝘵𝘩𝘦𝘯 𝘤𝘭𝘪𝘤𝘬 𝘵𝘩𝘦 💜. 𝘐𝘵 𝘩𝘦𝘭𝘱𝘴!

𝘐𝘧 𝘺𝘰𝘶 𝘬𝘯𝘰𝘸 𝘴𝘰𝘮𝘦𝘰𝘯𝘦 𝘦𝘭𝘴𝘦 𝘸𝘪𝘭𝘭 𝘣𝘦𝘯𝘦𝘧𝘪𝘵 𝘧𝘳𝘰𝘮 𝘵𝘩𝘪𝘴, ♻️ 𝘴𝘩𝘢𝘳𝘦 𝘵𝘩𝘪𝘴

💚 On WhatsApp

💙 On LinkedIn

🩵 Bluesky

1. You create a controller for your REST API.

2. That controller calls your use case to respond to the request.

3. You create an Interface for that use case, with one method, which receives the DTO, and returns a Domain object (for example).

4. Finally, you create a new class that implements the Interface you just created; implements the use case.

By the book, this appears to be correct. Why?

• You are decoupling your implementation thanks to the Interface.

• Tomorrow, when a new implementation is needed, you just need to create a new class that implements the Interface, and your Controller will not change.

• At the time of testing, you use that Interface and create a Mock implementation for your tests.

But, is this a good idea?

In today’s email, I will share with you when and why you should use and avoid Interfaces.

☝🏼 Before starting, I just wanted to let you know that I have a dedicated newsletter to the AI Native world, where you can learn how to become an AI Native developer and move your engineering organization from Cloud Native to AI Native. Folks are loving it.

Subscribe now 👇🏻

AI Native Compass

From Cloud Native to AI Native

By Marcos F. Lobo 🗻🧭

Now, let’s back to the topic of today.

✅ Do you really have multiple implementations upfront?

If the answer is yes, you definitely have to add an Interface.

For example, if you are using different services to send data to the outside world (Azure Log Analytics API and AWS Firehose), the Interface allows you to define that abstraction, simplifying the API for the user.

If you just have one single implementation, you are just duplicating code between the Interface and the actual implementation. No point in doing this.

✅ Do you have many things behind a boundary?

Interfaces are an excellent way to define a contract when establishing a boundary.

When there are many things behind a specific boundary, the interface defines a “joint” that is specific to a use case that other components or systems must consume.

In these scenarios, the contract that defines the Interface is fundamental, so that other developers or systems can interact with that complex behavior in a controlled and defined way.

🤔 Do you have one single implementation?

If there is only one concrete implementation, like sending data to AWS Firehose only (from the previous example), the interface is simply a reflection or a “mimicry” of that concrete implementation.

The abstraction is defined purely by the behaviors of the concrete implementation, creating a one-to-one relationship. In these situations, creating an interface changes nothing in terms of coupling, system fragility, or extensibility.

You say:

Well, it’s cheap to create an Interface for the just in case.

I would answer that saying:

What happened last time you did that? did another implementation came up later?

🤔 Do you want to change the behavior for testing?

That’s fair. You plan to implement a class, for testing purposes, that implements that Interface, and you inject it.

But for that, I have for you the Delegation Pattern. With that, you can do exactly the same, and you don’t require an Interface.

🤔 Are you writing code for your internal use?

If you are implementing code for being used by yourself, like an internal component that another internal component will use, and you are creating Interfaces, then you are creating contracts for yourself.

No need to have a contract with no one.

✨ Summary

I don’t want to be the party pooper, or maybe yes, but in today’s email, I just wanted to share with you my conclusions from many years of writing Interfaces all over the place.

I’ve been there, I feel you, you think that an Interface is the way to go; My 2 cents here:

Think twice… most likely you don’t need it.

I have much code written over there that, if I read it today, I would refactor it removing Interfaces.

I would love to hear your tales about coding with or without Interfaces. Reply to this email or comment if you are on Substack.

Thanks for your support and feedback, I really appreciate it!

You’re the best! 🖖🏼

𝘐𝘧 𝘺𝘰𝘶 𝘦𝘯𝘫𝘰𝘺𝘦𝘥 𝘵𝘩𝘪𝘴 𝘱𝘰𝘴𝘵, 𝘵𝘩𝘦𝘯 𝘤𝘭𝘪𝘤𝘬 𝘵𝘩𝘦 💜. 𝘐𝘵 𝘩𝘦𝘭𝘱𝘴!

𝘐𝘧 𝘺𝘰𝘶 𝘬𝘯𝘰𝘸 𝘴𝘰𝘮𝘦𝘰𝘯𝘦 𝘦𝘭𝘴𝘦 𝘸𝘪𝘭𝘭 𝘣𝘦𝘯𝘦𝘧𝘪𝘵 𝘧𝘳𝘰𝘮 𝘵𝘩𝘪𝘴, ♻️ 𝘴𝘩𝘢𝘳𝘦 𝘵𝘩𝘪𝘴

💚 On WhatsApp

💙 On LinkedIn

🩵 Bluesky

You can read the previous one here 👇🏻

As you grow as a Software Engineer or Tech Lead, you need to grow in people skills, of course, but also in technical depth. That’s why this series exists. I hope you find this one useful.

Today, you will read about incremental architectures. Let’s put a bit of common ground first.

An incremental architecture is an architecture designed to grow.

☝🏼 And why should you want that?

Two simple reasons.

1. You care about code.

2. Your code will evolve. Your system will evolve. That is natural. The question is whether it can evolve easily or not.

If your architecture makes change painful, you are in trouble. If it is easy to modify, you can iterate fast without rewriting everything every few months.

👉🏼 There is a set of ideas, patterns, and organizational moves that enable this. Below are the core ones you should know.

1. Mission of the architect

The job of an architect is not to impose diagrams.

The job of an architect is to teach and keep coherence across the system. Guide decisions, set constraints, and help teams move forward without blocking them.

From my personal perspective, this can be achieved by implementing code and solutions, and not just advising people from an Architecture role. In some special cases, when an engineering organization is in a concrete momentum, a dedicated and non-coding architect could be ok.

2. Organization before technology

Cross-functional teams deliver faster than layered organizations because each team owns a domain end-to-end. No waiting for “the backend team” or “the frontend team” to free up a sprint.

👉🏼 One team, one domain, full ownership.

Keep this in mind:

• If you want microservices, you need cross-functional teams first.

• If you want incremental architecture, you need independent teams first.

This is why Gall’s Law matters, and with it, one of the takeaways for you today 👇🏻

If you start with a complex system, you will end with a complex system that does not work. Start small, grow intentionally.

3. Change the conversation

Instead of asking:

“how long will this take?”

ask:

“Can we make this smaller?”

👉🏼 You shift from deadlines to value. You move from guessing time to defining learning.

The real loop of incremental architectures is very simple and very effective:

1. Build something small.

2. Ship it.

3. Get feedback.

4. Adjust.

5. Repeat.

This loop is architecture. Not diagrams.

4. When the team lacks skills

A common blocker is “we cannot move fast because not everyone knows this part.”

Try mob programming. It spreads knowledge incredibly fast and aligns the whole team around real code, not documentation that nobody reads.

Other techniques that I practice and get great success with are:

• Pair Programming.

• Pair Review.

5. Patterns, but at the right time

Patterns are solutions to problems.

👉🏼 Use them when you have a problem, not before.

Some helpful patterns for incremental growth:

• Layered architecture: good separation of concerns, but people skip layers, so be careful.

• Hexagonal or microkernel: other ways to separate responsibilities, more explicit.

• Distributed architectures: enable incremental deployment and elasticity, but recovery can be hard. Know the distributed systems fallacies before you embrace distribution.

👉🏼 Remember: patterns are tools, not the goal.

6. The Strangler Pattern

This one is for when you have a big ball of mud and you want something cleaner.

I have a dedicated email for you on this here 👇🏻

And here in a nutshell:

1. Build a new module inside the existing system.

2. Give it clean, explicit interfaces.

3. Extract it from the system.

4. Redirect the rest to use the new module.

It is not easy, but it is a safe migration strategy without big-bang rewrites.

7. Component Architecture: your growth engine

A good component has a few strict qualities:

• Small classes and single responsibility.

• High cohesion and a focused interface.

• Hard boundaries, a single entry point. No spaghetti.

• Replaceable implementation, so clients do not depend on internals (Liskov matters here).

👉🏼 A component is not a microservice. It is smaller.

The trick is this: if you build components well, you can extract them into microservices whenever you need to. Zero drama.

This is how incremental architecture feels in real life.

8. Event-Driven Architecture

Strong coupling is bad.

👉🏼 You cannot eliminate it completely, but you can reduce it.

Event-driven approaches help. Instead of orchestrating a whole flow from a central component that calls payment, billing, shipping, and so on, the core component simply emits an event, for example, “OrderPlaced”.

Other parts react if they need to.

Everyone listens, no one knows each other, and events go through a broker, not direct calls.

This reduces coupling dramatically.

9. Domain Driven Design

DDD is a technique, not just a pattern, and it fits beautifully with incremental architectures.

👉🏼 And pay attention to this: Ubiquitous language is not a luxury. It reduces confusion and accelerates change.

When you map events and work, try this mental model:

• Orange stickers: events, written in the past tense.

• Blue stickers: consequences, the actual work to do.

• Yellow stickers: who performs the work, your services.

• Red stickers: questions, gaps in your knowledge.

If you see many red stickers, do not code yet. Answer the questions first.

✨ Takeaways

Today has been a very focused email about incremental architectures, but with many important things to take into consideration.

Take this list with you:

• Can you make the work smaller?

• Do teams own domains end-to-end?

• Do you apply patterns when they solve real problems, not by default?

• Do components have hard boundaries and respect Liskov?

Incremental architecture is a discipline, not magic. It pairs organizational change with pragmatic technical choices so you can grow without breaking everything.

Thanks for your support and feedback, I really appreciate it!

You’re the best! 🖖🏼

𝘐𝘧 𝘺𝘰𝘶 𝘦𝘯𝘫𝘰𝘺𝘦𝘥 𝘵𝘩𝘪𝘴 𝘱𝘰𝘴𝘵, 𝘵𝘩𝘦𝘯 𝘤𝘭𝘪𝘤𝘬 𝘵𝘩𝘦 💜. 𝘐𝘵 𝘩𝘦𝘭𝘱𝘴!

𝘐𝘧 𝘺𝘰𝘶 𝘬𝘯𝘰𝘸 𝘴𝘰𝘮𝘦𝘰𝘯𝘦 𝘦𝘭𝘴𝘦 𝘸𝘪𝘭𝘭 𝘣𝘦𝘯𝘦𝘧𝘪𝘵 𝘧𝘳𝘰𝘮 𝘵𝘩𝘪𝘴, ♻️ 𝘴𝘩𝘢𝘳𝘦 𝘵𝘩𝘪𝘴

💚 On WhatsApp

💙 On LinkedIn

🩵 Bluesky

I need an S3 bucket so my application can send data to it.

Simple, right? But here is the problem you are facing. Either you:

• Have to open a ticket to another team’s Jira board, wait until the S3 is in place, and iterate because something was missing.

• Have to open a Pull Request in a repo from another team, typically, the “Infrastructure team”.

• You name it…

👉🏼 You don’t have ownership of the infrastructure you need for the applications you own.

And this is a huge problem, because:

1. One, you depend on another team.

2. Two, that makes you slow when shipping your code to Production.

In today’s email, I bring you a Cloud Native technology that would return power to developers by allowing them to own their own infrastructure.

Enter Crossplane

This is my definition:

In short:

• You deploy a Custom Resource, a YAML file, in Kubernetes,

• And that will be translated into a piece of infrastructure, like an S3 bucket.

More in detail:

• Crossplane works on top of the Custom Resource API from Kubernetes. It’s an application living in Kubernetes.

• Crossplane defines a set of Custom Resource Definitions.

• You, as a developer, can deploy new Custom Resources, a new Crossplane Composition, to tell “hey, I want an S3 bucket with the name ‘ACME’”.

• And Crossplane will connect to the corresponding API (AWS S3 API in this example) to create the infrastructure indicated in the Custom Resource you deployed.

Now that you have an idea about what Crossplane is, let’s get back to the problems it solves.

1. Lack of autonomy for developers

👉 Problem:

Every time you need a database or a bucket, you must open a ticket with the platform team.

👉 With Crossplane:

1. The Platform Team publishes internal declarative APIs (for example, kind: PostgreSQLInstance, kind: CacheCluster, or kind: MessageQueue) with secure defaults.

2. You, as a developer, just create a YAML manifest with their parameters, in the same way as you would deploy a Deployment or a Service.

Crossplane translates that into an RDS in AWS, a CloudSQL in GCP, or whatever is defined in the Composition.

2. Inconsistency and risk in infrastructure

👉 Problem:

You and your peers define your own infrastructure differently, sometimes with insecure configurations.

👉 With Crossplane:

1. The Platform Team defines Compositions that enforce corporate standards, such as instance types, policies, regions, tags, etc.

2. You, as a developer, can only choose allowed parameters.

You can guarantee that you deploy a secure and standardized (within your company) infrastructure without sacrificing autonomy.

3. Artificial separation between “infrastructure” and “application”

👉 Problem:

The code (application) lives in Kubernetes, but the infrastructure (DBs, queues, etc.) is defined in Terraform or another external tool. This breaks the declarative model and complicates GitOps deployments.

👉 With Crossplane:

1. Everything lives in the same Kubernetes control plane.

2. Your application manifest can include both the Deployment and the PostgreSQL instance or Bucket it needs.

This results in you having infra and app versioned and deployed together under a unified declarative model.

✨ Summary

In today’s email, I just wanted to make a heads-up about Crossplane, a technology that solves a real and important problem that the majority of us suffer from every day: Not owning our infrastructure.

In the last months, I’ve been implementing Crossplane to solve the problem described.

I’ve already learned a few things, and I’ll be sharing them with you here in the next few weeks.

If you know some friends who might be interested in this email, forward it!

Stay tuned!

Thanks for your support and feedback, I really appreciate it!

You’re the best! 🖖🏼

𝘐𝘧 𝘺𝘰𝘶 𝘦𝘯𝘫𝘰𝘺𝘦𝘥 𝘵𝘩𝘪𝘴 𝘱𝘰𝘴𝘵, 𝘵𝘩𝘦𝘯 𝘤𝘭𝘪𝘤𝘬 𝘵𝘩𝘦 💜. 𝘐𝘵 𝘩𝘦𝘭𝘱𝘴!

𝘐𝘧 𝘺𝘰𝘶 𝘬𝘯𝘰𝘸 𝘴𝘰𝘮𝘦𝘰𝘯𝘦 𝘦𝘭𝘴𝘦 𝘸𝘪𝘭𝘭 𝘣𝘦𝘯𝘦𝘧𝘪𝘵 𝘧𝘳𝘰𝘮 𝘵𝘩𝘪𝘴, ♻️ 𝘴𝘩𝘢𝘳𝘦 𝘵𝘩𝘪𝘴

💚 On WhatsApp

💙 On LinkedIn

🩵 Bluesky

📣 If you want to keep receiving this newsletter, I need you to click on the button below 👇🏻

✅ I want keep receiving your emails!

It just gets you to the main page of The Optimist Engineer newsletter. By clicking on that button, I will know that you are still interested in receiving my emails.

If you are no longer interested, I will kindly remove you from the list in 1 week's time, so you will no longer receive future newsletter issues from me. I also take this opportunity to thank you for the time you’ve been part of this newsletter.

Now, let’s go to today’s topic: Surprises are your enemy.

Let me ask you:

How many times have you gone to an operational meeting with your Engineering Manager or Engineering Director and ended up punished?

This is the picture:

• Your weekly operational meeting with your Engineering Director to report on the highlights that she/he needs to be aware of and understand the progress.

• But the meeting doesn't start as usual; the Engineering Director kicks off the conversation with an angry face.

• Yeah, she/he is pished off because has been told that your service lost data 2 days ago, and the Incident Report has not been filed yet.

• Your brain is telling you: Why is she/he so upset about a problem that we solved out in 10 minutes?

• You try to explain the situation, but it’s too late.

Summarizing: Because something you knew that happened and, in your vision, had low impact, you got a hit from your Engineering Director.

Spoiler: the problem isn’t that you didn’t fill out an Incident Report form.

Why is that? It was a surprise.

Upper management doesn’t want surprises.

Upper management doesn’t need surprises.

Let me tell you what happened under the hood.

• There was another meeting where your Engineering Director was invited with other upper management (CPO, CTO, other Engineering Directors, or similar).

• There, your Engineering Director was asked: Hey, one of your teams owns a service that had an incident by which we lost a significant piece of data, which was critical for our group. What happened?

• And your Engineering Director had no idea… at all.

So, she/he could just say:

I’ll check with the team in order to understand the problem and I will come back to you.

Meaning:

I don’t have f*ck*ng idea what are you talking about, you got me, I should know, and I will find it out ASAP.

👉🏼 Surprises put upper management in difficult situations. Without an explanation, the only thing they can do is put the shield on and wait for the hit to end.

I can tell you, I’ve been there a few times. Lucky me, I managed to understand the problem, so I could work on the solution, and now I’m sharing it with you to help you avoid the same struggle.

🎯 So, what’s the problem here?

Here we are talking about a misalignment about what is important:

• You thought the problem was neither important nor urgent. Or you just forgot to report it!

• Somebody in upper management considered the opposite.

Your upper management should be able to trust you to get notified when something she/he should know. Once that, your Engineering Director will know exactly what to do in the next meeting with the rest of upper management.

👉🏼 Trust is built through clear and straightforward communication, no matter the issue.

💡 What’s the solution for this?

It goes through two points.

1️⃣ First, you have to clearly understand what is important and urgent in your ownership.

👉🏼 Everything urgent or important must be reported.

When it’s related to the input & outputs of your team or services, it will most likely impact others, hence it’s important or urgent. For example:

• A commitment in the roadmap during a meeting with the other team, sure, this is important.

• A production incident, for sure, it’s urgent.

In the scenario I’ve described, the missing data was actually a problem for more people than just your team, which means it was important or even urgent, and your managers needed to know about it.

I can recommend that you use the Eisenhower Matrix to help you organize by importance. I wrote a template for you to copy/paste and use. You can freely download it here 👇🏻

2️⃣ Second, you have to communicate as soon as something important or urgent happens.

How to do this? I recommend a chat, but this depends on how your company works with the communication channel.

If we are talking about an incident in production, my template for you is:

1. Open a chat with your Engineering Manager + Engineering Director, indicating:

   1. The problem.

   2. The impact. Are customers affected?

   3. Current status.

   4. Next steps.

2. Keep the chat up to date, with 1-hour recurrence (less, if the impact is high), until the problem is solved.

Many times, we are not sure what we have to communicate. If this happens to you, my advice to you: Overcommunicate.

👉🏼 It’s better to overcommunicate than have information not reach the people.

Hope you find this email useful. If you have similar tales like this one, I would love to hear from you. Drop me a message or leave a comment.

Thanks for your support and feedback, I really appreciate it!

You’re the best! 🖖🏼

𝘐𝘧 𝘺𝘰𝘶 𝘦𝘯𝘫𝘰𝘺𝘦𝘥 𝘵𝘩𝘪𝘴 𝘱𝘰𝘴𝘵, 𝘵𝘩𝘦𝘯 𝘤𝘭𝘪𝘤𝘬 𝘵𝘩𝘦 💜. 𝘐𝘵 𝘩𝘦𝘭𝘱𝘴!

𝘐𝘧 𝘺𝘰𝘶 𝘬𝘯𝘰𝘸 𝘴𝘰𝘮𝘦𝘰𝘯𝘦 𝘦𝘭𝘴𝘦 𝘸𝘪𝘭𝘭 𝘣𝘦𝘯𝘦𝘧𝘪𝘵 𝘧𝘳𝘰𝘮 𝘵𝘩𝘪𝘴, ♻️ 𝘴𝘩𝘢𝘳𝘦 𝘵𝘩𝘪𝘴

💚 On WhatsApp

💙 On LinkedIn

🩵 Bluesky