Beyond LGTM: Bringing Speed and Quality to Code Reviews with AI Agents

Written By: Gurleen Kaur

The Pain and the Promise of Code Reviews

Here at CosX.ai, we make no compromises on shipping rock-solid code. As a team building complex backend infrastructure and AI-automated systems, we’ve always believed that code reviews are non-negotiable. They’re our safety net, our mentorship layer, and our first line of defence against bad decisions slipping into production.

So when we rolled out the policy - “Every PR must be reviewed by at least two engineers before it touches main”, it felt like a victory for quality.

And it was …for a while.

Quality vs. Velocity: The Hidden Cost

As our team grew and deadlines got tighter, we started to feel the pressure.

• PRs would sit in review for hours, sometimes days blocking peer engineers.
• Reviewers were juggling unfamiliar code with other tasks.
• To keep things moving, we began to see "LGTM :)"-style approvals that weren’t really reviews.

We were slowing ourselves down - not because we didn’t care, but because our code review process didn’t scale with our swift velocity.

And ironically, quality was slipping, despite the guardrails we’d put in place.

Time to Innovate

We knew we needed a better way - something that could give us speed and quality.

So we gave ourselves a challenge:
Could we build an AI agent that reviews code like an expert engineer?

Not a generic chatbot.
Not a spammy “looks good” bot.
But a real, helpful reviewer that runs on every PR.

How We Built the AI Review Agent

We used Open AI SDK and GitHub’s Model Context Protocol (MCP) to build a multi-agentic PR reviewer that lives alongside our repo and plays the role of an AI reviewer.

Every time a PR is created or updated, Github actions trigger a review api which further triggers our multi agentic PR reviewer. This then performs analysis on the PR diff and posts the review comments onto Github.

Designing Intelligent Agents

Creating intelligent agents starts with designing the right prompts. A well-written prompt acts as the agent's blueprint for thinking, reasoning, and decision-making. It must clearly define the agent’s role, boundaries, goals, and expected behaviours. It is almost like writing a job description, a guide, and a mission statement all in one.

You are a senior software engineer.
Your job is to carefully review the given pr diff and find
concrete bugs or logic issues.
    Focus ONLY on:
    - Logic errors or incorrect control flow
    - Improper null checks that may cause runtime exceptions
    - Misuse of language features 
    - Data consistency or synchronization issues

    Your output should include:
    - A clear explanation of the issue
    - The filename and code reference (function name etc)
    - A possible fix or suggestion if applicable
    - If you dont see any issues, respond: `No issues found in this file.

But writing the prompt is not a one-shot effort. It's an iterative process.
You start with a clear design, observe how the agent responds, and refine the prompt based on gaps, misunderstandings, or new possibilities you discover. Each iteration helps sharpen the agent’s intelligence, making it more aligned, reliable, and capable over time. This iterative approach helped us create agents that do not give spammy reviews!

Enhancing Team Communication: Google Chat Notifications for PRs

While the AI agent could instantly analyze and provide feedback on a PR, we didn’t want to sideline the importance of human reviews. After all, code reviews are more than just finding bugs - they’re a crucial part of the learning process. The back-and-forth, the debates, and the "Aha!" moments that come from collaborating are key to improving as engineers.

To keep everyone in the loop, we added a GitHub Action that sends a notification to our Google Chat channel whenever a new PR is created or updated. This way, everyone knows when to jump in and contribute, whether it’s right after the AI's review or during their own detailed review process.

It posts something like :

📢 New PR Created!
👥 Org: Catalogue AI
📁 Domain: Backend
🔗 PR: https://github.com/cosx-ai/catalogueai-backend/pull/7
📝 Description: TS-219 view folders
👩‍💻 Dev: @gurleenk-cosx
🧠 AI review in progress...

Now the whole squad sees it, can jump in early, or wait for the AI to surface feedback first.
Simple. Effective. Kind of fun.

The Impact: Faster Reviews, Better Feedback

Within a few days of going live, the results were obvious:

• First-pass feedback landed in seconds instead of hours.
• Reviewers had a smarter starting point for their own thoughts. Review coverage went up by nearly 80%.
• The average time-to-merge dropped by 70%, cutting down PR cycles by nearly 1.5 days.
• Estimated 30+ human hours saved per week across the team, freeing engineers to focus more on building rather than chasing reviews.

What’s Next

While with multiple iterations, adding code context and incorporating manual feedback we have been able to create a terrific agent, there is always scope for improvement. Our next step is to set up a seamless feedback loop: engineers will simply hit 👍 or 👎 on each suggestion, and we’ll turn those votes into regular “tune-ups,” so the agent learns exactly what works (and what doesn’t) in our codebase.

Final Thoughts

At Cosx.ai, our aim is to become the most autonomous AI agency, revolutionising how we approach complex tasks. As part of this mission, we’ve begun to design agents that not only assist, but transform entire workflows.

The PR review agent, as we like to call it, ‘Galactic Guardian’ - was one of the first agents we built for our cosmos. A testament to the power of AI in improving efficiency and communication, Galactic Guardian serves as a trusted ally in ensuring faster, more reliable reviews. Its ability to seamlessly integrate into our team’s workflow has freed up human hours, reduced review fatigue, and drastically sped up our time-to-merge.

But this is just the beginning. Galactic Guardian is only one piece of our larger plan. Each step we take in building and refining these agents brings us closer to our goal of being the first truly autonomous AI agency — one where technology empowers our teams to achieve more, faster and with greater precision.

For any technical details or deeper dives, feel free to reach out to us directly at admin@cosx.ai or stay tuned for upcoming blog posts.