How Our Engineering Team Uses AI Tools Daily to Ship Faster, Catch More Bugs, and Write Better Code — A Practitioner's Honest Breakdown

AI tools are not magic. They do not replace engineers, they do not understand your codebase, and they will confidently generate code that compiles but violates your business rules. What they do — when used with discipline — is compress repetitive tasks, surface patterns a tired human might miss, and reduce the friction between thinking about code and writing it.

This post is a candid breakdown of how our engineering team at Stripe Systems uses AI tools every day. We will cover what works, what does not, and the specific metrics we have tracked over six months of deliberate AI integration. No hype, no vendor evangelism — just what we have observed building production software for clients.

The Tool Inventory

We do not use a single AI tool. Different tasks call for different tools, and we have settled on a combination after months of trial and evaluation.

GitHub Copilot is our primary code completion tool. It runs inside VS Code and JetBrains IDEs, providing inline suggestions as engineers type. It is strongest at boilerplate code, repetitive patterns, and common API usage. We use the Business tier, which provides the data privacy guarantees we require.

Cursor is our AI-native editor for exploratory coding sessions. When an engineer is prototyping a new module or working through a complex refactor, Cursor's ability to hold multi-file context and generate edits across files is materially faster than switching between a traditional editor and a chat interface.

Claude and ChatGPT serve as architecture discussion partners and debugging assistants. When an engineer is stuck on a design decision or needs to reason through a complex interaction between systems, a structured conversation with an LLM often surfaces edge cases or alternative approaches faster than rubber-ducking with a colleague (who has their own work to do). We primarily use Claude for longer technical reasoning and ChatGPT for quick lookups.

Codeium fills a niche role: we use its free tier for internal tooling and scripts where Copilot's business tier is not justified. It handles shell scripts, CI configuration, and one-off data migration scripts well enough.

Where AI Helps Most

After six months of tracking, these are the areas where AI tools provide consistent, measurable productivity gains:

Boilerplate code. NestJS modules, Flutter widget scaffolding, Express middleware, React component shells — any code that follows a predictable structure benefits from AI completion. An engineer types the service name and Copilot fills in the module, controller, service, and DTO files. This saves 15-20 minutes per module, multiple times per week.

Test generation. AI is surprisingly competent at generating test scaffolding. Given a function signature and a brief description, Copilot or Claude generates a reasonable set of test cases covering happy paths, null inputs, and boundary values. The engineer then adds the business-specific assertions. More on this below.

Regex and SQL writing. Nobody enjoys writing complex regex patterns or multi-join SQL queries from scratch. Describing the pattern in natural language and letting the AI generate it, then validating against test cases, is both faster and less error-prone than manual construction.

API integration scaffolding. When integrating with a third-party API (Razorpay, AWS S3, Twilio), AI tools generate the HTTP client setup, authentication headers, request/response types, and basic error handling. The engineer then customizes the retry logic, error mapping, and business-specific validation.

Documentation drafting. API documentation, README sections, inline comments for complex algorithms — AI generates a reasonable first draft that the engineer edits for accuracy. This turns documentation from a dreaded chore into a 10-minute editing task.

Where AI Fails

This section matters more than the previous one. Knowing where AI tools break down prevents wasted time and dangerous over-reliance.

Complex business logic. AI has no understanding of your domain. It cannot infer that a payment reconciliation must handle partial refunds differently from full refunds, or that a specific client's SLA requires a different retry strategy. Any code involving business rules must be written by an engineer who understands the domain.

Architecture decisions without context. Asking an LLM "should I use microservices or a monolith?" produces a generic answer. Architecture decisions require understanding of team size, deployment constraints, scaling patterns, organizational structure, and operational maturity — context that does not fit in a prompt.

Security-sensitive code. Authentication flows, encryption implementations, input sanitization, and authorization logic are areas where a plausible-looking but subtly wrong implementation is worse than no implementation. AI-generated security code must be treated as untrusted input and reviewed with the same rigor as code from an unknown contributor.

Nuanced error handling. AI tends to generate generic try-catch blocks with logging. Real error handling requires understanding what the caller expects, whether the operation is idempotent, what the recovery path is, and what the user sees. This is judgment work that AI cannot do.

Code Completion Usage Patterns

We tracked code completion acceptance across our team for three months. The numbers stabilize after the first month as engineers learn what to expect:

• ✓Accepted as-is: ~40% of suggestions
• ✓Accepted with edits: ~30% of suggestions (usually changing variable names, adjusting logic, or fixing off-by-one issues)
• ✓Rejected: ~30% of suggestions

What does this mean for productivity? The 40% accepted as-is represents genuine time savings — code the engineer would have typed manually. The 30% accepted with edits is a smaller but still real saving — the structure is right, the details need adjustment. The 30% rejected is the cost of using AI: the time spent reading and evaluating a bad suggestion is not zero, but it is small (1-2 seconds per rejection).

Net productivity gain from code completion alone: we estimate 15-20% faster code writing speed. This is not a 10x improvement. It is a meaningful but modest gain that compounds over weeks and months.

Test Generation Workflow

This is where we have seen the highest return on AI investment. Our workflow:

Step 1: Engineer writes the function or method.

Step 2: Engineer opens the test file and writes a descriptive comment:

// Tests for PaymentReconciliationService.reconcileTransaction
// Should handle: successful match, partial refund, full refund,
// missing transaction, duplicate reconciliation attempt,
// amount mismatch, currency mismatch, expired transaction

Step 3: Copilot generates test scaffolding for each case:

describe('PaymentReconciliationService', () => {
  describe('reconcileTransaction', () => {
    it('should successfully reconcile a matching transaction', async () => {
      const payment = createMockPayment({ amount: 1000, currency: 'INR' });
      const bankRecord = createMockBankRecord({ amount: 1000, currency: 'INR' });

      const result = await service.reconcileTransaction(payment, bankRecord);

      expect(result.status).toBe('RECONCILED');
      expect(result.matchedAt).toBeDefined();
    });

    it('should handle partial refund reconciliation', async () => {
      const payment = createMockPayment({ amount: 1000, currency: 'INR' });
      const bankRecord = createMockBankRecord({ amount: 700, currency: 'INR' });

      const result = await service.reconcileTransaction(payment, bankRecord);

      expect(result.status).toBe('PARTIAL_MATCH');
      expect(result.discrepancy).toBe(300);
    });

    // ... more test cases generated
  });
});

Step 4: Engineer reviews each test case, adjusts assertions for business correctness, and adds the cases AI missed — typically edge cases that require domain knowledge:

it('should reject reconciliation when transaction is older than settlement window', async () => {
  const payment = createMockPayment({
    amount: 1000,
    currency: 'INR',
    createdAt: subDays(new Date(), 45), // Beyond 30-day settlement window
  });
  const bankRecord = createMockBankRecord({ amount: 1000, currency: 'INR' });

  await expect(service.reconcileTransaction(payment, bankRecord))
    .rejects.toThrow(SettlementWindowExpiredError);
});

Of 22 test cases in a typical service, AI generates usable scaffolding for 14. The engineer writes the remaining 8, which are invariably the most business-critical tests. This is the right division of labor: AI handles the predictable, humans handle the nuanced.

Debugging with AI

When an engineer hits a complex bug — one that is not obvious from the stack trace — we have a structured workflow for AI-assisted debugging:

Prompt template:

I'm debugging an issue in a NestJS application using TypeORM with PostgreSQL.

**Error:** [paste stack trace]

**Relevant code:** [paste the function and its immediate dependencies]

**Expected behavior:** [what should happen]

**Actual behavior:** [what actually happens]

**What I've already checked:**
- [list of hypotheses already eliminated]

**Environment:** Node 20, NestJS 10, TypeORM 0.3.x, PostgreSQL 15

Give me your top 3 hypotheses ranked by probability,
with a specific check I can run to confirm or eliminate each one.

This prompt structure works because it gives the LLM enough context to generate useful hypotheses rather than generic advice. The key is including what you have already checked — this prevents the AI from suggesting obvious things you have already tried.

In practice, this workflow saves 20-30 minutes per complex bug. Not because the AI always identifies the root cause (it does about 40% of the time), but because it generates a structured list of things to check, which is faster than the engineer's unstructured exploration.

PR Description Generation

Every PR in our workflow includes a structured description: what changed, why, how to test, and what risks exist. AI generates the first draft from the diff:

Given this git diff, generate a PR description with these sections:
## What Changed
## Why
## How to Test
## Risks and Rollback

Focus on the intent of the changes, not line-by-line description.

Engineers edit the output for accuracy — AI gets the "what" right about 80% of the time but often misses the "why" because it does not have ticket context. Still, editing a draft is faster than writing from scratch, and the consistent structure improves review efficiency.

Six-Month Metrics

We measured these metrics across a six-person team over six months, comparing against the six-month period before AI tool adoption. These are real numbers from our project management tooling, not estimates:

Sprint velocity: +22% — from 34 to 41.5 story points per sprint. The gain comes primarily from faster boilerplate generation and test writing. Complex features did not speed up significantly; simple and medium-complexity tasks did.

PR review time: -35% — AI-generated PR descriptions mean reviewers understand the change faster. AI pre-review (described in a companion post) catches style and common issues before the human reviewer sees the PR. Senior engineers report spending less time on mechanical review and more on architectural review.

Defect escape rate: -18% — AI-generated tests catch edge cases that engineers skip under time pressure: null inputs, empty arrays, boundary values, concurrent access. These are not complex bugs, but they are the kind that slip through when a deadline is approaching and the engineer writes only the happy path tests.

Time-to-first-commit for new team members: -40% — New engineers use AI to explore the codebase ("explain what this service does", "show me how authentication works in this project") instead of reading stale documentation or waiting for a senior engineer to be available. The codebase becomes self-documenting through AI interaction.

What We Do Not Use AI For

Deliberate exclusion is as important as deliberate adoption:

Security reviews. AI-generated security analysis produces false confidence. A human reviewer with security expertise evaluates authentication flows, authorization logic, input validation, and data handling. AI might flag obvious issues (SQL injection via string concatenation) but misses context-dependent vulnerabilities.

Architecture decisions. System design requires understanding organizational constraints, team capabilities, operational maturity, and business evolution. We use AI to explore options ("what are the tradeoffs of event sourcing vs. CRUD for this use case?") but the decision is always human.

Client communication. Emails, proposals, and status updates to clients are written by humans. AI-generated communication lacks the context of the relationship, the history of the project, and the tone appropriate for the situation.

Production debugging. When a production incident is in progress, speed and accuracy matter more than AI assistance. Our incident response relies on runbooks, monitoring dashboards, and experienced engineers. AI is too slow and too unreliable for real-time incident response.

Governance and Data Privacy

This is non-negotiable: no proprietary code is sent to public LLM APIs. Our governance rules:

1. ✓GitHub Copilot Business — code suggestions are not used for training, telemetry is limited, and data is not retained.
2. ✓Claude and ChatGPT — enterprise tiers only, with data processing agreements in place. When discussing client code, we abstract the business logic ("a payment processing function that needs to handle X") rather than pasting raw code.
3. ✓Codeium — used only for internal tooling and non-proprietary code.
4. ✓No copy-paste of client code into public chat interfaces. This is a fireable offense, not a suggestion.

Every engineer acknowledges these rules during onboarding, and we audit usage quarterly.

Case Study: Payment Reconciliation Module — Ticket to Production

To make this concrete, here is the complete journey of a single feature through our AI-augmented workflow. The feature: a payment reconciliation module that matches bank settlement records against internal payment records, flags discrepancies, and generates reconciliation reports.

Ticket Analysis (Claude)

The PM wrote the ticket with business requirements. Before estimation, the assigned engineer pasted the requirements into Claude with this prompt:

Here are the requirements for a payment reconciliation module.
Identify any gaps, ambiguities, or edge cases not covered:

[pasted requirements]

Consider: error handling, concurrency, data consistency,
reporting requirements, and operational concerns.

Claude identified three gaps the PM had not considered: (1) what happens when the bank file contains transactions not in our system (orphaned bank records), (2) how to handle timezone differences between bank timestamps and system timestamps, and (3) what the retry behavior should be when the bank API is temporarily unavailable. These were added to the ticket before development started.

Time spent: 15 minutes. Estimated time without AI: 45 minutes (engineer would discover these gaps during development, requiring back-and-forth with the PM).

Module Scaffolding (Copilot)

The engineer created the NestJS module structure using Copilot. Starting with the module file:

// reconciliation.module.ts
@Module({
  imports: [TypeOrmModule.forFeature([Reconciliation, ReconciliationItem])],
  controllers: [ReconciliationController],
  providers: [
    ReconciliationService,
    BankFileParser,
    DiscrepancyDetector,
    ReconciliationReportGenerator,
  ],
  exports: [ReconciliationService],
})
export class ReconciliationModule {}

Copilot generated the service skeleton, controller with CRUD endpoints, DTOs with class-validator decorators, and entity definitions. The engineer adjusted the entity relationships and added business-specific validation rules.

Time spent: 45 minutes for the full module structure. Estimated time without AI: 2 hours.

Core Business Logic (Engineer-Written)

The reconciliation matching algorithm, discrepancy detection rules, and settlement window logic were written entirely by the engineer. This is complex business logic where AI suggestions were consistently wrong — the AI did not understand the specific matching rules (amount tolerance, reference number format, timezone handling) that the business required.

async matchTransactions(
  payments: Payment[],
  bankRecords: BankSettlementRecord[],
): Promise<ReconciliationResult> {
  const matched: MatchedPair[] = [];
  const unmatched: UnmatchedRecord[] = [];
  const discrepancies: Discrepancy[] = [];

  // Index bank records by normalized reference for O(n) matching
  const bankIndex = this.buildBankIndex(bankRecords);

  for (const payment of payments) {
    const normalizedRef = this.normalizeReference(payment.bankReference);
    const bankRecord = bankIndex.get(normalizedRef);

    if (!bankRecord) {
      unmatched.push({ type: 'MISSING_BANK_RECORD', payment });
      continue;
    }

    // Amount comparison with tolerance (bank may round differently)
    const amountDiff = Math.abs(payment.amount - bankRecord.settledAmount);
    if (amountDiff > this.AMOUNT_TOLERANCE_PAISE) {
      discrepancies.push({
        type: 'AMOUNT_MISMATCH',
        payment,
        bankRecord,
        difference: amountDiff,
      });
      continue;
    }

    // Timezone-aware date comparison
    const settlementDate = this.normalizeToIST(bankRecord.settlementDate);
    if (!this.isWithinSettlementWindow(payment.createdAt, settlementDate)) {
      discrepancies.push({
        type: 'SETTLEMENT_WINDOW_EXCEEDED',
        payment,
        bankRecord,
        daysDifference: differenceInDays(settlementDate, payment.createdAt),
      });
      continue;
    }

    matched.push({ payment, bankRecord, matchedAt: new Date() });
    bankIndex.delete(normalizedRef);
  }

  // Remaining bank records are orphaned
  for (const [ref, bankRecord] of bankIndex) {
    unmatched.push({ type: 'ORPHANED_BANK_RECORD', bankRecord });
  }

  return { matched, unmatched, discrepancies };
}

Time spent: 4 hours (this is the core of the feature). AI contribution: near zero for this section.

Test Writing (Copilot + Engineer)

Following the workflow described above, Copilot generated 14 of 22 test cases. The engineer wrote 8 business-critical tests:

AI-generated tests (examples): successful match, null input handling, empty arrays, single payment with single bank record, large batch processing (1000 records).

Engineer-written tests: settlement window boundary (29 days vs 31 days), amount tolerance edge case (exactly at threshold), timezone edge case (transaction at 23:59 IST vs 00:01 UTC next day), concurrent reconciliation attempts on the same batch, partial bank file processing (file truncated mid-record), reference number normalization across different bank formats, orphaned bank record reporting format, idempotency of reconciliation runs.

it('should handle timezone edge case at IST/UTC day boundary', async () => {
  const payment = createMockPayment({
    amount: 5000,
    createdAt: new Date('2026-03-15T23:59:00+05:30'), // 15th March IST
  });
  const bankRecord = createMockBankRecord({
    amount: 5000,
    settlementDate: new Date('2026-03-15T18:31:00Z'), // 16th March IST
  });

  const result = await service.matchTransactions([payment], [bankRecord]);

  // Should match despite appearing to be different days in UTC
  expect(result.matched).toHaveLength(1);
  expect(result.discrepancies).toHaveLength(0);
});

Time spent: 2.5 hours. Estimated time without AI: 4.5 hours.

PR Review (AI + Human)

The PR was submitted with an AI-generated description. Before the senior engineer reviewed it, our AI pre-review (a GitHub Actions workflow) ran and flagged one issue:

⚠️ Potential race condition in ReconciliationService.processReconciliation():

The method reads from the database, processes in memory, then writes back.
If two reconciliation runs execute concurrently for the same date range,
they could both read the same unreconciled records and produce
duplicate matches.

Consider: database-level locking, a distributed lock, or
an idempotency check before writing results.

The engineer had missed this. They added a database advisory lock on the date range before the human reviewer saw the PR. The senior engineer's review focused on the matching algorithm correctness and the settlement window logic — the mechanical issues had already been addressed.

Time spent on review: 25 minutes (senior engineer). Estimated without AI pre-review: 50 minutes.

Documentation (AI-Drafted)

The engineer pasted the controller and service code into Claude with the prompt:

Generate API documentation for this NestJS controller in markdown format.
Include: endpoint, method, request body, response body,
error codes, and a curl example for each endpoint.

Claude produced documentation for all 5 endpoints. The engineer corrected two response type descriptions and added a section on authentication requirements that the AI had no context for.

Time spent: 20 minutes. Estimated without AI: 1.5 hours.

Total Timeline

The time savings came from the mechanical tasks: scaffolding, test generation, documentation. The core business logic — the hard part — took the same time regardless of AI assistance. This is the honest picture: AI compresses the easy work, not the hard work. But easy work still takes time, and compressing it means engineers spend more of their day on the problems that actually require engineering judgment.

Conclusion

AI tools have become a permanent part of our engineering workflow at Stripe Systems. They are not a silver bullet — they are more like a very fast junior developer who is good at pattern matching but has no business context and cannot be trusted with anything security-sensitive.

The key to productive AI use is knowing where the boundaries are. Use AI for boilerplate, scaffolding, test generation, and documentation. Write business logic, security code, and architecture decisions yourself. Review everything AI generates with the same rigor you would apply to code from a new team member.

The 22% velocity improvement is real, but it came from disciplined adoption, clear governance, and honest measurement — not from blindly accepting AI suggestions and hoping for the best.