Investigating a data migration bug with GitHub Copilot CLI and Git history

Template archival was failing with "Cleanup Handlers Invalid" for some older templates, while recently created ones worked. A database check showed CleanupFlagsJson was NULL for the affected records. I asked GitHub Copilot CLI to trace the creation and archival code through Git history and identify which records needed attention.

The investigation found a change that populated cleanup flags for new templates without migrating older data. Copilot produced a timeline, assessment queries and remediation options. I spent about five minutes writing the prompt, and it ran for roughly 25 minutes while I worked on other things.

#What needed investigating

The scenario: a document generation platform where some templates created between May and August couldn't be archived. Templates from September onwards were fine.

When you archive a template, the system runs cleanup workflows: detach shared resources, notify dependent systems, archive generated documents. Different templates need different cleanup based on their configuration. The CleanupFlagsJson field tells the system which workflows to run.

For the affected templates, that field was NULL. Not all templates from that period, just certain ones. An edge case we needed to understand.

DocumentTemplate.cs

public class DocumentTemplate
{
    public Guid Id { get; set; }
    public string Name { get; set; }
    public string Content { get; set; }
    public DateTime? ArchivedOn { get; set; }
    
    // This is NULL for some templates created May-August (edge case)
    // Should contain flags indicating which cleanup workflows to run
    public string? CleanupFlagsJson { get; set; }
}

I wanted to know when the missing data originated, whether a migration had been added, and which records were affected.

The repository had more than 50,000 lines of code and 200 commits across the previous six months, with more than 15 files potentially involved in template operations.

#The investigation prompt

I expected a manual investigation to take several hours, but that was an estimate. I didn't run the same investigation manually to measure the difference.

I ran Copilot CLI in a Docker container using my copilot_yolo -dotnet configuration (detailed here), which sandboxes Copilot's access while still letting it work with the codebase.

Terminal

copilot_yolo -dotnet

My prompt:

I have a bug where archiving templates created in June gives me the error "Cleanup Handlers Invalid". In the database, these templates have a null value for CleanupFlagsJson. The template was created on 2025-06-15. Can you investigate how the data could have become like this? Search through git history to identify changes to create/update/archive templates since May, and trace where it went wrong. I need to know the blast radius.

The prompt included the error, field name, an affected record date and a starting point for the history search. Those gave the agent concrete places to look.

#Tracing the code and history

#Phase 1: Find the current state

Copilot started by mapping what was there:

Terminal

# It ran commands like this in parallel
find /src -type f -name "*.cs" | grep -i "template"
find /src -type f -name "*.cs" | xargs grep -l "CleanupFlagsJson"

Found 12 files referencing CleanupFlagsJson, including CreateTemplate.cs, ArchiveTemplate.cs (where the error occurred), and CleanupWorkflowRegistry.cs.

#Phase 2: Search git history

Next, it traced the timeline through git commits:

Terminal

git log --all --since="2025-05-01" --oneline -- src/Templates/CreateTemplate.cs
git log --all --since="2025-05-01" --oneline -- src/Templates/ArchiveTemplate.cs

Two commits stood out across 6 months of history:

2ef4a91 (July 10, 2025): "Fix template preview rendering"
7c2b8a3 (Sept 3, 2025): "Add cleanup workflow registry for template archival"

#Phase 3: Code comparison across git history

Copilot then used git show to compare the code before and after the September refactor:

Before September 3rd (the edge case):

CreateTemplate.cs (before fix)

// CreateTemplate.cs - circa June 2025
public async Task<Result<TemplateResponse>> Execute(CreateTemplateRequest request)
{
    // Create the template
    var template = new DocumentTemplate
    {
        Id = Guid.NewGuid(),
        Name = request.Name,
        Content = request.Content,
        // CleanupFlagsJson not yet populated
    };
    
    // Determine template features
    var hasSharedAssets = request.SharedAssetIds?.Any() ?? false;
    var hasExternalIntegrations = request.Webhooks?.Any() ?? false;
    var hasScheduledJobs = request.ScheduleConfig != null;
    
    // Store feature flags for general use
    var features = new List<string>();
    if (hasSharedAssets) features.Add("SharedAssets");
    if (hasExternalIntegrations) features.Add("Webhooks");
    if (hasScheduledJobs) features.Add("Scheduled");
    
    await _db.TemplateFeatures.AddAsync(new TemplateFeatures
    {
        TemplateId = template.Id,
        Features = features
    });
    
    await _db.SaveChangesAsync();
    
    // ⚠️ Edge case: Some code paths didn't populate CleanupFlagsJson
    // This meant certain templates couldn't determine cleanup workflows on archival
    return Result.Ok(new TemplateResponse(template.Id));
}

After September 3rd (the fix):

CreateTemplate.cs (after fix)

// CreateTemplate.cs - after September 3, 2025
public async Task<Result<TemplateResponse>> Execute(CreateTemplateRequest request)
{
    var template = new DocumentTemplate
    {
        Id = Guid.NewGuid(),
        Name = request.Name,
        Content = request.Content,
    };
    
    // Determine template features
    var hasSharedAssets = request.SharedAssetIds?.Any() ?? false;
    var hasExternalIntegrations = request.Webhooks?.Any() ?? false;
    var hasScheduledJobs = request.ScheduleConfig != null;
    
    // Store feature flags for general use
    var features = new List<string>();
    if (hasSharedAssets) features.Add("SharedAssets");
    if (hasExternalIntegrations) features.Add("Webhooks");
    if (hasScheduledJobs) features.Add("Scheduled");
    
    await _db.TemplateFeatures.AddAsync(new TemplateFeatures
    {
        TemplateId = template.Id,
        Features = features
    });
    
    // ✅ NEW: Build cleanup flags for archival workflows
    var cleanupFlags = new CleanupFlags();
    
    if (hasSharedAssets)
    {
        // Need to detach shared assets before archiving
        cleanupFlags.RequireAssetDetachment = true;
    }
    
    if (hasExternalIntegrations)
    {
        // Need to notify webhooks and clean up external registrations
        cleanupFlags.RequireWebhookNotification = true;
        cleanupFlags.RequireExternalCleanup = true;
    }
    
    if (hasScheduledJobs)
    {
        // Need to cancel scheduled jobs
        cleanupFlags.RequireScheduleCancellation = true;
    }
    
    // ✅ NEW: Populate the cleanup flags JSON
    template.CleanupFlagsJson = JsonSerializer.Serialize(cleanupFlags);
    
    await _db.SaveChangesAsync();
    
    return Result.Ok(new TemplateResponse(template.Id));
}

#The archive operation (where it broke)

The archival endpoint expected cleanup flags to exist. These are excerpts of the relevant logic:

The snippets are excerpts rather than a runnable reproduction. In particular, the archive example calls JsonSerializer.Deserialize before checking the result; passing a null string can throw before reaching the displayed “Cleanup Handlers Invalid” branch. The null guard in Option 2 avoids that call. Check the actual error-handling path when applying this pattern to another codebase. Microsoft's Deserialize API reference documents the null-input exception.

ArchiveTemplate.cs

// ArchiveTemplate.cs
public async Task<Result> Execute(Guid templateId)
{
    var template = await _db.Templates
        .Include(t => t.Features)
        .FirstAsync(t => t.Id == templateId);
    
    // This fails for old templates
    var cleanupFlags = JsonSerializer.Deserialize<CleanupFlags>(
        template.CleanupFlagsJson // ← NULL for June templates
    );
    
    if (cleanupFlags == null)
    {
        return Result.Fail("Cleanup Handlers Invalid");
    }
    
    // Run cleanup workflows based on flags
    if (cleanupFlags.RequireAssetDetachment)
    {
        await _assetService.DetachFromTemplate(templateId);
    }
    
    if (cleanupFlags.RequireWebhookNotification)
    {
        await _webhookService.NotifyArchival(templateId);
    }
    
    if (cleanupFlags.RequireExternalCleanup)
    {
        await _integrationService.CleanupExternalRefs(templateId);
    }
    
    if (cleanupFlags.RequireScheduleCancellation)
    {
        await _scheduler.CancelTemplateJobs(templateId);
    }
    
    // Mark as archived
    template.ArchivedOn = _timeProvider.GetUtcNow();
    await _db.SaveChangesAsync();
    
    return Result.Ok();
}

#The complete timeline (built by Copilot)

Investigation timeline

May 15, 2025
 Template system launched
 Bug introduced: CleanupFlagsJson not populated

June 15, 2025  
 Affected template created

July 10, 2025 (Commit 2ef4a91)
 "Fix template preview rendering"
   └─ Fixed UI bug, data model issue remained

September 3, 2025 (Commit 7c2b8a3)
 "Add cleanup workflow registry for template archival"
   ├─ Added cleanup workflow orchestration
   ├─ New templates get CleanupFlagsJson populated
   └─ ❌ No migration for OLD templates

October 27, 2025
 Archival attempted, issue discovered

#Checking affected records

Copilot generated SQL to find active templates with missing cleanup flags in the relevant date range:

Find affected templates

-- Find affected templates
SELECT 
    t.Id,
    t.Name,
    t.CreatedOn,
    tf.Features
FROM Templates t
LEFT JOIN TemplateFeatures tf ON tf.TemplateId = t.Id
WHERE t.CleanupFlagsJson IS NULL
  AND t.CreatedOn >= '2025-05-01'
  AND t.CreatedOn < '2025-09-03'
  AND t.ArchivedOn IS NULL
ORDER BY t.CreatedOn;

The assessment identified 47 affected templates created between May 15 and September 2. They were still active and blocked from archival; other templates from that period were unaffected.

Copilot also checked related operations:

Check related operations

-- Check if template editing/duplication is affected
SELECT 
    t.Name,
    te.EditedOn,
    te.UserId
FROM Templates t
JOIN TemplateEdits te ON te.TemplateId = t.Id
WHERE t.CleanupFlagsJson IS NULL
  AND te.EditedOn > '2025-09-03'
ORDER BY te.EditedOn DESC;

Editing and duplication still worked. Only archival was blocked.

#The solution

Copilot gave three options:

#Option 1: Data migration (recommended)

Backfill the missing cleanup flags by reconstructing them from the TemplateFeatures table:

Backfill migration script

// Migration script
public async Task BackfillCleanupFlags()
{
    var affectedTemplates = await _db.Templates
        .Where(t => t.CleanupFlagsJson == null 
                 && t.CreatedOn >= new DateTime(2025, 5, 1)
                 && t.CreatedOn < new DateTime(2025, 9, 3))
        .Include(t => t.Features)
        .ToListAsync();
    
    foreach (var template in affectedTemplates)
    {
        var cleanupFlags = new CleanupFlags();
        
        // Reconstruct flags from existing feature data
        if (template.Features.Features.Contains("SharedAssets"))
        {
            cleanupFlags.RequireAssetDetachment = true;
        }
        
        if (template.Features.Features.Contains("Webhooks"))
        {
            cleanupFlags.RequireWebhookNotification = true;
            cleanupFlags.RequireExternalCleanup = true;
        }
        
        if (template.Features.Features.Contains("Scheduled"))
        {
            cleanupFlags.RequireScheduleCancellation = true;
        }
        
        template.CleanupFlagsJson = JsonSerializer.Serialize(cleanupFlags);
    }
    
    await _db.SaveChangesAsync();
}

#Option 2: Defensive archival

Make ArchiveTemplate handle NULL cleanup flags:

Defensive archival fallback

public async Task<Result> Execute(Guid templateId)
{
    var template = await _db.Templates
        .Include(t => t.Features)
        .FirstAsync(t => t.Id == templateId);

    CleanupFlags cleanupFlags;
    
    if (template.CleanupFlagsJson == null)
    {
        // Fallback: Reconstruct from TemplateFeatures
        cleanupFlags = ReconstructCleanupFlags(template.Features);
    }
    else
    {
        cleanupFlags = JsonSerializer.Deserialize<CleanupFlags>(
            template.CleanupFlagsJson
        );
    }
    
    // ... rest of archival with cleanup workflows
}

#Option 3: Block archival of old templates

Prevent archival of legacy templates:

Block legacy templates

if (template.CleanupFlagsJson == null)
{
    return Result.Fail(
        "This template uses a legacy format and cannot be archived automatically. " +
        "Please contact support for manual archival."
    );
}

We used Option 1 to backfill the missing data. For templates where the feature data was more complex or missing, we combined Options 1 and 2 to handle the different cases.

#Reviewing the investigation

Copilot's output was a roughly 3,200-word document covering five key commits, the timeline, SQL queries and three remediation options. The excerpts above show two of those commits. The original tally also recorded more than 50 commits covered within the broader six-month history.

It ran independent history searches concurrently, using commands such as:

Terminal

# All running at the same time
git log --since="2025-05-01" -- CreateTemplate.cs &
git log --since="2025-05-01" -- ArchiveTemplate.cs &
git log --since="2025-05-01" -- CleanupWorkflowRegistry.cs &
find . -name "*.cs" | xargs grep -l "CleanupFlagsJson" &

The useful connection was between the creation path, the later archival code and the absence of a backfill. That explained why fixing new records hadn't repaired the existing ones.

I estimated the manual work at four to six hours, so the result was useful, but that estimate isn't a measured saving. The recorded five minutes covered prompting, not every later review or remediation step. What I kept from the workflow was the documented investigation: a traceable change, queries to check its scope and code options I could review before choosing a fix.

Investigating a data migration bug with GitHub Copilot CLI and Git history

#What needed investigating

#The investigation prompt

#Tracing the code and history

#Phase 1: Find the current state

#Phase 2: Search git history

#Phase 3: Code comparison across git history

#The archive operation (where it broke)

#The complete timeline (built by Copilot)

#Checking affected records

#The solution

#Option 1: Data migration (recommended)

#Option 2: Defensive archival

#Option 3: Block archival of old templates

#Reviewing the investigation

Explaining the engineering work behind AI-generated prototypes

Managing OpenClaw From My Mac Without Running It Locally

Setting Up Docker-in-Docker in copilot_here