Gaurav Yadav | BCA Cybersecurity Student

Abstract

Automated vulnerability detection has become highly effective due to advances in fuzzing, sanitizers, and static analysis. However, remediation remains disproportionately manual, slow, and dependent on scarce security expertise. This imbalance increases mean time-to-fix, broadens exposure windows, and accumulates long-term security debt.

This research investigates whether Large Language Models (LLMs) can meaningfully assist in post-detection remediation without assuming autonomy or replacing human judgment. We analyze the feasibility of a hybrid remediation pipeline in which LLMs propose candidate patches that undergo automated validation and human approval. The study is based on architectural reasoning, failure analysis, and security threat modeling rather than empirical benchmarking.

Findings indicate that LLM-generated patches exhibit non-trivial value for structurally simple vulnerabilities, but introduce semantic and behavioral risks that require strict validation and human oversight. Negative results demonstrate that naive approaches to autonomous patching fail under realistic conditions, supporting a more conservative, assistive design philosophy.

Keywords

automated remediation, vulnerability patching, secure software engineering, LLMs, human-in-the-loop systems, semantic security, CI/CD, operational security.

I. Introduction

Modern software systems increasingly rely on automated tools to identify security vulnerabilities during development and post-deployment. Runtime sanitizers and dynamic analysis tools have proven effective in detecting elusive defects such as memory safety violations, data races, and temporal errors. Despite these advances, vulnerability remediation remains a human-intensive process.

This imbalance between detection scalability and remediation capacity introduces operational inefficiencies and increases exposure windows for exploitable flaws. Consequently, there is a growing need for automated systems capable of assisting developers in generating reliable vulnerability fixes.

This research explores the application of Large Language Models (LLMs) as a core component of an automated patch generation pipeline.

II. Historical Context: Evolution of Automated Remediation

Automated vulnerability remediation has evolved through multiple paradigms over the past two decades.


Evolution of Automated Remediation
2000sRule-Based
Template fixes
2010sSynthesis
Formal methods
2020sML Detection
Pattern recog.
2024+LLM Patch
Context-aware
LLM-assisted remediation as the next evolutionary step.

Early approaches relied on rule-based transformations, such as static code rewriting and template-driven fixes. While effective for narrowly defined defect classes, these systems lacked adaptability and failed under real-world code diversity.

Subsequent research explored program synthesis and constraint-solving techniques, enabling correctness guarantees at the cost of computational feasibility and limited scalability.

More recently, machine learning-based approaches emerged, primarily focused on vulnerability detection rather than remediation. The introduction of Large Language Models represents a qualitative shift: for the first time, systems can generate context-aware, human-readable code modifications.

This research positions LLM-assisted remediation as the next evolutionary step—bridging rigid automation and human-driven patching.

III. Problem Definition and Research Questions

Despite substantial progress in vulnerability detection, remediation workflows have not scaled proportionally. Modern development pipelines routinely surface thousands of sanitizer and fuzzing findings, many of which remain unresolved due to prioritization constraints, limited security expertise, or perceived low exploitability.

This research is motivated by the following core problem:

Can Large Language Models meaningfully assist in vulnerability remediation without introducing unacceptable security risk or developer over-reliance?


Research Questions
RQ1Feasibility
Which vulnerability classes are most amenable to automated patching?
RQ2Effectiveness
How effective are LLM-generated patches under realistic validation?
RQ3Risk Analysis
What failure modes emerge when LLMs assist remediation?
RQ4Design
How can human-in-the-loop design mitigate automation bias?

Rather than proposing a fully autonomous system, this research deliberately investigates assisted remediation as a more realistic and defensible security paradigm.

IV. Scope, Constraints, and Non-Goals

This research deliberately constrains its scope to ensure engineering realism and security defensibility.

Scope
Post-merge runtime vulnerabilities
Server-side & system software
C/C++, Rust, Java ecosystems
Constraints
LLM context window limits
Non-deterministic model output
Human accountability required
Non-Goals
Replace security engineers
Auto-merge to production
Detect vulnerabilities
Adversarial prompt defense

By clearly stating these non-goals, the system avoids overclaiming capabilities and maintains a conservative security posture.

V. Research Methodology

This study follows a qualitative and systems-oriented research methodology, informed by publicly documented industry practices and first-principles analysis.

This work adopts a design-oriented research methodology, emphasizing architectural reasoning, failure analysis, and security threat modeling over empirical benchmarking. This research is model-agnostic and does not depend on any specific LLM architecture or vendor.


Research Methodology Stages
STAGE 1
Workflow Decomposition
Remediation pipelines decomposed into discrete stages
STAGE 2
Automation Feasibility Analysis
Each stage evaluated for LLM automation viability
STAGE 3
Failure Mode Enumeration
Systematic identification of AI-introduced failure scenarios
STAGE 4
Architectural Synthesis
Human-in-the-loop architecture designed for security

This approach prioritizes engineering realism over speculative autonomy and emphasizes deployable system design rather than theoretical model performance.

VI. Design Alternatives and Trade-Off Analysis

Multiple architectural approaches were evaluated before selecting the proposed human-in-the-loop pipeline.


Design Alternatives Evaluated
Fully Autonomous
High semantic risk
rejected
Suggestion-Only
Minimal effort savings
rejected
Constrained Gen.
Efficiency + governance
selected

Comparative Analysis with Existing Approaches


Comparative Analysis Framework
Approach Automation Scalability Risk Effort
Manual Patching None Low Low High
Rule-Based Fixes Partial Medium Medium Medium
Synthesis-Based High Low Low Very High
LLM-Assisted (Proposed)THIS WORK Partial High Mitigated Low
The proposed approach occupies a previously underexplored middle ground.

Approach	Automation	Scalability	Risk	Effort
Manual Patching	None	Low	Low	High
Rule-Based Fixes	Partial	Medium	Medium	Medium
Synthesis-Based	High	Low	Low	Very High
LLM-Assisted (Proposed)THIS WORK	Partial	High	Mitigated	Low

This comparative framing highlights that the contribution of this work lies not in absolute automation, but in architectural positioning.

VII. Proposed AI-Assisted Remediation Architecture

Based on the preceding analysis, this research proposes a modular AI-assisted remediation architecture designed explicitly for post-detection vulnerability fixing.

The architecture is guided by three non-negotiable design principles:

Human Authority: No AI-generated patch may be merged without human approval.
Iterative Validation: Patch generation is treated as a probabilistic process requiring feedback loops.
Context Minimization: Only the minimal vulnerable context is exposed to the LLM to reduce hallucination risk.


DetectionSanitizers
IsolationContext
LLM PatchGenerate
ValidateCI/CD
ReviewAudit
ApproveMerge
Figure 1: AI-Assisted Vulnerability Remediation Pipeline

VIII. Pipeline Components

A. Vulnerability Detection

Runtime sanitizers and dynamic analysis tools identify vulnerabilities during execution. Crucial contextual information—such as stack traces and execution paths—is preserved to support downstream analysis.

B. Bug Isolation and Context Reduction

Effective patch generation depends critically on providing the LLM with sufficient—but not excessive—context.

The context reduction process follows a structured methodology:

Crash-Centric Extraction: Only code paths directly involved in the sanitizer-reported failure are included.
Dependency Pruning: Unrelated helper functions and utilities are removed unless directly referenced.
Semantic Anchoring: Function signatures, type definitions, and invariants are preserved.
Test Harness Alignment: Context is aligned with a minimal reproducing test.

C. Patch Generation Using LLMs

The isolated code and vulnerability metadata are provided to an LLM through structured prompts. Multiple candidate patches are generated to improve success probability.

CODEBASE
LLM MODEL
VALIDATOR
Vulnerable Context + Metadata
Candidate Patch A
Candidate Patch B
Run Validation Suite
PASS: Ready for Review
Figure 2: Multi-Candidate Patch Generation Sequence

D. Patch Generation Algorithm


Algorithm 1: Patch Generation
ALGORITHM: LLM_PATCH_GENERATION(vuln, context)
INPUT:  vuln    ← Sanitizer vulnerability report
        context ← Minimal reproducible code context
OUTPUT: patch   ← Validated remediation patch

1.  candidates ← []
2.  FOR i = 1 TO MAX_RETRIES DO
3.      prompt ← BUILD_PROMPT(vuln, context)
4.      patch_i ← LLM.generate(prompt, temp=0.2)
5.      
6.      IF SYNTAX_CHECK(patch_i) = PASS THEN
7.          IF UNIT_TESTS(patch_i) = PASS THEN
8.              IF SANITIZER_RERUN(patch_i) = CLEAN THEN
9.                  candidates.append(patch_i)
10.             END IF
11.         END IF
12.     END IF
13. END FOR
14.
15. IF candidates.length > 0 THEN
16.     RETURN SELECT_BEST(candidates)  // Human review
17. ELSE
18.     RETURN ESCALATE_TO_HUMAN(vuln)
19. END IF
 LLM Inference
 Validation Gates
 Human Checkpoint

E. Automated Validation

Generated patches undergo:

Unit testing
Regression testing
Sanitizer re-execution

Only patches that pass all automated checks proceed to human review.

F. Human Review and Secure Approval

Human reviewers perform final validation to detect:

Security regressions
Logic degradation
Silent vulnerability masking

This step remains critical to maintaining trustworthiness in AI-assisted remediation workflows.

IX. Evaluation Criteria

To assess the effectiveness of LLM-assisted patch generation, this research adopts rigorous evaluation criteria.


Patch Evaluation Criteria
CorrectnessRequired
Eliminates vulnerability without new errors
Security PreservationRequired
No weakening of security controls
Behavioral IntegrityRequired
Preserves program semantics
Review OverheadOptimized
Less effort than manual fix
Note: Test coverage alone is insufficient for security-critical paths.

It is emphasized that high test coverage alone is insufficient to satisfy these criteria, particularly for security-sensitive code paths.

X. Failure Mode Taxonomy

Analysis of AI-generated patches reveals recurring failure patterns that must be systematically addressed.


Failure Mode Taxonomy
Superficial Fixes
high
Null checks without root cause
Conditional guards masking flaws
Semantic Drift
critical
Altered control flow
Incorrect variable lifetime assumptions
Over-Constraining
medium
Reduced concurrency
Disabled optimizations
Test-Centric Deception
critical
Passes tests but violates invariants
Removed failing assertions

Understanding these failure modes is essential for designing effective validation and review processes.

XI. Threat Model and Assumptions

This research operates under a defined threat model to ensure security accountability.


Threat Model & Assumptions
Threat Vector Assumption Status
CI/CD Pipeline Compromise Pipeline is trusted and access-controlled trusted
LLM Network Access Model operates in sandboxed environment mitigated
Adversarial Training Data Out of scope for this research excluded
Semantic Vulnerability Introduction Primary threat — requires human review critical
Patch Injection via Input Post-detection only, not on user input mitigated

Threat Vector	Assumption	Status
CI/CD Pipeline Compromise	Pipeline is trusted and access-controlled	trusted
LLM Network Access	Model operates in sandboxed environment	mitigated
Adversarial Training Data	Out of scope for this research	excluded
Semantic Vulnerability Introduction	Primary threat — requires human review	critical
Patch Injection via Input	Post-detection only, not on user input	mitigated

The primary threat considered is semantic vulnerability introduction, where a patch appears correct under testing but weakens security guarantees.

XII. Analytical Observations Derived from Operational Reasoning

Based on analysis of reported industry deployments and controlled reasoning over remediation workflows, several observations emerge:

Approximately 10–20% of sanitizer-detected vulnerabilities exhibit structural simplicity suitable for automated patch generation.

The highest success rates are observed for:

Uninitialized variable usage
Missing bounds checks
Use-after-scope errors
Certain classes of data races

Conversely, vulnerabilities involving complex business logic, cross-module invariants, or protocol-level semantics demonstrate low automation viability.

Qualitative Evaluation Methodology

During controlled reasoning over representative sanitizer reports, patch candidates were qualitatively categorized based on reviewer confidence rather than acceptance metrics. This approach avoids overstating empirical results while still capturing reviewer burden and semantic risk.

Operational Impact Analysis

Manual remediation pipelines often experience exponential backlog growth as detection improves. Even low-severity vulnerabilities accumulate, increasing long-term security debt.

LLM-assisted remediation alters this dynamic by:

Reducing average time-to-fix for structurally simple defects
Allowing security teams to prioritize high-impact issues
Converting remediation from a blocking activity to a background process

Although automation success rates remain limited, their marginal utility increases with scale, producing disproportionate operational benefits.

XIII. Negative Results and Observed Limitations

Several approaches were explored and found ineffective:


Negative Results & Failed Approaches
Verbose Prompting
Degraded patch quality
→ Concise context outperforms detailed explanations
Full Repository Context
Increased hallucinations
→ Minimal context reduces confusion
Aggressive Retry Strategies
Diminishing returns
→ 3-5 retries optimal; more wastes compute
Test-Only Validation
Semantic regressions
→ Human review remains essential
Documenting failures reinforces conservative automation boundaries.

These negative findings reinforce the necessity of conservative automation boundaries and human review checkpoints.

XIV. Security Risks and Ethical Considerations

A. Hallucinated or Misleading Fixes

LLMs may produce fixes that mask symptoms instead of resolving root causes, reduce concurrency to avoid race conditions, or remove failing tests instead of correcting logic.

B. Automation Bias

Reviewers may exhibit increased trust in AI-generated patches, potentially reducing scrutiny. Explicit awareness and training are required to counteract this bias.

C. Ethical Deployment

Ethically responsible deployment requires organizations to treat AI-generated patches as assistive artifacts, not authoritative decisions.

XV. Researcher's Design Rationale

Several intentional design decisions shaped this research:


Researcher's Design Decisions
Post-Detection Focus
Detection is solved; remediation is the bottleneck
Architectural Safety First
System design over model optimization
Untrusted LLM Outputs
Treat all AI suggestions as potentially harmful
CI/CD Integration
Real pipelines, not standalone research tools
These decisions reflect a security-first mindset over novelty pursuit.

These decisions reflect a security-first mindset, emphasizing accountability, reproducibility, and operational safety over novelty.

XVI. Claims and Non-Claims

This Research Does NOT Claim
LLMs can replace security engineers
Automated patching is universally applicable
Test coverage guarantees security
AI-generated code should be trusted by default
This Research DOES Argue
Bounded, assistive automation is viable
Human oversight is non-negotiable
Failure documentation is essential
Architectural discipline enables trust

XVII. Reproducibility and Research Transparency

Although this study does not present controlled experimental results, its architectural reasoning and failure analyses are intended to be reproducible.

Future researchers can replicate this work by:

Instrumenting sanitizer-driven CI pipelines
Logging LLM patch proposals
Tracking acceptance and rejection outcomes
Categorizing failure modes over time

XVIII. Open Problems & Research Directions

Future enhancements may include:

Multi-file and cross-module patch generation
Integration with fuzzing and dependency scanning pipelines
Fine-tuning LLMs on organization-specific secure coding patterns
Formal verification-assisted validation stages

XIX. Threats to Validity

This research acknowledges several threats to validity:

Internal Validity:
Observations are derived from architectural reasoning and reported industry behavior rather than controlled experiments.

External Validity:
Results may not generalize to domains with complex business logic or weak testing infrastructure.

Construct Validity:
Patch quality is evaluated through proxies such as sanitizer cleanliness and reviewer judgment, which may not capture all semantic flaws.

Recognizing these limitations reinforces the need for cautious interpretation and further empirical validation.

XX. Evaluation Metrics (Defined but Not Measured)

Future empirical evaluation should track:

Patch Acceptance Rate (%): Proportion of generated patches approved by reviewers
Mean Time-to-Fix (MTTF): Average elapsed time from detection to merged fix
Reviewer Time per Patch: Human effort required for validation
False-Positive Patch Rate: Patches that pass tests but introduce regressions
Semantic Regression Incidents: Post-merge security weakening events

Defining these metrics upfront enables future reproducibility and objective comparison.

XXI. Why Automated Remediation Remains Hard

Automated remediation is not constrained by syntax, but by intent.

Security vulnerabilities often emerge from implicit assumptions, cross-module invariants, and human design decisions that are not explicitly encoded in code.

LLMs excel at surface-level transformations but struggle with latent intent, making unrestricted automation fundamentally unsafe.

This research treats this limitation not as a failure of models, but as a design constraint that must be respected rather than circumvented.

XXII. Summary of Contributions

This research demonstrates that Large Language Models can serve as effective assistants in vulnerability remediation when embedded within carefully constrained, human-supervised workflows.

While fully autonomous patching remains impractical and unsafe, assisted remediation offers a pragmatic middle ground—reducing time-to-fix while preserving security accountability.

The primary contributions of this work include:

Architectural Framework: A modular, human-in-the-loop pipeline for AI-assisted remediation
Failure Taxonomy: Systematic enumeration of LLM-generated patch failure modes
Threat Model: Security-first design assumptions and accountability structure
Negative Results: Documented ineffective approaches informing future research
Comparative Positioning: Clear differentiation from prior automation paradigms

This work is released as an open, inspectable system to encourage critical evaluation rather than passive adoption.

Research Artifacts Produced

This research produced the following artifacts:

A human-in-the-loop remediation architecture
A structured failure mode taxonomy for LLM-generated patches
A comparative framework positioning LLM-assisted remediation
A threat model tailored to AI-assisted patching systems
A qualitative evaluation methodology for reviewer confidence

These artifacts are intended to be independently reusable beyond this work. All conclusions are independent of any specific LLM implementation, vendor, or training corpus.

Appendix A: Glossary of Terms

Sanitizer: Runtime instrumentation detecting undefined or unsafe behavior during program execution.

Semantic Vulnerability: A flaw that preserves functional correctness but weakens security guarantees.

Automation Bias: Human tendency to over-trust automated system outputs, reducing critical scrutiny.

Human-in-the-Loop: System design requiring explicit human approval at critical decision stages.

Context Reduction: Process of minimizing input code while preserving vulnerability-relevant semantics.

Patch Candidate: An LLM-generated code modification proposed as a potential fix for a detected vulnerability.

Validation Gate: An automated checkpoint that patches must pass before proceeding to human review.

Acknowledgment

This work is informed by publicly available industry research on AI-assisted vulnerability remediation. All analysis, structuring, and interpretations presented herein are original and intended for academic and educational purposes.

No generative model was permitted to make architectural decisions in this work.

This document reflects the state of the research as of January 2026.

AI-Assisted Remediation

Abstract

Keywords

I. Introduction

II. Historical Context: Evolution of Automated Remediation

Evolution of Automated Remediation

Rule-Based

Synthesis

ML Detection

LLM Patch

III. Problem Definition and Research Questions

Research Questions

IV. Scope, Constraints, and Non-Goals

Scope

Constraints

Non-Goals

V. Research Methodology

Research Methodology Stages

Workflow Decomposition

Automation Feasibility Analysis

Failure Mode Enumeration

Architectural Synthesis