Task 5.2: Troubleshooting

Task 5.2: Troubleshoot GenAI applications

Skill 5.2.1: Content handling issues

Kiến thức cần nắm:

  • Context window overflow diagnostics
  • Dynamic chunking strategies
  • Prompt design optimization
  • Truncation-related error analysis

Common Issue: Context window overflow xảy ra khi input + context vượt quá giới hạn tokens của model. Giải pháp: dynamic chunking, prompt compression, hoặc chọn model có context window lớn hơn.

Giải thích chi tiết:

Context Window Overflow Troubleshooting:

SymptomNguyên nhânGiải pháp
ValidationExceptionInput vượt context limitGiảm input size, chọn model lớn hơn
Truncated responsesmax_tokens quá thấpTăng max_tokens
Missing contextRAG trả về quá nhiều chunksGiảm numberOfResults, improve chunking
Irrelevant responsesContext không liên quanImprove retrieval, better chunking

Dynamic Chunking Strategies:

StrategyMô tảBest For
Fixed-sizeChia theo số characters/tokensSimple documents
SemanticChia theo meaning boundariesComplex documents
HierarchicalParent-child chunksStructured documents
Sliding windowOverlapping chunksContinuous text

Skill 5.2.2: FM integration issues

Kiến thức cần nắm:

  • Error logging cho API calls
  • Request validation
  • Response analysis
  • Common error codes và troubleshooting steps

Giải thích chi tiết:

Common Bedrock API Errors:

Error CodeMô tảGiải pháp
ThrottlingExceptionRate limit exceededImplement exponential backoff
ValidationExceptionInvalid request formatCheck request body format
ModelTimeoutExceptionModel took too longReduce input size, use streaming
AccessDeniedExceptionIAM permission issueCheck IAM policies
ServiceUnavailableExceptionService issueRetry with backoff, try another region
ModelNotReadyExceptionModel not availableCheck model access, region availability

Debugging Checklist:

  1. Check CloudWatch Logs cho error details
  2. Verify IAM permissions
  3. Validate request format (JSON structure)
  4. Check model availability trong region
  5. Monitor throttling metrics
  6. Use X-Ray cho end-to-end tracing

Skill 5.2.3: Prompt engineering problems

Kiến thức cần nắm:

  • Prompt testing frameworks
  • Version comparison
  • Systematic refinement process
  • Prompt debugging techniques

Giải thích chi tiết:

Prompt Debugging Process:

  1. Identify issue — Response không đúng, không đầy đủ, hoặc hallucination
  2. Isolate variable — System prompt? User input? Context? Parameters?
  3. Test variations — Thay đổi 1 variable tại 1 thời điểm
  4. Compare versions — A/B test prompt versions
  5. Document findings — Track what works và what doesn’t

Common Prompt Issues:

IssueSymptomFix
Ambiguous instructionsInconsistent outputsBe more specific
Missing contextHallucinationAdd relevant context
Conflicting instructionsConfused responsesSimplify, prioritize
Too many constraintsRefusal to answerRelax constraints
Wrong formatUnstructured outputAdd format examples

Skill 5.2.4: Retrieval system issues

Kiến thức cần nắm:

  • Model response relevance analysis
  • Embedding quality diagnostics
  • Drift monitoring
  • Vectorization issue resolution
  • Chunking và preprocessing remediation
  • Vector search performance optimization

Giải thích chi tiết:

RAG Troubleshooting Flow:

Poor response quality
    ↓
Is retrieval returning relevant docs?
    ├── No → Fix retrieval
    │   ├── Check embedding quality
    │   ├── Improve chunking strategy
    │   ├── Tune search parameters
    │   └── Add metadata filters
    └── Yes → Fix generation
        ├── Improve prompt template
        ├── Add instructions for using context
        └── Tune model parameters

Embedding Quality Diagnostics:

  • Visualize embeddings (t-SNE, UMAP)
  • Check semantic similarity scores
  • Compare different embedding models
  • Verify embedding dimensions match index

Skill 5.2.5: Prompt maintenance issues

Kiến thức cần nắm:

  • Template testing với CloudWatch Logs
  • X-Ray cho prompt observability pipelines
  • Schema validation cho format inconsistencies
  • Systematic prompt refinement workflows

Giải thích chi tiết:

Prompt Maintenance Best Practices:

  • Version control cho tất cả prompts
  • Automated regression testing khi thay đổi prompts
  • Monitor prompt performance metrics over time
  • Use Bedrock Prompt Management cho centralized management
  • Set up alerts cho quality degradation

Tài liệu tham khảo